Opened 12 years ago

# move notebook to flask/wsgi-based notebook — at Version 173

Reported by: Owned by: Jason Grout jason, mpatel, was blocker sage-5.2 notebook sd31 sd35.5 Dan Drake, Jason Grout, Karl-Dieter Crisman, Mike Hansen, Radoslav Kirov, Tim Dumol, Ivan Andrus, Steven Trogdon Mike Hansen, Rado Kirov, William Stein, Jason Grout Rado Kirov, Dan Drake, Jason Grout N/A #11078, #11874, #12229, #11503, #12327

This ticket tracks the progress to move the notebook to a flask/wsgi-based notebook.

Relevant code repository: https://github.com/sagemath/sagenb

Explicit instructions starting with sage-5.0.beta13 including dependencies (replace $SAGE_ROOT with the root sage directory): cd$SAGE_ROOT
./sage -f http://boxen.math.washington.edu/home/keshav/files/sagenb-0.9.0.spkg
./sage -f http://www.uwosh.edu/faculty_staff/gutow/jmol-12.2.21.p0.spkg

./sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11874/trac-11874-remove-twisted.patch
./sage -hg qpush

./sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11503/trac-11503-jmol-spkg.2.patch
./sage -hg qpush

cd devel/sage
../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11874/sage-spkg-11874.patch
../../sage -hg qpush

../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11078/trac_11078.patch
../../sage -hg qpush

../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11503/trac-11503-jmol-commandline.patch
../../sage -hg qpush

../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/12229/trac-12229-sagenb-developer-doc.2.patch
../../sage -hg qpush

../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/12229/trac-12229-manifest.patch
../../sage -hg qpush

../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11080/trac-11080-notebook-docs.patch
../../sage -hg qpush

cd ../../
./sage -br


spkg: http://boxen.math.washington.edu/home/keshav/files/sagenb-0.9.0.spkg (this includes all dependencies not already in Sage)

apply: trac-11080-notebook-docs.patch to sage repository

This is a review version of the spkg. When we have finalized things, we'll make a finished spkg that has the appropriate tags in the revision history, no unnecessary revision branches, etc.

Testing release: http://boxen.math.washington.edu/home/release/sage-5.1.notebook

### comment:2 Changed 11 years ago by Jonathan Gutow

Description: modified (diff)

### comment:4 Changed 11 years ago by Jonathan Gutow

Description: modified (diff)

### comment:5 Changed 11 years ago by Jonathan Gutow

Description: modified (diff)

### comment:6 Changed 11 years ago by Jonathan Gutow

Description: modified (diff)

removed #11078 from list as the fix is included in the 11503 command line patch.

### comment:7 Changed 11 years ago by Jason Grout

Authors: Mike Hansen, Rado Kirov → Mike Hansen, Rado Kirov, William Stein, Jason Grout → #11078 modified (diff)

### comment:8 Changed 11 years ago by Jason Grout

Description: modified (diff)

### comment:9 Changed 11 years ago by Jason Grout

Description: modified (diff)

### comment:10 Changed 11 years ago by Jason Grout

Dependencies: #11078 → #11078, #11874

### comment:11 Changed 11 years ago by Jason Grout

Status: new → needs_review

### comment:12 Changed 11 years ago by Jason Grout

(for future reference, after this is merged, jmol is a high priority. These two tickets deal with these changes:

• #11496 (update jmol script in local/bin to reflect move of jmol directory to local)
• #11503 (updated spkg and paths for move of jmol directory)
• #9238 (for reference -original Jmol enhancement ticket but heavily laden with pre-flask notebook stuff)

Again, this is only FYI and for further reference.

### comment:13 Changed 11 years ago by Jason Grout

Description: modified (diff)

### comment:14 Changed 11 years ago by Jonathan Gutow

I get the following error when I try to apply the spkg dependencies patch. Everything else seems to work. I'm still checking functionality...

applying /home/jonathan/.sage/temp/VB_Ubuntu_11/26614/tmp_0.patch
unable to find 'spkg/install' for patching
1 out of 1 hunks FAILED -- saving rejects to file spkg/install.rej
unable to find 'spkg/standard/deps' for patching
4 out of 4 hunks FAILED -- saving rejects to file spkg/standard/deps.rej
spkg/install: No such file or directory
spkg/standard/deps: No such file or directory
abort: patch failed to apply


### comment:15 Changed 11 years ago by Jonathan Gutow

With 4.8.alpha4, what is described here does not work. I redid everything. The .spkgs seem to install without error. The patch for this ticket does not work and the notebook fails to launch with the error:

/home/jonathan/Documents/sage-4.8.alpha4/<ipython console> in <module>()

/home/jonathan/Documents/sage-4.8.alpha4/devel/sagenb/sagenb/notebook/notebook_object.py in __call__(self, *args, **kwds)
202     """
203     def __call__(self, *args, **kwds):
--> 204         return self.notebook(*args, **kwds)
205
206     notebook = run_notebook.notebook_twisted

/home/jonathan/Documents/sage-4.8.alpha4/devel/sagenb/sagenb/notebook/run_notebook.py in notebook_twisted(self, directory, port, interface, address, port_tries, secure, reset, require_login, accounts, openid, server_pool, ulimit, timeout, open_viewer, sagetex_path, start_path, fork, quiet, subnets)
442     if open_viewer:
--> 444     return run(port)
445

/home/jonathan/Documents/sage-4.8.alpha4/devel/sagenb/sagenb/notebook/run_notebook.py in run(port)
427         os.chdir(cwd)
428         if e == 256:
--> 429             raise socket.error
430
431         return True



### comment:16 Changed 11 years ago by Jason Grout

It's possible that the SAGE_ROOT patch needs to be rebased to 4.8.alpha4. Let me check. Thanks for the test

### comment:17 Changed 11 years ago by Jason Grout

Description: modified (diff)

Jonathan,

I think the problem is that you didn't apply the patch here to the SAGE_ROOT repository. I've updated the instructions above with explicit commands.

### comment:18 Changed 11 years ago by Jason Grout

I'm checking now into the error you posted in your second. It appears that there is indeed a problem with the sagenb spkg not overwriting the old notebook.

### comment:19 follow-up:  21 Changed 11 years ago by Jason Grout

Description: modified (diff)

For future reference, here are the instructions if we had a sagenb_dependencies spkg. DON'T USE THESE--use the instructions in the description of the ticket.

cd $SAGE_ROOT ./sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11080/trac-11080-sagenb_dependencies-spkg.patch ./sage -hg qpush ./sage -f http://sage.math.washington.edu/home/jason/sagenb_dependencies-20111216.spkg ./sage -f http://sage.math.washington.edu/home/jason/twisted-11.0.0.spkg ./sage -f http://sage.math.washington.edu/home/jason/sagenb-0.9.0.spkg cd devel/sage ../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11078/trac_11078.patch ../../sage -hg qpush cd ../../ ./sage -br  ### comment:20 Changed 11 years ago by Jason Grout Description: modified (diff) ### comment:21 in reply to: 19 Changed 11 years ago by Jonathan Gutow OK, with the changed instructions and this explicit instruction set, I get a running sage-4.8.alpha4. The notebook launches and everything I tried other than 3-D plotting worked in the notebook. 3-D plotting does not work because Jmol seems to be gone. Is #11503 supposed to be included as well? That would install the new Jmol in the$SAGE_ROOT/local/shared directory rather than in the sagenb/data directory. I note that there is no sagenb/data/jmol directory in the notebook created by this ticket, while there is in the old 4.8.alpha4 notebook.

I think it's almost there. Jonathan Replying to jason:

For future reference, here are the instructions if we had a sagenb_dependencies spkg. DON'T USE THESE--use the instructions in the description of the ticket.

### comment:107 follow-up:  108 Changed 11 years ago by Jonathan Gutow

Arrrgh...after rebuild (./sage -b). The notebook launches with an error

ImportError                               Traceback (most recent call last)

/home/jonathan/.sage/<ipython console> in <module>()

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/notebook_object.pyc in __call__(self, *args, **kwds)
215     """
216     def __call__(self, *args, **kwds):
--> 217         return self.notebook(*args, **kwds)
218
219     notebook = run_notebook.notebook_twisted

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/run_notebook.pyc in notebook_twisted(self, directory, port, interface, address, port_tries, secure, reset, accounts, require_login, server_pool, ulimit, timeout, open_viewer, sagetex_path, start_path, fork, quiet, subnets)
200         print '*' * 70
201
203
204     directory = nb._dir

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/notebook.pyc in load_notebook(dir, interface, port, secure)
1807     # mainly to avoid circular references, etc.  This also means
1808     # only one notebook can actually be used at any point.
-> 1809     import sagenb.notebook.twist
1810     sagenb.notebook.twist.notebook = nb
1811

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/twist.py in <module>()
40
41 from twisted.python import log
---> 42 from twisted.web2 import server, http, resource, channel
43 from twisted.web2 import static, http_headers, responsecode
44 from twisted.web2.filter import gzip

ImportError: No module named web2


Did I miss something or is there a new problem?

### comment:108 in reply to:  107 Changed 11 years ago by Dima Pasechnik

ImportError?: No module named web2 }}} Did I miss something or is there a new problem?

I bet you did, as there is no web2 in the new twisted (version 11).

### comment:109 Changed 11 years ago by Jason Grout

And there is no twist.py in the new notebook... Are you *sure* that you installed the sagenb spkg above? Does devel/sagenb-main/flask_version exist?

You may need to do sage -f instead of sage -i when you install the spkgs above so they overwrite old versions of the same spkg.

### comment:110 Changed 11 years ago by Jonathan Gutow

Yep...the sagenb spkg install failed on a missing header for ssl. Somehow I missed the error message. I have the ssl binaries but not the header files on my systems. I guess ssl.h be part of the package or they need to be listed as prerequisite for the sagenb spkg. I'm running on the same Ubuntu 11.04 that I successfully installed this on through 4.8.

Here's the relevant part of the log file.

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-pWFAnl/pyOpenSSL-0.13/egg-dist-tmp-4xiQaN
warning: no previously-included files matching '*.pyc' found anywhere in distribution
In file included from OpenSSL/crypto/crypto.h:30:0,
from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:111 Changed 11 years ago by Jonathan Gutow

sagenb spkg requires openssl and openssl-dev. After adding openssl-dev to my system the spkg installs fine. If I missed this dependency sorry. If not it needs to be clear somewhere.

### comment:112 Changed 11 years ago by Jason Grout

Should we put an explicit check for that file into the spkg-install? If so, what should that check look like? We should also clearly list this somewhere.

### comment:113 in reply to:  101 Changed 11 years ago by Punarbasu Purkayastha

I explicitly tried each of those steps, and things worked flawlessly. I'm trying this on a 5.0 beta (which is where the notebook should be tested). I wonder if that makes a difference (it shouldn't, but who knows?).

Just to make sure: did you explicitly do each of the steps in the description of this ticket?

Also, is there any error message printed out for you in the terminal?

Thanks to kini we have the answer to this. It seems the error is triggered only when the password length is less than 6 characters. This means that the min password length of 6 is hard-coded in some file. The new git revisions of the nb which has set the variable min_password_length to 6 effectively masks the problem without fixing it.

### comment:114 Changed 11 years ago by Steven Trogdon

I have a question about the following chunk of code

try:
# simplejson is faster, so try to import it first
import simplejson as json
except ImportError:
import json


in sagenb/notebook/misc.py. It would seem that Sage is not shipped with simplejson. In testing the flask notebook on sage-on-gentoo the sagenb/notebook/misc.py test is the only failing notebook test because simplejson, which is pulled in by werkzeug as a runtime dependency, is found. The failure, roughly, is

sage -t -long  -force_lib /storage/strogdon/gentoo/usr/lib/python2.7/site-packages/sagenb/notebook/misc.py
**********************************************************************
File "/storage/strogdon/gentoo/usr/lib/python2.7/site-packages/sagenb/notebook/misc.py", line 221:
sage: print encode_response(d, separators = (', ', ': '), indent = 4)
Exception raised:
Traceback (most recent call last):
...
File "element.pyx", line 1305, in sage.structure.element.RingElement.__add__ (sage/structure/element.c:11569)
File "coerce.pyx", line 797, in sage.structure.coerce.CoercionModel_cache_maps.bin_op (sage/structure/coerce.c:7467)
TypeError: unsupported operand parent(s) for '+': '<type 'str'>' and 'Integer Ring'
**********************************************************************


If I remove the conditional the test passes. So, the question is, is the conditional reference to simplejson needed? Of course, the s-o-g flask notebook can always be patched.

### comment:115 Changed 11 years ago by Keshav Kini

Patching the sage-on-gentoo flask notebook is not the solution. Test failures should not occur no matter what python modules, or indeed other programs, the user has installed into Sage's directory.

But there's actually a new failure, on 5.0.beta4:

sage -t  misc/sageinspect.py
**********************************************************************
File "/home/fs/src/sagenb/sagenb/misc/sageinspect.py", line 653:
sage: sage_getargspec(Poset)
Expected:
(['data', 'element_labels', 'cover_relations'], None, None, (None, None, False))
Got:
(['data', 'element_labels', 'cover_relations', 'category', 'facade', 'key'], None, None, (None, None, False, None, None, None))
**********************************************************************
1 of   7 in __main__.example_16
***Test Failed*** 1 failures.


I presume this is caused by #10998.

### comment:116 Changed 11 years ago by Keshav Kini

(Why the heck is sageinspect in sagenb anyway...)

### comment:117 Changed 11 years ago by Keshav Kini

I submitted a pull request to update the failing doctest. Looking at the simple_json one now...

### comment:118 Changed 11 years ago by Keshav Kini

The problem is as follows. In the json module, we have

If indent is a non-negative integer, then JSON array elements and
object members will be pretty-printed with that indent level. An indent
level of 0 will only insert newlines. None is the most compact
representation.


On the other hand, in the simple_json module, we have

If indent is a string, then JSON array elements and object members
will be pretty-printed with a newline followed by that string repeated
for each level of nesting. None (the default) selects the most compact
representation without any newlines. For backwards compatibility with
versions of simplejson earlier than 2.1.0, an integer is also accepted
and is converted to a string with that many spaces.


Sage integers do not trigger the following lines in simple_json (simplejson/encoder.py:173):

        if isinstance(indent, (int, long)):
indent = ' ' * indent
self.indent = indent


Unfortunately it doesn't seem like the json module supports this new string form of the indent argument, which is why it works with our code - it just does ' ' * indent without caring that it's a Sage integer. Apparently the json module is just an old version of the simplejson module which is bundled with Python, so it will get the new string-based indent parameter eventually, but for now we'll either have to require an up-to-date simplejson for the new notebook, or remember/detect which of the two modules we imported and call the function accordingly.

### comment:120 Changed 11 years ago by Keshav Kini

The pull request has been merged into simplejson, so the test will no longer fail once the next version of simplejson is released.

### comment:121 follow-up:  122 Changed 11 years ago by Keshav Kini

The new version of simplejson has been released. strogdon, if you upgrade simplejson to 2.3.3 the test failure should be gone :)

### comment:122 in reply to:  121 Changed 11 years ago by Steven Trogdon

The new version of simplejson has been released. strogdon, if you upgrade simplejson to 2.3.3 the test failure should be gone :)

Good deal. I just upgraded to 2.3.3 and the test passes. Thanks.

### comment:123 follow-up:  124 Changed 11 years ago by Steven Trogdon

It's not clear from this ticket which of the twisted patches from #11874 should be applied. I believe it should be trac-11874-remove-twisted.3.patch . At least that one as well as the indicated jmol-spkg patch apply on top of 5.0.beta6 and I'm able to proceed to the "sage -br" step to get the new notebook working.

### comment:124 in reply to:  123 Changed 11 years ago by Jeroen Demeyer

It's not clear from this ticket which of the twisted patches from #11874 should be applied.

Considering that ticket needs_work, the "correct" patch for #11874 still needs to be written.

### comment:125 Changed 11 years ago by Jason Grout

Current status:

#11874 needs to be fixed and reviewed. There is a failing doctest, but I think the failing doctest may be resolved by using the sage.misc.fpickle module (I'm not sure, though)

#12229 needs review (and possibly a fix for the git_links)

#11503 needs review (and looks like it is ready to go...)

Merge pull request https://github.com/sagemath/sagenb/pull/40

### comment:126 follow-up:  128 Changed 11 years ago by Dima Pasechnik

It appears that when we build Python 2.7.2 we automatically pull the latest Twisted (currently version 12) from the net and install it into SAGE_LOCAL.

We definitely, definitely do NOT want TWO twisted frameworks installed!!!

### comment:127 Changed 11 years ago by Jason Grout

Description: modified (diff)

Change urls to boxen instead of sage.math since sage.math is down.

### comment:128 in reply to:  126 Changed 11 years ago by Dima Pasechnik

It appears that when we build Python 2.7.2 we automatically pull the latest Twisted (currently version 12) from the net and install it into SAGE_LOCAL.

Please ignore this comment. It was somehow due to my confusion when I recently looked at #11874.

Please also note that #11874 needs review now.

### comment:129 Changed 11 years ago by Dima Pasechnik

running Explicit instructions from the ticket description gives one Twisted 12, not (as intended) Twisted 11.1. This needs to be fixed: please see http://trac.sagemath.org/sage_trac/ticket/11874#comment:62

### comment:130 Changed 11 years ago by Jeroen Demeyer

Status: needs_info → needs_review

There is a doctest error:

sage -t  -force_lib devel/sagenb-main/sagenb/misc/sageinspect.py
**********************************************************************
sage: sage_getargspec(Poset)
Expected:
(['data', 'element_labels', 'cover_relations'], None, None, (None, None, False))
Got:
(['data', 'element_labels', 'cover_relations', 'category', 'facade', 'key'], None, None, (None, None, False, None, None, None))
**********************************************************************


### comment:131 follow-up:  132 Changed 11 years ago by Jeroen Demeyer

Status: needs_review → needs_work

### comment:132 in reply to:  131 ; follow-up:  135 Changed 11 years ago by Dima Pasechnik

Replying to jdemeyer: as mentioned here one just needs to update the spkg using this Git commit

### comment:133 Changed 11 years ago by Keshav Kini

Well, you need to update the spkg to the current master branch, which happens to contain that commit in its ancestry, and thus contains the code change. (Commits are not something you apply individually like patches.)

### comment:134 Changed 11 years ago by Keshav Kini

Description: modified (diff) needs_work → needs_review

Here's a new spkg which should pass doctests: http://boxen.math.washington.edu/home/keshav/files/sagenb-0.9.0.spkg

Are there any other work issues?

### comment:135 in reply to:  132 Changed 11 years ago by Keshav Kini

Status: needs_review → needs_work

Hmm, there is something weird when I install the spkg and run sage -t \$SAGE_ROOT/devel/sagenb-main: 1) README.rst is getting doctested, which throws up a password prompt; 2) there's a tab character in flask_version/base.py .

Why were these not showing up in earlier doctests? They're not new problems, as far as I can tell.

### comment:136 Changed 11 years ago by Keshav Kini

Status: needs_work → needs_review

OK, uploaded a new SPKG to the same URL. Now it definitely does pass all its doctests.

### comment:137 Changed 11 years ago by Jason Grout

Just curious, the new spkg probably has Twisted 12.0, right? Will people at #11874 complain that Twisted was updated in the spkg?

### comment:138 follow-up:  140 Changed 11 years ago by Keshav Kini

Yes, it contains Twisted 12. Yes, they might complain... feel free to create your own spkg with Twisted 11 instead - I might have messed something up anyway :)

### comment:139 follow-up:  141 Changed 11 years ago by Jason Grout

I won't have very much time this week to work on this, so I'm happy letting you make an spkg.

### comment:140 in reply to:  138 Changed 11 years ago by Dima Pasechnik

Yes, it contains Twisted 12. Yes, they might complain... feel free to create your own spkg with Twisted 11 instead - I might have messed something up anyway :)

No complaints from me. Actually, with the old sagenb 0.9.0 (ie with Twisted 11), when I replaced sagenb spkg in Sage 5.0.beta8 with 0.9.0, and then followed the procedure on #11874, I got Sage built with Twisted 11.1, instead of Twisted 12. So Twisted 12 was sneaking in as it was getting installed before the installation of the old sagenb 0.9.0.

### comment:141 in reply to:  139 Changed 11 years ago by Keshav Kini

I won't have very much time this week to work on this, so I'm happy letting you make an spkg.

OK, just checking that I did everything right... first I cloned the git repo to another git repo (to just get master and no other commits), then used hg-git to pull that clone into a newly-initialized hg repo. Then I ran ./spkg-dist to create the spkg. Did I do anything wrong?

### comment:142 Changed 11 years ago by Jeroen Demeyer

Status: needs_review → needs_work

I again get

sage -t  -force_lib devel/sagenb-main/sagenb/misc/sageinspect.py
**********************************************************************
File "/scratch/jdemeyer/merger/sage-5.0.notebook/devel/sagenb-main/sagenb/misc/sageinspect.py", line 653:
sage: sage_getargspec(Poset)
Expected:
(['data', 'element_labels', 'cover_relations'], None, None, (None, None, False))
Got:
(['data', 'element_labels', 'cover_relations', 'category', 'facade', 'key'], None, None, (None, None, False, None, None, None))
**********************************************************************


### comment:143 Changed 11 years ago by Jason Grout

Description: modified (diff)

Update instructions to reflect another patch on #11874 (I didn't test the instructions, though!)

### comment:144 Changed 11 years ago by Punarbasu Purkayastha

Description: modified (diff)

Update to new jmol patch for SAGE_ROOT.

Also, after applying the changes from #11874, #11503 and this ticket, the doctest of devel/sagenb-main/sagenb passes.

After these changes however, there is a problem with typesetting maths and I get this notice on evaluation of every cell, whether it is plot or maths.

Error typesetting mathematics.

And sure enough, something as simple as view(2+2) shows the latex code instead of the maths. jsmath is loaded and the button is present on the right hand corner.

UPDATE: I have same problem with sagenb from github with mathjax instead of jsmath. Seems like something is broken in sage and not sagenb.

Last edited 11 years ago by Punarbasu Purkayastha (previous) (diff)

### comment:145 follow-up:  152 Changed 10 years ago by Jeroen Demeyer

Milestone: sage-5.0 → sage-5.1

We would like to release sage-5.0 reasonably soon, and it's too late to get this still in sage-5.0.

### comment:146 follow-up:  147 Changed 10 years ago by Keshav Kini

Description: modified (diff) needs_work → needs_review

Jeroen: Either the version you installed was from the URL in the ticket description (the one Jason uploaded a while ago) and not the SPKG I posted in a comment recently, or I screwed up making my SPKG and mistakenly packaged an old version of the sagenb code. So to cover both possibilities, I changed the link in the ticket description to point to the SPKG I'm hosting, and I also repacked the SPKG and reuploaded it. It passes doctests on my machine. Can you try it again?

### comment:147 in reply to:  146 Changed 10 years ago by Jeroen Demeyer

Jeroen: Either the version you installed was from the URL in the ticket description (the one Jason uploaded a while ago)

of course the version I installed was from the URL in the ticket description. I'm not going to read 146 comments to find out the URL of the spkg to test.

### comment:148 follow-up:  149 Changed 10 years ago by Keshav Kini

Right, sorry about that. From your statement that you "again" saw the doctest errors, I assumed that at some point you had not seen the errors, which would have meant that you had installed my SPKG at some point. But I guess you just meant that you were continuing to see the doctest errors. My mistake :)

### comment:149 in reply to:  148 Changed 10 years ago by Jeroen Demeyer

Right, sorry about that. From your statement that you "again" saw the doctest errors, I assumed that at some point you had not seen the errors, which would have meant that you had installed my SPKG at some point.

Maybe I had installed your SPKG at some point, then later forgot about your SPKG and installed Jason's SPKG again. I don't remember...

### comment:150 Changed 10 years ago by Keshav Kini

Fair enough - anyway it's my fault for not putting the correct info in the ticket description.

### comment:151 Changed 10 years ago by Jonathan Gutow

Description: modified (diff)

update jmol spkg to match #11503

### comment:152 in reply to:  145 ; follow-up:  153 Changed 10 years ago by William Stein

We would like to release sage-5.0 reasonably soon, and it's too late to get this still in sage-5.0.

I agree with this decision.

### comment:153 in reply to:  152 Changed 10 years ago by Jason Grout

We would like to release sage-5.0 reasonably soon, and it's too late to get this still in sage-5.0.

I agree with this decision.

Same here. I had to put the proverbial rallying flag down due to work, but no one has really picked it back up again and finished the job. I'll be able to work on this more in a few weeks after our classes get out, if someone else hasn't started pushing it again by then.

### comment:154 Changed 10 years ago by Keshav Kini

What exactly remains to be done, other than the minor hiccup on #11503?

### comment:155 follow-up:  157 Changed 10 years ago by Jason Grout

What I would do to answer that question is go through each ticket dependency and see if it is positive-reviewed. If not, then that ticket needs to be done :). I don't think there is much left to be done.

### comment:156 Changed 10 years ago by Jonathan Gutow

Description: modified (diff)

### comment:157 in reply to:  155 ; follow-up:  158 Changed 10 years ago by Dima Pasechnik

What I would do to answer that question is go through each ticket dependency and see if it is positive-reviewed. If not, then that ticket needs to be done :). I don't think there is much left to be done.

I think one should provide a recipe for preparing a sage distribution with all these patches applied to be built from scratch, rather than patching an already built distribution. (Assuming one has hg (or a previous version of Sage) installed system-wide). That is, most of these patches are to be applied to sage-??? spkg.

It makes more sense this way, as the changes are so extensive that it's time-consuming, and more error-prone, I think, to apply these to an already built installation of Sage.

### comment:158 in reply to:  157 Changed 10 years ago by Jeroen Demeyer

I think one should provide a recipe for preparing a sage distribution with all these patches applied to be built from scratch, rather than patching an already built distribution. (Assuming one has hg (or a previous version of Sage) installed system-wide). That is, most of these patches are to be applied to sage-??? spkg.

I have made http://boxen.math.washington.edu/home/release/sage-5.0.notebook/sage-5.0.notebook.tar to test this ticket and its dependencies.

### comment:159 Changed 10 years ago by Jeroen Demeyer

Status: needs_review → needs_work

This fails to install on OS X 10.4 PPC:

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-h8pJu5/pyOpenSSL-0.13/egg-dist-tmp-Awaj3a
warning: no previously-included files matching '*.pyc' found anywhere in distribution
OpenSSL/ssl/connection.c: In function 'ssl_Connection_set_context':
OpenSSL/ssl/connection.c:289:5: warning: implicit declaration of function 'SSL_set_SSL_CTX' [-Wimplicit-function-declaration]
OpenSSL/ssl/connection.c: In function 'ssl_Connection_get_servername':
OpenSSL/ssl/connection.c:313:16: error: 'TLSEXT_NAMETYPE_host_name' undeclared (first use in this function)
OpenSSL/ssl/connection.c:313:16: note: each undeclared identifier is reported only once for each function it appears in
OpenSSL/ssl/connection.c:320:5: warning: implicit declaration of function 'SSL_get_servername' [-Wimplicit-function-declaration]
OpenSSL/ssl/connection.c:320:10: warning: assignment makes pointer from integer without a cast [enabled by default]
OpenSSL/ssl/connection.c: In function 'ssl_Connection_set_tlsext_host_name':
OpenSSL/ssl/connection.c:346:5: warning: implicit declaration of function 'SSL_set_tlsext_host_name' [-Wimplicit-function-declaration]
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:160 Changed 10 years ago by Jeroen Demeyer

This also fails to install on the Skynet machine silius (SUSE ES 11 SP1 ppc64):

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-_3ma5D/pyOpenSSL-0.13/egg-dist-tmp-efVFsy
warning: no previously-included files matching '*.pyc' found anywhere in distribution
In file included from OpenSSL/crypto/crypto.h:30:0,
from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:161 Changed 10 years ago by Keshav Kini

According to http://stackoverflow.com/questions/7340784/easy-install-pyopenssl-error we should be able to fix this by requiring pyOpenSSL 0.12 or lower.

Last edited 10 years ago by Keshav Kini (previous) (diff)

### comment:162 follow-up:  166 Changed 10 years ago by Jeroen Demeyer

What's the situation with OpenSSL support anyway? I believe it was decided not to require OpenSSL as dependency? But there have been many discussions about this, I might have lost track.

### comment:163 Changed 10 years ago by Jeroen Demeyer

Also on the Skynet machine cleo (RHEL 5.3 ia64):

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-PQFiHf/pyOpenSSL-0.13/egg-dist-tmp-XsFtPR
warning: no previously-included files matching '*.pyc' found anywhere in distribution
OpenSSL/ssl/connection.c: In function ssl_Connection_set_context:
OpenSSL/ssl/connection.c:289:5: warning: implicit declaration of function SSL_set_SSL_CTX [-Wimplicit-function-declaration]
OpenSSL/ssl/connection.c: In function ssl_Connection_get_servername:
OpenSSL/ssl/connection.c:313:16: error: TLSEXT_NAMETYPE_host_name undeclared (first use in this function)
OpenSSL/ssl/connection.c:313:16: note: each undeclared identifier is reported only once for each function it appears in
OpenSSL/ssl/connection.c:320:5: warning: implicit declaration of function SSL_get_servername [-Wimplicit-function-declaration]
OpenSSL/ssl/connection.c:320:10: warning: assignment makes pointer from integer without a cast [enabled by default]
OpenSSL/ssl/connection.c: In function ssl_Connection_set_tlsext_host_name:
OpenSSL/ssl/connection.c:346:5: warning: implicit declaration of function SSL_set_tlsext_host_name [-Wimplicit-function-declaration]
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:164 Changed 10 years ago by Jeroen Demeyer

On the Skynet machine iras (SUSE ES10 ia64):

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-9gn9yE/pyOpenSSL-0.13/egg-dist-tmp-4CCNMQ
warning: no previously-included files matching '*.pyc' found anywhere in distribution
In file included from OpenSSL/crypto/crypto.h:30:0,
from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:165 Changed 10 years ago by Jeroen Demeyer

On UGA rosemary (RHEL 5.6 x86_64):

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-urCg6W/pyOpenSSL-0.13/egg-dist-tmp-fw0DPw
warning: no previously-included files matching '*.pyc' found anywhere in distribution
OpenSSL/ssl/connection.c: In function ‘ssl_Connection_set_context’:
OpenSSL/ssl/connection.c:289:5: warning: implicit declaration of function ‘SSL_set_SSL_CTX’ [-Wimplicit-function-declaration]
OpenSSL/ssl/connection.c: In function ‘ssl_Connection_get_servername’:
OpenSSL/ssl/connection.c:313:16: error: ‘TLSEXT_NAMETYPE_host_name’ undeclared (first use in this function)
OpenSSL/ssl/connection.c:313:16: note: each undeclared identifier is reported only once for each function it appears in
OpenSSL/ssl/connection.c:320:5: warning: implicit declaration of function ‘SSL_get_servername’ [-Wimplicit-function-declaration]
OpenSSL/ssl/connection.c:320:10: warning: assignment makes pointer from integer without a cast [enabled by default]
OpenSSL/ssl/connection.c: In function ‘ssl_Connection_set_tlsext_host_name’:
OpenSSL/ssl/connection.c:346:5: warning: implicit declaration of function ‘SSL_set_tlsext_host_name’ [-Wimplicit-function-declaration]
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:166 in reply to:  162 Changed 10 years ago by Dima Pasechnik

What's the situation with OpenSSL support anyway? I believe it was decided not to require OpenSSL as dependency? But there have been many discussions about this, I might have lost track.

so we'll require openssl, but we need to sort out the versions issue, requiring pyOpenSSL 0.12 or lower, as Keshav suggests.

### comment:167 Changed 10 years ago by Keshav Kini

New SPKG with pyOpenSSL 0.12 uploaded. All tests pass on sage.math. Jeroen, can you test it on the buildbots again? It seems like all the errors you reported are coming from pyOpenSSL so I hope downgrading it will solve them...

### comment:168 Changed 10 years ago by Keshav Kini

Status: needs_work → needs_review

Sorry, I guess I should mark this needs_review again.

### comment:169 Changed 10 years ago by Jeroen Demeyer

Description: modified (diff)

### comment:170 Changed 10 years ago by Simon King

I tried to follow the instructions, but the patch from #11874 fails to apply. Interesting that it was possible to create a testing release, then...

### comment:171 Changed 10 years ago by Keshav Kini

Huh, really? For me, the only dependency that didn't apply was #12327. #11874 worked fine for me.

### comment:172 Changed 10 years ago by Simon King

I think I start to understand. It does not apply with sage-5.0.prealpha0. But in sage-5.0.beta13, the files look totally different.

So, I may (at some point) try again with the latest beta.

### comment:173 Changed 10 years ago by Simon King

Description: modified (diff)

The instructions did not tell that one is supposed to start with the latest beta version. Starting with 5.0.prealpha0, things fail to apply.

Note: See TracTickets for help on using tickets.