Opened 10 years ago

Closed 9 years ago

# move notebook to flask/wsgi-based notebook

Reported by: Owned by: jason blocker sage-5.2 notebook sd31 sd35.5 ddrake, jason, kcrisman, mhansen, rkirov, timdumol, iandrus, strogdon sage-5.2.beta0 Mike Hansen, Radoslav Kirov, William Stein, Jason Grout, Jeroen Demeyer Radoslav Kirov, Dan Drake, Jason Grout, Simon King, Dmitrii Pasechnik, John Palmieri, Punarbasu Purkayastha N/A #13126, #11078, #11874, #12229, #11503, #12327, #12899, #13270

This ticket tracks the progress to move the notebook to a flask/wsgi-based notebook.

Relevant code repository: https://github.com/sagemath/sagenb

Explicit instructions to apply relevant patches for this whole group of tickets, starting from a working installation of sage-5.0.beta13 or later (replace $SAGE_ROOT with the root sage directory): cd$SAGE_ROOT
./sage -i openssl # optional if you have systemwide SSL dev headers installed
./sage -f http://boxen.math.washington.edu/home/keshav/files/sagenb-0.9.0.spkg
./sage -f http://www.uwosh.edu/faculty_staff/gutow/jmol-12.2.21.p0.spkg

./sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11874/trac-11874-remove-twisted.patch
./sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11503/trac-11503-jmol-spkg.2.patch

cd local/bin
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11080/trac_11080-scripts-hgignore-dulwich.patch

cd ../../devel/sage
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11874/sage-spkg-11874.patch
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11078/trac_11078.patch
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11503/trac-11503-jmol-commandline.patch
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/12229/trac_12229-sagenb-developer-doc.3.patch
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/12229/trac-12229-manifest.patch
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11080/trac-11080-notebook-docs.patch

### NOTE: Skip the following one line if you are using Sage 5.0.rc1 or later
../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/12899/12899_install_guide.patch

../../sage -hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11080/trac_11080-openssl-in-installation-docs.patch

cd ../../
./sage -b


Alternatively, use the testing release http://boxen.math.washington.edu/home/release/sage-5.2.alpha0/sage-5.2.alpha0.tar (which includes this tickets and its dependencies but not #12229).

spkg: http://boxen.math.washington.edu/home/keshav/files/sagenb-0.9.0.spkg includes all dependencies not already in Sage. Please install it as part of the process described above, as it depends on a bunch of patches.

apply:

1. trac-11080-notebook-docs.patch to the Sage library
2. trac_11080-openssl-in-installation-docs.patch to the Sage library
3. trac_11080-scripts-hgignore-dulwich.patch to sage_scripts.

This is a review version of the spkg. When we have finalized things, we'll make a finished spkg that has the appropriate tags in the revision history, no unnecessary revision branches, etc.

### comment:1 Changed 10 years ago by jason

• Cc ddrake added

### comment:2 Changed 10 years ago by gutow

• Description modified (diff)

### comment:3 Changed 10 years ago by gutow

• Description modified (diff)
• Keywords sd31 added

### comment:4 Changed 10 years ago by gutow

• Description modified (diff)

### comment:5 Changed 10 years ago by gutow

• Description modified (diff)

### comment:6 Changed 10 years ago by gutow

• Description modified (diff)

removed #11078 from list as the fix is included in the 11503 command line patch.

### comment:7 Changed 9 years ago by jason

• Authors changed from Mike Hansen, Rado Kirov to Mike Hansen, Rado Kirov, William Stein, Jason Grout
• Dependencies set to #11078
• Description modified (diff)

### comment:8 Changed 9 years ago by jason

• Description modified (diff)

### comment:9 Changed 9 years ago by jason

• Description modified (diff)

### comment:10 Changed 9 years ago by jason

• Dependencies changed from #11078 to #11078, #11874

### comment:11 Changed 9 years ago by jason

• Status changed from new to needs_review

### comment:12 Changed 9 years ago by jason

(for future reference, after this is merged, jmol is a high priority. These two tickets deal with these changes:

• #11496 (update jmol script in local/bin to reflect move of jmol directory to local)
• #11503 (updated spkg and paths for move of jmol directory)
• #9238 (for reference -original Jmol enhancement ticket but heavily laden with pre-flask notebook stuff)

Again, this is only FYI and for further reference.

### comment:13 Changed 9 years ago by jason

• Description modified (diff)

### comment:14 Changed 9 years ago by gutow

I get the following error when I try to apply the spkg dependencies patch. Everything else seems to work. I'm still checking functionality...

applying /home/jonathan/.sage/temp/VB_Ubuntu_11/26614/tmp_0.patch
unable to find 'spkg/install' for patching
1 out of 1 hunks FAILED -- saving rejects to file spkg/install.rej
unable to find 'spkg/standard/deps' for patching
4 out of 4 hunks FAILED -- saving rejects to file spkg/standard/deps.rej
spkg/install: No such file or directory
spkg/standard/deps: No such file or directory
abort: patch failed to apply


### comment:15 Changed 9 years ago by gutow

With 4.8.alpha4, what is described here does not work. I redid everything. The .spkgs seem to install without error. The patch for this ticket does not work and the notebook fails to launch with the error:

/home/jonathan/Documents/sage-4.8.alpha4/<ipython console> in <module>()

/home/jonathan/Documents/sage-4.8.alpha4/devel/sagenb/sagenb/notebook/notebook_object.py in __call__(self, *args, **kwds)
202     """
203     def __call__(self, *args, **kwds):
--> 204         return self.notebook(*args, **kwds)
205
206     notebook = run_notebook.notebook_twisted

/home/jonathan/Documents/sage-4.8.alpha4/devel/sagenb/sagenb/notebook/run_notebook.py in notebook_twisted(self, directory, port, interface, address, port_tries, secure, reset, require_login, accounts, openid, server_pool, ulimit, timeout, open_viewer, sagetex_path, start_path, fork, quiet, subnets)
442     if open_viewer:
443         "Open viewer automatically isn't fully implemented.  You have to manually open your web browser to the above URL."
--> 444     return run(port)
445

/home/jonathan/Documents/sage-4.8.alpha4/devel/sagenb/sagenb/notebook/run_notebook.py in run(port)
427         os.chdir(cwd)
428         if e == 256:
--> 429             raise socket.error
430
431         return True



### comment:16 Changed 9 years ago by jason

It's possible that the SAGE_ROOT patch needs to be rebased to 4.8.alpha4. Let me check. Thanks for the test

### comment:17 Changed 9 years ago by jason

• Description modified (diff)

Jonathan,

I think the problem is that you didn't apply the patch here to the SAGE_ROOT repository. I've updated the instructions above with explicit commands.

### comment:18 Changed 9 years ago by jason

I'm checking now into the error you posted in your second. It appears that there is indeed a problem with the sagenb spkg not overwriting the old notebook.

### comment:19 follow-up: ↓ 21 Changed 9 years ago by jason

• Description modified (diff)

For future reference, here are the instructions if we had a sagenb_dependencies spkg. DON'T USE THESE--use the instructions in the description of the ticket.

cd $SAGE_ROOT ./sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11080/trac-11080-sagenb_dependencies-spkg.patch ./sage -hg qpush ./sage -f http://sage.math.washington.edu/home/jason/sagenb_dependencies-20111216.spkg ./sage -f http://sage.math.washington.edu/home/jason/twisted-11.0.0.spkg ./sage -f http://sage.math.washington.edu/home/jason/sagenb-0.9.0.spkg cd devel/sage ../../sage -hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11078/trac_11078.patch ../../sage -hg qpush cd ../../ ./sage -br  ### comment:20 Changed 9 years ago by jason • Description modified (diff) ### comment:21 in reply to: ↑ 19 Changed 9 years ago by gutow OK, with the changed instructions and this explicit instruction set, I get a running sage-4.8.alpha4. The notebook launches and everything I tried other than 3-D plotting worked in the notebook. 3-D plotting does not work because Jmol seems to be gone. Is #11503 supposed to be included as well? That would install the new Jmol in the$SAGE_ROOT/local/shared directory rather than in the sagenb/data directory. I note that there is no sagenb/data/jmol directory in the notebook created by this ticket, while there is in the old 4.8.alpha4 notebook.

I think it's almost there. Jonathan Replying to jason:

For future reference, here are the instructions if we had a sagenb_dependencies spkg. DON'T USE THESE--use the instructions in the description of the ticket.

### comment:107 follow-up: ↓ 108 Changed 9 years ago by gutow

Arrrgh...after rebuild (./sage -b). The notebook launches with an error

ImportError                               Traceback (most recent call last)

/home/jonathan/.sage/<ipython console> in <module>()

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/notebook_object.pyc in __call__(self, *args, **kwds)
215     """
216     def __call__(self, *args, **kwds):
--> 217         return self.notebook(*args, **kwds)
218
219     notebook = run_notebook.notebook_twisted

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/run_notebook.pyc in notebook_twisted(self, directory, port, interface, address, port_tries, secure, reset, accounts, require_login, server_pool, ulimit, timeout, open_viewer, sagetex_path, start_path, fork, quiet, subnets)
200         print '*' * 70
201
--> 202     nb = notebook.load_notebook(directory)
203
204     directory = nb._dir

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/notebook.pyc in load_notebook(dir, interface, port, secure)
1807     # mainly to avoid circular references, etc.  This also means
1808     # only one notebook can actually be used at any point.
-> 1809     import sagenb.notebook.twist
1810     sagenb.notebook.twist.notebook = nb
1811

/home/jonathan/Documents/sage-5.0.beta2/devel/sagenb-main/sagenb/notebook/twist.py in <module>()
40
41 from twisted.python import log
---> 42 from twisted.web2 import server, http, resource, channel
43 from twisted.web2 import static, http_headers, responsecode
44 from twisted.web2.filter import gzip

ImportError: No module named web2


Did I miss something or is there a new problem?

### comment:108 in reply to: ↑ 107 Changed 9 years ago by dimpase

ImportError?: No module named web2 }}} Did I miss something or is there a new problem?

I bet you did, as there is no web2 in the new twisted (version 11).

### comment:109 Changed 9 years ago by jason

And there is no twist.py in the new notebook... Are you *sure* that you installed the sagenb spkg above? Does devel/sagenb-main/flask_version exist?

You may need to do sage -f instead of sage -i when you install the spkgs above so they overwrite old versions of the same spkg.

### comment:110 Changed 9 years ago by gutow

Yep...the sagenb spkg install failed on a missing header for ssl. Somehow I missed the error message. I have the ssl binaries but not the header files on my systems. I guess ssl.h be part of the package or they need to be listed as prerequisite for the sagenb spkg. I'm running on the same Ubuntu 11.04 that I successfully installed this on through 4.8.

Here's the relevant part of the log file.

Processing pyOpenSSL-0.13.tar.gz
Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-pWFAnl/pyOpenSSL-0.13/egg-dist-tmp-4xiQaN
warning: no previously-included files matching '*.pyc' found anywhere in distribution
In file included from OpenSSL/crypto/crypto.h:30:0,
from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.13.tar.gz.


### comment:111 Changed 9 years ago by gutow

sagenb spkg requires openssl and openssl-dev. After adding openssl-dev to my system the spkg installs fine. If I missed this dependency sorry. If not it needs to be clear somewhere.

### comment:112 Changed 9 years ago by jason

Should we put an explicit check for that file into the spkg-install? If so, what should that check look like? We should also clearly list this somewhere.

### comment:113 in reply to: ↑ 101 Changed 9 years ago by ppurka

I explicitly tried each of those steps, and things worked flawlessly. I'm trying this on a 5.0 beta (which is where the notebook should be tested). I wonder if that makes a difference (it shouldn't, but who knows?).

Just to make sure: did you explicitly do each of the steps in the description of this ticket?

Also, is there any error message printed out for you in the terminal?

Thanks to kini we have the answer to this. It seems the error is triggered only when the password length is less than 6 characters. This means that the min password length of 6 is hard-coded in some file. The new git revisions of the nb which has set the variable min_password_length to 6 effectively masks the problem without fixing it.

### comment:114 Changed 9 years ago by strogdon

I have a question about the following chunk of code

try:
# simplejson is faster, so try to import it first
import simplejson as json
except ImportError:
import json


in sagenb/notebook/misc.py. It would seem that Sage is not shipped with simplejson. In testing the flask notebook on sage-on-gentoo the sagenb/notebook/misc.py test is the only failing notebook test because simplejson, which is pulled in by werkzeug as a runtime dependency, is found. The failure, roughly, is

sage -t -long  -force_lib /storage/strogdon/gentoo/usr/lib/python2.7/site-packages/sagenb/notebook/misc.py
**********************************************************************
File "/storage/strogdon/gentoo/usr/lib/python2.7/site-packages/sagenb/notebook/misc.py", line 221:
sage: print encode_response(d, separators = (', ', ': '), indent = 4)
Exception raised:
Traceback (most recent call last):
...
File "element.pyx", line 1305, in sage.structure.element.RingElement.__add__ (sage/structure/element.c:11569)
File "coerce.pyx", line 797, in sage.structure.coerce.CoercionModel_cache_maps.bin_op (sage/structure/coerce.c:7467)
TypeError: unsupported operand parent(s) for '+': '<type 'str'>' and 'Integer Ring'
**********************************************************************


If I remove the conditional the test passes. So, the question is, is the conditional reference to simplejson needed? Of course, the s-o-g flask notebook can always be patched.

### comment:115 Changed 9 years ago by kini

Patching the sage-on-gentoo flask notebook is not the solution. Test failures should not occur no matter what python modules, or indeed other programs, the user has installed into Sage's directory.

But there's actually a new failure, on 5.0.beta4:

sage -t  misc/sageinspect.py
**********************************************************************
File "/home/fs/src/sagenb/sagenb/misc/sageinspect.py", line 653:
sage: sage_getargspec(Poset)
Expected:
(['data', 'element_labels', 'cover_relations'], None, None, (None, None, False))
Got:
(['data', 'element_labels', 'cover_relations', 'category', 'facade', 'key'], None, None, (None, None, False, None, None, None))
**********************************************************************
1 items had failures:
1 of   7 in __main__.example_16
***Test Failed*** 1 failures.


I presume this is caused by #10998.

### comment:116 Changed 9 years ago by kini

(Why the heck is sageinspect in sagenb anyway...)

### comment:117 Changed 9 years ago by kini

I submitted a pull request to update the failing doctest. Looking at the simple_json one now...

### comment:118 Changed 9 years ago by kini

The problem is as follows. In the json module, we have

If indent is a non-negative integer, then JSON array elements and
object members will be pretty-printed with that indent level. An indent
level of 0 will only insert newlines. None is the most compact
representation.


On the other hand, in the simple_json module, we have

If indent is a string, then JSON array elements and object members
will be pretty-printed with a newline followed by that string repeated
for each level of nesting. None (the default) selects the most compact
representation without any newlines. For backwards compatibility with
versions of simplejson earlier than 2.1.0, an integer is also accepted
and is converted to a string with that many spaces.


Sage integers do not trigger the following lines in simple_json (simplejson/encoder.py:173):

        if isinstance(indent, (int, long)):
indent = ' ' * indent
self.indent = indent


Unfortunately it doesn't seem like the json module supports this new string form of the indent argument, which is why it works with our code - it just does ' ' * indent without caring that it's a Sage integer. Apparently the json module is just an old version of the simplejson module which is bundled with Python, so it will get the new string-based indent parameter eventually, but for now we'll either have to require an up-to-date simplejson for the new notebook, or remember/detect which of the two modules we imported and call the function accordingly.

### comment:119 Changed 9 years ago by kini

FWIW I made a pull request on simplejson addressing this: https://github.com/simplejson/simplejson/pull/29

### comment:120 Changed 9 years ago by kini

The pull request has been merged into simplejson, so the test will no longer fail once the next version of simplejson is released.

### comment:121 follow-up: ↓ 122 Changed 9 years ago by kini

The new version of simplejson has been released. strogdon, if you upgrade simplejson to 2.3.3 the test failure should be gone :)

### comment:122 in reply to: ↑ 121 Changed 9 years ago by strogdon

The new version of simplejson has been released. strogdon, if you upgrade simplejson to 2.3.3 the test failure should be gone :)

Good deal. I just upgraded to 2.3.3 and the test passes. Thanks.

### comment:123 follow-up: ↓ 124 Changed 9 years ago by strogdon

It's not clear from this ticket which of the twisted patches from #11874 should be applied. I believe it should be trac-11874-remove-twisted.3.patch . At least that one as well as the indicated jmol-spkg patch apply on top of 5.0.beta6 and I'm able to proceed to the "sage -br" step to get the new notebook working.

### comment:124 in reply to: ↑ 123 Changed 9 years ago by jdemeyer

It's not clear from this ticket which of the twisted patches from #11874 should be applied.

Considering that ticket needs_work, the "correct" patch for #11874 still needs to be written.

### comment:125 Changed 9 years ago by jason

Current status:

#11874 needs to be fixed and reviewed. There is a failing doctest, but I think the failing doctest may be resolved by using the sage.misc.fpickle module (I'm not sure, though)

#12229 needs review (and possibly a fix for the git_links)

#11503 needs review (and looks like it is ready to go...)

Merge pull request https://github.com/sagemath/sagenb/pull/40

### comment:126 follow-up: ↓ 128 Changed 9 years ago by dimpase

Please check out http://trac.sagemath.org/sage_trac/ticket/11874#comment:44

It appears that when we build Python 2.7.2 we automatically pull the latest Twisted (currently version 12) from the net and install it into SAGE_LOCAL.

We definitely, definitely do NOT want TWO twisted frameworks installed!!!

### comment:127 Changed 9 years ago by jason

• Description modified (diff)

Change urls to boxen instead of sage.math since sage.math is down.

### comment:128 in reply to: ↑ 126 Changed 9 years ago by dimpase

Please check out http://trac.sagemath.org/sage_trac/ticket/11874#comment:44

It appears that when we build Python 2.7.2 we automatically pull the latest Twisted (currently version 12) from the net and install it into SAGE_LOCAL.

Please ignore this comment. It was somehow due to my confusion when I recently looked at #11874.

Please also note that #11874 needs review now.

### comment:129 Changed 9 years ago by dimpase

running Explicit instructions from the ticket description gives one Twisted 12, not (as intended) Twisted 11.1. This needs to be fixed: please see http://trac.sagemath.org/sage_trac/ticket/11874#comment:62

### comment:130 Changed 9 years ago by jdemeyer

• Status changed from needs_info to needs_review

There is a doctest error:

sage -t  -force_lib devel/sagenb-main/sagenb/misc/sageinspect.py
**********************************************************************
File "/padic/scratch/jdemeyer/merger/sage-5.0.notebook/devel/sagenb-main/sagenb/misc/sageinspect.py", line 653:
sage: sage_getargspec(Poset)
Expected:
(['data', 'element_labels', 'cover_relations'], None, None, (None, None, False))
Got:
(['data', 'element_labels', 'cover_relations', 'category', 'facade', 'key'], None, None, (None, None, False, None, None, None))
**********************************************************************


### comment:131 follow-up: ↓ 132 Changed 9 years ago by jdemeyer

• Status changed from needs_review to needs_work

### comment:132 in reply to: ↑ 131 ; follow-up: ↓ 135 Changed 9 years ago by dimpase

Replying to jdemeyer: as mentioned here one just needs to update the spkg using this Git commit

### comment:133 Changed 9 years ago by kini

Well, you need to update the spkg to the current master branch, which happens to contain that commit in its ancestry, and thus contains the code change. (Commits are not something you apply individually like patches.)

### comment:134 Changed 9 years ago by kini

• Description modified (diff)
• Status changed from needs_work to needs_review

Here's a new spkg which should pass doctests: http://boxen.math.washington.edu/home/keshav/files/sagenb-0.9.0.spkg

Are there any other work issues?

### comment:135 in reply to: ↑ 132 Changed 9 years ago by kini

• Status changed from needs_review to needs_work

Hmm, there is something weird when I install the spkg and run sage -t $SAGE_ROOT/devel/sagenb-main: 1) README.rst is getting doctested, which throws up a password prompt; 2) there's a tab character in flask_version/base.py . Why were these not showing up in earlier doctests? They're not new problems, as far as I can tell. ### comment:136 Changed 9 years ago by kini • Status changed from needs_work to needs_review OK, uploaded a new SPKG to the same URL. Now it definitely does pass all its doctests. ### comment:137 Changed 9 years ago by jason Just curious, the new spkg probably has Twisted 12.0, right? Will people at #11874 complain that Twisted was updated in the spkg? ### comment:138 follow-up: ↓ 140 Changed 9 years ago by kini Yes, it contains Twisted 12. Yes, they might complain... feel free to create your own spkg with Twisted 11 instead - I might have messed something up anyway :) ### comment:139 follow-up: ↓ 141 Changed 9 years ago by jason I won't have very much time this week to work on this, so I'm happy letting you make an spkg. ### comment:140 in reply to: ↑ 138 Changed 9 years ago by dimpase Replying to kini: Yes, it contains Twisted 12. Yes, they might complain... feel free to create your own spkg with Twisted 11 instead - I might have messed something up anyway :) No complaints from me. Actually, with the old sagenb 0.9.0 (ie with Twisted 11), when I replaced sagenb spkg in Sage 5.0.beta8 with 0.9.0, and then followed the procedure on #11874, I got Sage built with Twisted 11.1, instead of Twisted 12. So Twisted 12 was sneaking in as it was getting installed before the installation of the old sagenb 0.9.0. ### comment:141 in reply to: ↑ 139 Changed 9 years ago by kini Replying to jason: I won't have very much time this week to work on this, so I'm happy letting you make an spkg. OK, just checking that I did everything right... first I cloned the git repo to another git repo (to just get master and no other commits), then used hg-git to pull that clone into a newly-initialized hg repo. Then I ran ./spkg-dist to create the spkg. Did I do anything wrong? ### comment:142 Changed 9 years ago by jdemeyer • Status changed from needs_review to needs_work I again get sage -t -force_lib devel/sagenb-main/sagenb/misc/sageinspect.py ********************************************************************** File "/scratch/jdemeyer/merger/sage-5.0.notebook/devel/sagenb-main/sagenb/misc/sageinspect.py", line 653: sage: sage_getargspec(Poset) Expected: (['data', 'element_labels', 'cover_relations'], None, None, (None, None, False)) Got: (['data', 'element_labels', 'cover_relations', 'category', 'facade', 'key'], None, None, (None, None, False, None, None, None)) **********************************************************************  ### comment:143 Changed 9 years ago by jason • Description modified (diff) Update instructions to reflect another patch on #11874 (I didn't test the instructions, though!) ### comment:144 Changed 9 years ago by ppurka • Description modified (diff) Update to new jmol patch for SAGE_ROOT. Also, after applying the changes from #11874, #11503 and this ticket, the doctest of devel/sagenb-main/sagenb passes. After these changes however, there is a problem with typesetting maths and I get this notice on evaluation of every cell, whether it is plot or maths. Error typesetting mathematics. And sure enough, something as simple as view(2+2) shows the latex code instead of the maths. jsmath is loaded and the button is present on the right hand corner. UPDATE: I have same problem with sagenb from github with mathjax instead of jsmath. Seems like something is broken in sage and not sagenb. Last edited 9 years ago by ppurka (previous) (diff) ### comment:145 follow-up: ↓ 152 Changed 9 years ago by jdemeyer • Milestone changed from sage-5.0 to sage-5.1 We would like to release sage-5.0 reasonably soon, and it's too late to get this still in sage-5.0. ### comment:146 follow-up: ↓ 147 Changed 9 years ago by kini • Description modified (diff) • Status changed from needs_work to needs_review Jeroen: Either the version you installed was from the URL in the ticket description (the one Jason uploaded a while ago) and not the SPKG I posted in a comment recently, or I screwed up making my SPKG and mistakenly packaged an old version of the sagenb code. So to cover both possibilities, I changed the link in the ticket description to point to the SPKG I'm hosting, and I also repacked the SPKG and reuploaded it. It passes doctests on my machine. Can you try it again? ### comment:147 in reply to: ↑ 146 Changed 9 years ago by jdemeyer Replying to kini: Jeroen: Either the version you installed was from the URL in the ticket description (the one Jason uploaded a while ago) of course the version I installed was from the URL in the ticket description. I'm not going to read 146 comments to find out the URL of the spkg to test. ### comment:148 follow-up: ↓ 149 Changed 9 years ago by kini Right, sorry about that. From your statement that you "again" saw the doctest errors, I assumed that at some point you had not seen the errors, which would have meant that you had installed my SPKG at some point. But I guess you just meant that you were continuing to see the doctest errors. My mistake :) ### comment:149 in reply to: ↑ 148 Changed 9 years ago by jdemeyer Replying to kini: Right, sorry about that. From your statement that you "again" saw the doctest errors, I assumed that at some point you had not seen the errors, which would have meant that you had installed my SPKG at some point. Maybe I had installed your SPKG at some point, then later forgot about your SPKG and installed Jason's SPKG again. I don't remember... ### comment:150 Changed 9 years ago by kini Fair enough - anyway it's my fault for not putting the correct info in the ticket description. ### comment:151 Changed 9 years ago by gutow • Description modified (diff) update jmol spkg to match #11503 ### comment:152 in reply to: ↑ 145 ; follow-up: ↓ 153 Changed 9 years ago by was Replying to jdemeyer: We would like to release sage-5.0 reasonably soon, and it's too late to get this still in sage-5.0. I agree with this decision. ### comment:153 in reply to: ↑ 152 Changed 9 years ago by jason Replying to was: Replying to jdemeyer: We would like to release sage-5.0 reasonably soon, and it's too late to get this still in sage-5.0. I agree with this decision. Same here. I had to put the proverbial rallying flag down due to work, but no one has really picked it back up again and finished the job. I'll be able to work on this more in a few weeks after our classes get out, if someone else hasn't started pushing it again by then. ### comment:154 Changed 9 years ago by kini What exactly remains to be done, other than the minor hiccup on #11503? ### comment:155 follow-up: ↓ 157 Changed 9 years ago by jason What I would do to answer that question is go through each ticket dependency and see if it is positive-reviewed. If not, then that ticket needs to be done :). I don't think there is much left to be done. ### comment:156 Changed 9 years ago by gutow • Description modified (diff) ### comment:157 in reply to: ↑ 155 ; follow-up: ↓ 158 Changed 9 years ago by dimpase Replying to jason: What I would do to answer that question is go through each ticket dependency and see if it is positive-reviewed. If not, then that ticket needs to be done :). I don't think there is much left to be done. I think one should provide a recipe for preparing a sage distribution with all these patches applied to be built from scratch, rather than patching an already built distribution. (Assuming one has hg (or a previous version of Sage) installed system-wide). That is, most of these patches are to be applied to sage-??? spkg. It makes more sense this way, as the changes are so extensive that it's time-consuming, and more error-prone, I think, to apply these to an already built installation of Sage. ### comment:158 in reply to: ↑ 157 Changed 9 years ago by jdemeyer Replying to dimpase: I think one should provide a recipe for preparing a sage distribution with all these patches applied to be built from scratch, rather than patching an already built distribution. (Assuming one has hg (or a previous version of Sage) installed system-wide). That is, most of these patches are to be applied to sage-??? spkg. I have made http://boxen.math.washington.edu/home/release/sage-5.0.notebook/sage-5.0.notebook.tar to test this ticket and its dependencies. ### comment:159 Changed 9 years ago by jdemeyer • Status changed from needs_review to needs_work This fails to install on OS X 10.4 PPC: Processing pyOpenSSL-0.13.tar.gz Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-h8pJu5/pyOpenSSL-0.13/egg-dist-tmp-Awaj3a warning: no previously-included files matching '*.pyc' found anywhere in distribution OpenSSL/ssl/connection.c: In function 'ssl_Connection_set_context': OpenSSL/ssl/connection.c:289:5: warning: implicit declaration of function 'SSL_set_SSL_CTX' [-Wimplicit-function-declaration] OpenSSL/ssl/connection.c: In function 'ssl_Connection_get_servername': OpenSSL/ssl/connection.c:313:16: error: 'TLSEXT_NAMETYPE_host_name' undeclared (first use in this function) OpenSSL/ssl/connection.c:313:16: note: each undeclared identifier is reported only once for each function it appears in OpenSSL/ssl/connection.c:320:5: warning: implicit declaration of function 'SSL_get_servername' [-Wimplicit-function-declaration] OpenSSL/ssl/connection.c:320:10: warning: assignment makes pointer from integer without a cast [enabled by default] OpenSSL/ssl/connection.c: In function 'ssl_Connection_set_tlsext_host_name': OpenSSL/ssl/connection.c:346:5: warning: implicit declaration of function 'SSL_set_tlsext_host_name' [-Wimplicit-function-declaration] error: Setup script exited with error: command 'gcc' failed with exit status 1 Error installing pyOpenSSL-0.13.tar.gz.  ### comment:160 Changed 9 years ago by jdemeyer This also fails to install on the Skynet machine silius (SUSE ES 11 SP1 ppc64): Processing pyOpenSSL-0.13.tar.gz Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-_3ma5D/pyOpenSSL-0.13/egg-dist-tmp-efVFsy warning: no previously-included files matching '*.pyc' found anywhere in distribution In file included from OpenSSL/crypto/crypto.h:30:0, from OpenSSL/crypto/crypto.c:16: OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory compilation terminated. error: Setup script exited with error: command 'gcc' failed with exit status 1 Error installing pyOpenSSL-0.13.tar.gz.  ### comment:161 Changed 9 years ago by kini According to http://stackoverflow.com/questions/7340784/easy-install-pyopenssl-error we should be able to fix this by requiring pyOpenSSL 0.12 or lower. Edit: er, I meant the problem on OS X - dunno about this silius one. Last edited 9 years ago by kini (previous) (diff) ### comment:162 follow-up: ↓ 166 Changed 9 years ago by jdemeyer What's the situation with OpenSSL support anyway? I believe it was decided not to require OpenSSL as dependency? But there have been many discussions about this, I might have lost track. ### comment:163 Changed 9 years ago by jdemeyer Also on the Skynet machine cleo (RHEL 5.3 ia64): Processing pyOpenSSL-0.13.tar.gz Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-PQFiHf/pyOpenSSL-0.13/egg-dist-tmp-XsFtPR warning: no previously-included files matching '*.pyc' found anywhere in distribution OpenSSL/ssl/connection.c: In function ssl_Connection_set_context: OpenSSL/ssl/connection.c:289:5: warning: implicit declaration of function SSL_set_SSL_CTX [-Wimplicit-function-declaration] OpenSSL/ssl/connection.c: In function ssl_Connection_get_servername: OpenSSL/ssl/connection.c:313:16: error: TLSEXT_NAMETYPE_host_name undeclared (first use in this function) OpenSSL/ssl/connection.c:313:16: note: each undeclared identifier is reported only once for each function it appears in OpenSSL/ssl/connection.c:320:5: warning: implicit declaration of function SSL_get_servername [-Wimplicit-function-declaration] OpenSSL/ssl/connection.c:320:10: warning: assignment makes pointer from integer without a cast [enabled by default] OpenSSL/ssl/connection.c: In function ssl_Connection_set_tlsext_host_name: OpenSSL/ssl/connection.c:346:5: warning: implicit declaration of function SSL_set_tlsext_host_name [-Wimplicit-function-declaration] error: Setup script exited with error: command 'gcc' failed with exit status 1 Error installing pyOpenSSL-0.13.tar.gz.  ### comment:164 Changed 9 years ago by jdemeyer On the Skynet machine iras (SUSE ES10 ia64): Processing pyOpenSSL-0.13.tar.gz Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-9gn9yE/pyOpenSSL-0.13/egg-dist-tmp-4CCNMQ warning: no previously-included files matching '*.pyc' found anywhere in distribution In file included from OpenSSL/crypto/crypto.h:30:0, from OpenSSL/crypto/crypto.c:16: OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory compilation terminated. error: Setup script exited with error: command 'gcc' failed with exit status 1 Error installing pyOpenSSL-0.13.tar.gz.  ### comment:165 Changed 9 years ago by jdemeyer On UGA rosemary (RHEL 5.6 x86_64): Processing pyOpenSSL-0.13.tar.gz Running pyOpenSSL-0.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-urCg6W/pyOpenSSL-0.13/egg-dist-tmp-fw0DPw warning: no previously-included files matching '*.pyc' found anywhere in distribution OpenSSL/ssl/connection.c: In function ‘ssl_Connection_set_context’: OpenSSL/ssl/connection.c:289:5: warning: implicit declaration of function ‘SSL_set_SSL_CTX’ [-Wimplicit-function-declaration] OpenSSL/ssl/connection.c: In function ‘ssl_Connection_get_servername’: OpenSSL/ssl/connection.c:313:16: error: ‘TLSEXT_NAMETYPE_host_name’ undeclared (first use in this function) OpenSSL/ssl/connection.c:313:16: note: each undeclared identifier is reported only once for each function it appears in OpenSSL/ssl/connection.c:320:5: warning: implicit declaration of function ‘SSL_get_servername’ [-Wimplicit-function-declaration] OpenSSL/ssl/connection.c:320:10: warning: assignment makes pointer from integer without a cast [enabled by default] OpenSSL/ssl/connection.c: In function ‘ssl_Connection_set_tlsext_host_name’: OpenSSL/ssl/connection.c:346:5: warning: implicit declaration of function ‘SSL_set_tlsext_host_name’ [-Wimplicit-function-declaration] error: Setup script exited with error: command 'gcc' failed with exit status 1 Error installing pyOpenSSL-0.13.tar.gz.  ### comment:166 in reply to: ↑ 162 Changed 9 years ago by dimpase Replying to jdemeyer: What's the situation with OpenSSL support anyway? I believe it was decided not to require OpenSSL as dependency? But there have been many discussions about this, I might have lost track. so we'll require openssl, but we need to sort out the versions issue, requiring pyOpenSSL 0.12 or lower, as Keshav suggests. ### comment:167 Changed 9 years ago by kini New SPKG with pyOpenSSL 0.12 uploaded. All tests pass on sage.math. Jeroen, can you test it on the buildbots again? It seems like all the errors you reported are coming from pyOpenSSL so I hope downgrading it will solve them... ### comment:168 Changed 9 years ago by kini • Status changed from needs_work to needs_review Sorry, I guess I should mark this needs_review again. ### comment:169 Changed 9 years ago by jdemeyer • Description modified (diff) ### comment:170 Changed 9 years ago by SimonKing I tried to follow the instructions, but the patch from #11874 fails to apply. Interesting that it was possible to create a testing release, then... ### comment:171 Changed 9 years ago by kini Huh, really? For me, the only dependency that didn't apply was #12327. #11874 worked fine for me. ### comment:172 Changed 9 years ago by SimonKing I think I start to understand. It does not apply with sage-5.0.prealpha0. But in sage-5.0.beta13, the files look totally different. So, I may (at some point) try again with the latest beta. ### comment:173 follow-up: ↓ 174 Changed 9 years ago by SimonKing • Description modified (diff) The instructions did not tell that one is supposed to start with the latest beta version. Starting with 5.0.prealpha0, things fail to apply. ### comment:174 in reply to: ↑ 173 Changed 9 years ago by dimpase Replying to SimonKing: The instructions did not tell that one is supposed to start with the latest beta version. Starting with 5.0.prealpha0, things fail to apply. Aren't the milestone setting to 5.1 and the fact that it was only marked as "needs review" a day or so ago indications that one has to use a very recent beta? ### comment:175 Changed 9 years ago by kini According to Jeroen's official policy, all tickets are required to have instructions that apply to the latest beta at all times. So this really should not be surprising... regardless of how recently this was marked as "needs review" or the milestone setting. ### comment:176 Changed 9 years ago by jdemeyer I think the "dependencies" field is even more important than the explicit instructions. The dependencies are an easy way to see that a ticket with positive_review cannot be merged because it depends on tickets which don't have positive_review. ### comment:177 follow-up: ↓ 178 Changed 9 years ago by jdemeyer Installation on Skynet silius (SUSE ES 11 SP1 ppc64) still fails, presumably because OpenSSL isn't installed: Processing pyOpenSSL-0.12.tar.gz Running pyOpenSSL-0.12/setup.py -q bdist_egg --dist-dir /tmp/easy_install-cWTpWy/pyOpenSSL-0.12/egg-dist-tmp-iS5i6u warning: no files found matching 'COPYING' warning: no previously-included files matching '*.pyc' found anywhere in distribution In file included from OpenSSL/crypto/crypto.h:17:0, from OpenSSL/crypto/crypto.c:16: OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory compilation terminated. error: Setup script exited with error: command 'gcc' failed with exit status 1 Error installing pyOpenSSL-0.12.tar.gz.  I'm going to try to install OpenSSL and try again. ### comment:178 in reply to: ↑ 177 ; follow-up: ↓ 179 Changed 9 years ago by SimonKing Replying to jdemeyer: Error installing pyOpenSSL-0.12.tar.gz. When trying to build sage-5.1.notebook, I get a similar error with king@mpc622:~$ uname -a
Linux mpc622 2.6.34.linuxpool #0 SMP PREEMPT Wed May 19 16:32:19 CEST 2010 x86_64 GNU/Linux
king@mpc622:~$cat /etc/issue Debian GNU/Linux 6.0 \n \l king@mpc622:~$ cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 37
model name	: Intel(R) Core(TM) i3 CPU         530  @ 2.93GHz
stepping	: 2
cpu MHz		: 1197.000
cache size	: 4096 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5852.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 37
model name	: Intel(R) Core(TM) i3 CPU         530  @ 2.93GHz
stepping	: 2
cpu MHz		: 1197.000
cache size	: 4096 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5851.87
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 37
model name	: Intel(R) Core(TM) i3 CPU         530  @ 2.93GHz
stepping	: 2
cpu MHz		: 1197.000
cache size	: 4096 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 2
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5851.87
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 37
model name	: Intel(R) Core(TM) i3 CPU         530  @ 2.93GHz
stepping	: 2
cpu MHz		: 1197.000
cache size	: 4096 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 2
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5851.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:



I have posted the install log.

Last edited 9 years ago by SimonKing (previous) (diff)

### comment:179 in reply to: ↑ 178 ; follow-up: ↓ 180 Changed 9 years ago by dimpase

Error installing pyOpenSSL-0.12.tar.gz.

When trying to build sage-5.1.notebook, I get a similar error with

it looks as if you don't have openssl installed.

### comment:180 in reply to: ↑ 179 Changed 9 years ago by SimonKing

Error installing pyOpenSSL-0.12.tar.gz.

When trying to build sage-5.1.notebook, I get a similar error with

it looks as if you don't have openssl installed.

How can I install it when I am not root? Is there an openssl spkg that I could install instead?

### comment:181 Changed 9 years ago by SimonKing

Apparently there *is* an openssl spkg, I am installing it now.

The installation starts with many warnings.

Question: Shouldn't openssl be a standard spkg, since it is a dependency for the new notebook?

### comment:182 Changed 9 years ago by SimonKing

I manually installed the optional openssl spkg and typed "make" again. Bug installation of sagenb-0.9.0.spkg did still not work. I have updated the install log.

### comment:183 follow-up: ↓ 184 Changed 9 years ago by jason

we can't distribute OpenSSL with Sage since it is not GPL-compatible.

### comment:184 in reply to: ↑ 183 Changed 9 years ago by dimpase

we can't distribute OpenSSL with Sage since it is not GPL-compatible.

I presume one can have an optional package. Does anyone test the scenario Simon is talking about? I don't think it's easy to find a system without openssl installed in the usual way...

### comment:185 Changed 9 years ago by SimonKing

By the way: I just found that openssl is installed on the machine in question!

king@mpc622:~$which openssl /usr/bin/openssl  And note that installing the optional openssl spkg did not help. In other words, I doubt that the installation failure of sage-5.1.notebook comes from a lack of openssl. ### comment:186 follow-up: ↓ 188 Changed 9 years ago by kini You need libSSL development headers. On Ubuntu this is called libssl-dev, not openssl. This is a new systemwide dependency for Sage (see https://groups.google.com/d/msg/sage-devel/iwrF8_kGLzM/32HIts7EPKQJ ) and should be noted in the Sage docs - I'll upload a patch to do just this. ### comment:187 Changed 9 years ago by kini • Status changed from needs_review to needs_work Er, sorry, I meant on Debian, not Ubuntu. Marking as needs work until I have that patch uploaded... ### comment:188 in reply to: ↑ 186 ; follow-up: ↓ 190 Changed 9 years ago by SimonKing Replying to kini: You need libSSL development headers. OK. Is there a libSSL spkg? If not, then I will not be able to build Sage any more on the machine in my office. ### comment:189 Changed 9 years ago by SimonKing I now tried the pyopenssl spkg, but this hasn't helped either... ### comment:190 in reply to: ↑ 188 ; follow-up: ↓ 191 Changed 9 years ago by dimpase Replying to SimonKing: Replying to kini: You need libSSL development headers. OK. Is there a libSSL spkg? If not, then I will not be able to build Sage any more on the machine in my office. Why? I imagine building openssl from source is not such a big deal. As well, why can't you ask your sysadmin to install openssl headers? ### comment:191 in reply to: ↑ 190 ; follow-up: ↓ 192 Changed 9 years ago by SimonKing Replying to dimpase: Why? I imagine building openssl from source is not such a big deal. Does the optional openssl spkg contain the necessary sources? Then the openssl spkg should be modified such that the development headers are installed. Then, I think it would be helpful if the installation script of sagenb-0.9.0.spkg would check for the presence of the headers, and would print a clear message, if there are missing. Such as:  **************************************************************** ** The libSSL development headers are missing. For licensing ** ** reasons, we can not distribute it in Sage by default. But ** ** you can install the missing headers with an optional Sage ** ** package, by doing ** ** ./sage -i openssl ** ** in the Sage root directory. ** ****************************************************************  As well, why can't you ask your sysadmin to install openssl headers? I think that introducing yet another nontrivial dependency for Sage is a bad idea. ### comment:192 in reply to: ↑ 191 ; follow-up: ↓ 193 Changed 9 years ago by dimpase Replying to SimonKing: As well, why can't you ask your sysadmin to install openssl headers? I think that introducing yet another nontrivial dependency for Sage is a bad idea. I think this is a trivial dependency. Every modern Linux distro has openssl, and OSX's Xcode comes with openssl preinstalled. I don't see a need to jump hoops just in order to overcome a non-problem. ### comment:193 in reply to: ↑ 192 ; follow-up: ↓ 194 Changed 9 years ago by SimonKing Replying to dimpase: I think that introducing yet another nontrivial dependency for Sage is a bad idea. I think this is a trivial dependency. Every modern Linux distro has openssl, ... Sure. My computer has openssl as well. But it has not the development headers. Meanwhile, I have king@mpc622:/mnt/local/king/SAGE/experiment/notebook/sage-5.1.notebook$ ls local/include/openssl/
aes.h          conf_api.h     ec.h           mdc2.h         pqueue.h       ssl.h
asn1.h         conf.h         engine.h       modes.h        rand.h         stack.h
asn1_mac.h     crypto.h       e_os2.h        objects.h      rc2.h          symhacks.h
asn1t.h        des.h          err.h          obj_mac.h      rc4.h          tls1.h
bio.h          des_old.h      evp.h          ocsp.h         ripemd.h       ts.h
blowfish.h     dh.h           hmac.h         opensslconf.h  rsa.h          txt_db.h
bn.h           dsa.h          idea.h         opensslv.h     safestack.h    ui_compat.h
buffer.h       dso.h          krb5_asn.h     ossl_typ.h     seed.h         ui.h
camellia.h     dtls1.h        kssl.h         pem2.h         sha.h          whrlpool.h
cast.h         ebcdic.h       lhash.h        pem.h          ssl23.h        x509.h
cms.h          ecdh.h         md4.h          pkcs12.h       ssl2.h         x509v3.h
comp.h         ecdsa.h        md5.h          pkcs7.h        ssl3.h         x509_vfy.h


These are many headers, and they come from the openssl spkg. But apparently the development headers are still not there.

In other words, since about two hours I try to find out where I could get the development headers from, without being root. So, it is indeed a nontrivial dependency.

### comment:194 in reply to: ↑ 193 Changed 9 years ago by dimpase

I think that introducing yet another nontrivial dependency for Sage is a bad idea.

I think this is a trivial dependency. Every modern Linux distro has openssl, ...

Sure. My computer has openssl as well. But it has not the development headers.

OK, so openssl headers need to be listed as a dependency for building Sage from source. It is trivial in the sense that installing these headers is trivial (for a root). While I understand that sysadmins might be slow, or picky (at the place I did my PhD at, the sysadmin only replied to emails which had "BEER" mentioned in the subject---I'm not kidding!), I cannot imagine this being hard to overcome...

These are many headers, and they come from the openssl spkg. But apparently the development headers are still not there.

In other words, since about two hours I try to find out where I could get the development headers from, without being root. So, it is indeed a nontrivial dependency.

Looks like a bug in openssl spkg (or in whatever place that need to compile openssl-related stuff).

### comment:195 Changed 9 years ago by kini

No, those are the development headers... it looks like you have some case sensitivity problem or something:

In file included from OpenSSL/crypto/crypto.h:17,
from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: error: openssl/ssl.h: Datei oder Verzeichnis nicht gefunden


Maybe the pyOpenSSL installer is picking up your systemwide openssl headers, which are in some funkily capitalized directory, rather than the ones which the ssl spkg seems to have installed into your sage directory?

### comment:196 Changed 9 years ago by kini

Oh, sorry, I'm stupid - those are filenames inside the pyOpenSSL package, I guess.

### comment:197 Changed 9 years ago by SimonKing

If these are the development headers, then it seems that the sagenb spkg is looking for them in the wrong place. Is SAGE_ROOT/local/include part of the include path, when sagenb is installed?

### comment:198 Changed 9 years ago by SimonKing

Or more precisely: The sagenb spkg contains a file pyOpenSSL-0.12.tar.gz, and the error message indicates that the problem lies in installing that sub-package.

### comment:199 follow-up: ↓ 202 Changed 9 years ago by kini

OK, so the following text appears in the pyOpenSSL INSTALL file:

If your OpenSSL header files aren't in /usr/include, you may need to supply
the -I flag to let the setup script know where to look. The same goes for the
libraries of course, use the -L flag. Note that build won't accept these
flags, so you have to run first build_ext and then build! Example:

$python setup.py build_ext -I/usr/local/ssl/include -L/usr/local/ssl/lib$ python setup.py build


So that's why pyOpenSSL isn't seeing the dev headers that are installed by the OpenSSL package. That means we should make a separate pyOpenSSL SPKG and micromanage the build process as described above. (Judging by comments above, I guess one already exists, and so it just needs to be modified.)

By the way, can we really distribute OpenSSL as an optional SPKG? Doesn't that count as distributing it with Sage? Sure, it's not shipped literally with Sage... well, I don't know.

### comment:200 Changed 9 years ago by kini

Um, OK... the pyOpenSSL SPKG already does this. So I guess the problem is that sagenb saw there was a newer version of pyOpenSSL sitting right there and decided to install it even though you already had 0.8 installed. I'll upgrade the pyOpenSSL SPKG to 0.12 and then everything should hopefully work...

### comment:201 Changed 9 years ago by SimonKing

Hi Keshav,

I was just about to point to the same text.

I don't know about licensing, but I think I was told that an optional spkg is fine for non-GPL packages.

I see in spkg-install that the pyOpenSSL package is installed via easy_install --allow-hosts=None pyOpenSSL-0.12.tar.gz. Isn't there a way to tell easy_install what include directory to use?

### comment:202 in reply to: ↑ 199 Changed 9 years ago by dimpase

By the way, can we really distribute OpenSSL as an optional SPKG? Doesn't that count as distributing it with Sage? Sure, it's not shipped literally with Sage... well, I don't know.

some optional packages have licenses which are incompatible with GPL, so this should not be a problem, if überhaupt we can distribute openssl...

### comment:203 Changed 9 years ago by SimonKing

I see in spkg-install that the pyOpenSSL package ...

I mean "spkg-install in the sagenb spkg", and pyOpenSSL should rather be named a sub-package of sagenb.

### comment:204 follow-up: ↓ 207 Changed 9 years ago by kini

Right, I get what you meant, Simon. It doesn't seem like easy_install has the functionality you mentioned, unfortunately. Not that surprising since most Python packages don't have a C component anyway...

### comment:205 follow-up: ↓ 206 Changed 9 years ago by SimonKing

By the way: pyOpenSSL has the APL2 licence. Is that compatible with GPL?

It is distributed as sub-package of a standard spkg (namely the sagenb spkg) - so there is a potential violation of GPL, isn't it?

### comment:206 in reply to: ↑ 205 Changed 9 years ago by SimonKing

By the way: pyOpenSSL has the APL2 licence. Is that compatible with GPL?

Google told me: "The Free Software Foundation considers the Apache License, Version 2.0 to be a free software license, compatible with version 3 of the GPL." At least one good news...

### comment:207 in reply to: ↑ 204 Changed 9 years ago by dimpase

Right, I get what you meant, Simon. It doesn't seem like easy_install has the functionality you mentioned, unfortunately. Not that surprising since most Python packages don't have a C component anyway...

one would modify package's setup.py to accomplish this, looking for custom header locations. E.g. look at the one of cvxopt.

### comment:208 Changed 9 years ago by kini

Supplying command line arguments to setup.py, as Simon and I discussed earlier, is less intrusive than modifying setup.py .

I am making a new pyOpenSSL SPKG.

### comment:209 Changed 9 years ago by SimonKing

At stackoverflow, the variables C_INCLUDE_PATH and CPLUS_INCLUDE_PATH are mentioned as an option to make easy_install understand the location of headers.

Worth a try, I'd say!

And why should there be a new pyOpenSSL spkg? I think one should instead modify spkg-install in the sagenb spkg (using C_INCLUDE_PATH), so that the pyOpenSSL sub-package (which is different from the pyOpenSSL spkg) is correctly installed.

### comment:210 Changed 9 years ago by SimonKing

• Owner changed from jason, mpatel, was to (none)

YESSS!

export C_INCLUDE_PATH="$SAGE_ROOT"/local/include  to sagenb's spkg-install script, and that seems to have solved the problem!! I guess that what one really wants is to add "$SAGE_ROOT"/local/include to an already existing include path. But I guess you know better than I how that can be done in a script...

My suggestions:

• Add "$SAGE_ROOT"/local/include to C_INCLUDE_PATH, in the sagenb spkg-install • If installation of the notebook fails, inform the user about the optional openssl package. ### comment:212 Changed 9 years ago by kini I would like to use systemwide headers if at all possible. Otherwise we are basically requiring people to install the OpenSSL SPKG, which means that we must make the OpenSSL SPKG standard, which is not possible. I don't see a way to use$SAGE_LOCAL/include as a fallback only, other than just putting

if [ ! -d /usr/include/openssl ]; then
python setup.py build_ext -I"$SAGE_LOCAL"/include -L"$SAGE_LOCAL"/lib
else
python setup.py build_ext
fi


in an spkg-install script somewhere. That's why I elected to do so in the pyOpenSSL SPKG. I could as well do so in the sagenb SPKG, but that would involve unpacking the pyOpenSSL tarball that's there and messing around with it instead of using easy_install. This way we can make the pyOpenSSL SPKG standard and make the OpenSSL SPKG optional, and still tell people we prefer them to install OpenSSL dev headers systemwide.

### comment:213 follow-up: ↓ 214 Changed 9 years ago by kini

Wait, I misunderstood. I guess I could just put

if [ ! -d /usr/include/openssl ]; then
export C_INCLUDE_PATH="$SAGE_ROOT"/local/include fi easy_install --allow-hosts=None pyOpenSSL-0.12.tar.gz  into sagenb's spkg_install, haha. Last edited 9 years ago by kini (previous) (diff) ### comment:214 in reply to: ↑ 213 Changed 9 years ago by SimonKing Replying to kini: Wait, I misunderstood. I guess I could just put if [ ! -d /usr/include/openssl ]; then export C_INCLUDE_PATH="$SAGE_ROOT"/local/include
fi
easy_install --allow-hosts=None pyOpenSSL-0.12.tar.gz


into sagenb's spkg_install, haha.

### comment:219 follow-up: ↓ 220 Changed 9 years ago by SimonKing

If it is indeed true that setting C_INCLUDE_PATH gives priority to the given path, but uses the default patch as fallback, then I think setting C_INCLUDE_PATH for all sub-packages (not just for pyOpenSSL - in fact I added the C_INCLUDE_PATH already to the second line of the spkg-install script) would be a valid solution.

And concerning warning message: Is it really a good solution to try to install pyOpenSSL and look whether it fails? Should one not rather start by searching for the headers, and immediately raise the warning (and stop installation) if they are missing, without the attempt to install any sub-package?

### comment:220 in reply to: ↑ 219 ; follow-up: ↓ 221 Changed 9 years ago by jason

And concerning warning message: Is it really a good solution to try to install pyOpenSSL and look whether it fails? Should one not rather start by searching for the headers, and immediately raise the warning (and stop installation) if they are missing, without the attempt to install any sub-package?

What if someone has openssl installed in a non-standard place? Looking for openssl in one specific place seems brittle to me.

### comment:221 in reply to: ↑ 220 Changed 9 years ago by SimonKing

What if someone has openssl installed in a non-standard place? Looking for openssl in one specific place seems brittle to me.

If someone has openssl in a non-standard place, then pyOpenSSL will only pick it up if an appropriate environment variable or setup option is used. Ideally, we would respect that variable/option. This is what I meant by adding $SAGE_ROOT/local/include - adding is not replacing. ### comment:222 Changed 9 years ago by kini Simon: I think Jason was objecting to your suggestion to check for headers first and fail out if they were not found. We'd have to search in every possible place GCC might search, and that seems like too much trouble to engineer - why not just let GCC try to find the headers, I guess is what Jason is saying. ### comment:223 Changed 9 years ago by jason Kini: that's what I was thinking. The problem I had was the -d test for a specific header directory to determine whether we'd even try installing pyopenssl. ### comment:224 Changed 9 years ago by SimonKing If the attempt to build it takes not too much time, then it is ok to simply wait if installation of pyOpenSSL works or not. By the way, meanwhile building sage-5.1.notebook has finished. I can not do make test, but that's a problem fixed elsewhere (#12568, I think). What I did to build it boils down to the following: • Patch sagenb's spkg-install by adding export C_INCLUDE_PATH="$SAGE_ROOT"/local/include as the second line. Note that the sagenb spkg has plenty of uncommitted changes - that needs to be fixed before release. Also note that my fix may be too simplistic. It is just a proof of concept.
• make, and wait until it fails
• ./sage -i openssl
• make

Now I am adding the patch from #12568 and try make test.

### comment:225 Changed 9 years ago by jdemeyer

Testing whether openssl exists and raising an error message if not is better done in prereq, not in sagenb.

### comment:226 follow-up: ↓ 227 Changed 9 years ago by kini

Jeroen: I think we have agreed not to do such a check, and instead just try to build.

Simon: Note that the sagenb spkg doesn't even have a repository ;) Since the SPKG is generated by version-controlled code in the actual sagenb git repository, I guess someone decided that having a repository in the SPKG was not necessary...

By the way I think we should use CPATH instead of C_INCLUDE_PATH. Apparently this will allow the extra paths provided to apply to C++ and Objective-C files as well as C files. Could come in handy in the future.

### comment:227 in reply to: ↑ 226 Changed 9 years ago by jdemeyer

Jeroen: I think we have agreed not to do such a check, and instead just try to build.

Why that? I much prefer to see a nice error message like Simon suggested instead of a cryptic gcc error. Also, I much prefer to see it immediatly (in prereq) instead of after hours of building. The point of prereq is precisely to check dependencies of Sage.

### comment:228 Changed 9 years ago by kini

Because as Jason said we don't know where to look for SSL headers. It could be frustrating to the user if Sage refuses to build just because our code is too stupid to find the user's SSL headers, even though GCC would be able to find them. Since we are not assuming that OpenSSL headers are going to be in the Sage directory, we should wash our hands of trying to find them outside the Sage directory, where there be monsters! :)

### comment:229 Changed 9 years ago by kini

Our other dependencies are all command line utilities if I'm not mistaken so a quick command -v is sufficient to find them, but that is not true for include dirs... right? Well I guess we could just write a simple C file that tries to link in openssl and attempt to compile it as our "check for SSL" in prereq...

### comment:230 follow-up: ↓ 238 Changed 9 years ago by jdemeyer

These kind of checks is precisely why autoconf was invented...

### comment:231 Changed 9 years ago by SimonKing

FWIW: All tests pass!

### comment:232 follow-up: ↓ 233 Changed 9 years ago by kini

Jeroen: Well, sadly I don't know how to use autoconf :( And apparently, according to people on sage-devel, you cannot just muddle your way through writing autoconf...

Anyway, new SPKG is up. Diff at http://git.io/mM8qnQ . Simon, if you could test this SPKG it would be great. (A complete test would I guess involve putting this SPKG into a freshly extracted copy of the testing release Jeroen made, i.e. replacing the sagenb-0.9.0.spkg that is already there, and then running make and make ptestlong.) Good night for now, it's almost dawn here :) I'll work on editing the installation manual tomorrow. Er, today...

### comment:233 in reply to: ↑ 232 ; follow-up: ↓ 260 Changed 9 years ago by jdemeyer

Jeroen: Well, sadly I don't know how to use autoconf :(

I don't mind doing that. However, I'd rather wait until the whole openSSL thing in the Notebook has been settled.

### comment:234 Changed 9 years ago by ppurka

IIRC, this check for a package can be done with pkgconfig. Something like

if pkg-config openssl; then
:
else
echo >&2 "Get some ssl!"
exit 1
fi

Last edited 9 years ago by ppurka (previous) (diff)

### comment:235 follow-up: ↓ 236 Changed 9 years ago by jason

pkgconfig isn't installed by default on OSX, but the openssl headers are. So wouldn't the above test return a false result on stock OSX?

### comment:236 in reply to: ↑ 235 Changed 9 years ago by SimonKing

pkgconfig isn't installed by default on OSX, but the openssl headers are. So wouldn't the above test return a false result on stock OSX?

If pkgconfig is nonstandard on OSX, I think we can not use it. In addition: Would pkgconfig openssl just detect openssl (which is installed on my machine), or would it detect the headers (which are not installed on my machine, by default)?

I just tested: pkg-config does not work in the way we need it. Namely, I did pkg-config openssl in the sage shell of sage-5.1.notebook. Apparently openssl is installed (namely, the headers are in $SAGE_ROOT/local/include), but pkg-config openssl returns nothing. ### comment:238 in reply to: ↑ 230 Changed 9 years ago by dimpase Replying to jdemeyer: These kind of checks is precisely why autoconf was invented... indeed. And I am also very much for this check being moved to prereq. ### comment:239 Changed 9 years ago by jason Simon, you need to use pkg-config correctly. It returns an exit code, it doesn't print out something, IIRC. Do pkg-config --help to see the options. ### comment:240 Changed 9 years ago by jason +1 to using autoconf, if someone knows how to write the necessary test. How would we then get the result when doing the sagenb spkg? We apparently want the sagenb to work without openssl too. ### comment:241 in reply to: ↑ 237 Changed 9 years ago by ppurka Replying to SimonKing: I just tested: pkg-config does not work in the way we need it. Namely, I did pkg-config openssl in the sage shell of sage-5.1.notebook. Apparently openssl is installed (namely, the headers are in$SAGE_ROOT/local/include), but pkg-config openssl returns nothing.

pkg-config will return nothing unless you ask it to print out.

And I am surprised that it is not present in OSX. According to its homepage it runs on all *nix systems.

### comment:242 Changed 9 years ago by jason

It runs just fine on OSX. It is one of the first things I install. It just doesn't come standard.

### comment:243 follow-up: ↓ 244 Changed 9 years ago by kini

We apparently want the sagenb to work without openssl too.

Er, we do? Then why is pyOpenSSL a dependency?

### comment:244 in reply to: ↑ 243 Changed 9 years ago by dimpase

We apparently want the sagenb to work without openssl too.

Er, we do? Then why is pyOpenSSL a dependency?

No, I don't think we really need an openssl-less sagenb. Given the triviality of openssl installation, if needed, this looks like a makework project, to strip this functionality out of sagenb.

### comment:245 follow-up: ↓ 248 Changed 9 years ago by kini

The dev headers, unlike OpenSSL itself, are not usually installed, in binary Linux distributions. If requiring OpenSSL headers for sagenb results in a majority of Sage users installing the optional OpenSSL SPKG, then calling it "optional" starts to become a bit indefensible... so I can see why allowing sagenb to function without OpenSSL is attractive. After all, we only use it for basically openid and HTTPS connections, am I right?

### comment:246 Changed 9 years ago by ppurka

If it is only for openid and https then it does make sense to have it as an optional package. Most people who download Sage, will be downloading for personal use. The few who are running servers, will have to install the extra package. This can be documented in an "upgrade guide for sagenb" perhaps.

### comment:247 Changed 9 years ago by SimonKing

• Reviewers changed from Rado Kirov, Dan Drake, Jason Grout to Rado Kirov, Dan Drake, Jason Grout, Simon King

Above, I had claimed that pkg-config did not recognise the headers installed by the optional openssl spkg. I stand corrected, since inside a sage shell I get

(sage-sh) king@mpc622:sage-5.1.notebook$pkg-config --cflags openssl -I/mnt/local/king/SAGE/experiment/notebook/sage-5.1.notebook/local/include  So, testing for the presence of openssl using pkg-config during configuration (i.e., before starting the installation) makes sense. If it is true that sagenb would - in principle - also work without openssl, I agree with ppurka that we could easily avoid openssl to become de-facto standard: • Make sure that sagenb spkg installs and the notebook basically works, even if openssl is not installed. Of course, the notebook will then lack some features. • If one tries to use the openid and https features, then a runtime error should be raised, with a clear error message along the lines Install openssl with development headers. If they are not available for your operating system, you could install the optional openssl Sage package. This is possible, e.g., with the command install_package("openssl") during a Sage session. Then, re-install the Sage notebook package, e.g., by install_package("sagenb", forced=True) during a Sage session.  Would that be enough, actually? Or would a recompilation of Sage be needed after re-installation of sagenb? ### comment:248 in reply to: ↑ 245 Changed 9 years ago by dimpase Replying to kini: The dev headers, unlike OpenSSL itself, are not usually installed, in binary Linux distributions. If requiring OpenSSL headers for sagenb results in a majority of Sage users installing the optional OpenSSL SPKG, then calling it "optional" starts to become a bit indefensible... so I can see why allowing sagenb to function without OpenSSL is attractive. After all, we only use it for basically openid and HTTPS connections, am I right? a Linux binary distro doesn't come with gcc and make either, you'd have to install it. Nothing prevents you from installing openssh-dev at this moment, too. The only real advantage of openssl-less sagenb is inclreased modularity... But, could we postpond this to the next milestone, please? ### comment:249 Changed 9 years ago by kini Wow, really? ... It seems you are correct, Ubuntu 12.04 doesn't come with gcc preinstalled! Well then I agree that we should just encourage it as a dependency and provide an SPKG as well. ### comment:250 Changed 9 years ago by SimonKing FWIW, make test passes again. What I did: • Open the tar ball, cd sage-5.1.notebook; replace the existing sagenb spkg with the new one posted here. • make; wait until it fails. • ./sage -i openssl • make • make test Can someone else test whether the new spkg is able to pick up a system wide libssl-dev? ### comment:251 Changed 9 years ago by SimonKing For the record: The problem reported at #11913 is not fixed by the new notebook. But I think, since the new notebook is probably close to being used, that #11913 should start with the new notebook. ### comment:252 follow-up: ↓ 253 Changed 9 years ago by SimonKing And another criticism: When starting the notebook, I see sage: notebook() The notebook files are stored in: sage_notebook.sagenb Upgrading model version to version 1 Done upgrading to model version 1 ************************************************** * * * Open your web browser to http://localhost:8080 * * * ************************************************** 2012-04-17 11:08:31+0200 [-] Log opened. 2012-04-17 11:08:31+0200 [-] twistd 12.0.0 (/mnt/local/king/SAGE/experiment/notebook/sage-5.1.notebook/local/bin/python 2.7.2) starting up. 2012-04-17 11:08:31+0200 [-] reactor class: twisted.internet.pollreactor.PollReactor. 2012-04-17 11:08:31+0200 [-] QuietSite starting on 8080 2012-04-17 11:08:31+0200 [-] Starting factory <__builtin__.QuietSite instance at 0x3da77e8>  It is not mentioned that one is supposed to type Ctrl-c in order to return to Sage's command line version. I think there was a recent thread on sage-support (or sage-devel?) started by a user who was confused about that point. William stated that there used to be a clearer instruction when starting the notebook. Hence, I believe that this ticket is a good occasion to make the notebook instructions clearer. ### comment:253 in reply to: ↑ 252 Changed 9 years ago by jdemeyer Replying to SimonKing: I believe that this ticket is a good occasion to make the notebook instructions clearer. I completely disagree with this. Fix it, certainly. But not on this ticket. ### comment:254 Changed 9 years ago by jdemeyer On Skynet iras (SUSE ES10 ia64), I built OpenSSL in my $HOME directory (so not in Sage) and I get the following weird error:

Processing pyOpenSSL-0.12.tar.gz
Running pyOpenSSL-0.12/setup.py -q bdist_egg --dist-dir /tmp/easy_install-GTqwq9/pyOpenSSL-0.12/egg-dist-tmp-yGKR4i
warning: no files found matching 'COPYING'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
/usr/bin/ld: /home/buildbot/local/iras/lib/libcrypto.a(obj_dat.o): @gprel relocation against dynamic symbol obj_cleanup_defer
/usr/bin/ld: /home/buildbot/local/iras/lib/libcrypto.a(obj_dat.o): @gprel relocation against dynamic symbol obj_cleanup_defer
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.12.tar.gz.


### comment:255 Changed 9 years ago by jdemeyer

Installing the optional openSSL spkg did work on iras.

### comment:256 Changed 9 years ago by kini

That is a strange error indeed.

Simon: I agree with Jeroen - that can be taken care of later. There are actually several things people are doing with sagenb that will not make it into 0.9.0 (the version of sagenb we are gearing up to release, as soon as this ticket has positive review). They will go into the following release of sagenb. This includes stuff like the switch from jsmath to mathjax, Ivan Andrus's code to allow opening .sws files from the command line, some LDAP implementation stuff which people are still hashing out, and a Russian translation that someone on github kindly made for us. See https://github.com/sagemath/sagenb/pulls for some stuff that people are working on but hasn't been accepted yet because we're still making 0.9.0. You can make your own edit of the startup message and add it to this list, if you wish :) Here is the relevant code: https://github.com/sagemath/sagenb/blob/master/sagenb/misc/misc.py#L26

(Of course, we should technically have a separate branch for feature freezes such as this, but we'll get around to that later.)

### comment:257 Changed 9 years ago by jdemeyer

One small thing still to be fixed in sagenb is to merge the patch from #12677 (I reported this at http://code.google.com/p/sagenb/issues/detail?id=91 before being told that Google Code isn't used anymore).

### comment:258 Changed 9 years ago by kini

Done. Building and uploading new SPKG now...

### comment:259 Changed 9 years ago by kini

The new SPKG is up. The commit I added is here.

### comment:260 in reply to: ↑ 233 ; follow-up: ↓ 261 Changed 9 years ago by kini

So what are we going to do about SSL? It seems to me that since one cannot run sage -i openssl from a freshly unpacked tarball, we basically need to require SSL to be installed systemwide. Or we can expect the user to run make, wait for it to fail (!), then run sage -i openssl (hopefully by this time the machinery for sage -i is in place), and then run make again. Should I add a note to this effect to the "install from source" docs page?

### comment:261 in reply to: ↑ 260 Changed 9 years ago by jdemeyer

hopefully by this time the machinery for sage -i is in place

If you make SageNB depend on sage_scripts, this is guaranteed.

### comment:262 follow-up: ↓ 263 Changed 9 years ago by kini

Surely sage -i must be installed by the time the sagenb SPKG fails, because the sagenb SPKG installation itself would have been started via sage -i. No?

### comment:263 in reply to: ↑ 262 Changed 9 years ago by jdemeyer

Surely sage -i must be installed by the time the sagenb SPKG fails, because the sagenb SPKG installation itself would have been started via sage -i. No?

No. The spkg installation could be started by make. Moreover, the part of sage -i openssl which wouldn't work is the downloading part, not the installation part.

OK, I see.

### comment:265 Changed 9 years ago by kini

• Description modified (diff)
• Status changed from needs_work to needs_review

### comment:266 Changed 9 years ago by kini

I've uploaded a patch which causes the "install from source code" docs to mention OpenSSL as a requirement. Generally the whole page needs a cleanup though - see #11237. Marking as needs review again since I don't think there were any other issues (?).

### comment:267 Changed 9 years ago by kini

Updated the SPKG due to merging a bugfix from Jason.

### comment:268 Changed 9 years ago by jason

Don't the OpenSSL libraries need to be installed, no matter what? And the dev headers installed if you are compiling from source. Are the libraries mentioned as prerequisites somewhere for running the notebook?

### comment:269 Changed 9 years ago by kini

• Description modified (diff)

Yup, that's what the latest patch does - mentions OpenSSL as a requirement for build Sage from source. I should probably also put that in the ticket description for testers/reviewers though.

### comment:270 Changed 9 years ago by jason

My point is that OpenSSL libraries are needed for *running* the notebook, are they not? If so, are they listed somewhere as runtime dependencies?

### comment:271 Changed 9 years ago by kini

Oh, very good point. No, they are not.

### comment:272 Changed 9 years ago by jason

They are not required? Or they are not listed as runtime dependencies?

### comment:273 Changed 9 years ago by kini

They are not listed as runtime dependencies. Though I suppose it might be possible that they are not required either - and I think I'd rather not remove OpenSSL from my system to check this... :P I'll expand the latest patch to also mention this as a runtime dependency, if that's OK with you.

### comment:274 Changed 9 years ago by kini

• Description modified (diff)
• Status changed from needs_review to needs_work

### comment:275 Changed 9 years ago by kini

• Description modified (diff)
• Status changed from needs_work to needs_review

New patch is up.

### comment:276 follow-up: ↓ 277 Changed 9 years ago by ppurka

I don't see any problems with the set of patches and spkgs in this ticket at present.

I checked jmol on the command line and on the notebook and checked some other stuff (plot, interacts, math, etc), and they all work. I would give it positive review except that I haven't faced any openssl problems (in any of the earlier iterations of the patches).

### comment:277 in reply to: ↑ 276 Changed 9 years ago by gutow

I've got a systems I can set up without OpenSSL to test.  However, I won't be able to do it until I finish with exams and grading (roughly May 12).  If we still need this part reviewed at that time, someone should remind me and I will do it.

I don't see any problems with the set of patches and spkgs in this ticket at present. I checked jmol on the command line and on the notebook and checked some other stuff (plot, interacts, math, etc), and they all work. I would give it positive review except that I haven't faced any openssl problems (in any of the earlier iterations of the patches).

### comment:278 Changed 9 years ago by kini

• Dependencies changed from #11078, #11874, #12229, #11503, #12327 to #11078, #11874, #12229, #11503, #12327, #12899
• Status changed from needs_review to needs_work
• Work issues set to rebase on #12899

apply to $SAGE_ROOT/devel/sage (on top of #12899) ### comment:279 Changed 9 years ago by kini • Description modified (diff) • Status changed from needs_work to needs_review • Work issues rebase on #12899 deleted Rebased patch up. ### Changed 9 years ago by ppurka Original on top, with this ticket on bottom ### comment:280 follow-up: ↓ 283 Changed 9 years ago by ppurka Installed it with openssl dev headers missing in Ubuntu-12.04, 64bits. The instructions provided are good. Everything working good except for a slight glitch in live documentation - tested interacts, jsmath, plots, jmol (both in command line and in notebook). In the live documentation, the original output does not contain newlines. See this screenshot. ### comment:281 Changed 9 years ago by kini • Description modified (diff) Edited the instructions since #12899 was merged in 5.0.rc1 ### comment:282 Changed 9 years ago by ppurka • Description modified (diff) Update to refer to the new patch with fixed git links in #12229 ### comment:283 in reply to: ↑ 280 ; follow-up: ↓ 285 Changed 9 years ago by dimpase Replying to ppurka: Installed it with openssl dev headers missing in Ubuntu-12.04, 64bits. The instructions provided are good. Everything working good except for a slight glitch in live documentation - tested interacts, jsmath, plots, jmol (both in command line and in notebook). In the live documentation, the original output does not contain newlines. See this screenshot. This must be browser/OS-specific, as I can't reproduce this MacOSX. ### comment:284 follow-ups: ↓ 286 ↓ 288 Changed 9 years ago by jason @kini, when did you last update the spkg? The other day I fixed some serious scaling bugs that have been around for years (see the latest commits). In the process, I eliminated the need for twisted. We can merge this reviewed version in and then immediately follow up with a new version, or we can try to cram yet a few more commits in. What do you think? ### comment:285 in reply to: ↑ 283 Changed 9 years ago by ppurka Replying to dimpase: Replying to ppurka: Installed it with openssl dev headers missing in Ubuntu-12.04, 64bits. The instructions provided are good. Everything working good except for a slight glitch in live documentation - tested interacts, jsmath, plots, jmol (both in command line and in notebook). In the live documentation, the original output does not contain newlines. See this screenshot. This must be browser/OS-specific, as I can't reproduce this MacOSX. I had tried it on both opera and firefox on Gentoo Linux. If it is not reproducible, then it's even better. We can finally close this ticket. :) ### comment:286 in reply to: ↑ 284 Changed 9 years ago by ppurka Replying to jason: @kini, when did you last update the spkg? The other day I fixed some serious scaling bugs that have been around for years (see the latest commits). In the process, I eliminated the need for twisted. We can merge this reviewed version in and then immediately follow up with a new version, or we can try to cram yet a few more commits in. What do you think? I think it would be better to merge this ticket at the earliest. It has come to a stage (after so many months!) where it is finally satisfactory. The scalability problems can be addressed in a later ticket since the problems are evident only when there are thousands of accounts. From an end-user point of view, the speedup won't be very evident. So, mathjax + uwsgi can be the target of the next big nb ticket. ### comment:287 Changed 9 years ago by jason That sounds good. I wonder if we'll reach 300 comments before this ticket is merged... ### comment:288 in reply to: ↑ 284 Changed 9 years ago by dimpase • Status changed from needs_review to positive_review Replying to jason: @kini, when did you last update the spkg? The other day I fixed some serious scaling bugs that have been around for years (see the latest commits). In the process, I eliminated the need for twisted. We can merge this reviewed version in and then immediately follow up with a new version, or we can try to cram yet a few more commits in. What do you think? boxen:/home/keshav/files$ ls -l sagenb-0.9.0.spkg
-rw-r--r-- 1 keshav keshav 26699829 2012-05-03 23:13 sagenb-0.9.0.spkg


Anyway, IMHO it's time to close this ticket. I hereby give it a positive review...

### comment:289 Changed 9 years ago by jdemeyer

• Milestone changed from sage-5.1 to sage-pending

### comment:290 follow-up: ↓ 296 Changed 9 years ago by kcrisman

• Reviewers changed from Rado Kirov, Dan Drake, Jason Grout, Simon King to Rado Kirov, Dan Drake, Jason Grout, Simon King, Dmitrii Pasechnik

After looking at #12229 fairly carefully, I believe that it should remain a blocker for whatever Sage release the new notebook goes in, but not for this actual ticket. The instructions are so generic and even currently inaccurate, that it would be counterproductive to keep that from allowing people to test this ticket in an early beta of some Sage version.

I know you all know I wouldn't say this lightly ;-) so as long as nobody has a problem with #12229 having that new status

• Blocker for ... Sage 5.2, I think Jeroen was estimating?
• No longer a prereq for this ticket being merged

then let's get this in.

### comment:291 Changed 9 years ago by jdemeyer

• Milestone changed from sage-pending to sage-5.2

### comment:292 Changed 9 years ago by jdemeyer

• Description modified (diff)

### comment:293 in reply to: ↑ description ; follow-up: ↓ 298 Changed 9 years ago by jdemeyer

When we have finalized things, we'll make a finished spkg that has the appropriate tags in the revision history, no unnecessary revision branches, etc.

Maybe now would be the time to do precisely this?

### comment:294 follow-up: ↓ 305 Changed 9 years ago by jdemeyer

• Status changed from positive_review to needs_work

The sagenb's SPKG.txt should list OpenSSL as dependency.

### comment:295 Changed 9 years ago by jdemeyer

• Description modified (diff)

### comment:296 in reply to: ↑ 290 ; follow-up: ↓ 297 Changed 9 years ago by jdemeyer

After looking at #12229 fairly carefully, I believe that it should remain a blocker for whatever Sage release the new notebook goes in, but not for this actual ticket.

This has a high probability of leading to the situation where #11080 gets merged but nobody looks after #12229. Then sage-5.2 (or whichever version) cannot be released. Whenever there is a group of tickets which must be released together (even if they don't strictly depend on eachother), I prefer to wait until all of them have positive review.

### comment:297 in reply to: ↑ 296 Changed 9 years ago by kcrisman

After looking at #12229 fairly carefully, I believe that it should remain a blocker for whatever Sage release the new notebook goes in, but not for this actual ticket.

This has a high probability of leading to the situation where #11080 gets merged but nobody looks after #12229. Then sage-5.2 (or whichever version) cannot be released. Whenever there is a group of tickets which must be released together (even if they don't strictly depend on eachother), I prefer to wait until all of them have positive review.

In this case, because #12229 does not need that much additional work, and because we would like people to TEST the new notebook at an early stage a lot (which #12229 is totally unnecessary for), I disagree.

In any case, most of what ails #12229 is for nb.sagemath.org to be updated slightly so we can change some references there, and I've already sent Harald two emails with specific changes (which hopefully he'll get back to me on soon). So I think it would be penny-wise and pound-foolish to wait for #12229. Also, having this ticket in will put more motivation behind the people at #12229 to get it done :)

### comment:298 in reply to: ↑ 293 Changed 9 years ago by kini

When we have finalized things, we'll make a finished spkg that has the appropriate tags in the revision history, no unnecessary revision branches, etc.

Maybe now would be the time to do precisely this?

I've done all of that stuff already except tagging the release. Stand by...

### comment:299 Changed 9 years ago by kini

John Palmieri, who is sitting next to me at sage days 41, suggests that we just not package a repository with sagenb at all. What do you think? After all, sagenb is considered an upstream package at this point, and we don't package repositories for those. Plus it's using a revision control system that isn't included with sage anyway (yet). Also, due to various decisions by people over the lifetime of the repository, the repository is really large - 14 MB at maximum compression, at the current master. So removing it would slim down the sage source and binary distributions by a non-negligible amount. The docs added at #12229 already give instructions on how to clone the repository from github anyway. So why do we need to ship a repo?

### comment:300 Changed 9 years ago by kcrisman

Please, let's just get this positively reviewed. Also, I'm not really interested in touching #12229 again, which we would have to do if you removed the hg repo this time around. "Slimming down" sounds fine after Jeroen has definitively put this in 5.2.

### comment:301 Changed 9 years ago by jhpalmieri

I was really suggesting this "slimming" for the followup ticket, #13121, not for this one.

### comment:302 Changed 9 years ago by kini

Sure, that's fine.

### comment:303 Changed 9 years ago by kini

New SPKG is up at the same URL.

### comment:304 Changed 9 years ago by kini

• Status changed from needs_work to needs_review

### comment:305 in reply to: ↑ 294 Changed 9 years ago by jhpalmieri

The sagenb's SPKG.txt should list OpenSSL as dependency.

It doesn't look like this has been addressed.

### comment:306 follow-up: ↓ 307 Changed 9 years ago by jhpalmieri

By the way, as far as openssl goes, it probably shouldn't be a dependency, since openssl (at least our current spkg) doesn't seem to build on OS X Lion. So the role of openssl should be spelled out in SPKG.txt: under what circumstances do you need to install it?

### comment:307 in reply to: ↑ 306 ; follow-up: ↓ 308 Changed 9 years ago by jdemeyer

By the way, as far as openssl goes, it probably shouldn't be a dependency, since openssl (at least our current spkg) doesn't seem to build on OS X Lion.

Isn't OpenSSL part of the OS X operating system?

So the role of openssl should be spelled out in SPKG.txt: under what circumstances do you need to install it?

If and only if it is not already supplied on the system.

### comment:308 in reply to: ↑ 307 ; follow-up: ↓ 309 Changed 9 years ago by jhpalmieri

By the way, as far as openssl goes, it probably shouldn't be a dependency, since openssl (at least our current spkg) doesn't seem to build on OS X Lion.

Isn't OpenSSL part of the OS X operating system?

I guess it is, but perhaps not for much longer.

So the role of openssl should be spelled out in SPKG.txt: under what circumstances do you need to install it?

If and only if it is not already supplied on the system.

Okay, sounds good, but this needs to be in SPKG.txt, in clear enough terms so someone like me reading it knows not to try to install the optional openssl spkg on OS X, rather than trying sage -i openssl and then worrying when it doesn't build.

### comment:309 in reply to: ↑ 308 Changed 9 years ago by gutow

Are there any issues left other than the openssl stuff? I've been running this for almost a year on the server I use with my students and have had good luck with it (it is at least as good as the standard notebook). Additionally, I've developed #12299 on top of #11080, so have looked at some fraction of the code in #11080. I'm willing to give it a positive review once the openssl stuff is settled.

### comment:310 Changed 9 years ago by jdemeyer

For me, all that is needed for positive review is to document in SPKG.txt the fact that this depends on OpenSSL.

### comment:311 follow-up: ↓ 312 Changed 9 years ago by kini

Jeroen: the SPKG.txt file here is extremely out of date and there is a lot more wrong with it than the fact that the OpenSSL dependency is not mentioned. I have more or less completely rewritten the SPKG.txt at github #63, which I hope will go into #13121 (in fact the new SPKG.txt is in the preliminary 0.9.1 SPKG I have put up on that ticket). More importantly, the OpenSSL dependency is already explained in the installation docs (by trac_11080-openssl-in-installation-docs.patch). Is it really necessary to edit this old version of SPKG.txt when OpenSSL is already stated as a dependency of Sage as a whole?

John makes a good point, though - it should be made clear that installing OpenSSL is unnecessary if OpenSSL is already on your system. The current patch to the installation docs could be updated to say that more explicitly.

### comment:312 in reply to: ↑ 311 ; follow-up: ↓ 320 Changed 9 years ago by jdemeyer

• Status changed from needs_review to positive_review

Jeroen: the SPKG.txt file here is extremely out of date and there is a lot more wrong with it than the fact that the OpenSSL dependency is not mentioned. I have more or less completely rewritten the SPKG.txt at github #63, which I hope will go into #13121 (in fact the new SPKG.txt is in the preliminary 0.9.1 SPKG I have put up on that ticket). More importantly, the OpenSSL dependency is already explained in the installation docs (by trac_11080-openssl-in-installation-docs.patch). Is it really necessary to edit this old version of SPKG.txt when OpenSSL is already stated as a dependency of Sage as a whole?

Okay, that's good for me.

### comment:313 follow-up: ↓ 317 Changed 9 years ago by nborie

Hi everybody,

I just post this comment to tell you that my machine (core 2 duo with Ubuntu 12.04 32 bits installation) asked to install uuid-dev package when installing sagecell-0.9.0.spkg After the good apt-get install, it works now fine with the 5.0

...
checking for sem_init in -lrt... yes
checking for uuid_generate in -luuid... no
configure: error: cannot link with -luuid, install uuid-dev.
Error configuring zeromq.


The Sage Cell Server works now fine on my machine. On the same machine, I didn't manage to install the sage-5.2.alpha0 which didn't compiled.

Thanks for this nice work. As all this go too far for my computer skills, i can just test your work but no more...

### comment:314 Changed 9 years ago by jdemeyer

• Status changed from positive_review to needs_work

Unfortunately, this fails on hawk (OpenSolaris 06/2009 i86pc):

****************************************************
Host system:
SunOS hawk 5.11 snv_134 i86pc i386 i86pc
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
Target: i386-pc-solaris2.11
Configured with: ../gcc-4.4-20100112/configure --with-as=/usr/local/binutils-2.20/bin/as --with-ld=/usr/ccs/bin/ld --with-gmp=/usr/local --with-mpfr=/usr/local
gcc version 4.4.3 20100112 (prerelease) (GCC)
****************************************************
Processing Twisted-12.1.0.tar.bz2
Running Twisted-12.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-hCklWr/Twisted-12.1.0/egg-dist-tmp-5538SU
conftest.c:1:23: error: sys/epoll.h: No such file or directory
twisted/python/sendmsg.c: In function ‘sendmsg_sendmsg’:
twisted/python/sendmsg.c:198: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:199: error: ‘struct msghdr’ has no member named ‘msg_controllen’
twisted/python/sendmsg.c:201: error: ‘struct msghdr’ has no member named ‘msg_flags’
twisted/python/sendmsg.c:243: warning: implicit declaration of function ‘CMSG_SPACE’
twisted/python/sendmsg.c:268: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:269: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:274: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:276: error: ‘struct msghdr’ has no member named ‘msg_controllen’
twisted/python/sendmsg.c:285: warning: implicit declaration of function ‘CMSG_FIRSTHDR’
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function ‘CMSG_LEN’
twisted/python/sendmsg.c:316: warning: implicit declaration of function ‘CMSG_DATA’
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function ‘CMSG_NXTHDR’
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:352: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:353: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c: In function ‘sendmsg_recvmsg’:
twisted/python/sendmsg.c:414: error: ‘struct msghdr’ has no member named ‘msg_control’
twisted/python/sendmsg.c:416: error: ‘struct msghdr’ has no member named ‘msg_controllen’
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:477: error: ‘struct msghdr’ has no member named ‘msg_flags’
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing Twisted-12.1.0.tar.bz2.


I seem to recall that Twisted-11 did work, but I'm not sure.

### comment:315 follow-up: ↓ 316 Changed 9 years ago by jhpalmieri

The file twisted/topfiles/setup.py has this entry in the variable extensions:

    Extension("twisted.python._epoll",
["twisted/python/_epoll.c"],
condition=lambda builder: (_isCPython and _hasEpoll(builder) and
sys.version_info[:2] < (2, 6))),


The function _hasEpoll (defined in twisted/python/dist.py) should return False if epoll is not present, for example on Solaris, so I don't understand what's going on. (The other parts of the code seem to have fall-backs if epoll is not present.) Anyway, maybe we should try patching this file to remove this extension, when on Solaris or OpenSolaris. I can't log in to hawk right now, so I can't see if this would help.

### comment:316 in reply to: ↑ 315 Changed 9 years ago by jdemeyer

I'm not actually sure the error is related to epoll. The real errors refer to struct msghdr which should exist on OpenSolaris.

I can't log in to hawk right now, so I can't see if this would help.

### comment:317 in reply to: ↑ 313 Changed 9 years ago by kini

Hi everybody,

I just post this comment to tell you that my machine (core 2 duo with Ubuntu 12.04 32 bits installation) asked to install uuid-dev package when installing sagecell-0.9.0.spkg After the good apt-get install, it works now fine with the 5.0

...
checking for sem_init in -lrt... yes
checking for uuid_generate in -luuid... no
configure: error: cannot link with -luuid, install uuid-dev.
Error configuring zeromq.


The Sage Cell Server works now fine on my machine. On the same machine, I didn't manage to install the sage-5.2.alpha0 which didn't compiled.

Thanks for this nice work. As all this go too far for my computer skills, i can just test your work but no more...

Thanks, but you should probably post this on the sage-devel mailing list, or maybe on the issue tracker for the sage cell server here. This ticket is for the Sage notebook, not the cell server. And in particular it's for version 0.9.0 of sagenb.

### comment:318 Changed 9 years ago by jdemeyer

• Dependencies changed from #11078, #11874, #12229, #11503, #12327, #12899 to #13126, #11078, #11874, #12229, #11503, #12327, #12899

Added OpenSSL fix (#13126) as dependency, as the old OpenSSL spkg doesn't build on all supported systems.

### comment:319 follow-up: ↓ 323 Changed 9 years ago by jdemeyer

On hawk, adding the compiler flag -D_XOPEN_SOURCE=500 fixes the problem. Any objections to adding this unconditionally on all architectures?

### comment:320 in reply to: ↑ 312 Changed 9 years ago by kcrisman

Jeroen: the SPKG.txt file here is extremely out of date and there is a lot more wrong with it than the fact that the OpenSSL dependency is not mentioned. I have more or less completely rewritten the SPKG.txt at github #63, which I hope will go into #13121 (in fact the new SPKG.txt is in the preliminary 0.9.1 SPKG I have put up on that ticket). More importantly, the OpenSSL dependency is already explained in the installation docs (by trac_11080-openssl-in-installation-docs.patch). Is it really necessary to edit this old version of SPKG.txt when OpenSSL is already stated as a dependency of Sage as a whole?

Okay, that's good for me.

Though I'd caution - see this ask.sage.math.org post. It's not necessarily all about the notebook, but apparently people using Linux often have this problem. Do we say that this is a dependency for Sage? If this isn't relevant, my apologies.

### comment:321 follow-up: ↓ 322 Changed 9 years ago by kini

That's a problem with Python 2.6 which is fixed in 2.7 if I'm not mistaken - in Sage terms, that means that 4.x will consistently have that problem on newer distros (especially Debian/Ubuntu it seems) unless the user has purposely installed an older-than-default version of OpenSSL systemwide.

Last edited 9 years ago by kini (previous) (diff)

### comment:322 in reply to: ↑ 321 Changed 9 years ago by kcrisman

That's a problem with Python 2.6 which is fixed in 2.7 if I'm not mistaken - in Sage terms, that means that 4.x will consistently have that problem on newer distros (especially Debian/Ubuntu it seems) unless the user has purposely installed an older-than-default version of OpenSSL systemwide.

Hmm, but these all refer to 5.0 and 5.0.1.

Anyway, it may have nothing to do with the notebook and something to do with us needing to have better-documented prereqs. On the other hand, most of those seem to have been using Sage for 10.04 on newer Ubuntu, maybe that's the problem. Sorry if this was noise, it just seemed possibly related because one of the posts was about the notebook.

### comment:323 in reply to: ↑ 319 Changed 9 years ago by jhpalmieri

On hawk, adding the compiler flag -D_XOPEN_SOURCE=500 fixes the problem. Any objections to adding this unconditionally on all architectures?

Do you mean something like this?

• ## spkg-install

 old [ -z "$LIBRARY_PATH" ] || LIBRARY_PATH="$LIBRARY_PATH": export CPATH="$CPATH""$SAGE_LOCAL"/include export LIBRARY_PATH="$LIBRARY_PATH""$SAGE_LOCAL"/lib export CFLAGS="$CFLAGS -D_XOPEN_SOURCE=500" easy_install --allow-hosts=None Twisted-12.1.0.tar.bz2 if [$? -ne 0 ]; then

This seems to allow compilation on hawk and on mark, and compilation still works on OS X Lion and sage.math.

### comment:324 Changed 9 years ago by jdemeyer

Normally such things should be added to $CPPFLAGS, not $CFLAGS.

### comment:325 Changed 9 years ago by jhpalmieri

• Status changed from needs_work to needs_review

I propose using the spkg here, which is identical to Keshav's except for this change:

• ## spkg-install

 old [ -z "$LIBRARY_PATH" ] || LIBRARY_PATH="$LIBRARY_PATH": export CPATH="$CPATH""$SAGE_LOCAL"/include export LIBRARY_PATH="$LIBRARY_PATH""$SAGE_LOCAL"/lib # Fix a build error on (Open)Solaris: export CPPFLAGS="$CPPFLAGS -D_XOPEN_SOURCE=500" easy_install --allow-hosts=None Twisted-12.1.0.tar.bz2 if [$? -ne 0 ]; then

(It's a little frustrating that spkg-install is not under revision control, but this is fixed in #13121.) I think this should be easy to review positively, since the only problem in the earlier spkg was its failure to build on (Open)Solaris. Then we can move on to #13121, which we should try to get into Sage 5.2 as well.

I have not changed the link in the ticket description to point to this new spkg, but if this sounds like a good approach, please go ahead and do that.

### comment:326 Changed 9 years ago by kini

Sounds good to me, except I'd call it 0.9.0, not 0.9.0.p0. Since no 0.9.0 has gone up on the Sage mirrors yet, there's no need for a patch version yet - and a patch version should technically have changes to SPKG.txt too, but as I said before, I want to avoid changing SPKG.txt since it's already been rewritten for #13121.

So I've put your SPKG at the URL in the ticket description, renaming it sagenb-0.9.0.spkg and renaming the directory inside it to sagenb-0.9.0.

### comment:327 Changed 9 years ago by jhpalmieri

It's fine to incorporate my change and still call it 0.9.0; while my change was still pending, I just wanted to distinguish mine from the one you posted earlier.

Jeroen: positive review now?

### comment:328 Changed 9 years ago by kcrisman

• Authors changed from Mike Hansen, Rado Kirov, William Stein, Jason Grout to Mike Hansen, Rado Kirov, William Stein, Jason Grout, Jeroen Demeyer
• Reviewers changed from Rado Kirov, Dan Drake, Jason Grout, Simon King, Dmitrii Pasechnik to Rado Kirov, Dan Drake, Jason Grout, Simon King, Dmitrii Pasechnik, John Palmieri, Punarbasu Purkayastha

Since jdemeyer said:

On hawk, adding the compiler flag -D_XOPEN_SOURCE=500 fixes the problem. Normally such things should be added to $CPPFLAGS, not$CFLAGS.

so the two changes are from him, and jhpalmieri said:

This seems to allow compilation on hawk and on mark, and compilation still works on OS X Lion and sage.math.

so the review of that is from him, I don't quite understand why that isn't enough for positive review. I assume that the CPPFLAGS version was also actually tested, correct?

The current version of the spkg has this in it, so this sounds good enough.

Of course there is this in the SPKG.txt, and presumably other misleading stuff. Does it matter?

== Changelog ==

* For an accurate log of changes, run "hg log" in src/sagenb.


I wonder if we'll reach 300 comments before this ticket is merged...

I guess so, Jason :)

### comment:329 Changed 9 years ago by jdemeyer

• Status changed from needs_review to positive_review

Note that the OpenSSL spkg at #13126 still needs review.

### comment:330 follow-up: ↓ 331 Changed 9 years ago by kini

kcrisman: the line from SPKG.txt which you quoted remains true for the SPKG on this ticket, which does contain a mercurial repository.

Last edited 9 years ago by kini (previous) (diff)

### comment:331 in reply to: ↑ 330 Changed 9 years ago by kcrisman

kcrisman: the line from SPKG which you quoted remains true for the SPKG on this ticket, which does contain a mercurial repository.

I misread it. The line I quote indeed is correct, but there is no top-level hg repository, unlike every other (well, every other recently updated) spkg.

$cd Downloads/sagenb-0.9.0/$ hg st
abort: There is no Mercurial repository here (.hg not found)!

### comment:332 Changed 9 years ago by jdemeyer

• Merged in set to sage-5.2.beta0
• Resolution set to fixed
• Status changed from positive_review to closed

### comment:333 Changed 9 years ago by ddrake

...and there was much rejoicing.

### comment:334 Changed 9 years ago by jdemeyer

• Merged in sage-5.2.beta0 deleted
• Milestone changed from sage-5.2 to sage-pending
• Resolution fixed deleted
• Status changed from closed to new

Unmerging this from sage-5.2 due to the serious security issue at #13270.

### comment:335 Changed 9 years ago by jdemeyer

• Status changed from new to needs_review

### comment:336 Changed 9 years ago by jdemeyer

• Dependencies changed from #13126, #11078, #11874, #12229, #11503, #12327, #12899 to #13126, #11078, #11874, #12229, #11503, #12327, #12899, #13270, to be merged with #13121
• Status changed from needs_review to positive_review

### comment:337 follow-up: ↓ 339 Changed 9 years ago by ppurka

Can you confirm if the patch above does the job? Then we don't need to prolong the release of 5.2. You can apply it to devel/sagenb by the following command

....beta1.11080/devel/sagenb» patch --dry-run -p1 -i /path/to/trac_11080-user_registration.patch
patching file sagenb/notebook/run_notebook.py
....beta1.11080/devel/sagenb» patch -p1 -i /path/to/trac_11080-user_registration.patch
patching file sagenb/notebook/run_notebook.py


If the first command returns success, then proceed with the second command.

Version 1, edited 9 years ago by ppurka (previous) (next) (diff)

### Changed 9 years ago by ppurka

(temporary fix) apply to devel/sagenb

### comment:338 follow-up: ↓ 340 Changed 9 years ago by ppurka

Hmm.. the above was not complete. Typing <url>/register would still go to the registration. The new patch disables user registration. (Again, also taken from the git diff).

### comment:339 in reply to: ↑ 337 Changed 9 years ago by dimpase

Can you confirm if the patch above does the job? Then we don't need to prolong the release of 5.2. You can apply it to devel/sagenb by the following command

....beta1.11080/devel/sagenb» patch --dry-run -p1 -i /path/to/trac_11080-user_registration.patch
patching file sagenb/notebook/run_notebook.py
....beta1.11080/devel/sagenb» patch -p1 -i /path/to/trac_11080-user_registration.patch
patching file sagenb/notebook/run_notebook.py


If the first command returns success, then proceed with the second command.

yes, it fixes it. I was actually pondering the same place here:

Last edited 9 years ago by dimpase (previous) (diff)

### comment:340 in reply to: ↑ 338 Changed 9 years ago by dimpase

Last edited 9 years ago by dimpase (previous) (diff)

### comment:341 follow-up: ↓ 342 Changed 9 years ago by jason

Manage Users is an admin feature. It should *always* work, regardless of the accounts option. Normal users should NOT have the Manage Users page.

Last edited 9 years ago by jason (previous) (diff)

### comment:342 in reply to: ↑ 341 ; follow-up: ↓ 343 Changed 9 years ago by dimpase

Manage Users is an admin feature. It should *always* work, regardless of the accounts option. Normal users should NOT have the Manage Users page.

oh, OK. I'll test this again more carefully, under different users.

### comment:343 in reply to: ↑ 342 Changed 9 years ago by dimpase

Manage Users is an admin feature. It should *always* work, regardless of the accounts option. Normal users should NOT have the Manage Users page.

oh, OK. I'll test this again more carefully, under different users.

YES, the patch works. Let's beg Jeroen to remerge the stuff...

### comment:344 Changed 9 years ago by ppurka

Thanks for confirming. I have the patch in a new branch branch0.9 at github that is created from 0.9.0.

@kini: I sent you a pull request. But if you apply it, then apply to a separate branch, created from 0.9.0.

### comment:345 Changed 9 years ago by jdemeyer

Please discuss #13270 on that ticket, not here.

### comment:346 Changed 9 years ago by jdemeyer

• Dependencies changed from #13126, #11078, #11874, #12229, #11503, #12327, #12899, #13270, to be merged with #13121 to #13126, #11078, #11874, #12229, #11503, #12327, #12899, #13270

### comment:347 Changed 9 years ago by jdemeyer

• Merged in set to sage-5.2.beta0
• Milestone changed from sage-pending to sage-5.2
• Resolution set to fixed
• Status changed from positive_review to closed

### comment:348 Changed 5 years ago by chapoton

• Authors changed from Mike Hansen, Rado Kirov, William Stein, Jason Grout, Jeroen Demeyer to Mike Hansen, Radoslav Kirov, William Stein, Jason Grout, Jeroen Demeyer
• Reviewers changed from Rado Kirov, Dan Drake, Jason Grout, Simon King, Dmitrii Pasechnik, John Palmieri, Punarbasu Purkayastha to Radoslav Kirov, Dan Drake, Jason Grout, Simon King, Dmitrii Pasechnik, John Palmieri, Punarbasu Purkayastha
Note: See TracTickets for help on using tickets.