r-2.10.1.p4 may fail to configure with readline (BLOCKS Sage from building on Ubuntu 11.10)

[Snark]

I was having a hard time compiling sage on my ARM box : the R spkg, which until then hadn't given any issue, didn't even want to configure.

The reason was that if ncurses is detected, then termcap isn't searched for, and hence detecting rl_callback_read_char fails (missing -ltermacp means missing symbols) ; which means the configuration failed because --with-readline was given on the commandline, and it wasn't correctly detected.

I got around it by uninstalling the libncurses5-dev package from my system.

Looking at the configure.ac, I think the issue isn't ARM-specific ; perhaps that should be pushed upstream.

---

New spkg:

• ​http://sage.math.washington.edu/home/palmieri/SPKG/readline-6.2.p3.spkg

[leif]

The problem isn't really R itself, but our readline spkg. (Cf. discussions on sage-release where this configure error was reported for Ubuntu 11.10 systems that have libreadline-dev installed.)

Here's a preliminary new readline spkg:

​http://sage.math.washington.edu/home/leif/Sage/spkgs/readline-6.2.p2.spkg

[leif]

P.S.: You should give more details about your configuration.

On many systems, libtermcap.so is (or used to be) just a symbolic link to libncurses.so. On Oneiric, with libreadline-dev installed, libncurses.so is a linker script, which causes problems.

[leif]

Btw., since Sage also ships a termcap package, we should perhaps also build a shared library of that as well, at least if the systems lacks one.

Or, if the system does have a "devel version" of libtermcap (headers, static library and an "unversioned" .so), don't build / install Sage's at all.

[Snark]

As discussed on irc, the readline package compiled, and I'm now managing to compile R with even with the previously problematic headers installed.

[leif]

Replying to Snark:

As discussed on irc, the readline package compiled, and I'm now managing to compile R with even with the previously problematic headers installed.

Fine. Michael Anselmi also reported on ​sage-release:

Good news!  To have sage-4.7.2.rc1 build properly on Ubuntu 11.10, all I
had to do was replace $SAGE_ROOT/spkg/standard/readline-6.1.pkg with
readline-6.2.p2.spkg (the spkg you provided) before running `make'.

(Although he did get an apparently unrelated doctest error, related to LaTeX.)

I'll commit the changes, update the spkg (in-place) and update the ticket's description later.

[jdemeyer]

Milestone sage-4.7.3 deleted

[was]

I just tried building the latest sage-4.8.alpha5 on the latest Ubuntu-11.10. It includes an spkg called "readline-6.2.p2". Nonetheless, the build failed with:

...
checking for history_truncate_file... no
configure: error: --with-readline=yes (default) and headers/libs are not available
Error configuring R.
************************************************************************
Error installing package r-2.14.0.p1
************************************************************************

This is a different readline-6.2.p2 than the one leif posted above. Installing leif's package, I was then able to build R.

I'm bumping the priority of this ticket up to BLOCKER, since we can't release Sage with it not even building on the current version of the most popular Linux distribution.

[ddrake]

I used Leif's spkg above with 4.8.alpha4, and although the build finished, Sage doesn't start; I get a segfault from IPython. I'm using Ubuntu 11.10. Is anyone else seeing this?

[was]

Replying to ddrake:

I used Leif's spkg above with 4.8.alpha4, and although the build finished, Sage doesn't start; I get a segfault from IPython. I'm using Ubuntu 11.10. Is anyone else seeing this?

Dan, I saw exactly the same thing. I posted on sage-devel about this just now. I was able to get around all problems and now have a seemingly functional version of Sage running on 11.10.

[ddrake]

Replying to was:

Dan, I saw exactly the same thing. I posted on sage-devel about this just now. I was able to get around all problems and now have a seemingly functional version of Sage running on 11.10.

Hmmm...in your message (​https://groups.google.com/d/msg/sage-release/sXdDXME5Ds4/4CpiS04wT28J) you get a segfault from some matrix code. But I don't even get a traceback. This is what I get:

drake@klee:/scratch/sage-4.8.alpha4$ ./sage
----------------------------------------------------------------------
| Sage Version 4.8.alpha4, Release Date: 2011-12-13                  |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
local/bin/sage-sage: line 305: 28355 Segmentation fault      sage-ipython "$@" -i

Although I see you used alpha5, and I'm using alpha4. That might be relevant, but maybe not.

Interestingly, sage -ipython (note the space!) works fine. So my problem is somewhere in sage-ipython (note no space!); something with the library flags??

[vbraun]

The readline-6.2.p2 is from #12131, I think.

Leif: Can you merge the one-line change from that readline into a .p3 and clean it up for review? Right now there are not-yet checked in changes...

[fbissey]

Replying to ddrake:

Replying to was:

Dan, I saw exactly the same thing. I posted on sage-devel about this just now. I was able to get around all problems and now have a seemingly functional version of Sage running on 11.10.

Hmmm...in your message (​https://groups.google.com/d/msg/sage-release/sXdDXME5Ds4/4CpiS04wT28J) you get a segfault from some matrix code. But I don't even get a traceback. This is what I get:

drake@klee:/scratch/sage-4.8.alpha4$ ./sage
----------------------------------------------------------------------
| Sage Version 4.8.alpha4, Release Date: 2011-12-13                  |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
local/bin/sage-sage: line 305: 28355 Segmentation fault      sage-ipython "$@" -i

Although I see you used alpha5, and I'm using alpha4. That might be relevant, but maybe not.

Interestingly, sage -ipython (note the space!) works fine. So my problem is somewhere in sage-ipython (note no space!); something with the library flags??

That's the kind of output you get with the failure from matrix_modn_float if you don't start sage with "-gdb" see my own follow up in that thread. alpha3,4,5 is irrelevant here. And this is not related to R or readline but #4260.

[ddrake]

Replying to fbissey:

That's the kind of output you get with the failure from matrix_modn_float if you don't start sage with "-gdb" see my own follow up in that thread. alpha3,4,5 is irrelevant here. And this is not related to R or readline but #4260.

Ah, you are right. And I can confirm that the patch you included (​https://groups.google.com/d/msg/sage-release/sXdDXME5Ds4/pteQHfW2APYJ) allows 4.8.alpha4 to start and pass all doctests in sage/matrix on my Ubuntu 11.10 machine!

[ddrake]

These issues with segfaults on startup are being addressed at #12157.

[jdemeyer]

bump Leif: Can you merge the one-line change from that readline into a .p3 (starting from #12131) and clean it up for review?

[jhpalmieri]

I've posted an updated version of Leif's spkg, although without the changes to spkg-check.

[jhpalmieri]

patch for readline spkg. For review only.

[cremona]

Is this one of the blockers for 4.8 release? In which case some concerted effort to get it reviewed is surely urgent?

Using readline-6.2.p2.spkg I just helped my colleague Bruce Westbury build Sage on an old laptop on which he had installed ubuntu 11.10 (which is still building) but realised later that he was building 4.7.2 and not any 4.8.alpha. The only ubuntu 11.10 machine I have is a slow netbook on which I built 4.7.2 before upgrading from ubuntu 11.04. I can try this patch out on that but it may take a few days.

[jdemeyer]

Replying to cremona:

Is this one of the blockers for 4.8 release? In which case some concerted effort to get it reviewed is surely urgent?

Yes, it is. And I certainly hope this will get fixed rather quickly.

[jdemeyer]

The spkg looks good on first sight. I can of course not test it on every system. So I'm willing to give it positive_review if somebody can confirm that it actually fixes the problem on Ubuntu 11.10. I asked Paul Zimmermann to do this.

[cremona]

Bruce and I used readline-6.2.p2.spkg not the p3, and also used it on a fresh build of 4.7.2 (which had previously stalled building R). So the following might not be relevant: after building, when we run Sage it gives a 2-line warning that readline is not available -- and indeed Sage has no history functions. What did we do wrong?

Bruce also tells me that although he installed all the ubuntu packages listed on the installation web page, he also had to install dpkg-dev before Sage would build.

[ddrake]

Replying to cremona:

Bruce and I used readline-6.2.p2.spkg not the p3, and also used it on a fresh build of 4.7.2 (which had previously stalled building R). So the following might not be relevant: after building, when we run Sage it gives a 2-line warning that readline is not available -- and indeed Sage has no history functions. What did we do wrong?

I got the same thing using the "p3" spkg. But the build finished and (I think) passed tests.

[zimmerma]

I also tried the p3 spkg, and indeed the build finished, but:

tiramisu% ./sage
----------------------------------------------------------------------
| Sage Version 4.8.alpha5, Release Date: 2011-12-21                  |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
WARNING: Readline services not available on this platform.
WARNING: The auto-indent feature requires the readline library

However I only applied the spkg, not the patch.

Paul

[jdemeyer]

On sage.math with:

• the new readline spkg from this ticket
• compiled with gcc-4.2.4
• #12263 applied

the following happens (and all 3 conditions above are needed):

$ ./sage -c True
mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libcsage.so(print_backtrace+0x2b)[0x7f22311cd3a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libcsage.so(sigdie+0x14)[0x7f22311cd3ce]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libcsage.so(sage_signal_handler+0x201)[0x7f22311cd031]
/lib/libpthread.so.0[0x7f2232dd37d0]
/lib/libc.so.6(strcmp+0x2)[0x7f2232455b42]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libreadline.so.6(tgetent+0xc4)[0x7f222bb2d18c]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libreadline.so.6(_rl_init_terminal_io+0x306)[0x7f222bb24fc6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libreadline.so.6(rl_initialize+0x87)[0x7f222bb0f5b7]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/python2.6/lib-dynload/readline.so(initreadline+0x15f)[0x7f222bd4248f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(_PyImport_LoadDynamicModule+0xc2)[0x7f22330ddc42]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbfb2]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x12a)[0x7f22330dc1ca]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x169)[0x7f22330dc209]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x169)[0x7f22330dc209]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x12a)[0x7f22330dc1ca]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x6452)[0x7f22330c3552]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dd018]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbfb2]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x12a)[0x7f22330dc1ca]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x12a)[0x7f22330dc1ca]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x12a)[0x7f22330dc1ca]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x169)[0x7f22330dc209]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x7f22330d8b52]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330db368]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dba59]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330dbf2b]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyImport_ImportModuleLevel+0x169)[0x7f22330dc209]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0[0x7f22330bbdcd]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyObject_Call+0x68)[0x7f2233026f48]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x7f22330bc4f6]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x214f)[0x7f22330bf24f]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x891)[0x7f22330c49a1]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x32)[0x7f22330c4a72]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyRun_FileExFlags+0xc1)[0x7f22330e8871]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(PyRun_SimpleFileExFlags+0x1f9)[0x7f22330e8b49]
/mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libpython2.6.so.1.0(Py_Main+0xa71)[0x7f22330f4591]
/lib/libc.so.6(__libc_start_main+0xf4)[0x7f22323f91c4]
python[0x400679]

------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred in Sage.
This probably occurred because a *compiled* component of Sage has a bug
in it and is not properly wrapped with sig_on(), sig_off(). You might
want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate.
------------------------------------------------------------------------
local/bin/sage-sage: line 766: 23516 Segmentation fault      sage-eval "$@"

[leif]

Replying to cremona:

Bruce and I used readline-6.2.p2.spkg not the p3, and also used it on a fresh build of 4.7.2 (which had previously stalled building R). So the following might not be relevant: after building, when we run Sage it gives a 2-line warning that readline is not available -- and indeed Sage has no history functions. What did we do wrong?

Before building Sage (or [re-]installing Sage's readline), you need to

$ sudo apt-get install libncurses5-dev libncursesw5-dev

(The latter depends on the former, so you actually have to specify only one of them). You do not have to install any libreadline package, as Sage builds and uses its own -- the one we're talking about here.

Bruce also tells me that although he installed all the ubuntu packages listed on the installation web page, he also had to install dpkg-dev before Sage would build.

I'm pretty sure the instructions on the web page are still outdated (and by the way have been wrong anyway), as discussed with Harald on sage-release a while ago.

dpkg-dev is needed since Ubuntu 11.04, in order to make Python build (because the locations of some libraries were changed in Debian/Ubuntu?).

---

I'm sorry I'm unable to work much on Sage at the moment; thanks John for providing an updated and rebased spkg of my preliminary one.

[leif]

Replying to jdemeyer:

On sage.math with:

• the new readline spkg from this ticket
  
• compiled with gcc-4.2.4
  
• #12263 applied
  

the following happens (and all 3 conditions above are needed):

...
> /lib/libc.so.6(strcmp+0x2)[0x7f2232455b42]
> /mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libreadline.so.6(tgetent+0xc4)[0x7f222bb2d18c]
> /mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libreadline.so.6(_rl_init_terminal_io+0x306)[0x7f222bb24fc6]
> /mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/libreadline.so.6(rl_initialize+0x87)[0x7f222bb0f5b7]
> /mnt/usb1/scratch/jdemeyer/merger/sage-4.8.rc1/local/lib/python2.6/lib-dynload/readline.so(initreadline+0x15f)[0x7f222bd4248f]
...

Can you track this further down?

I tried tgetent(NULL, NULL) on sage.math (with GCC 4.2.4), which doesn't segfault but properly returns 1. Could you try setting TERM to something else (rather than unsetting it) in sage-sage, as suggested at #12263?

[jdemeyer]

Here is a first interesting data point:

Let tgetent_test.c be following file:

#include <termcap.h>
#include <stdlib.h>

int main(int argc, char** argv)
{
    char* buf = malloc(100000);
    tgetent(buf, "dumb");
}

Compiling this with

$ gcc -ltermcap tgetent_test.c -o tgetent_test

and running it works flawlessly.

However, the same gcc command with the Sage environment variables gives

$ ( source local/bin/sage-env; gcc -ltermcap tgetent_test.c -o tgetent_test )
/tmp/ccwDi3Oi.o: In function `main':
tgetent_test.c:(.text+0x27): undefined reference to `tgetent'
collect2: ld returned 1 exit status

With gcc-4.5.1 (which I happen to have installed in my own $HOME directory), the compilation works fine again.

[jdemeyer]

Deleting the file $SAGE_ROOT/local/lib/libtermcap.a solves the problem. Somehow that static-only library confuses the linker. Not sure whether this is a linker bug or feature.

[jdemeyer]

Note: the output of readline-6.2.p3.log is exactly the same whether or not local/lib/libtermcap.a exists. From my observations above, I would expect the readline build to fail when local/bin/libtermcap.a exists. But it doesn't, so maybe it finds a broken tgetent() somewhere somehow.

[jdemeyer]

That's it for now, going to sleep...

[leif]

Replying to jdemeyer:

Here is a first interesting data point:

Let tgetent_test.c be following file:

#include <termcap.h>
#include <stdlib.h>

int main(int argc, char** argv)
{
    char* buf = malloc(100000);
    tgetent(buf, "dumb");
}

FWIW, I did #include <term.h> and used -lcurses, since termcap is part of ncurses on Ubuntu/Debian?.

Compiling this with

$ gcc -ltermcap tgetent_test.c -o tgetent_test

and running it works flawlessly.

Libraries should in general be specified after the modules that use (parts of) them. This is (again) mandatory with newer Linux distros as you will know (cf. the Symmetrica and Singular changes for Ubuntu 11.10).

Some linkers seem to make a difference w.r.t. whether they find a static library [only] (or depending on whether LIBRARY_PATH is set? -- which sage-env does).

However, the same gcc command with the Sage environment variables gives

$ ( source local/bin/sage-env; gcc -ltermcap tgetent_test.c -o tgetent_test )
/tmp/ccwDi3Oi.o: In function `main':
tgetent_test.c:(.text+0x27): undefined reference to `tgetent'
collect2: ld returned 1 exit status

This problem vanishes if you

• put -ltermcap last (as one would expect), or
• unset LIBRARY_PATH after sourcing sage-env (in which case Sage's libtermcap.a of course isn't used), or
• both.

Anyway, I don't think this is related to our problem; Sage's readline shouldn't^TM use Sage's termcap library at all; on Ubuntu, libtermcap* is a symlink to libncurses*.

[zimmerma]

I confirm that -ltermcap should be last. With gcc 4.6.1:

tiramisu% gcc tgetent_test.c -o tgetent_test -ltermcap
tiramisu% gcc -ltermcap tgetent_test.c -o tgetent_test
/tmp/ccqQuCr3.o: In function `main':
tgetent_test.c:(.text+0x2a): undefined reference to `tgetent'
collect2: ld returned 1 exit status

Paul

[leif]

Replying to jdemeyer:

On sage.math with:

• the new readline spkg from this ticket
  
• compiled with gcc-4.2.4
  
• #12263 applied
  

the following happens (and all 3 conditions above are needed) [...]

Anything else that might affect this (besides #12263, which was merged into rc0), i.e., does the same happen with 4.8.alpha{5,6} + #12263 (with GCC 4.2.4 on Ubuntu 08.04 / sage.math), and does my change (on top of readline-6.2.p1) make the difference?

My current internet connection is quite poor, so I don't know if I can fully build Sage and experiment on sage.math myself.

Otherwise you could try unsetting LIBRARY_PATH in readline's spkg-install (with the same configuration as above that segfaulted) and see whether that makes any difference. I didn't know that sage-env sets LIBRARY_PATH, so didn't expect that Sage's libtermcap might get picked up despite us intentionally not using -I$SAGE_LOCAL/include nor -L$SAGE_LOCAL/lib in readline's spkg-install.

[leif]

FWIW, I've made a p4 (changes not yet committed) that unsets LIBRARY_PATH:

​http://sage.math.washington.edu/home/leif/Sage/spkgs/readline-6.2.p4.spkg

[leif]

Ok, I've now successfully built Sage 4.8.alpha6 on sage.math (with GCC 4.2.4); sage -c True works (doesn't segfault)

• with the vanilla installation
• with my new readline-6.2.p4
• with my p4 and #12263 applied (i.e., TERM unset).

Jeroen, can you check / confirm that it also works with your 4.8.rc1?

[leif]

There's also a "debug" version which outputs some make variables when building the shared library (modified patch to upstream):

​http://sage.math.washington.edu/home/leif/Sage/spkgs/readline-6.2.p4-debug.spkg

src/shlib/Makefile.in

@@ -166 +166 @@
 
 $(SHARED_READLINE):     $(SHARED_OBJ)
         $(RM) $@
-        $(SHOBJ_LD) ${SHOBJ_LDFLAGS} ${SHLIB_XLDFLAGS} -o $@ $(SHARED_OBJ) $(SHLIB_LIBS)
+        # $(SHOBJ_LD) ${SHOBJ_LDFLAGS} ${SHLIB_XLDFLAGS} -o $@ $(SHARED_OBJ) $(SHLIB_LIBS)
+        @echo "**************************************************************************"
+        @echo "Sage: Shared readline library gets linked to \"$(TERMCAP_LIB)\"."
+        @echo
+        @echo "\$$(SHOBJ_LD)="$(SHOBJ_LD)
+        @echo "\$$(SHOBJ_LDFLAGS)="$(SHOBJ_LDFLAGS)
+        @echo "\$$(SHOBJ_XLDFLAGS)="$(SHOBJ_XLDFLAGS)
+        @echo
+        @echo "\$$(SHLIB_LIBS)="$(SHLIB_LIBS)
+        @echo "**************************************************************************"
+        $(SHOBJ_LD) ${SHOBJ_LDFLAGS} ${SHLIB_XLDFLAGS} -o $@ $(SHARED_OBJ) $(SHLIB_LIBS) $(TERMCAP_LIB)
 
 $(SHARED_HISTORY):      $(SHARED_HISTOBJ) xmalloc.so xfree.so
         $(RM) $@

[zimmerma]

I tried ​http://sage.math.washington.edu/home/leif/Sage/spkgs/readline-6.2.p4.spkg under Ubuntu 11.10 but still get:

tiramisu% ./sage
----------------------------------------------------------------------
| Sage Version 4.8.alpha5, Release Date: 2011-12-21                  |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
WARNING: Readline services not available on this platform.
WARNING: The auto-indent feature requires the readline library

However I had previously installed the previous spkg (readline-6.2.p3.spkg). Could it be that there was an interference between both? If so, how could I get a fresh Sage version without starting "make" from scratch?

Paul

[jdemeyer]

I think I got to the bottom of the tgetent() segmentation fault: it is a bug in the termcap library. Under normal circumstances, strcmp() can be called with a NULL pointer argument.

[jdemeyer]

See #12282 for the termcap issue.

[jdemeyer]

Replying to zimmerma:

I tried ​http://sage.math.washington.edu/home/leif/Sage/spkgs/readline-6.2.p4.spkg under Ubuntu 11.10 but still get:

tiramisu% ./sage
----------------------------------------------------------------------
| Sage Version 4.8.alpha5, Release Date: 2011-12-21                  |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
WARNING: Readline services not available on this platform.
WARNING: The auto-indent feature requires the readline library

John Cremona reports that building Sage from scratch with the new readline p3 solves this warning.

[leif]

Replying to jdemeyer:

Replying to zimmerma:

I tried ​http://sage.math.washington.edu/home/leif/Sage/spkgs/readline-6.2.p4.spkg under Ubuntu 11.10 but still get:

...
WARNING: Readline services not available on this platform.
WARNING: The auto-indent feature requires the readline library

John Cremona reports that building Sage from scratch with the new readline p3 solves this warning.

You just have to:

1. sudo apt-get install libncurses5-dev as mentioned above
2. (Re)install the readline spkg:
   ./sage -f /path/to/readline-6.2.p{2,3,4}
3. Reinstall the Python spkg:
   ./sage -f spkg/standard/python-* (or ./sage -f python-2.6.4.p13)

[leif]

Replying to jdemeyer:

• status changed from needs_review to positive_review

Which spkg got the positive review?

I'd prefer also unsetting LIBRARY_PATH in readline's spkg-install (which the p4 does), since in any case we don't want to use Sage's static libtermcap there.

[leif]

Diff between the p3 and the p4. For reference / review only.

[leif]

I've attached a diff between the p3 and the p4; the changes aren't yet committed though.

[zimmerma]

thanks Leif, I missed the last step (reinstalling the python spkg). It works now.

Paul

[jdemeyer]

Replying to leif:

Which spkg got the positive review?

The p3 version (the spkg mentioned in the ticket description).

since in any case we don't want to use Sage's static libtermcap there.

Why not? The fixed one from #12282 should be fine.