Upgrade to Pynac-0.7.8

[rws]

From the release page of Pynac-0.7.8:

• fix series bug (#22733)
• fix polylog return type for some arguments (#18386)
• fix Python3 compile (thanks François)
• fix symbol.set_domain(); fix power.is_real() (#23093)
• add limit function to functions with bound symbols (#20179)
• numeric power speedup and cleanup

​https://github.com/pynac/pynac/releases/download/pynac-0.7.8/pynac-0.7.8.tar.bz2

[rws]

New commits:

​bd732ec	23134: version/chksum/delete old patch
​31340ea	23134: doctest fixes

[tscrim]

LGTM and fixes the issues noted in the ticket (except possibly the Python3, which I did not check).

[rws]

Thanks.

[vbraun]

Segfaults at the end of docbuild:

[dochtml] [tutorial ] build succeeded.
[dochtml] Build finished.  The built documents can be found in /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/share/doc/sage/html/es/tutorial
[dochtml] Elapsed time: 3167.9 seconds.
[dochtml] Done building the documentation!
[dochtml] Fatal Python error: PyThreadState_Get: no current thread
[dochtml] ------------------------------------------------------------------------
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x4cdf)[0x7f9637730cdf]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x4e84)[0x7f9637730e84]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x46ee)[0x7f96377306ee]
[dochtml] /lib64/libpthread.so.0(+0x115c0)[0x7f9646e365c0]
[dochtml] /lib64/libc.so.6(gsignal+0x9f)[0x7f964638491f]
[dochtml] /lib64/libc.so.6(abort+0x16a)[0x7f964638651a]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x1958d7)[0x7f96471d88d7]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyThreadState_Get+0x1f)[0x7f96471d3e2e]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyErr_Fetch+0x19)[0x7f96471b7fbb]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/number_field/number_field_element_quadratic.so(+0x54b97)[0x7f8e2751cb97]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/libpython2.7.so.1.0(_Py_Dealloc+0x35)[0x7f964710073e]
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/libpynac.so.13(+0x1600b1)[0x7f8e18fc60b1]
[dochtml] /lib64/libc.so.6(+0x3a410)[0x7f9646389410]
[dochtml] /lib64/libc.so.6(+0x3a46a)[0x7f964638946a]
[dochtml] /lib64/libc.so.6(__libc_start_main+0xf8)[0x7f964636f408]
[dochtml] python(_start+0x2a)[0x40071a]
[dochtml] ------------------------------------------------------------------------
[dochtml] Unhandled SIGABRT: An abort() occurred.
[dochtml] This probably occurred because a *compiled* module has a bug
[dochtml] in it and is not properly wrapped with sig_on(), sig_off().
[dochtml] Python will now terminate.
[dochtml] ------------------------------------------------------------------------
[dochtml] /mnt/disk/home/buildslave-sage/slave/sage_git/build/build/bin/sage-logger: line 89: 22392 Aborted                 (core dumped) ./sage --docbuild --no-pdf-links all html -j
Makefile:993: recipe for target 'doc-html-mathjax' failed

[fbissey]

Did you get that everywhere, or just on some machines? I did successfully build sage and its documentation at ​https://github.com/vbraun/sage/commit/a813afe4b362285d831dd68e68f5b637e4d1b7fa and it included pynac-0.7.8.

[vbraun]

I got that on all buildbots; might be in combination with other tickets (try the next beta)

[fbissey]

Oh, I see it is building the spanish documentation, and I only build english and french, regularly (on the assumption that all the translation do the same thing as the english master anyway). So there could be a problem there that I missed. Specific to spanish and possibly other languages.

[rws]

How to reproduce from the command line? ./sage --docbuild all html will not show it here.

[rws]

(I did merge the latest beta)

[rws]

As a wildly guessing fix I'll merge the (independently opened) #23199.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c83582f	Merge branch 'develop' into t/23134/upgrade_to_pynac_0_7_8
​7f69a64	23199: missing sig_on/off in symbolic arithmetics
​caf7065	Merge branch 'u/rws/23199-1' of git://trac.sagemath.org/sage into t/23134/upgrade_to_pynac_0_7_8

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​af5ef8b	23199: fix previous commit
​e46e07c	Merge branch 'u/rws/23199-1' of git://trac.sagemath.org/sage into t/23134/upgrade_to_pynac_0_7_8

[tscrim]

You should be able to do it from the command line by sage --docbuild es/tutorial html. IIRC, make doc/sage --docbuild all html would build everything, including the non-English docs.

[rws]

With this branch (which has beta10 merged) on my system:

make doc-clean
./sage --docbuild es/tutorial html
...
[tutorial ] build succeeded.
Build finished.  The built documents can be found in /home/ralf/sage/local/share/doc/sage/html/es/tutorial

If someone other could confirm the error using these commands then at least it would restrict the explanation space. Without a confirmation we have only the hypothesis that something is different on the buildbots.

[rws]

More exactly, on my system make doc-clean; ./sage --docbuild es/tutorial html passes OK with this branch both at the commit c83582f8 and HEAD, that is, with or without the #23199 dependency.

[tscrim]

From Volker's posted error, #23199 should fix the problem.

@fbissey Were you able to reproduce the error Volker is getting? If so, can you test this to see if it now works? Otherwise, I'd say set it to positive review to go back to the buildbots and see if they come out with errors again.

[tscrim]

Let's send it back to the buildbots.

[vbraun]

Still segfaults on some buildbots:

[dochtml] [tutorial ] Writing js search indexes...writing additional pages... search
[dochtml] [tutorial ] copying static files... done
[dochtml] [tutorial ] copying extra files... done
[dochtml] [tutorial ] dumping search index in Russian (code: ru) ... done
[dochtml] [tutorial ] dumping object inventory... done
[dochtml] [tutorial ] build succeeded.
[dochtml] Build finished.  The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/html/ru/tutorial
[dochtml] Elapsed time: 1349.9 seconds.
[dochtml] Done building the documentation!
[dochtml] ------------------------------------------------------------------------
[dochtml] /home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x5c15)[0x2b605689bc15]
[dochtml] /home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x5c65)[0x2b605689bc65]
[dochtml] /home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x8af7)[0x2b605689eaf7]
[dochtml] /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x2b604f9860a0]
[dochtml] /home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyErr_Fetch+0xa)[0x2b604f65afca]
[dochtml] /home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/number_field/number_field_element_quadratic.so(+0x80b1)[0x2b665bfdd0b1]
[dochtml] /home/buildbot/slave/sage_git/build/local/lib/libpynac.so.13(+0x161401)[0x2b6669e48401]
[dochtml] /lib/x86_64-linux-gnu/libc.so.6(+0x36aa2)[0x2b6050252aa2]
[dochtml] /lib/x86_64-linux-gnu/libc.so.6(+0x36af5)[0x2b6050252af5]
[dochtml] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x104)[0x2b605023aeb4]
[dochtml] python[0x4006f1]
[dochtml] ------------------------------------------------------------------------
[dochtml] Attaching gdb to process id 15562.
[dochtml] Cannot find gdb installed
[dochtml] GDB is not installed.
[dochtml] Install gdb for enhanced tracebacks.
[dochtml] ------------------------------------------------------------------------
[dochtml] Unhandled SIGSEGV: A segmentation fault occurred.
[dochtml] This probably occurred because a *compiled* module has a bug
[dochtml] in it and is not properly wrapped with sig_on(), sig_off().
[dochtml] Python will now terminate.
[dochtml] ------------------------------------------------------------------------
[dochtml] /home/buildbot/slave/sage_git/build/build/bin/sage-logger: line 89: 15562 Segmentation fault      ./sage --docbuild --no-pdf-links all html -j
make[1]: *** [doc-html-mathjax] Error 139
make[1]: Leaving directory `/home/buildbot/slave/sage_git/build/build/make'

[thansen]

On our sage 8.0.beta11 builds with Debian packages and pynac 0.7.8 this segfault went away after we uninstalled the old pynac library (libpynac10 from version 0.7.5).

[rws]

OK then, this might be the/a reason: ​https://github.com/pynac/pynac/issues/242 As said there I can't change the past so we can only treat symptoms. Remove all libpynac before installation?

[infinity0]

I rebuilt it again (without libpynac10) just to check and it segfaulted again. So unfortunately it seems that removing libpynac10 does not fix the problem.

I did earlier somehow avoid the segfault though, so it's not a deterministic failure.

[rws]

How many out of 10 will this fail on your system:

make doc-clean
./sage --docbuild es/tutorial html

[infinity0]

This does not fail for me at all, after 10+ tries.

The full docbuild failed 2/3 times for me. The time that it succeeded, I didn't clean anything beforehand, I reran that build directly after the first failure. After it, I cleaned everything and then got a second failure.

[vbraun]

"make clean && make doc-html" crashes reproducably in Debian 7...

[vbraun]

Debug backtrace:

GNU gdb (GDB) 7.8
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
To enable execution of this file add
	add-auto-load-safe-path /home/buildbot/slave/sage_git/build/local/lib/libstdc++.so.6.0.21-gdb.py
line to your configuration file "/var/lib/buildbot/.gdbinit".
To completely disable this security protection add
	set auto-load safe-path /
line to your configuration file "/var/lib/buildbot/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
	info "(gdb)Auto-loading safe path"
0x00002b72a1f90c4e in waitpid () from /lib/x86_64-linux-gnu/libpthread.so.0

Stack backtrace
---------------
No symbol table info available.
#1  0x00002b72a8ea5c78 in print_enhanced_backtrace ()
    at build/src/cysignals/implementation.c:394
        parent_pid = 15910
        pid = <optimized out>
#2  0x00002b72a8ea6c6a in sigdie (sig=sig@entry=11, 
    s=s@entry=0x2b72a8eafc40 "Unhandled SIGSEGV: A segmentation fault occurred.") at build/src/cysignals/implementation.c:413
No locals.
#3  0x00002b72a8ea9af7 in cysigs_signal_handler (sig=11)
    at build/src/cysignals/implementation.c:212
        inside = <optimized out>
#4  <signal handler called>
No symbol table info available.
#5  PyErr_Fetch (p_type=p_type@entry=0x7ffe01771d18, 
    p_value=p_value@entry=0x7ffe01771d20, 
    p_traceback=p_traceback@entry=0x7ffe01771d28) at Python/errors.c:248
        tstate = 0x0
#6  0x00002b78ae5e80b1 in __pyx_tp_dealloc_4sage_5rings_12number_field_30number_field_element_quadratic_NumberFieldElement_quadratic (o=0x2b791475cb18)
    at /home/buildbot/slave/sage_git/build/src/build/cythonized/sage/rings/number_field/number_field_element_quadratic.cpp:23195
        etype = 0x0
        eval = 0x0
        etb = 0x0
        p = 0x2b791475cb18
#7  0x00002b78bc5af808 in GiNaC::numeric::~numeric (
    this=0x2b78bc8e7ce0 <GiNaC::numeric::power(GiNaC::numeric const&) const::a+288>, __in_chrg=<optimized out>) at numeric.cpp:783
No locals.
#8  0x00002b78bc5b3c66 in __tcf_0 () at numeric.cpp:1503
No locals.
#9  0x00002b72a285daa2 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#10 0x00002b72a285daf5 in exit () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#11 0x00002b72a2845eb4 in __libc_start_main ()
   from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#12 0x00000000004006f1 in _start ()
No symbol table info available.


Cython backtrace
----------------
#0  0x00002b72a1f90c30 in waitpid()
#1  0x00002b72a8ea5c30 in print_enhanced_backtrace()at /tmp/pip-TRKsSp-build/build/src/cysignals/implementation.c:394
#2  0x00002b72a8ea6c50 in sigdie()at /tmp/pip-TRKsSp-build/build/src/cysignals/implementation.c:413
#3  0x00002b72a8ea9960 in cysigs_signal_handler()at /tmp/pip-TRKsSp-build/build/src/cysignals/implementation.c:212
#4  0x00002b72a1f910a0 in __restore_rt()
#5  0x00002b72a1c65fc0 in PyErr_Fetch()at /home/buildbot/slave/sage_git/build/local/var/tmp/sage/build/python2-2.7.13.p1/src/Python/errors.c:248
#6  0x00002b78ae5e8090 in __pyx_tp_dealloc_4sage_5rings_12number_field_30number_field_element_quadratic_NumberFieldElement_quadratic()at /home/buildbot/slave/sage_git/build/src/build/cythonized/sage/rings/number_field/number_field_element_quadratic.cpp:23195
  23190      }
  23191      #endif
  23192      PyObject_GC_UnTrack(o);
  23193      {
  23194        PyObject *etype, *eval, *etb;
> 23195        PyErr_Fetch(&etype, &eval, &etb);
  23196        ++Py_REFCNT(o);
  23197        __pyx_pw_4sage_5rings_12number_field_30number_field_element_quadratic_28NumberFieldElement_quadratic_11__dealloc__(o);
  23198        --Py_REFCNT(o);
  23199        PyErr_Restore(etype, eval, etb);
#7  0x0000000000000000 in GiNaC::numeric::~numeric()at /home/buildbot/slave/sage_git/build/local/var/tmp/sage/build/pynac-0.7.8/src/ginac/numeric.cpp:783
   778    }
   779    
   780    numeric::~numeric() {
   781            switch (t) {
   782                    case PYOBJECT:
>  783                            Py_DECREF(v._pyobject);
   784                            return;
   785                    case DOUBLE:
   786                            return;
   787                    case MPZ:
#8  0x00002b78bc5b3c33 in __tcf_0()at /home/buildbot/slave/sage_git/build/local/var/tmp/sage/build/pynac-0.7.8/src/ginac/numeric.cpp:1503
  1498                    rational_power_parts(*this, expo, c, d, c_unit);
  1499                    if (d.is_one())
  1500                            return c;
  1501                    else if (d.is_minus_one()
  1502                                    and expo.denom().is_equal(*_num2_p)) {
> 1503                            static numeric a[] = { *_num1_p, I, *_num_1_p, -I };
  1504                            return a[expo.numer().to_long() % 4] * c;
  1505                    }
  1506                    else if (not c_unit)
  1507                            return d.power(expo) * c;


warning: File "/home/buildbot/slave/sage_git/build/local/lib/libstdc++.so.6.0.21-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
Traceback (most recent call last):
  File "<string>", line 131, in <module>
  File "<string>", line 106, in invoke
  File "<string>", line 113, in newest_first_order
  File "<string>", line 92, in print_stackframe
  File "/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/Cython/Debugger/libcython.py", line 88, in wrapper
    raise NoFunctionNameInFrameError()
NoFunctionNameInFrameError: C function name could not be determined in the current C stack frame
close failed in file object destructor:
IOError: [Errno 9] Bad file descriptor

[vbraun]

The static numeric a looks weird. It keeps the _num1_p alive until one happens to enter that branch again; Then we segfault in the dtor of that previous _num1_p.

[vbraun]

Actually, we segfault during program shutdown. The static a dtors are running during shutdown, in reverse order of construction. The python interpreter might very well be gone by that time, I think.

[fbissey]

I can't reproduce it in sage-on-gentoo, I am using gcc 5.4.0 and binutils 2.26.1.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5f9a991	Merge branch 'develop' into t/23134/upgrade_to_pynac_0_7_8
​86f8933	23134: last minute patch for doc build failures on Debian

[rws]

Thanks for synopsis. Static Python objects considered stupidity. The patch should fix it. It's tested but not fully.

[thansen]

I just did a test build with this fix and Debian packages and it didn't segfault. Thanks!