Upgrade Singular

[jdemeyer]

Upgrade Singular to ​version 3-1-5 (released 12 July 2012).

Upstream bugs discovered:

• ​#437: negate() should be NTL::negate() (little feedback)
• ​#438: Singular configured --without-debug doesn't build (developers acknowledge the bug)
• ​#439: factorize() returns wrong constant (fixed upstream, but not in a stable release)
• ​#440: Factoring over a finite field returns reducible factor (fixed upstream, but not in a stable release)
• ​#441: Factoring over a finite field sometimes returns wrong result (silently fixed upstream, but not in a stable release)

Apply 13237_singular_3_1_5.patch​

New spkg: ​http://boxen.math.washington.edu/home/jdemeyer/spkg/singular-3-1-5.p0.spkg

singular-3-1-5.p0 (Jeroen Demeyer, 1 August 2012)

• Trac #13237: Upgrade to version 3-1-5.
• Removed patches which are now upstreamed:
  • patches/Singular.Makefile.in.shared.patch
  • patches/Singular.configure.patch
  • patches/factory.GNUmakefile.in.patch
  • patches/factory_configure
  • patches/make_parallel.patch
  • patches/os_x_ppc.patch
• In spkg-install, simplify apply_patches(), automatically apply all patches in patches/*.patch, move conditional patches to patches/conditional
• Rename some patches such that they all have the extension '.patch'
• Put the two patches for SAGE_DEBUG (Singular.Makefile.in.debug.patch and kernel.Makefile.in.debug.patch) into one file conditional/sage_debug.patch.
• Add several patches (see above):
  • NTL_negate.patch
  • singular_trac_439.patch
  • singular_trac_440.patch
• In spkg-install, remove distclean() step; merge clean_headers() and part of the old distclean() into remove_old_version().
• Only unset LD on Darwin.
• Remove the unsetting of TMPDIR (bug fixed upstream).
• Do not override user-set CFLAGS and CXXFLAGS.
• Echo all error messages to stderr instead of stdout.
• Fix various ./configure options and remove some unsupported options.
• Always configure --with-debug, as --without-debug doesn't work, see ​http://www.singular.uni-kl.de:8002/trac/ticket/438
• Don't create sage_singular symlink, which wasn't used anyway.
• Slightly change the $SAGE_LOCAL/bin/Singular script, use "$@" instead of $*.
• Remove workaround for GCC-4.0.x on Darwin (obsolete by the GCC spkg).
• Fix formatting of spkg-install (consistent indentation, no TABs).

[leif]

checking for FLINT >= 2.3... not found
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)

:P

And lots of other warnings...

[leif]

Replying to leif:

And lots of other warnings...

These are the ones I get from configure and config.status:

configure: warning: Building of doc might fail. Need Texinfo
configure: WARNING: unrecognized options: --with-apint, --with-ntl, --without-MP, --without-lex, --without-bison, --without-Boost, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-factory, --with-libfac
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)
config.status: WARNING:  'GNUmakefile.in' seems to ignore the --datarootdir setting
configure: WARNING: unrecognized options: --with-apint, --with-ntl, --without-MP, --without-lex, --without-bison, --without-Boost, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-factory, --with-libfac
configure: warning: factory.h not found! Install factory before building libfac!
configure: WARNING: unrecognized options: --with-gmp, --with-ntl, --without-bison, --enable-gmp, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-Singular
configure: WARNING: building without lex -- make might fail
configure: WARNING: building without python support
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)
configure: WARNING: unrecognized options: --with-gmp, --with-ntl, --without-bison, --enable-gmp, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-Singular
configure: warning: Building of doc might fail. Need Texinfo
configure: WARNING: unrecognized options: --with-apint, --with-ntl, --without-MP, --without-lex, --without-bison, --without-Boost, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-factory, --with-libfac
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)
config.status: WARNING:  'GNUmakefile.in' seems to ignore the --datarootdir setting
configure: WARNING: unrecognized options: --with-apint, --with-ntl, --without-MP, --without-lex, --without-bison, --without-Boost, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-factory, --with-libfac
configure: warning: factory.h not found! Install factory before building libfac!
configure: WARNING: unrecognized options: --with-gmp, --with-ntl, --without-bison, --enable-gmp, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-Singular
configure: WARNING: building without lex -- make might fail
configure: WARNING: building without python support
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)
configure: WARNING: unrecognized options: --with-gmp, --with-ntl, --without-bison, --enable-gmp, --enable-Singular, --enable-factory, --enable-libfac, --enable-IntegerProgramming, --disable-doc, --with-malloc, --disable-debug, --enable-omalloc, --with-external-config_h, --with-track-custom, --enable-Plural, --with-Singular
configure: WARNING: unrecognized options: --with-apint, --with-ntl, --disable-debug
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)
config.status: WARNING:  'GNUmakefile.in' seems to ignore the --datarootdir setting
configure: WARNING: unrecognized options: --with-apint, --with-ntl, --disable-debug

The unrecognized options should be addressed I think.

No idea how to interpret "factory.h not found! Install factory before building libfac!", as the build succeeded.

"Building without python support" can safely be ignored I guess, as we use our own Cython wrappers.

[jdemeyer]

Replying to leif:

The unrecognized options should be addressed I think.

I fixed the fixable warnings. Many of them are caused by recursive calling of configure, for example, the top-level configure calls omalloc/configure. Many options are supported by some (but not all) configure scripts. These will give warnings and I don't see how to avoid these.

[leif]

I assume that this is work in progress (i.e., you haven't tried to fix doctest errors yet).

Anyway, here's what I got on Ubuntu 10.04.4 LTS x86_64, with Sage 5.2.rc0 and GCC 4.7.0:

The following tests failed:

	sage -t  --long -force_lib devel/sage/doc/en/constructions/polynomials.rst # 1 doctests failed
	sage -t  --long -force_lib devel/sage/sage/rings/function_field/function_field.py # 1 doctests failed
	sage -t  --long -force_lib devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx # 3 doctests failed
	sage -t  --long -force_lib devel/sage/sage/rings/polynomial/multi_polynomial_ideal.py # 1 doctests failed
	sage -t  --long -force_lib devel/sage/sage/calculus/wester.py # 1 doctests failed
	sage -t  --long -force_lib devel/sage/sage/libs/singular/function.pyx # 2 doctests failed
	sage -t  --long -force_lib devel/sage/sage/combinat/words/words.py # 0 doctests failed

(The last failure was a segfault with a large traceback IIRC. Somewhat surprisingly successfully compiled with -O3, although many if not most compile commands override this by -O2; looks as if they use CPPFLAGS instead of CXXFLAGS in some parts.)

Let me know in case you're interested in further output; I guess you can reproduce most of the errors.

[jdemeyer]

Replying to leif:

I assume that this is work in progress (i.e., you haven't tried to fix doctest errors yet).

Exactly, I mainly worked on spkg-install.

[jdemeyer]

This is a new bug, the new answer is incorrect:

sage -t  --long -force_lib devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx
**********************************************************************
File "/release/merger/sage-5.2.rc1/devel/sage-main/sage/rings/polynomial/multi_polynomial_libsingular.pyx", line 3856:
    sage: p.factor()
Expected:
    (-2*v^2*u + 4*u^3 + v^2)^2
Got:
    (4) * (-2*v^2*u + 4*u^3 + v^2)^2
**********************************************************************

[leif]

Replying to jdemeyer:

This is a new bug, the new answer is incorrect:

Expected:
    (-2*v^2*u + 4*u^3 + v^2)^2
Got:
    (4) * (-2*v^2*u + 4*u^3 + v^2)^2

Yes, the most obvious one... :-)

Just noticed the failure in words.py apparently isn't reproducible. Traceback was

${SAGE_LOCAL}/lib/libcsage.so(print_backtrace+0x31)[0x2abd9e8f6664]
${SAGE_LOCAL}/lib/libcsage.so(sigdie+0x14)[0x2abd9e8f6696]
${SAGE_LOCAL}/lib/libcsage.so(sage_signal_handler+0x218)[0x2abd9e8f6250]
/lib/libpthread.so.0(+0xf8f0)[0x2abd9c9158f0]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(+0x135020)[0x2abd9c628020]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(+0x76aa7)[0x2abd9c569aa7]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(+0x1352b7)[0x2abd9c6282b7]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(_PyObject_GC_Malloc+0x115)[0x2abd9c6292b5]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(_PyObject_GC_New+0xd)[0x2abd9c6292ed]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(PyCFunction_NewEx+0xc5)[0x2abd9c584f05]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(Py_InitModule4_64+0x140)[0x2abd9c60bc60]
${SAGE_LOCAL}/lib/python/site-packages/numpy/core/multiarray.so(initmultiarray+0x2b)[0x2abdcd07186b]
${SAGE_LOCAL}/lib/libpython2.7.so.1.0(_PyImport_LoadDynamicModule+0x99)[0x2abd9c6073b9]
...
Segmentation fault

[jdemeyer]

Another Singular bug:

sage -t --long "devel/sage/sage/rings/function_field/function_field.py"
**********************************************************************
File "/release/merger/sage-5.2.rc1/devel/sage/sage/rings/function_field/function_field.py", line 1132:
    sage: f.factor()
Expected:
    (1/t) * (X + 3*t) * (X + 5*t) * (X + 6*t) * (X^2 + 1/t) * (X^2 + 6/t)
Got:
    (1/t) * (X + 3*t) * (X + 5*t) * (X + 6*t) * (X^4 + 6/t^2)
**********************************************************************

Reported upstream at ​http://www.singular.uni-kl.de:8002/trac/ticket/440.

[jdemeyer]

New spkg, adds fixes for Singular bugs 439 and 440.

[jdemeyer]

Reproducible doctest failure:

**********************************************************************
File "/release/merger/sage-5.3.beta0/devel/sage-main/sage/libs/singular/function.pyx", line 1569:
    sage: singular_list(2, 3, 6, ring=P)
Exception raised:
    Traceback (most recent call last):
      File "/release/merger/sage-5.3.beta0/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/release/merger/sage-5.3.beta0/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/release/merger/sage-5.3.beta0/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_72[16]>", line 1, in <module>
        singular_list(Integer(2), Integer(3), Integer(6), ring=P)###line 1569:
    sage: singular_list(2, 3, 6, ring=P)
      File "function.pyx", line 1248, in sage.libs.singular.function.SingularFunction.__call__ (sage/libs/singular/function.cpp:11875)
        return call_function(self, args, ring, interruptible, attributes)
      File "function.pyx", line 1438, in sage.libs.singular.function.call_function (sage/libs/singular/function.cpp:13425)
        raise RuntimeError("Error in Singular function call '%s':\n%s"%(self._name,
    RuntimeError: Error in Singular function call 'list':
     list(`int`,`int`,`int`) failed
**********************************************************************
File "/release/merger/sage-5.3.beta0/devel/sage-main/sage/libs/singular/function.pyx", line 1585:
    sage: singular_list((1,2,3),3,[1,2,3], ring=P)
Exception raised:
    Traceback (most recent call last):
      File "/release/merger/sage-5.3.beta0/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/release/merger/sage-5.3.beta0/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/release/merger/sage-5.3.beta0/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_72[26]>", line 1, in <module>
        singular_list((Integer(1),Integer(2),Integer(3)),Integer(3),[Integer(1),Integer(2),Integer(3)], ring=P)###line 1585:
    sage: singular_list((1,2,3),3,[1,2,3], ring=P)
      File "function.pyx", line 1248, in sage.libs.singular.function.SingularFunction.__call__ (sage/libs/singular/function.cpp:11875)
        return call_function(self, args, ring, interruptible, attributes)
      File "function.pyx", line 1438, in sage.libs.singular.function.call_function (sage/libs/singular/function.cpp:13425)
        raise RuntimeError("Error in Singular function call '%s':\n%s"%(self._name,
    RuntimeError: Error in Singular function call 'list':
     list(`intvec`,`int`,`list`) failed
**********************************************************************

Non-reproducible doctest failure:

**********************************************************************
File "/release/merger/sage-5.3.beta0/devel/sage-main/sage/rings/polynomial/multi_polynomial_libsingular.pyx", line 3921:
    sage: f.factor()
Expected:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a)*x^8 + (a)*y^4)
Got:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a + 1)*x^8 + (a + 1)*y^4)
**********************************************************************

[leif]

So far I only tested the files that previously failed for me (with the new spkg and the Sage library patch applied).

I get the same list() errors (although prepended by some debug output it seems, before the doctest error is printed), and sporadically the f.factor() error -- with varying results:

sage -t --long -force_lib "devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx"
**********************************************************************
File "${SAGE_ROOT}/devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx", line 3910:
    sage: f.factor()
Expected:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a)*x^8 + (a)*y^4)
Got:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a + 1)*x^8 + (a + 1)*y^4)
**********************************************************************
1 items had failures:
   1 of  86 in __main__.example_77
***Test Failed*** 1 failures.

sage -t --long -force_lib "devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx"
**********************************************************************
File "${SAGE_ROOT}/devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx", line 3910:
    sage: f.factor()
Expected:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a)*x^8 + (a)*y^4)
Got:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9)
**********************************************************************
1 items had failures:
   1 of  86 in __main__.example_77
***Test Failed*** 1 failures.

sage -t --long -force_lib "devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx"
**********************************************************************
File "${SAGE_ROOT}/devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx", line 3910:
    sage: f.factor()
Expected:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a)*x^8 + (a)*y^4)
Got:
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + x^8 + y^4)
**********************************************************************
1 items had failures:
   1 of  86 in __main__.example_77
***Test Failed*** 1 failures.

(For me it's line 3910; presumably due to some other patch. The first variant seems to occur more often. All tests run sequentially, FWIW.)

P.S.: Run 100 times from the shell, the test failed 51 times.

[leif]

Replying to leif:

P.S.: Run 100 times from the shell, the test failed 51 times.

(Running sage -t ... in a for loop.)

25x
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + (a + 1)*x^8 + (a + 1)*y^4)

15x
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9 + x^8 + y^4)

11x
    x * y^3 * (y^8 + (a)*y^7 + (a + 1)*x) * (x^7*y^3 + x*y^9)

[jdemeyer]

Non-reproducible factoring bug reported upstream: ​http://www.singular.uni-kl.de:8002/trac/ticket/441

[jdemeyer]

Patch for the Sage library

[jdemeyer]

New patch fixes all doctest errors (except for the upstream factoring bug).