Opened 9 years ago

Closed 8 years ago

Last modified 8 years ago

#9958 closed enhancement (fixed)

Upgrade python to 2.7.x

Reported by: mhampton Owned by: tbd
Priority: major Milestone: sage-5.0
Component: packages: standard Keywords:
Cc: jhpalmieri, leif, jason, kcrisman, kini Merged in: sage-5.0.beta0
Authors: François Bissey, Steven Trogdon, Jeroen Demeyer Reviewers: John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé, François Bissey
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #5852, #11986, #12085, #12096, #12124, #11915 Stopgaps:

Description (last modified by jdemeyer)

From the release notes:

Python 2.7.2 was released on June 11th, 2011.

The Python 2.7 series is scheduled to be the last major version in the 2.x series before 2.x moves into an extended maintenance period. The 2.7 series contains many of the features that were first released in Python 3.1. Improvements in this release include:

  • An ordered dictionary type
  • New unittest features including test skipping, new assert methods, and test discovery
  • A much faster io module
  • Automatic numbering of fields in the str.format() method
  • Float repr improvements backported from 3.x
  • Tile support for Tkinter
  • A backport of the memoryview object from 3.x
  • Set literals
  • Set and dictionary comprehensions
  • Dictionary views
  • New syntax for nested with statements
  • The sysconfig module

Fixes #1159

New spkg: ​http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p1.spkg (diffs for reviewing (without deleted files): python-2.7.2.p0.diff​ and python-2.7.2.p0-p1.diff​)

Apply:

Attachments (46)

trac_9958-reenable-sage-deprecationwarnings.patch​ (635 bytes) - added by fbissey 8 years ago.
small patch to sage-doctest in sage_script to reenable sage's deprecation warnings. Idea by my friend Steve Trogdon.
trac_9958-fix_cmp.patch​ (626 bytes) - added by fbissey 8 years ago.
this form of comparison is removed in python 2.6.5 and later, refreshed using --git.
trac_9958-fixing_numericalnoise-part1.patch​ (34.5 KB) - added by fbissey 8 years ago.
fix numerical noise part 1
trac_9958-fixing_numericalnoise-part3.patch​ (8.2 KB) - added by fbissey 8 years ago.
fix numerical noise part 3
trac_9958-fixing_numericalnoise-part4.patch​ (3.5 KB) - added by fbissey 8 years ago.
fix numerical noise part 4: the forgotten bits
trac_9958-fixing_colorspy.patch​ (13.6 KB) - added by fbissey 8 years ago.
49 failures in colors.py, numerical noise except one.
trac_9958-fix-list_index.patch​ (1.4 KB) - added by fbissey 8 years ago.
fix the element reported by list index
trac_9958-mixedfix.patch​ (2.1 KB) - added by fbissey 8 years ago.
odds and ends missed in the other patches
trac_9958-build_module_listpy.patch​ (19.8 KB) - added by fbissey 8 years ago.
Not strictly necessary but this file needed a scrub. (refreshed for syntax errors)
trac_9958-build-sage_clib.patch​ (2.0 KB) - added by fbissey 8 years ago.
let's not forget the hardcoded version of python in SConstruct for sage_clib
trac_9958-build_setuppy.patch​ (3.8 KB) - added by fbissey 8 years ago.
change needed in setup.py made it python version smart. I also scrubbed the debian stuff
trac_9958-build_misc_cythonpy.patch​ (2.4 KB) - added by fbissey 8 years ago.
Some minor changes. The biggest change is to the doctest which explicitly refereed to python2.6 also we have extra include folders in a doctest apparently. [updated 20110525]
trac_9958-fix-misc_cythonpy.patch​ (1.2 KB) - added by fbissey 8 years ago.
fix doctest in sage/misc/cython.py, needs to be applied after #11376.
trac_9958-fixing_numericalnoise-part2.patch​ (26.7 KB) - added by fbissey 8 years ago.
fix numerical noise part 2 (refreshed against 4.7.1.alpha3 -20110625)
trac_9958-e_one_star.patch​ (985 bytes) - added by fbissey 8 years ago.
fix numerical noise in e_one_star.py
trac_9958-finite_crystals.patch​ (745 bytes) - added by fbissey 8 years ago.
fix error message in finite_crystals.py
trac_9958-symbolic_callable.patch​ (1.3 KB) - added by fbissey 8 years ago.
fix error message in symbolics/callable.py
Doctest_failures_Sage-4.7.2.alpha2_Linux_x86_SSE3-individual_tests_rerun.log​ (11.5 KB) - added by leif 8 years ago.
Pentium4 Prescott (SSE3 / PNI, 32-bit), GCC 4.5.1, native code, -mfpmath=sse, Ubuntu 9.04 x86
Doctest_failures_Sage-4.7.2.alpha2_Linux_x86_SSE3-individual_tests_rerun-segfaults-verbose.log​ (74.9 KB) - added by leif 8 years ago.
Pentium4 Prescott (SSE3 / PNI, 32-bit), GCC 4.5.1, native code, -mfpmath=sse, Ubuntu 9.04 x86 (The three segfaulting files tested in verbose mode.)
trac_9958-fixing_numericalnoise-part1_p1.patch​ (33.4 KB) - added by fbissey 8 years ago.
fixing numerical noise part 1 - updated 20110830
trac_9958-32_64bit_messages.patch​ (1.8 KB) - added by fbissey 8 years ago.
fix some 32/64 bits messages
trac_9958-fix-real_mpfr.patch​ (1.5 KB) - added by fbissey 8 years ago.
all fixes to real_mpfr.pyx split from everything else
trac_9958-mixedfix_p1.patch​ (1.2 KB) - added by fbissey 8 years ago.
split real_mpfr.pyx from the mixed fixes patch
trac_9958-fix_transcendental.patch​ (1.0 KB) - added by fbissey 8 years ago.
fixing numerical noise in transcendental.py- split from numerical noise pt1 - updated 2011092
trac_9958_junk_valueerror.patch​ (695 bytes) - added by fbissey 8 years ago.
Fix doctest in matrix_double_dense.pyx about junk
trac_9958_enumerate64bit.patch​ (806 bytes) - added by fbissey 8 years ago.
fix the doctest on enumeration order for 64bit system
trac_9958-fix-pureAssertError.patch​ (1.9 KB) - added by fbissey 8 years ago.
fix pure assertErrors failures - updated for python-2.7.2
trac_9958-sage_unittest.patch​ (2.3 KB) - added by fbissey 8 years ago.
fix sage_unittest.py failures - updated for python-2.7.2
trac_9958-fixing_numericalnoise-part1_p2.patch​ (31.6 KB) - added by fbissey 8 years ago.
fixing numerical noise part 1 - updated for 4.8.alpha3
trac_9958-gsl_integration.patch​ (1.8 KB) - added by fbissey 8 years ago.
fix numerical noise in gsl/integration.pyx
trac_9958-matrix_mod2e_dense.patch​ (1004 bytes) - added by fbissey 8 years ago.
fix numerical noise introduced by m4rie
python-2.6.4.p12-2.7.2.p0.diff​ (1.6 MB) - added by jdemeyer 8 years ago.
Diff for the Python spkg, for review only
trac_9958-suffix_trees-variations.patch​ (1.1 KB) - added by fbissey 8 years ago.
test against more valid dictionaries in suffix_trees
trac_9958-fix_randstate.pyx​ (5.9 KB) - added by fbissey 8 years ago.
fix all numerical noise in randstate
trac_9958-fixing_numericalnoise-part1_p3.patch​ (25.1 KB) - added by fbissey 8 years ago.
fixing numerical noise part 1 - updated 20111211: separate randstate and eis_modn_series
trac_9958-nfactor_enumerable_word-randomness.patch​ (1.4 KB) - added by fbissey 8 years ago.
take into account the randomness of it.next sequence of call - put random in the right place this time.
trac_9958-fix_randstate.patch​ (6.4 KB) - added by fbissey 8 years ago.
fix all numerical noise in randstate
trac_9958-suffix_trees-variations-sl.patch​ (1.1 KB) - added by slabbe 8 years ago.
Fix machine dependant doctest in suffix tree
trac_9958-fixing_numericalnoise-part1_p4.patch​ (24.8 KB) - added by fbissey 8 years ago.
fixing numerical noise part 1 - updated 20111214 for 4.8.alpha4
trac_9958_fix_rpy.patch​ (605 bytes) - added by fbissey 8 years ago.
numerical noise in interface/r.py updated for R-2.14.0
python-2.7.2.p0.diff​ (17.4 KB) - added by jdemeyer 8 years ago.
Diff for the Python spkg (without deleted files), for review only
python-2.7.2.p0-p1.diff​ (4.6 KB) - added by jdemeyer 8 years ago.
Diff for the Python spkg 2.7.2.p0 -> 2.7.2.p1, for review only
9958_float_printing.patch​ (85.9 KB) - added by jdemeyer 8 years ago.
9958_exceptions.patch​ (10.3 KB) - added by jdemeyer 8 years ago.
9958_misc.patch​ (1.6 KB) - added by jdemeyer 8 years ago.
9958_combinat.patch​ (1.9 KB) - added by jdemeyer 8 years ago.

Change History (334)

comment:1 follow-up: Changed 9 years ago by mhampton

First attempt is at: ​http://sage.math.washington.edu/home/palmieri/SPKG/python-2.7.p0.spkg

John (jhpalmieri) reports that this builds but Sage does not on sage.math, and "packages for twisted, zodb, pygments, and numpy don't build correctly."

Apparently the fix for ​http://bugs.python.org/issue7491 causes some of these problems.

On the numpy/scipy lists Ralf Gommers says "Numpy 1.5 should work with Python 2.7 and 3.1 and not be too far off. In August hopefully".

So it looks like ​http://trac.sagemath.org/sage_trac/ticket/9808 should fix the numpy issues.

comment:2 Changed 9 years ago by jhpalmieri

  • Cc jhpalmieri added
  • Component changed from PLEASE CHANGE to packages

comment:3 Changed 9 years ago by leif

  • Cc leif added

comment:4 Changed 9 years ago by leif

  • Type changed from defect to enhancement

comment:5 in reply to: ↑ 1 Changed 9 years ago by fbissey

Replying to mhampton:

First attempt is at: ​http://sage.math.washington.edu/home/palmieri/SPKG/python-2.7.p0.spkg

John (jhpalmieri) reports that this builds but Sage does not on sage.math, and "packages for twisted, zodb, pygments, and numpy don't build correctly."

Apparently the fix for ​http://bugs.python.org/issue7491 causes some of these problems.

This fix has been included in python-2.6.5 onwards. We are using 2.6.5 which includes this fix and everything builds (haven't tried 2.7 yet). The only problem we have is with ​http://github.com/cschwan/sage-on-gentoo/issues#issue/1 so I think it is unfair to single out this issue as the source of problems.

comment:6 Changed 9 years ago by novoselt

Is there still any interest in this upgrade? I wanted to use ordered dictionaries, which require 2.7. Unfortunately, active participation in this ticket is beyond my current capabilities...

comment:7 Changed 9 years ago by fbissey

It will happen at least in sage-on-gentoo as we are following the system python. I have a couple of things in our tree to fix problems with python-2.6.5 and 2.6.6 we are looking into 2.7 now. I should post any patch to have it working here.

We are not talking about using the new capabilities just porting, I imagine my counterpart in Mandriva does the same.

comment:8 Changed 9 years ago by fbissey

  • Status changed from new to needs_work

I just attached a patch that is needed for python 2.6.5 and later.

comment:9 Changed 9 years ago by jason

  • Cc jason added

comment:10 Changed 9 years ago by fbissey

Build sage-4.6.2 (and dependency) against python-2.7.1 on OS X. I get a lot of the following 

Expected nothing
Got:
    Failure in _test_pickling:
    Traceback (most recent call last):
      File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/site-packages/sage/misc/sage_unittest.py", line 275, in run
        test_method(tester = tester)
      File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/site-packages/sage/misc/sage_unittest.py", line 498, in _test_pickling
        tester.assertEqual(loads(dumps(self._instance)), self._instance)
      File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/unittest/case.py", line 493, in assertEqual
        assertion_func = self._getAssertEqualityFunc(first, second)
      File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/unittest/case.py", line 476, in _getAssertEqualityFunc
        asserter = self._type_equality_funcs.get(type(first))
    AttributeError: 'InstanceTester' object has no attribute '_type_equality_funcs'
    ------------------------------------------------------------

I patched python with the cpickle patch. Any ideas?

comment:11 Changed 9 years ago by fbissey

A lot of "doctest... DeprecationWarning?: ..." lines that were expected are just gone. Which fails the test.

comment:12 Changed 9 years ago by fbissey

Lots of numerical and a little bit of formatting noise.

comment:13 Changed 8 years ago by fbissey

Ok so now I understand the differences between unittest and unittest2 which is shipped with python-2.7. This will require a massive number of non-backward compatible changes. I am starting to experiment with a few sage components but that promise to be long and boring.

comment:14 Changed 8 years ago by kcrisman

  • Cc kcrisman added

comment:15 Changed 8 years ago by fbissey

I have attached a log of sage -testall. This is a sage-on-gentoo install a few tests are expected to fail ​https://github.com/cschwan/sage-on-gentoo/wiki/Known-test-failures but it should give you an idea of the problems we face.

comment:16 Changed 8 years ago by fbissey

  • Description modified (diff)

comment:17 Changed 8 years ago by fbissey

I attached a log of test failures with 4.7.alpha4 (+ #7377). It is mostly number of decimals and messages. I added PYTHONWARNINGS=default to sage-env so I collected extra messages. The most important one being the deprecations of "sets".

There are 3 tests killed reason currently unknown. And a few that may need special attention.

comment:18 Changed 8 years ago by fbissey

A little bit more details, these are curious: 

sage -t -long  -force_lib devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx", line 2297:
    sage: hash(R(-1)) 
Expected:
    95367431640624
Got:
    1977800240

sage -t -long  -force_lib devel/sage-main/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/suffix_trees.py", line 1345:
    sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
    True
Got:
    False
**********************************************************************
sage -t -long  -force_lib devel/sage-main/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/suffix_trees.py", line 1345:
    sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
    True
Got:
    False
**********************************************************************

This one doesn't worry me as much but should be looked at 

sage -t -long  -force_lib devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 22:
    sage: it.next()
Expected:
    word: 5645
Got:
    word: 4564
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 24:
    sage: it.next()
Expected:
    word: 4564
Got:
    word: 5645
**********************************************************************

sage -t -long  -force_lib devel/sage-main/sage/structure/parent.pyx
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/structure/parent.pyx", line 634:
    sage: CCls()._test_eq()
Expected:
    Traceback (most recent call last):
    ...
    AssertionError: <class '__main__.CCls'> == None
Got nothing

The following were killed: 

sage -t -long  -force_lib devel/sage-main/sage/rings/homset.py # Killed/crashed
sage -t -long  -force_lib devel/sage-main/sage/schemes/generic/scheme.py # Killed/crashed

In my original run 

sage -t -long -force_lib "devel/sage-main/sage/rings/morphism.pyx"

also got killed but not in a subsequent run after I adopted a small change in sage-doctest to get rid of PYTHONWARNINGS=default in sage-env.

Example of killed test: 

sage -t -long -verbose -force_lib "devel/sage-main/sage/rings/homset.py"
...

Trying:
    phi = S.hom([b,a])###line 141:_sage_    >>> phi = S.hom([b,a])
Expecting nothing
ok
Trying:
    phi == loads(dumps(phi))###line 142:_sage_    >>> phi == loads(dumps(phi))
Expecting:
    True
/usr/lib64/libcsage.so(print_backtrace+0x24)[0x7f83de940534]
/usr/lib64/libcsage.so(sigdie+0x1d)[0x7f83de9405cd]
/usr/lib64/libcsage.so(sage_signal_handler+0x131)[0x7f83de940741]
/lib64/libpthread.so.0(+0xfae0)[0x7f83e2385ae0]
/usr/lib64/libsingular.so.3(_Z9id_DeletePP10sip_sidealP9sip_sring+0x53)[0x7f83c5c48383]
/usr/lib64/python2.7/site-packages/sage/libs/singular/groebner_strategy.so(+0x388d)[0x7f83ad27488d]
/usr/lib64/libpython2.7.so.1.0(PyDict_Clear+0xfc)[0x7f83e261597c]
/usr/lib64/libpython2.7.so.1.0(+0x82a09)[0x7f83e2615a09]
/usr/lib64/libpython2.7.so.1.0(+0x112e7e)[0x7f83e26a5e7e]
/usr/lib64/libpython2.7.so.1.0(_PyObject_GC_Malloc+0x11c)[0x7f83e26a684c]
/usr/lib64/libpython2.7.so.1.0(_PyObject_GC_New+0xd)[0x7f83e26a685d]
/usr/lib64/libpython2.7.so.1.0(+0x7fe11)[0x7f83e2612e11]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/python2.7/site-packages/sage/rings/polynomial/polydict.so(+0x168c3)[0x7f83c8f378c3]
/usr/lib64/libpython2.7.so.1.0(+0xa0468)[0x7f83e2633468]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/python2.7/site-packages/sage/rings/polynomial/polydict.so(+0xa451)[0x7f83c8f2b451]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47)[0x7f83e2670ca7]
/usr/lib64/python2.7/lib-dynload/cPickle.so(+0x5f06)[0x7f83dcb2df06]
/usr/lib64/python2.7/lib-dynload/cPickle.so(+0xaf47)[0x7f83dcb32f47]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/python2.7/site-packages/sage/structure/sage_object.so(+0x152fc)[0x7f83dc7012fc]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x56bc)[0x7f83e267692c]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f83e26782f2]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5a78)[0x7f83e2676ce8]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(+0x71282)[0x7f83e2604282]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(+0x71282)[0x7f83e2604282]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(+0x7138b)[0x7f83e260438b]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f83e26782f2]
/usr/lib64/libpython2.7.so.1.0(+0xff25c)[0x7f83e269225c]
/usr/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x90)[0x7f83e2693090]
/usr/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0x1ff)[0x7f83e2693c6f]
/usr/lib64/libpython2.7.so.1.0(Py_Main+0xb53)[0x7f83e26a4fc3]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f83e201cb6d]
/usr/bin/python2.7[0x4008a9]

------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred in Sage.
This probably occurred because a *compiled* component of Sage has a bug
in it and is not properly wrapped with sig_on(), sig_off(). You might
want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate.
------------------------------------------------------------------------
The doctested process was killed by signal 11
         [2.5 s]

Changed 8 years ago by fbissey

small patch to sage-doctest in sage_script to reenable sage's deprecation warnings. Idea by my friend Steve Trogdon.

Changed 8 years ago by fbissey

this form of comparison is removed in python 2.6.5 and later, refreshed using --git.

comment:19 Changed 8 years ago by fbissey

It looks like the crash problems originates somewhere in libsingular 

sage: f = Zmod(6).cover()
sage: f.kernel()
Principal ideal (6) of Integer Ring
sage: R.<x,y> = PolynomialRing(QQ, 2)
sage: S.<a,b> = R.quo(x^2 + y^2)
sage: phi = S.cover()
sage: phi == loads(dumps(phi))
True
sage: phi == R.quo(x^2 + y^3).cover()

Program received signal SIGSEGV, Segmentation fault.
id_Delete (h=0x50687d8, r=0x0) at ideals.cc:127
127     ideals.cc: No such file or directory.
        in ideals.cc
(gdb) bt
#0  id_Delete (h=0x50687d8, r=0x0) at ideals.cc:127
#1  0x00007fffb9670b49 in __pyx_pf_4sage_4libs_8singular_17groebner_strategy_16GroebnerStrategy_1__dealloc__ (
    o=<value optimized out>) at sage/libs/singular/groebner_strategy.cpp:2570
#2  __pyx_tp_dealloc_4sage_4libs_8singular_17groebner_strategy_GroebnerStrategy (o=<value optimized out>)
    at sage/libs/singular/groebner_strategy.cpp:3190
#3  0x00007ffff7aa98c7 in PyDict_Clear (op=<value optimized out>) at Objects/dictobject.c:891
#4  0x00007ffff7aa990f in dict_tp_clear (op=<value optimized out>) at Objects/dictobject.c:2088
#5  0x00007ffff7b32f27 in delete_garbage (generation=0) at Modules/gcmodule.c:769
#6  collect (generation=0) at Modules/gcmodule.c:930
#7  0x00007ffff7b33616 in collect_generations (basicsize=<value optimized out>) at Modules/gcmodule.c:996
#8  _PyObject_GC_Malloc (basicsize=<value optimized out>) at Modules/gcmodule.c:1457
#9  0x00007ffff7ac4a96 in PyType_GenericAlloc (type=0x7ffff7d9ad40, nitems=0) at Objects/typeobject.c:744
#10 0x00007ffff7a8d256 in BaseException_new (type=<value optimized out>, args=<value optimized out>, kwds=<value optimized out>)
    at Objects/exceptions.c:34
#11 0x00007ffff7ac57b5 in type_call (type=0x7ffff7d9ad40, args=0x4ae46d0, kwds=0x0) at Objects/typeobject.c:712
#12 0x00007ffff7a77539 in PyObject_Call (func=0x7ffff7d9ad40, arg=0x4ae46d0, kw=0x0) at Objects/abstract.c:2529
#13 0x00007ffff7b00b45 in PyEval_CallObjectWithKeywords (func=0x7ffff7d9ad40, arg=0x4ae46d0, kw=0x0) at Python/ceval.c:3881
#14 0x00007ffff7b10443 in PyErr_NormalizeException (exc=0x7fffffffa778, val=0x7fffffffa770, tb=0x7fffffffa768)
    at Python/errors.c:190
#15 0x00007fffe433021c in __Pyx_GetException (type=0x7fffffffa7c0, value=0x7fffffffa7b8, tb=0x7fffffffa7c8)
    at sage/structure/parent.c:20909
#16 0x00007fffe433e4d1 in __pyx_pf_4sage_9structure_6parent_6Parent_2element_class (__pyx_v_self=0x4aea500, 
    unused=<value optimized out>) at sage/structure/parent.c:4251
#17 0x00007ffff7aac1e2 in PyCFunction_Call (func=0x504ebd8, arg=0x7ffff7f81050, kw=<value optimized out>)
    at Objects/methodobject.c:90
#18 0x00007ffff7a77539 in PyObject_Call (func=0x504ebd8, arg=0x7ffff7f81050, kw=0x0) at Objects/abstract.c:2529
#19 0x00007ffff7b00b45 in PyEval_CallObjectWithKeywords (func=0x504ebd8, arg=0x7ffff7f81050, kw=0x0) at Python/ceval.c:3881
#20 0x00007ffff7a8b51f in methoddescr_call (descr=<value optimized out>, args=0x7ffff7f81050, kwds=0x0)
    at Objects/descrobject.c:246
#21 0x00007ffff7a77539 in PyObject_Call (func=0x138e950, arg=0x7ffff7e9c050, kw=0x0) at Objects/abstract.c:2529
#22 0x00007ffff7b05efe in do_call (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4230
#23 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4035
#24 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665
#25 0x00007ffff7b07b65 in PyEval_EvalCodeEx (co=0x1159e30, globals=<value optimized out>, locals=<value optimized out>, 
    args=<value optimized out>, argcount=3, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3252
#26 0x00007ffff7a99a97 in function_call (func=0x115d5f0, arg=0xfadeb0, kw=0x0) at Objects/funcobject.c:526
#27 0x00007ffff7a77539 in PyObject_Call (func=0x115d5f0, arg=0xfadeb0, kw=0x0) at Objects/abstract.c:2529
#28 0x00007ffff7a77cca in PyObject_CallFunctionObjArgs (callable=0x115d5f0) at Objects/abstract.c:2760
#29 0x00007ffff7ac8a8e in slot_tp_descr_get (self=0x13179d0, obj=0x4aea500, type=0x19e91f0) at Objects/typeobject.c:5621
#30 0x00007ffff7aae433 in _PyObject_GenericGetAttrWithDict (obj=0x4aea500, name=0x13c5a78, dict=0x5068fb0) at Objects/object.c:1432
#31 0x00007ffff7aae4d1 in PyObject_GenericGetAttr (obj=<value optimized out>, name=<value optimized out>) at Objects/object.c:1454
#32 0x00007fffe434ed62 in __pyx_tp_getattro_4sage_9structure_6parent_Parent (o=0x4aea500, n=0x13c5a78)
    at sage/structure/parent.c:18702
#33 0x00007ffff7aad8f4 in PyObject_GetAttr (v=0x4aea500, name=<value optimized out>) at Objects/object.c:1189
#34 0x00007ffff7afdbab in builtin_hasattr (self=<value optimized out>, args=<value optimized out>) at Python/bltinmodule.c:885
#35 0x00007ffff7aac1a0 in PyCFunction_Call (func=0x7ffff7fae3b0, arg=0x503d680, kw=<value optimized out>)
    at Objects/methodobject.c:81
#36 0x00007ffff7b05ba3 in call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4012
#37 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665
#38 0x00007ffff7b05cfa in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4098
#39 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4033
#40 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665
#41 0x00007ffff7b07b65 in PyEval_EvalCodeEx (co=0x1159e30, globals=<value optimized out>, locals=<value optimized out>, 
    args=<value optimized out>, argcount=3, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3252
#42 0x00007ffff7a99a97 in function_call (func=0x115d5f0, arg=0xfadbe0, kw=0x0) at Objects/funcobject.c:526

comment:20 Changed 8 years ago by fbissey

Same two doctests killed after upgrade to alpha5.

comment:21 Changed 8 years ago by fbissey

Over-eager garbage collection in python-2.7.1! Preceding the sequence by 

sage: import gc
sage: gc.disable()

Makes everything go smoothly.

comment:22 Changed 8 years ago by fbissey

  • Description modified (diff)

comment:23 follow-up: Changed 8 years ago by nthiery

trac_9958-fix_cmp.patch is perfectly reasonable, since this is the idiom used everywhere else in similar situations. I don't have enough background to judge on the warning patch.

comment:24 Changed 8 years ago by fbissey

Put an updated list of tests failing with 4.7.alpha5 and the latest patches from Nicolas M. Thiery. I think the next area do need some work is the problem of garbage collection in polynomial rings that leads to "random" failures in libsingular. See: ​http://groups.google.com/group/sage-devel/browse_thread/thread/8c165c887d6b9e54

comment:25 Changed 8 years ago by fbissey

  • Description modified (diff)

Split the deprecation issue in its own ticket in #11244 - this is a different patch.

comment:26 in reply to: ↑ 23 Changed 8 years ago by fbissey

Replying to nthiery:

trac_9958-fix_cmp.patch is perfectly reasonable, since this is the idiom used everywhere else in similar situations. I don't have enough background to judge on the warning patch.

I will put the cmp patch in its own ticket would you review it?

Changed 8 years ago by fbissey

fix numerical noise part 1

Changed 8 years ago by fbissey

fix numerical noise part 3

Changed 8 years ago by fbissey

fix numerical noise part 4: the forgotten bits

Changed 8 years ago by fbissey

49 failures in colors.py, numerical noise except one.

Changed 8 years ago by fbissey

fix the element reported by list index

Changed 8 years ago by fbissey

odds and ends missed in the other patches

comment:27 Changed 8 years ago by fbissey

I have attached some patches to fix all the noise for numerics and formatting changes. What's left should be looked at before patching.

There are several failures because hashes have changed but I cannot say it is allright to just change the doctest: 

sage -t -long -force_lib "devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx", line 2339:
    sage: hash(R(-1)) 
Expected:
    95367431640624
Got:
    1977800240
**********************************************************************

sage -t -long -force_lib "devel/sage-main/sage/rings/integer.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 2995:
    sage: n = -920390823904823094890238490238484; n.__hash__()                                                                      
Expected:                                                                                                                           
    6874330978542788722                                                                                                             
Got:                                                                                                                                
    -2623069716                                                                                                                     
**********************************************************************                                                              
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3010:                                                           
    sage: hash(n)                                                                                                                   
Expected:                                                                                                                           
    -9223372036854767616                                                                                                            
Got:                                                                                                                                
    8192                                                                                                                            
**********************************************************************                                                              
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3013:                                                           
    sage: hash(n) == hash(int(n))                                                                                                   
Expected:                                                                                                                           
    True
Got:
    False
**********************************************************************

That hash(n) == hash(int(n)) is now false should be looked into carefully. 

sage -t -long -force_lib "devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 20:
    sage: it.next()
Expected:
    word: 6456
Got:
    word: 5645
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 22:
    sage: it.next()
Expected:
    word: 5645
Got:
    word: 6456
**********************************************************************

I think that one is OK, just a change of order but a second opinion would be nice.

Finally there are still three tests killed in libsingular's id_delete because a ring has been garbage collected even so there are some precautions in place for it: ​http://groups.google.com/group/sage-devel/browse_thread/thread/8c165c887d6b9e54

Next we need an updated python spkg so that more people can play and even try the patchbot on it.

comment:28 Changed 8 years ago by fbissey

  • Description modified (diff)

The replacement of type.cmp by cmp in strata.py is now in its own ticket #11264 as it can be applied independently of a python upgrade.

comment:29 Changed 8 years ago by fbissey

I forgot because I build on sage-on-gentoo but I'll post some patch to build against python-2.7 a little bit later.

comment:30 Changed 8 years ago by fbissey

Unless someone steps in I'll make a new python-2.7.1 spkg by the end of the week. I'll have to figure out if the latest change going in for sage-4.7 will be needed for that one.

comment:31 Changed 8 years ago by fbissey

I won't be able to do that this week.

comment:32 Changed 8 years ago by fbissey

  • Description modified (diff)

Our lead sage-on-gentoo debugger Martin tracked down the segfault problem to cython, this is now #11339

comment:33 Changed 8 years ago by kcrisman

I'd like to try this but don't have time to figure out exactly what patches are needed in what order - can someone update the description with this? Also, is it the same spkg that Marshall originally posted that one should try?

comment:34 follow-up: Changed 8 years ago by kcrisman

Also, is it possible to test this by ./sage -f something, or would one have to take a fresh tarball and replace the Python spkg?

comment:35 in reply to: ↑ 34 Changed 8 years ago by fbissey

Replying to kcrisman:

Also, is it possible to test this by ./sage -f something, or would one have to take a fresh tarball and replace the Python spkg?

Hi Karl,

I am mainly developing from sage-on-gentoo at the moment. I need to make a current python-2.7.1 spkg to match. I will update the description and give the patch order when there is a spkg (feel free to make one). The patch have initially been made against 4.7.1.alpha0 and the current version works against 4.7.1.alpha1. You would get failures against a 4.7 because #7377 finally landed.

I think the best course of action will be to start from scratch, it may even be useful to host an alternative tarball somewhere because the change in python version will be extremely messy. I don't know if we can do it without breaking sage -upgrade or being overly complicated.

In the meantime you could have a look at #11244 (which I need to update for 4.7.1.alpha1 because of #10334).

Francois

comment:36 Changed 8 years ago by fbissey

OK if you want to look at #11244 it can be tested against either the latest 4.7 or 4.7.1.alpha1 now.

comment:37 follow-up: Changed 8 years ago by kcrisman

I see. Well, I just want to make sure that it all works fine on older Macs, of course. I definitely don't have time to make an spkg now, though I do have time to do ./sage -f and slap a few patches on it :) Just let me know when you get around to it; I don't think there is a huge rush.

I should probably test 4.7.1.alpha1 on my box. I'm amazed at Jeroen's stamina on this, especially with several different versions simultaneously existing.

comment:38 in reply to: ↑ 37 Changed 8 years ago by fbissey

Replying to kcrisman:

I see. Well, I just want to make sure that it all works fine on older Macs, of course. I definitely don't have time to make an spkg now, though I do have time to do ./sage -f and slap a few patches on it :) Just let me know when you get around to it; I don't think there is a huge rush.

I should probably test 4.7.1.alpha1 on my box. I'm amazed at Jeroen's stamina on this, especially with several different versions simultaneously existing.

That's why I asked about #11244 it is an issue I split from this ticket because it should apply on a sage running python-2.6 without side effects. So it could be in place when we come around to the upgrade. Any ideas on #11339 which was recently split from this ticket is also welcome.

comment:39 Changed 8 years ago by fbissey

  • Description modified (diff)

My friend Steven Trogdon made some quick and dirty spkgs. His early tests indicate that setuptools needs to be updated. This is now #11363 it also revealed that I missed something in misc/cython.py watch out for an updated patch sometimes this weekend. There were also a few things that needed to be done in two other patches which I refreshed yesterday. I have a python-2.7.1 spkg ready to be uploaded probably sometimes today. If you want fun before that you can always use Steve's spkg (they are not in a reviewable state) at: ​http://www.d.umn.edu/~strogdon/sage/

comment:40 Changed 8 years ago by fbissey

  • Description modified (diff)

Didn't take long. I have been naughty and committed two changes to patches that needed touching after my first commit of python-2.7.1 so the log will show 3 entries for just the one SPKG.txt entry. They were minor changes. I dropped a number of patches because they were applied upstream. However in the case of two patches I dropped them because the upstream has changed in a way they don't apply anymore but I don't know if patching is still necessary for the underlying issues. If so new patches will need to be developed. The issues are

  • readline on itanium
  • multiprocessing on solaris

As promised I will give a patch order. The next to last one needs updating as stated earlier and the last one is really optional. The patches do not have to be taken n that order as they don't overlap.

comment:41 Changed 8 years ago by fbissey

I forgot apply the patches on 4.7.1.alpha1, there may be rejection on 4.7.rcX.

comment:42 Changed 8 years ago by fbissey

misc/cython.py patch updated for people who want to test it.

comment:43 Changed 8 years ago by fbissey

corrected a syntax error in the last patch.

Changed 8 years ago by fbissey

Not strictly necessary but this file needed a scrub. (refreshed for syntax errors)

comment:44 Changed 8 years ago by fbissey

Had to redo trac_9958-build_module_listpy.patch again as I messed up the syntax of one bit.

comment:45 follow-up: Changed 8 years ago by fbissey

My friend Steve Trogdon made a prepatched sage-4.7.1.alpha1 spkg (4.7.1.alpha1.p0 we named it) available to make testing easier, find it with hos own test.log here ​http://www.d.umn.edu/~strogdon/sage/ It has all the necessary patches from #11244 and this this ticket. We are a bit puzzled by the cause of failure in sage/misc/cython.py, white spaces?

comment:46 follow-up: Changed 8 years ago by kcrisman

I am getting the twisted install fail that jhpalmieri refers to. So my guess is that the Gentoo system has some updated things along those lines that vanilla 4.7.1.alpha1 does not. Is that possible?

comment:47 in reply to: ↑ 46 Changed 8 years ago by fbissey

Replying to kcrisman:

I am getting the twisted install fail that jhpalmieri refers to. So my guess is that the Gentoo system has some updated things along those lines that vanilla 4.7.1.alpha1 does not. Is that possible?

Well sage-on-gentoo uses a newer twisted. On the other I and Steve have build twisted in vanilla sage with python-2.7.1. Did you get the new setuptools spkg from #11363? It is necessary to build twisted with python-2.7.1.

comment:48 Changed 8 years ago by kcrisman

No, I didn't realize there was an spkg involved with that. I installed that first, and it seems to be working now.

comment:49 Changed 8 years ago by fbissey

  • Description modified (diff)

comment:50 in reply to: ↑ 45 Changed 8 years ago by kcrisman

Replying to fbissey:

My friend Steve Trogdon made a prepatched sage-4.7.1.alpha1 spkg (4.7.1.alpha1.p0 we named it) available to make testing easier, find it with hos own test.log here ​http://www.d.umn.edu/~strogdon/sage/ It has all the necessary patches from #11244 and this this ticket.

Is one of these patches needed before doing the Sage spkg itself? I get an immediate failure 

gcc -o src/convert.os -c -fPIC -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/python2.6 -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/NTL -Iinclude src/convert.c
In file included from include/memory.h:23,
                 from /Users/student/Desktop/sage-4.7.1.alpha1/local/include/pari/pari.h:42,
                 from include/convert.h:12,
                 from src/convert.c:14:
include/interrupt.h:32:20: error: Python.h: No such file or directory
scons: *** [src/convert.os] Error 1
*** TOUCHING ALL CYTHON (.pyx) FILES ***
gcc -o src/convert.os -c -fPIC -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/python2.6 -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/NTL -Iinclude src/convert.c
In file included from include/memory.h:23,
                 from /Users/student/Desktop/sage-4.7.1.alpha1/local/include/pari/pari.h:42,
                 from include/convert.h:12,
                 from src/convert.c:14:
include/interrupt.h:32:20: error: Python.h: No such file or directory
scons: *** [src/convert.os] Error 1

----------------------------------------------------------
sage: Building and installing modified Sage library files.


Installing c_lib
gcc -o src/convert.os -c -fPIC -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/python2.6 -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/NTL -Iinclude src/convert.c
In file included from include/memory.h:23,
                 from /Users/student/Desktop/sage-4.7.1.alpha1/local/include/pari/pari.h:42,
                 from include/convert.h:12,
                 from src/convert.c:14:
include/interrupt.h:32:20: error: Python.h: No such file or directory
scons: *** [src/convert.os] Error 1
ERROR: There was an error building c_lib.

which I assume is related to one of the patches involved here, though I couldn't say which one...

comment:51 Changed 8 years ago by fbissey

Did you rebuild cython after the python upgrade? In fact I have now added a list of package that needs to be rebuild after the python upgrade. Not all of them are necessary for building sage (scipy could be done after) but cython, numpy, matplotlib and pynac really need to be rebuilt.

Once you have done that apply all the patches with build in their names:

That should help sage build but will not correct many of the doctests. To fix most of the doctests apply the patches in #11244 as well as the one in the description of this bug. I have made a list now, have you seen it?

comment:52 follow-up: Changed 8 years ago by kcrisman

Well, I'm building from scratch, so that shouldn't affect it from that side.

Are you saying that before I build the Sage library, I have to apply these patches? That's a little tricky in a scratch build (for me, at least).

Yes, I saw the lists, but I thought they were only relevant for an upgrade/after actually building Sage. My apologies. I may not be able to finish testing this properly for a while now.

comment:53 in reply to: ↑ 52 Changed 8 years ago by fbissey

Replying to kcrisman:

Well, I'm building from scratch, so that shouldn't affect it from that side.

Are you saying that before I build the Sage library, I have to apply these patches? That's a little tricky in a scratch build (for me, at least).

Yes, I saw the lists, but I thought they were only relevant for an upgrade/after actually building Sage. My apologies. I may not be able to finish testing this properly for a while now.

That's all right there are still things to sort out. But you could use the sage spkg prepared by Steve with all patches included rather than patch yourself: ​http://www.d.umn.edu/~strogdon/sage/sage-4.7.1.alpha1.p0.spkg

And yes it is tricky from scratch unless you have a premade spkg.

comment:54 follow-up: Changed 8 years ago by jhpalmieri

I may have made mistakes, but I tried to do this, starting with sage-4.7.rc4:

  • install the new python spkg
  • install the new setuptools spkg (from #11363)
  • apply the patches from #11244 to the main Sage library
  • apply the patches from this ticket with "build" in their name -- some of these didn't apply cleanly, so I had to work around that.

I used that to build a new version of Sage (using "sage -sdist ..."). The tar file is here:

On sage.math, I've been able to untar this and build Sage, although tests haven't run yet. You can also try upgrading, although I haven't tested this at all: run 

$ sage -upgrade http://sage.math.washington.edu/home/palmieri/misc/python2.7/sage-4.7.rc4-python2.7/

comment:55 in reply to: ↑ 54 Changed 8 years ago by fbissey

Replying to jhpalmieri:

I may have made mistakes, but I tried to do this, starting with sage-4.7.rc4:

  • install the new python spkg
  • install the new setuptools spkg (from #11363)
  • apply the patches from #11244 to the main Sage library
  • apply the patches from this ticket with "build" in their name -- some of these didn't apply cleanly, so I had to work around that.

I used that to build a new version of Sage (using "sage -sdist ..."). The tar file is here:

On sage.math, I've been able to untar this and build Sage, although tests haven't run yet. You can also try upgrading, although I haven't tested this at all: run 

$ sage -upgrade http://sage.math.washington.edu/home/palmieri/misc/python2.7/sage-4.7.rc4-python2.7/

I have tried to keep abreast of everything so the patches are based on 4.7.1.alpha1 there are enough differences that a lot of patches for 4.7 would need rebasing.

So I decided to go straight for the bleeding edge. I am doing some more work on the "build" patches, I am trying to do something smarter with the python version number so some change could be applied independently of this ticket. The module_list,py patch definitely won't apply cleanly on 4.7.rc mainly because of #10334. What's more #11264 and #11236 which are needed here are merged in 4.7.1.alpha0.

That's all a bit of hassle to start from 4.7.1.alpha, but I would have to start again if I was starting from 4.7.rc.

Changed 8 years ago by fbissey

let's not forget the hardcoded version of python in SConstruct for sage_clib

comment:56 Changed 8 years ago by fbissey

I have updated trac_9958-build-sage_clib.patch it could now be splited in its own ticket. It enables sage-clib to be smarter about the python version used. It would figure it on its own. I am looking at doing the same for setup.py, anyone would review that in its own ticket? Once in, we could at least build sage against python2.6 or 2.7 without changes.

Changed 8 years ago by fbissey

change needed in setup.py made it python version smart. I also scrubbed the debian stuff

comment:57 Changed 8 years ago by fbissey

OK I also made setup.py python version smart any volunteer to review a separate ticket?

comment:58 Changed 8 years ago by kcrisman 

g++ -o libcsage.dylib -single_module -flat_namespace -undefined dynamic_lookup -dynamiclib src/convert.os src/interrupt.os src/memory.os src/mpn_pylong.os src/mpz_pylong.os src/mpz_longlong.os src/stdsage.os src/gmp_globals.os src/ZZ_pylong.os src/ntl_wrap.os -L/Users/student/Desktop/sage-4.7.1.alpha1/local/lib -L/Users/student/Desktop/sage-4.7.1.alpha1/local/lib/python/config -lntl -lpari -lgmp -lpython2.7
Traceback (most recent call last):
  File "setup.py", line 18, in <module>
    from module_list import ext_modules
  File "/Users/student/Desktop/sage-4.7.1.alpha1/devel/sage-main/module_list.py", line 1529, in <module>
    flint_depends],
TypeError: cannot concatenate 'str' and 'list' objects
sage: There was an error installing modified sage library code.


real    0m34.577s
user    0m19.519s
sys     0m6.901s
Error building new version of SAGE.
You might try typing 'sage -ba' or write to sage-support with as much information as possible.

real    2m42.066s
user    0m52.903s
sys     0m25.133s
sage: An error occurred while installing sage-4.7.1.alpha1.p0

Note this is the p0 package referred to above.

comment:59 follow-up: Changed 8 years ago by strogdon

Well, there was a problem with this but it was corrected. I've checked the present sage-4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage-4.7.1.alpha1.p0.spkg) should be 

b54da7bee966eaf15942964a05a8b6b1  sage-4.7.1.alpha1.p0.spkg

If the spkg you have is not this could you download again and see if that fixed things.

Changed 8 years ago by fbissey

Some minor changes. The biggest change is to the doctest which explicitly refereed to python2.6 also we have extra include folders in a doctest apparently. [updated 20110525]

comment:60 Changed 8 years ago by fbissey

  • Description modified (diff)

I am splitting the ticket again: in #11376 we make sage building system smart and work out the python version by itself. If we get this in the next alpha that will make testing easier as you won't need a patched sage spkg to build against python-2.7.1. One will still need to patch the doctests afterwards and any other issues but it will build.

In #11377 I put the clean up of module_list.py this is not necessary for python-2.7.1 per see but it would be a huge favour to me if it was going in (it makes work on sage-on-gentoo and sage-prefix much easier).

comment:61 in reply to: ↑ 59 ; follow-up: Changed 8 years ago by kcrisman

Replying to strogdon:

Well, there was a problem with this but it was corrected. I've checked the present sage-4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage-4.7.1.alpha1.p0.spkg) should be 

b54da7bee966eaf15942964a05a8b6b1  sage-4.7.1.alpha1.p0.spkg

If the spkg you have is not this could you download again and see if that fixed things.

It didn't seem to fix things (and I did check the md5 sum). I'm now starting a build from scratch, having replaced the python and sage spkgs.

By the way, now that 4.7.1.alpha1 is "official", one may have to upgrade the drop-in spkg.

comment:62 in reply to: ↑ 61 Changed 8 years ago by strogdon

Replying to kcrisman:

Replying to strogdon:

Well, there was a problem with this but it was corrected. I've checked the present sage-4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage-4.7.1.alpha1.p0.spkg) should be b54da7bee966eaf15942964a05a8b6b1 sage-4.7.1.alpha1.p0.spkg If the spkg you have is not this could you download again and see if that fixed things.

It didn't seem to fix things (and I did check the md5 sum). I'm now starting a build from scratch, having replaced the python and sage spkgs. By the way, now that 4.7.1.alpha1 is "official", one may have to upgrade the drop-in spkg.

OK, for those interested I have a new drop-in spkg based on the released 4.7.1.alpha1. This spkg includes the patches from this ticket as well as the patches from tickets #11244, #11373, #11376 and #11377 . I'm presently building Sage to make sure everything is as it should be. If successful, I'll post links to the drop-in spkg later today.

comment:63 Changed 8 years ago by fbissey

I was hoping that #11376 and #11363 would have made it in alpha2 which would have made things easier. But I think Jeroen has closed alpha2 and is starting on alpha3. We should try to have at least these two in 4.7.1 to make testing easier.

comment:64 Changed 8 years ago by strogdon

The following have been uploaded:

​http://www.d.umn.edu/~strogdon/sage/install.log-4.7.1.alpha1

​http://www.d.umn.edu/~strogdon/sage/test.log-4.7.1.alpha1

​http://www.d.umn.edu/~strogdon/sage/sage-4.7.1.alpha1.p0.spkg

​http://www.d.umn.edu/~strogdon/sage/sage-4.7.1.alpha1.p0.spkg.md5

for comparisons, perhaps just under the wire for alpha2. Note, the drop-in sage spkg has the same revision .p0 extension, so I've provided the md5 sum for the spkg.

comment:65 Changed 8 years ago by fbissey

  • Description modified (diff)

Changed the cython.py patch in the list to accommodate new changes made in #11376.

comment:66 follow-up: Changed 8 years ago by kcrisman

End of a looong traceback while building from scratch on PPC 10.4 with the spkgs dropped in (though not the absolute most recent alpha1.p0 spkg, but that hasn't even been gotten to yet, so it shouldn't matter). Just FYI in case that helps. 

  File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/genericpath.py", line 44, in isdir
    return stat.S_ISDIR(st.st_mode)
  File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/stat.py", line 41, in S_ISDIR
    return S_IFMT(mode) == S_IFDIR
RuntimeError: maximum recursion depth exceeded
Error installing Zope/Twisted interface.

real    0m16.055s
user    0m3.349s
sys     0m4.671s
sage: An error occurred while installing twisted-9.0.p2

comment:67 in reply to: ↑ 66 Changed 8 years ago by fbissey

Replying to kcrisman:

End of a looong traceback while building from scratch on PPC 10.4 with the spkgs dropped in (though not the absolute most recent alpha1.p0 spkg, but that hasn't even been gotten to yet, so it shouldn't matter). Just FYI in case that helps. 

  File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/genericpath.py", line 44, in isdir
    return stat.S_ISDIR(st.st_mode)
  File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/stat.py", line 41, in S_ISDIR
    return S_IFMT(mode) == S_IFDIR
RuntimeError: maximum recursion depth exceeded
Error installing Zope/Twisted interface.

real    0m16.055s
user    0m3.349s
sys     0m4.671s
sage: An error occurred while installing twisted-9.0.p2

Were you using the new setuptools from #11363? If so it may be worth trying a newer twisted from #8741.

comment:68 Changed 8 years ago by fbissey

  • Dependencies set to #11156 #11236 #11244 #11264 #11339 #11363 #11376

Changed 8 years ago by fbissey

fix doctest in sage/misc/cython.py, needs to be applied after #11376.

comment:69 Changed 8 years ago by fbissey

Updated trac_9958-fix-misc_cythonpy.patch to take into account changes in #11376. For some reason the numpy include line appears after the application of the other patches over there. Why it didn't appear before is a mistery.

comment:70 Changed 8 years ago by jhpalmieri

By the way, I don't know how feasible this is, but it would be very nice if the new python spkg passed self-tests: set SAGE_CHECK to "yes" and install the spkg. Previous versions of Python have failed the self-tests, but it's perhaps the only standard spkg for which this is true. With the spkg here, on an OS X box: 

344 tests OK.
4 tests failed:
    test__locale test_ctypes test_distutils test_locale
38 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb3
    test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dl test_epoll test_gdb test_gdbm test_gl test_imageop
    test_imgfile test_largefile test_linuxaudiodev test_macos
    test_macostools test_ossaudiodev test_scriptpackages test_smtpnet
    test_socketserver test_startfile test_sunaudiodev test_timeout
    test_tk test_ttk_guionly test_urllib2net test_urllibnet
    test_winreg test_winsound test_zipfile64
5 skips unexpected on darwin:
    test_aepack test_applesingle test_dl test_macos
    test_scriptpackages
make: *** [test] Error 1
An error occurred while testing Python

On sage.math: 

346 tests OK.
1 test failed:
    test_distutils
39 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_dl test_gdb test_gdbm test_gl test_imageop
    test_imgfile test_kqueue test_linuxaudiodev test_macos
    test_macostools test_ossaudiodev test_scriptpackages test_smtpnet
    test_socketserver test_startfile test_sunaudiodev test_timeout
    test_tk test_ttk_guionly test_urllib2net test_urllibnet
    test_winreg test_winsound test_zipfile64
7 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_tk
    test_ttk_guionly
make: *** [test] Error 1
An error occurred while testing Python

(I didn't upgrade the distutils package or anything else, just ran "sage -f python-2.7.1.spkg", so some of these may go away.)

comment:71 Changed 8 years ago by kini

  • Cc kini added

comment:72 Changed 8 years ago by fbissey

python testing is difficult. Even in gentoo the procedure is complicated. I'll have a look at it but I cannot make promise. There may be modules that we shouldn't build in the first place (nothing useful to sage, don't worry).

We have this for example in the ebuild: 

	if use berkdb; then
		ewarn "\"bsddb\" module is out-of-date and no longer maintained inside dev-lang/python. It has"
		ewarn "been additionally removed in Python 3. You should use external, still maintained \"bsddb3\""
		ewarn "module provided by dev-python/bsddb3 which supports both Python 2 and Python 3."
	fi

gdb, gdm, dbm and tk require extra things on the host system so they may very well fail without. Actually according to the ebuild distutils and gdm are known to fail.

comment:73 follow-up: Changed 8 years ago by leif

Just for the record:

​http://trac.sagemath.org/sage_trac/ticket/8664#comment:33 gives an example on how to automatically rebuild packages that depend on a newly added spkg (here: Python).

(Artificial version bumps aren't necessary, and simply touching spkg/installed/<spkg-that-others-depend-on> alone doesn't work.)

In general:

  1. Copy the new spkg(s) to $SAGE_ROOT/spkg/standard/.
  2. export SAGE_UPGRADING=yes
  3. (Optionally set MAKE and SAGE_PARALLEL_SPKG_BUILD, perhaps SAGE_CHECK.)
  4. Run make (or better make build) rather than doing ./sage -f ...
  5. Apply patches to the Sage library.
  6. Run ./sage -b (or ./sage -ba-force in case not all dependencies are covered by module_list.py, which in general I'd consider a bug, but this might be necessary upon a Python upgrade)
  7. Run tests or whatever.

Note that Sage switches to the "main" branch whenever the Sage spkg gets reinstalled.

Of course patches to the build system are usually a bit trickier, but as far as I can see Francois already sorted these out.

comment:74 Changed 8 years ago by fbissey

Thanks for the pointer, it will make testing easier for everyone. The bad news is that all the bits that makes testing easier won't be in 4.7.1. The good news is that everything is merged in 4.7.2.alpha0 at the moment. So once the next release cycle begins things will get easier. No patch will be needed to the build system. All that will remain will be doctest fix and issues like #11339.

I will update trac_9958-fixing_numericalnoise-part2.patch shortly as #11255 broke it. Which means an extra patch may appear later once I have put all the pieces back together.

Changed 8 years ago by fbissey

fix numerical noise part 2 (refreshed against 4.7.1.alpha3 -20110625)

comment:75 Changed 8 years ago by fbissey

OK. There are a few more patches needed for numerical and formatting noise in 4.7.2.alpha1 but now anyone should be able to upgrade python without a premade sage spkg. All fixes to be able to build sage against python-2.7 out of the box are in!

comment:76 in reply to: ↑ 73 ; follow-up: Changed 8 years ago by kcrisman

(Artificial version bumps aren't necessary, and simply touching spkg/installed/<spkg-that-others-depend-on> alone doesn't work.)

In general:

  1. Copy the new spkg(s) to $SAGE_ROOT/spkg/standard/.
  2. export SAGE_UPGRADING=yes

Huh, I didn't know about that, and it would be quite useful. I can't find it in either the installation guide or the developer guide, though - is this documented yet? Just curious.

Francois, I'll try to give this a test run on PPC OS X and/or Cygwin if you like.

comment:77 in reply to: ↑ 76 Changed 8 years ago by fbissey

Replying to kcrisman:

Francois, I'll try to give this a test run on PPC OS X and/or Cygwin if you like.

Be my guest, I thought that the fact things just became easier warranted an announcement.

comment:78 Changed 8 years ago by kini

Wonderful, this is great news! :) Thanks!

comment:79 Changed 8 years ago by kcrisman

So far Mac PPC is doing fine (I just swapped out the spkgs in a source tar). Numpy, mercurial, various other python-related spkgs apparently built ok, though it isn't at Scipy or Sage yet. I did NOT check with SAGE_CHECK=yes since I didn't want to have to watch the build and turn that off during the Python build... Anyway, looks good so far. If it finishes I'll apply all the patches :)

comment:80 Changed 8 years ago by jhpalmieri

I tried this (new python spkg plus all of the patches, then built from scratch) and I'm seeing a bunch of doctest failures. Are some of these due to #11339? 

The following tests failed:

        sage -t  -long -force_lib devel/sage/sage/categories/finite_crystals.py # 1 doctests failed
        sage -t  -long -force_lib devel/sage/sage/rings/morphism.pyx # 0 doctests failed
        sage -t  -long -force_lib devel/sage/sage/rings/integer.pyx # 3 doctests failed
        sage -t  -long -force_lib devel/sage/sage/rings/homset.py # 0 doctests failed
        sage -t  -long -force_lib devel/sage/sage/rings/padics/padic_capped_relative_element.pyx #  1 doctests failed
        sage -t  -long -force_lib devel/sage/sage/combinat/e_one_star.py # 1 doctests failed
        sage -t  -long -force_lib devel/sage/sage/combinat/words/nfactor_enumerable_word.py # 2 doctests failed
        sage -t  -long -force_lib devel/sage/sage/geometry/polyhedra.py # 1 doctests failed
        sage -t  -long -force_lib devel/sage/sage/schemes/generic/scheme.py # 0 doctests failed
        sage -t  -long -force_lib devel/sage/sage/symbolic/callable.py # 5 doctests failed

I think I did everything right. I put together a Sage distribution with the python spkg and the patches, based on 4.7.2.alpha1:

Upgrade path:

comment:81 Changed 8 years ago by fbissey

Any failures from #11339 gives you a killed doctest, so 

sage -t  -long -force_lib devel/sage/sage/rings/morphism.pyx
sage -t  -long -force_lib devel/sage/sage/rings/homset.py
sage -t  -long -force_lib devel/sage/sage/schemes/generic/scheme.py

could all be, a verbose run would help figure it out. By nature #11339 is somewhat random, not everyone see the same failures but all these are familiar.

Just checking the others with my current list.

  • sage/categories/finite_crystals.py is a new one introduced in 4.7.2 where the error message has changed slightly when using python-2.7.
  • sage/rings/integer.pyx is definitely python-2.7 related but not patched yet because I am not sure what should be done there.
  • sage/rings/padics/padic_capped_relative_element.pyx same thing
  • sage/combinat/e_one_star.py I had a patch but it became to fuzzy haven't done a new one.
  • sage/combinat/words/nfactor_enumerable_word.py python-2.7 two successive results are inverted, I want someone else to look at that one to make sure we can just do the obvious thing.
  • sage/symbolic/callable.py new in 4.7.2 change in the error message because we are running python-2.7
  • sage/geometry/polyhedra.py never seen that one before, can you re-run it and give more info on the failure?

As you can see a few more patches are needed and apart from #11339 which is a big show stopper because of the crashes at least two other tests point to some potential problems that should be looked at.

comment:82 follow-up: Changed 8 years ago by strogdon

Here are the failures I get on x86: 

sage -t -long  -force_lib devel/sage-main/sage/libs/cremona/newforms.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/misc/sageinspect.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/misc/randstate.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/rings/real_mpfr.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/stats/intlist.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/combinat/e_one_star.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/geometry/polyhedra.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed
sage -t -long  -force_lib devel/sage-main/sage/symbolic/callable.py # 5 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/functions/transcendental.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/categories/finite_crystals.py # 1 doctests failed

The failures newforms.pyx, real_mpfr.pyx and intlist.pyx are all of the form: 

File "/storage/sage/sage-4.7.2.alpha1/devel/sage-main/sage/libs/cremona/newforms.pyx", line 53:
    sage: ECModularSymbol(E)
Expected:
    Traceback (most recent call last):
    ...
    OverflowError: long int too large to convert to int
Got:
    Traceback (most recent call last):
      File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/storage/sage/sage-4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_1[11]>", line 1, in <module>
        ECModularSymbol(E)###line 53:
    sage: ECModularSymbol(E)
      File "newforms.pyx", line 69, in sage.libs.cremona.newforms.ECModularSymbol.__init__ (sage/libs/cremona/newforms.cpp:1820)
        a6 = new_bigint(int(E.a6()))
    OverflowError: Python int too large to convert to C long

These tests do not fail on amd64 because "long int too large to convert to int" is returned as the OverflowError? instead of "Python int too large to convert to C long".

comment:83 Changed 8 years ago by fbissey

I had forgotten about these. We need different test results for 32 and 64 bits for these.

comment:84 Changed 8 years ago by jhpalmieri

Here's the failure for sage/geometry/polyhedra.py: 

sage -t -long -force_lib "devel/sage/sage/geometry/polyhedra.py"
**********************************************************************
File "/mnt/usb1/scratch/palmieri/9958/sage-4.7.2.9958/devel/sage/sage/geometry/polyhedra.py", line 4948:
    sage: ppoints[0]
Expected:
    (-1.92296268638e-15, -1.92296268638e-15)
Got:
    (0.0, 0.0)
**********************************************************************

It looks like numerical noise, but is it an acceptable level of accuracy?

comment:85 Changed 8 years ago by fbissey

Now I remember that test, it is even in the log sage-4.7.alpha5-test.log.bz2 I produced a while ago. Strangely enough it doesn't happen anymore on my sage-on-gentoo install. I also have pari-2.5.0 and maxima-5.25.0 on this install, I suspect one of them may be responsible for the disparition of that failure

comment:86 Changed 8 years ago by fbissey

I am wrong! Not maxima or pari, there is a patch for it in trac_9958-fixing_numericalnoise-part1.patch​. So it looks like that bit may not be applied properly.

Changed 8 years ago by fbissey

fix numerical noise in e_one_star.py

Changed 8 years ago by fbissey

fix error message in finite_crystals.py

Changed 8 years ago by fbissey

fix error message in symbolics/callable.py

comment:87 Changed 8 years ago by fbissey

  • Description modified (diff)

Added three new patches. I will rework the patches that need adjusting between 32 and 64 bits next.

comment:88 in reply to: ↑ 82 ; follow-up: Changed 8 years ago by fbissey

Replying to strogdon:

Here are the failures I get on x86: 

sage -t -long  -force_lib devel/sage-main/sage/libs/cremona/newforms.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/misc/sageinspect.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/misc/randstate.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/rings/real_mpfr.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/stats/intlist.pyx # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/combinat/e_one_star.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/geometry/polyhedra.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed
sage -t -long  -force_lib devel/sage-main/sage/symbolic/callable.py # 5 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/functions/transcendental.py # 1 doctests failed
sage -t -long  -force_lib devel/sage-main/sage/categories/finite_crystals.py # 1 doctests failed

The failures newforms.pyx, real_mpfr.pyx and intlist.pyx are all of the form: 

File "/storage/sage/sage-4.7.2.alpha1/devel/sage-main/sage/libs/cremona/newforms.pyx", line 53:
    sage: ECModularSymbol(E)
Expected:
    Traceback (most recent call last):
    ...
    OverflowError: long int too large to convert to int
Got:
    Traceback (most recent call last):
      File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/storage/sage/sage-4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_1[11]>", line 1, in <module>
        ECModularSymbol(E)###line 53:
    sage: ECModularSymbol(E)
      File "newforms.pyx", line 69, in sage.libs.cremona.newforms.ECModularSymbol.__init__ (sage/libs/cremona/newforms.cpp:1820)
        a6 = new_bigint(int(E.a6()))
    OverflowError: Python int too large to convert to C long

These tests do not fail on amd64 because "long int too large to convert to int" is returned as the OverflowError? instead of "Python int too large to convert to C long".

Steve, what about 

sage -t -long  -force_lib devel/sage-main/sage/rings/polynomial/polynomial_rational_flint.pyx

comment:89 in reply to: ↑ 88 ; follow-up: Changed 8 years ago by strogdon

Replying to fbissey:

Replying to strogdon:

Steve, what about 

sage -t -long  -force_lib devel/sage-main/sage/rings/polynomial/polynomial_rational_flint.pyx

Here on x86 and amd64 the test passes. So, there seems to be some sort of inconsistency, doesn't it?

comment:90 in reply to: ↑ 89 Changed 8 years ago by fbissey

Replying to strogdon:

Replying to fbissey:

Replying to strogdon:

Steve, what about 

sage -t -long  -force_lib devel/sage-main/sage/rings/polynomial/polynomial_rational_flint.pyx

Here on x86 and amd64 the test passes. So, there seems to be some sort of inconsistency, doesn't it?

Yes! I'll fix it but it should be investigated. Gut feeling is on variation in cython definitions.

comment:91 Changed 8 years ago by leif

  • Description modified (diff)

comment:92 Changed 8 years ago by kcrisman

I've tried to remove the failures known above here. Note that I didn't get the last few patches in time, but I removed those failures as well, assuming they were the same. The timeouts/killed are probably unrelated - my computer is < 1 GHz, so I get a lot of timeouts and I forgot to set SAGE_TIMEOUT_LONG before I ran the tests - but at any rate I hope this will be useful. After the hurricane I'll start this up again and run the remaining tests. 

        sage -t -long "devel/sage/sage/crypto/mq/mpolynomialsystem.py" # Killed/crashed
        sage -t -long "devel/sage/sage/functions/transcendental.py"
        sage -t -long "devel/sage/sage/groups/generic.py" # Killed/crashed
        sage -t -long "devel/sage/sage/interfaces/maxima.py" # Time out
        sage -t -long "devel/sage/sage/interfaces/singular.py" # Killed/crashed
        sage -t -long "devel/sage/sage/libs/cremona/newforms.pyx"
        sage -t -long "devel/sage/sage/libs/ppl.pyx" # Time out
        sage -t -long "devel/sage/sage/libs/singular/ring.pyx" # Killed/crashed
        sage -t -long "devel/sage/sage/misc/randstate.pyx"
        sage -t -long "devel/sage/sage/misc/sageinspect.py"
        sage -t -long "devel/sage/sage/rings/multi_power_series_ring_element.py" # Killed/crashed
        sage -t -long "devel/sage/sage/rings/polynomial/multi_polynomial_ideal.py" # Killed/crashed
        sage -t -long "devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx" # Killed/crashed
        sage -t -long "devel/sage/sage/rings/polynomial/polynomial_singular_interface.py" # Killed/crashed
        sage -t -long "devel/sage/sage/rings/real_mpfr.pyx"
        sage -t -long "devel/sage/sage/rings/tests.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/BSD.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_curve_isogeny.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_field.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_finite_field.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_generic.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_local_data.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_number_field.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_point.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py" # Time out
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/gal_reps.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/heegner.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/elliptic_curves/period_lattice.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/generic/scheme.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/plane_conics/con_finite_field.py" # Killed/crashed
        sage -t -long "devel/sage/sage/schemes/plane_curves/projective_curve.py" # Killed/crashed
        sage -t -long "devel/sage/sage/stats/intlist.pyx"
        sage -t -long "devel/sage/sage/structure/sage_object.pyx" # Killed/crashed

comment:93 follow-up: Changed 8 years ago by kcrisman

Here is one example. 

sage -t -long "devel/sage/sage/functions/transcendental.py" 
**********************************************************************
File "/Users/student/Desktop/sage-4.7.2.alpha1/devel/sage/sage/functions/transcendental.py", line 83:
    sage: w = exponential_integral_1(2,4); w
Expected:
    [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843924751e-05] 
Got:
    [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.766562284392475e-05]
**********************************************************************

Here is one of the killed ones (not a timeout, I'm surprised): 

sage -t -long "devel/sage/sage/crypto/mq/mpolynomialsystem.py"
The doctested process was killed by signal 10
         [352.0 s]

Could this be an unrelated 4.7.2.alpha1 issue on my platform?

comment:94 Changed 8 years ago by fbissey

I am particularly interested in the killed/crashed tests. And also those for which there is already a patch. 

sage -t -long "devel/sage/sage/libs/cremona/newforms.pyx"
sage -t -long "devel/sage/sage/rings/real_mpfr.pyx"
sage -t -long "devel/sage/sage/stats/intlist.pyx"

are the 32/64 bits message difference already mentioned by Steve.

The timeouts are the regular suspect on ppc if I am not mistaken.

The transcendental.py doctest you are quoting is worrying me. It is patched in trac_9958-fixing_numericalnoise-part1.patch​ but you have a slightly different result again (one less digit on the last number). We may have to consider a different patch.

All killed/crashed doctest are candidates for #11339 but you'll have to run sage -gdb on them to be sure.

comment:95 in reply to: ↑ 93 Changed 8 years ago by leif

Replying to kcrisman:

Here is one of the killed ones (not a timeout, I'm surprised): 

sage -t -long "devel/sage/sage/crypto/mq/mpolynomialsystem.py"
The doctested process was killed by signal 10
         [352.0 s]

Could this be an unrelated 4.7.2.alpha1 issue on my platform?

Does SIGUSR1 (#10) have a special meaning on Darwin?

comment:96 Changed 8 years ago by fbissey

Almost anything with "polynomial" in its name is likely to use singular (and crypto/mq/mpolynomialsystem.py is no exception), signal 10 is apparently a user defined signal so it could be anything.

We'll have to wait for the hurricane to move on before Karl can answer us on any of this points.

Looking back at you list Karl I am also very curious about 

sage -t -long "devel/sage/sage/structure/sage_object.pyx
sage -t -long "devel/sage/sage/misc/sageinspect.py"
sage -t -long "devel/sage/sage/misc/randstate.pyx"

two of these have already been patched (sage_object and randstate) in this ticket so I am quite curious about what's wrong with them.

comment:97 Changed 8 years ago by kcrisman

Yeah, I'm sorry I couldn't provide more information. The tests only just finished after classes, and I had one hour to copy/paste the above, restart just doing the failed tests, write a lecture, check email, and then turn off and unplug my PPC machine. I just couldn't do any more. I certainly wasn't taking an eMac home on the bike/train with me!

But we are just due for heavy rain and wind, so hopefully Monday I can get back on it and tell you more, save an extended power outage. Much better than an earthquake, I think fbissey would heartily agree. I wish I'd had a chance to run tests after #11339, but I just knew it would take eons. I'd do it on one of my home PPC machines (believe it or not, I have two more) but one I can no longer get !XCode for, and the other is also slow enough it wouldn't be ready by the time we have to unplug it... sigh.

Anyway, I'll keep this ticket on the priority list for the machine I did these tests on.

comment:98 Changed 8 years ago by leif

So, I did test this with Sage 4.7.2.alpha2 on my "last" 32-bit machine, a Pentium4 Prescott (which has SSE3) running Ubuntu 9.04; my other Pentium4 (Northwood, without SSE3 / PNI) recently died, and I won't revive it in the near future.

The good news are: The patches all still apply to 4.7.2.alpha2, though many hunks with partially large offsets, but no fuzz.

The bad news: I expected some numerical noise because of 32-bit architecture with (there rather rare) SSE3 (-mfpmath=sse), but also experienced at least one segfault (in sage/rings/homset.py). I'll have to investigate the rest, for now just: 

...
sage -t  -long -force_lib "devel/sage/sage/geometry/polyhedra.py"
**********************************************************************
File "/media/H-1TB-P6-linux2/Sage/sage-4.7.2.alpha2-gcc-4.5.1/devel/sage/sage/geometry/polyhedra.py", line 4948:
    sage: ppoints[0]
Expected:
    (-1.92296268638e-15, -1.92296268638e-15)
Got:
    (0.0, 0.0)
**********************************************************************
1 items had failures:
   1 of   8 in __main__.example_171
***Test Failed*** 1 failures.
For whitespace errors, see the file /home/leif/.sage//tmp/.doctest_polyhedra.py
	 [166.0 s]
 
----------------------------------------------------------------------
The following tests failed:


	sage -t  -long -force_lib "devel/sage/sage/libs/cremona/newforms.pyx"
	sage -t  -long -force_lib "devel/sage/sage/functions/transcendental.py"
	sage -t  -long -force_lib "devel/sage/sage/misc/randstate.pyx"
	sage -t  -long -force_lib "devel/sage/sage/combinat/words/suffix_trees.py"
	sage -t  -long -force_lib "devel/sage/sage/combinat/words/nfactor_enumerable_word.py"
	sage -t  -long -force_lib "devel/sage/sage/rings/morphism.pyx"
	sage -t  -long -force_lib "devel/sage/sage/rings/real_mpfr.pyx"
	sage -t  -long -force_lib "devel/sage/sage/rings/homset.py"
	sage -t  -long -force_lib "devel/sage/sage/stats/intlist.pyx"
	sage -t  -long -force_lib "devel/sage/sage/schemes/generic/scheme.py"
	sage -t  -long -force_lib "devel/sage/sage/geometry/polyhedra.py"
Total time for all tests: 26263.8 seconds
make: *** [testlong] Error 128

real	439m23.357s
user	393m40.644s
sys	31m7.485s
leif@californication:~/Sage/sage-4.7.2.alpha2-gcc-4.5.1$

(The tail of the log. More to come later.)

Note that I didn't test in parallel, although with some other CPU-greedy process running, but time-outs are unlikely. Vanilla 4.7.2.alpha2 passed all tests in exactly the same setting (and I did rebuild all dependent packages when installing the Python 2.7 spkg, modulo missing extension module dependencies in module_list.py).

comment:99 follow-up: Changed 8 years ago by leif

Ok, three segfaults (two of them without backtrace / Sage message, for whatever reason), some Python ints too large ("as usual" I guess), some numerical noise, the rest apparently related to dictionaries / hashing(?).

Attaching short logs of rerun tests (all reproducable)...

Changed 8 years ago by leif

Pentium4 Prescott (SSE3 / PNI, 32-bit), GCC 4.5.1, native code, -mfpmath=sse, Ubuntu 9.04 x86

Changed 8 years ago by leif

Pentium4 Prescott (SSE3 / PNI, 32-bit), GCC 4.5.1, native code, -mfpmath=sse, Ubuntu 9.04 x86 (The three segfaulting files tested in verbose mode.)

comment:100 in reply to: ↑ 99 Changed 8 years ago by leif

Replying to leif:

Attaching short logs of rerun tests (all reproducable)...

Done.

Haha, the first segfault is an especially nice one: 

...
  10 tests in __main__.example_7
   6 tests in __main__.example_8
   7 tests in __main__.example_9
511 tests in 55 items.
511 passed and 0 failed.
Test passed.
Segmentation fault
	 [9.8 s]
 
----------------------------------------------------------------------
The following tests failed:


	sage -t -long -verbose "devel/sage/sage/rings/morphism.pyx"
Total time for all tests: 9.8 seconds

real	0m10.013s
user	0m6.980s
sys	0m1.068s

comment:101 follow-ups: Changed 8 years ago by fbissey

Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these. 

sage -t  -long -force_lib "devel/sage/sage/combinat/words/nfactor_enumerable_word.py"

is known to fail but is probably a minor adjustment. 

sage -t  -long -force_lib "devel/sage/sage/combinat/words/suffix_trees.py"

is new. The rest look like #11339 material.

comment:102 in reply to: ↑ 101 Changed 8 years ago by leif

Replying to fbissey:

Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.

The failing doctests are all in Doctest_failures_Sage-4.7.2.alpha2_Linux_x86_SSE3-individual_tests_rerun.log​.

[...] The rest look like #11339 material.

I'm just to apply the patches from there; the machine is dead slow, doing a ./sage -ba-force atm to exclude missing extension module dependencies, as I've not built from scratch.

comment:103 Changed 8 years ago by fbissey

I was too fast. Some of the failures are already reported by Steve. I may try to update the patch set a bit tonight.

comment:104 Changed 8 years ago by leif

Just experienced some new(?) error when cloning the Sage library for #11339: 

...
byte-compiling /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage/misc/copying.py to copying.pyc
byte-compiling /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage/version.py to version.pyc
running install_egg_info
Removing /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage-0.0.0-py2.7.egg-info
Writing /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage-0.0.0-py2.7.egg-info
Exception in thread Thread-4 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 530, in __bootstrap_inner
  File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 483, in run
  File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/multiprocessing/pool.py", line 272, in _handle_workers
<type 'exceptions.TypeError'>: 'NoneType' object is not callable
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 530, in __bootstrap_inner
  File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 483, in run
  File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/multiprocessing/pool.py", line 272, in _handle_workers
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

real	1m35.816s
user	1m35.794s
sys	0m5.648s

After cloning, if you change any Sage library files and want to rebuild
the html version of the reference manual, use the command
    sage -docbuild reference html
(after running 'sage -b').
...

comment:105 Changed 8 years ago by leif

Waaaaaah! 

leif@californication:~/Sage/sage-4.7.2.alpha2-gcc-4.5.1/devel/sage-9958-11339$ ../../sage -hg import -v ~/Sage/patches/trac_11339_refcount_singular_polynomials.patch 
applying /home/leif/Sage/patches/trac_11339_refcount_singular_polynomials.patch
patching file sage/rings/polynomial/multi_polynomial_libsingular.pxd
patching file sage/rings/polynomial/multi_polynomial_libsingular.pyx
Hunk #13 FAILED at 727
Hunk #14 FAILED at 741
Hunk #15 FAILED at 763
Hunk #16 FAILED at 773
Hunk #17 FAILED at 825
Hunk #18 succeeded at 910 with fuzz 2 (offset 72 lines).
Hunk #19 FAILED at 875
Hunk #20 succeeded at 961 (offset 75 lines).
Hunk #21 succeeded at 980 (offset 75 lines).
Hunk #22 succeeded at 1006 (offset 75 lines).
...

(The first patch still applies, with offsets.)

I'm not going to fix that...

comment:106 in reply to: ↑ 101 ; follow-up: Changed 8 years ago by strogdon

Replying to fbissey:

Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.

I'm sure the polyhedra.py patch was needed at one time but it now (?) appears that 

@@ -4946,7 +4946,7 @@
         sage: proj = ProjectionFuncStereographic([1.1,1.1,1.1])
         sage: ppoints = [proj(vector(x)) for x in cube]
         sage: ppoints[0]
-        (0.0, 0.0)
+        (-1.92296268638e-15, -1.92296268638e-15)
     """
     def __init__(self, projection_point):
         """

from trac_9958-fixing_numericalnoise-part1.patch is no longer needed.

comment:107 in reply to: ↑ 106 Changed 8 years ago by fbissey

Replying to strogdon:

Replying to fbissey:

Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.

I'm sure the polyhedra.py patch was needed at one time but it now (?) appears that 

@@ -4946,7 +4946,7 @@
         sage: proj = ProjectionFuncStereographic([1.1,1.1,1.1])
         sage: ppoints = [proj(vector(x)) for x in cube]
         sage: ppoints[0]
-        (0.0, 0.0)
+        (-1.92296268638e-15, -1.92296268638e-15)
     """
     def __init__(self, projection_point):
         """

from trac_9958-fixing_numericalnoise-part1.patch is no longer needed.

You are right I am reading the test results backwards. It now seems to go to 0 properly. One less patch in the patch and one less worry.

comment:108 follow-up: Changed 8 years ago by kcrisman

Okay, you can get a full list of current failure and what they looked like at ​this link. Some are definitely known above, but others may be unknown. The timeouts are probably just due to this machine (and the Maxima one), but others I don't know whether they are here or #11339.

Changed 8 years ago by fbissey

fixing numerical noise part 1 - updated 20110830

comment:109 follow-up: Changed 8 years ago by fbissey

  • Description modified (diff)

Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.

comment:110 in reply to: ↑ 108 Changed 8 years ago by fbissey

Replying to kcrisman:

Okay, you can get a full list of current failure and what they looked like at ​this link. Some are definitely known above, but others may be unknown. The timeouts are probably just due to this machine (and the Maxima one), but others I don't know whether they are here or #11339.

You have some nasty stuff in polyhedra.py 

File "/Users/student/Desktop/sage-4.7.2.alpha1/devel/sage/sage/geometry/polyhedra.py", line 5250:
    sage: _.show()
Exception raised:
    Traceback (most recent call last):
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_183[5]>", line 1, in <module>
        _.show()###line 5250:
    sage: _.show()
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python/site-packages/sage/misc/displayhook.py", line 174, in displayhook
        print_obj(sys.stdout, obj)
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python/site-packages/sage/misc/displayhook.py", line 142, in print_obj
        print >>out_stream, `obj`
      File "base.pyx", line 80, in sage.plot.plot3d.base.Graphics3d.__repr__ (sage/plot/plot3d/base.c:2526)
      File "base.pyx", line 1124, in sage.plot.plot3d.base.Graphics3d.show (sage/plot/plot3d/base.c:10344)
      File "base.pyx", line 663, in sage.plot.plot3d.base.Graphics3d.export_jmol (sage/plot/plot3d/base.c:5867)
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python2.7/zipfile.py", line 1138, in writestr
        self.fp.write(zinfo.FileHeader())
      File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python2.7/zipfile.py", line 348, in FileHeader
        len(filename), len(extra))
    error: integer out of range for 'H' format code

You get an extra digit in randstate compared to x86 :( [note to self it is in numerical_noise_part4].

sage-inspect looks bad :( but not hopeless. The message is just overly long. I will have to postpone the rest for later.

comment:111 Changed 8 years ago by fbissey

sage-inspect is actually an example merged in 4.7.2.alpha1 from #11298 it may not be a python2.7 problems unless you already tested that ticket. I wonder if #11734 would help, the result should be python code but you have cython code instead.

comment:112 follow-up: Changed 8 years ago by fbissey

In polyhedra.py it looks like you have a problem with jmol printing 3D images to file. It involves zip, looking at the pyx there is an instance of testing zip. Could you paste the cython code that produced sage/plot/plot3d/base.c:5867 ?

Changed 8 years ago by fbissey

fix some 32/64 bits messages

Changed 8 years ago by fbissey

all fixes to real_mpfr.pyx split from everything else

Changed 8 years ago by fbissey

split real_mpfr.pyx from the mixed fixes patch

comment:113 Changed 8 years ago by fbissey

  • Description modified (diff)

Updated the patch set again. split real_mpfr.pyx from the mixedfix patch. The 32/64 bits issues reported is separate from the message that was already patched. Added a patch to fix 32/64 bit messages.

comment:114 in reply to: ↑ 112 Changed 8 years ago by kcrisman

Replying to fbissey:

In polyhedra.py it looks like you have a problem with jmol printing 3D images to file. It involves zip, looking at the pyx there is an instance of testing zip. Could you paste the cython code that produced sage/plot/plot3d/base.c:5867 ? 

  /* "sage/plot/plot3d/base.pyx":663
 *         f.write("isosurface fullylit; pmesh o* fullylit; set antialiasdisplay on;\n")
 * 
 *         render_params.output_archive.writestr('SCRIPT', f.getvalue())             # <<<<<<<<<<<<<<
 *         render_params.output_archive.close()
 * 
 */
  __pyx_t_3 = PyObject_GetAttr(__pyx_v_render_params, __pyx_n_s__output_archive); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_t_2 = PyObject_GetAttr(__pyx_t_3, __pyx_n_s__writestr); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_2);
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  __pyx_t_3 = PyObject_GetAttr(__pyx_v_f, __pyx_n_s__getvalue); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_t_4 = PyObject_Call(__pyx_t_3, ((PyObject *)__pyx_empty_tuple), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_4);
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  __pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(((PyObject *)__pyx_t_3));
  __Pyx_INCREF(((PyObject *)__pyx_n_s__SCRIPT));
  PyTuple_SET_ITEM(__pyx_t_3, 0, ((PyObject *)__pyx_n_s__SCRIPT));
  __Pyx_GIVEREF(((PyObject *)__pyx_n_s__SCRIPT));
  PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_4);
  __Pyx_GIVEREF(__pyx_t_4);
  __pyx_t_4 = 0;
  __pyx_t_4 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_t_3), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_4);
  __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
  __Pyx_DECREF(((PyObject *)__pyx_t_3)); __pyx_t_3 = 0;
  __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;

5867 is 

  __pyx_t_4 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_t_3), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}

I was also surprised by the polyhedra.py additional weirdness. Anything I can do to help identify it? Warning: I'm planning on building and testing alpha2 without Python 2.7 now, which will tie up most resources on this machine for a couple days.

comment:115 in reply to: ↑ 109 ; follow-up: Changed 8 years ago by strogdon

Replying to fbissey:

Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.

Here the fix_transcendental patch fixed the associated test on x86 but now the test fails on amd64: 

sage -t -long  -force_lib devel/sage-main/sage/functions/transcendental.py
**********************************************************************
File "/storage/sage/sage-4.7.2.alpha2/devel/sage-main/sage/functions/transcendental.p
y", line 83:
    sage: w = exponential_integral_1(2,4); w
Expected:
    [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843
924...e-05]
Got:
    [0.04890051070806112, 0.0037793524098489063, 0.00036008245216265873, 3.7665622843
924534e-05]
**********************************************************************

This is odd! I wonder what changed? Previously this test must have returned ...67 and not ...63 for it to pass.

comment:116 in reply to: ↑ 115 Changed 8 years ago by fbissey

Replying to strogdon:

Replying to fbissey:

Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.

Here the fix_transcendental patch fixed the associated test on x86 but now the test fails on amd64: 

sage -t -long  -force_lib devel/sage-main/sage/functions/transcendental.py
**********************************************************************
File "/storage/sage/sage-4.7.2.alpha2/devel/sage-main/sage/functions/transcendental.p
y", line 83:
    sage: w = exponential_integral_1(2,4); w
Expected:
    [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843
924...e-05]
Got:
    [0.04890051070806112, 0.0037793524098489063, 0.00036008245216265873, 3.7665622843
924534e-05]
**********************************************************************

This is odd! I wonder what changed? Previously this test must have returned ...67 and not ...63 for it to pass.

It's my fault for not noticing this difference between 32 and 64 bit results. It was there before but I didn't notice it. I thought the last number was the only thing different so when I truncated the last number to the common digits I dropped the difference between 32 and 64 bit. So it is entirely my fault.

Changed 8 years ago by fbissey

fixing numerical noise in transcendental.py- split from numerical noise pt1 - updated 2011092

comment:117 follow-up: Changed 8 years ago by fbissey

transcendental patch updated.

comment:118 in reply to: ↑ 117 ; follow-up: Changed 8 years ago by leif

Replying to fbissey:

transcendental patch updated.

Literally?

comment:119 in reply to: ↑ 118 Changed 8 years ago by fbissey

Replying to leif:

Replying to fbissey:

transcendental patch updated.

Literally?

Literally.

comment:120 follow-up: Changed 8 years ago by kcrisman

Just for reference, on the PPC machine all tests pass with alpha2 except three timeouts which are long-standing (I didn't bother to set SAGE_TIMEOUT_LONG but I've seen these many times).

So all the rest are due to this ticket or #11339.

Would it be helpful for me to do #11339 on this new build (without Python 2.7), or is that in limbo right now anyway? Alternately, is there any way to decouple this ticket from that one?

comment:121 in reply to: ↑ 120 Changed 8 years ago by fbissey

Replying to kcrisman:

Just for reference, on the PPC machine all tests pass with alpha2 except three timeouts which are long-standing (I didn't bother to set SAGE_TIMEOUT_LONG but I've seen these many times).

So all the rest are due to this ticket or #11339.

That's a very useful point of reference. Thank you for providing it.

Would it be helpful for me to do #11339 on this new build (without Python 2.7), or is that in limbo right now anyway? Alternately, is there any way to decouple this ticket from that one?

Technically you can apply #11339 without python 2.7 - the behavior we want to correct is only revealed when we use python 2.7 but I guess it would be useful to know if there are any side effects if you apply the patch on a python 2.6 install. Bear in mind that the current patch are not necessarily the best solution to the problem, what's more they are a whack a mole game. Fix something here and some new fault appears in a completely unexpected location.

comment:122 Changed 8 years ago by fbissey

I will update the python spkg to 2.7.3 when it will be released. 2.7.3 will include the patch to python that we have been carrying in sage for the longest time. See: ​http://bugs.python.org/issue7689

Does anyone know if the apocalypse is about to happen?

comment:123 Changed 8 years ago by jason

  • Description modified (diff)

Fixes #1159, according to that ticket.

comment:124 Changed 8 years ago by fbissey

That's really weird. Some of the hashing problems I encounter with python 2.7 are almost the same as reported in ​http://trac.sagemath.org/sage_trac/ticket/4957#comment:3 

sage -t -long -force_lib "devel/sage-main/sage/rings/integer.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3046:
    sage: n = -920390823904823094890238490238484; n.__hash__()
Expected:
    6874330978542788722   
Got:
    -2623069716
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3061:
    sage: hash(n)
Expected:
    -9223372036854767616      
Got:
    8192
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3064:
    sage: hash(n) == hash(int(n))
Expected:
    True
Got:
    False
**********************************************************************

comment:125 Changed 8 years ago by fbissey

With python-2.6: 

sage: n=2^63+2^13
sage: n
9223372036854784000
sage: int(n)
9223372036854784000L
sage: hash(n)
-9223372036854767616
sage: hash(int(n))
-9223372036854767616

with python-2.7 

sage: n=2^63+2^13
sage: n
9223372036854784000
sage: hash(n)
8192
sage: int(n)
9223372036854784000L
sage: hash(int(n))
-9223372036854767616

Furthermore in both case: 

sage: type(n)
<type 'sage.rings.integer.Integer'>
sage: type(int(n))
<type 'long'>

So I would expect different hashes naively.

comment:126 Changed 8 years ago by fbissey

Regarding 

sage -t -long -force_lib "devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 20:
    sage: it.next()
Expected:
    word: 6456
Got:
    word: 5645
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 22:
    sage: it.next()
Expected:
    word: 5645
Got:
    word: 6456
**********************************************************************

With python-2.7 

sage: w = Word([4,5,6])^7
sage: w
word: 456456456456456456456
sage: it = w.factor_iterator(4)
sage: it.next()
word: 5645
sage: it.next()
word: 6456
sage: it.next()
word: 4564

with python-2.6 

sage: w = Word([4,5,6])^7
sage: w
word: 456456456456456456456
sage: it = w.factor_iterator(4)
sage: it.next()
word: 6456
sage: it.next()
word: 5645
sage: it.next()
word: 4564

So it looks like with python 2.6 we where moving in the series of number with a stride 2 while python 2.7 works through the sequence with a stride 1.

comment:127 Changed 8 years ago by strogdon

Here on x86 I get only two failures: 

sage -t -long  -force_lib devel/sage-main/sage/misc/randstate.pyx
**********************************************************************
File "/storage/sage/sage-4.7.2.rc0/devel/sage-main/sage/misc/randstate.pyx", line 61:
    sage: rtest()
Expected:
    (978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 603
59, 0.83350776541997362)
Got:
    (978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 603
59, 0.8335077654199736)
**********************************************************************

and 

sage -t -long  -force_lib devel/sage-main/sage/matrix/matrix_double_dense.pyx
**********************************************************************
File "/storage/sage/sage-4.7.2.rc0/devel/sage-main/sage/matrix/matrix_double_dense.py
x", line 1046:
    sage: A.singular_values(eps='junk')
Expected:
    Traceback (most recent call last):
    ...
    ValueError: invalid literal for float(): junk
Got:
    Traceback (most recent call last):
      File "/storage/sage/sage-4.7.2.rc0/local/bin/ncadoctest.py", line 1231, in run_
one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/storage/sage/sage-4.7.2.rc0/local/bin/sagedoctest.py", line 38, in run_o
ne_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags
)
      File "/storage/sage/sage-4.7.2.rc0/local/bin/ncadoctest.py", line 1172, in run_
one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_19[37]>", line 1, in <module>
        A.singular_values(eps='junk')###line 1046:
    sage: A.singular_values(eps='junk')
      File "matrix_double_dense.pyx", line 1075, in sage.matrix.matrix_double_dense.M
atrix_double_dense.singular_values (sage/matrix/matrix_double_dense.c:6338)
        eps = RDF(eps)
      File "parent.pyx", line 988, in sage.structure.parent.Parent.__call__ (sage/str
ucture/parent.c:7326)
      File "coerce_maps.pyx", line 82, in sage.structure.coerce_maps.DefaultConvertMa
p_unique._call_ (sage/structure/coerce_maps.c:3268)
      File "coerce_maps.pyx", line 77, in sage.structure.coerce_maps.DefaultConvertMa
p_unique._call_ (sage/structure/coerce_maps.c:3171)
      File "real_double.pyx", line 548, in sage.rings.real_double.RealDoubleElement._
_init__ (sage/rings/real_double.c:5567)
    ValueError: could not convert string to float: junk
**********************************************************************

The latter also fails here the same way on amd64 and is documented on sage-devel

​http://groups.google.com/group/sage-devel/browse_thread/thread/8e1d7f665ac5be37

but with some slight differences.

comment:128 Changed 8 years ago by fbissey

OK Steve, I got the matrix_double_dense but you don't get the numerical noise. I believe the noise may come from python-2.7.2. Interesting that the hashing and enumeration problem are 64bit only. I'll check on the mac.

Changed 8 years ago by fbissey

Fix doctest in matrix_double_dense.pyx about junk

comment:129 Changed 8 years ago by fbissey

  • Description modified (diff)

New patch to fix the message about junk.

comment:130 Changed 8 years ago by fbissey

  • Dependencies changed from #11156 #11236 #11244 #11264 #11339 #11363 #11376 to #11156 #11236 #11244 #11264 #11339 #11363 #11376 #11986

I have now split the hashing issue in #11986

Changed 8 years ago by fbissey

fix the doctest on enumeration order for 64bit system

comment:131 Changed 8 years ago by fbissey

  • Description modified (diff)

One more patch added to fix combinat/words/nfactor_enumerable_word.py on 64 bit system. I believe is output is equivalent in both case so it is not important.

comment:132 Changed 8 years ago by jdemeyer

  • Dependencies changed from #11156 #11236 #11244 #11264 #11339 #11363 #11376 #11986 to #11986
  • Description modified (diff)

Removed old dependencies.

comment:133 follow-up: Changed 8 years ago by jdemeyer

  • Milestone set to sage-4.8

The spkg needs to be rebased to #11447.

comment:134 Changed 8 years ago by jdemeyer

Permissions is the spkg should be 0644 (or 0755 for executables), currently many files are 0640 or 0750.

comment:135 in reply to: ↑ 133 ; follow-up: Changed 8 years ago by fbissey

Replying to jdemeyer:

The spkg needs to be rebased to #11447.

I looked at that before making my spkg. I believe the strategy used in #11447 is in python-2.7. Someone with a debian system proves me wrong but I think that works out of the box.

comment:136 in reply to: ↑ 135 Changed 8 years ago by leif

Replying to fbissey:

Replying to jdemeyer:

The spkg needs to be rebased to #11447.

I looked at that before making my spkg. I believe the strategy used in #11447 is in python-2.7. Someone with a debian system proves me wrong but I think that works out of the box.

Well, we did take the patch from Python upstream, but IIRC there are further changes to spkg-install, and the new spkg needs the previous one to be mentioned in SPKG.txt anyway (to pass the merger's sanity check).

comment:137 Changed 8 years ago by leif

Indeed, there've been a lot of changes inbetween:

  • spkg-install 

    diff -Nu python-2.7.1/spkg-install python-2.6.4.p11/spkg-install
    old new  
    55
    66CUR=`pwd`
    77
    8 if [ "$SAGE_LOCAL" = "" ]; then
    9    echo "SAGE_LOCAL undefined ... exiting";
    10    echo "Maybe run 'sage -sh'?"
    11    exit 1
     8if [ -z "$SAGE_LOCAL" ]; then
     9    echo >&2 "SAGE_LOCAL undefined ... exiting"
     10    echo >&2 "Maybe run 'sage -sh'?"
     11    exit 1
    1212fi
    1313
    1414# PATCH
    1515
    16 # Due to issues building Python on Ubuntu 11.04, a file is patched
    17 # on these systems.
     16# When building Python 2.6 on Debian and derivatives with multiarch,
     17# system library directories are not correctly setup.  This patch
     18# fixes this, with no-op on other systems.
     19# The fix requires 'dpkg-architecture' to be installed there though
     20# (see below).
     21echo "Patching src/setup.py for multiarch."
     22patch -p0 < patches/setup.py.multiarch.patch
     23if [ $? -ne 0 ]; then
     24    echo >&2 "Error patching src/setup.py"
     25    exit 1
     26fi
    1827
    19 if [ -f /etc/issue ] && grep "Ubuntu 11.04" /etc/issue ; then
    20    echo "Patching src/Modules/Setup.dist as this is Ubuntu 11.04"
    21    patch -p0 < patches/Modules.Setup.dist.patch
     28# Due to a python bug with Solaris
     29# see http://bugs.python.org/issue1759169
     30# it is necessary to apply a patch to configure.in
     31# then run autoconf. This not only generates a
     32# new 'configure' script, but some subdirectories too
     33# so these will be copied.
     34
     35if [ "x`uname`" = xSunOS ] ; then
     36   echo "Applying a revised 'configure' script for Solaris"
     37   echo "See http://bugs.python.org/issue1759169"
     38   echo "http://trac.sagemath.org/sage_trac/ticket/7867"
     39   cp -r patches/autom4te.cache patches/configure patches/configure.in src
    2240   if [ $? -ne 0 ]; then
    23       echo "Error patching src/Modules/Setup.dist"
     41      echo >&2 "Failed to apply the Solaris patches needed for"
     42      echo >&2 "http://bugs.python.org/issue1759169"
     43      echo >&2 "http://trac.sagemath.org/sage_trac/ticket/7867"
    2444      exit 1
    2545   fi
     46   echo "Setting  HAVE_FD_TRANSFER=0 for Solaris to allow"
     47   echo "the python module '_multiprocessing' to build"
     48   echo "See: http://trac.sagemath.org/sage_trac/ticket/8440"
     49   cp patches/setup.py src
     50   if [ $? -ne 0 ]; then
     51      echo >&2 "Failed to apply the Solaris patch needed for"
     52      echo >&2 "http://trac.sagemath.org/sage_trac/ticket/8440"
     53      exit 1
     54   fi
     55fi
     56
     57
     58cp patches/locale.py src/Lib/locale.py
     59if [ $? -ne 0 ]; then
     60    echo >&2 "Error copying patched locale.py"
     61    exit 1
    2662fi
    2763
    28 patch -p0 < patches/Lib.distutils.command.sdist.patch
     64cp patches/Makefile.pre.in src/Makefile.pre.in
    2965if [ $? -ne 0 ]; then
    30     echo "Error copying patched sdist.py"
     66    echo >&2 "Error copying patched Makefile.pre.in"
    3167    exit 1
    3268fi
    3369
    34 patch -p0 < patches/Lib.socket.patch
     70cp patches/sdist.py src/Lib/distutils/command/sdist.py
    3571if [ $? -ne 0 ]; then
    36     echo "Error copying patched socket.py"
     72    echo >&2 "Error copying patched sdist.py"
    3773    exit 1
    3874fi
    3975
    40 patch -p0 < patches/dynamic_class_copyreg_py.patch
     76cp patches/socket.py src/Lib/socket.py
    4177if [ $? -ne 0 ]; then
    42     echo "Error copying patched pickle.py"
     78    echo >&2 "Error copying patched socket.py"
    4379    exit 1
    4480fi
    4581
    46 patch -p0 < patches/dynamic_class_copyreg_c.patch
     82cp patches/pickle.py src/Lib/pickle.py
    4783if [ $? -ne 0 ]; then
    48     echo "Error copying patched cPickle.c"
     84    echo >&2 "Error copying patched pickle.py"
    4985    exit 1
    5086fi
    5187
     88cp patches/cPickle.c src/Modules/cPickle.c
     89if [ $? -ne 0 ]; then
     90    echo >&2 "Error copying patched cPickle.c"
     91    exit 1
     92fi
     93
     94# Due to a problem with _socket not building on OpenSolaris on x64
     95# see http://bugs.python.org/issue8852
     96# http://trac.sagemath.org/sage_trac/ticket/9041
     97# http://trac.sagemath.org/sage_trac/ticket/9022
     98# Modules/socketmodule.c needs patching. The patch consists of
     99# only checking if things are defined before trying to build with them
     100# so it is safe (and desirable) on any platform.
     101
     102cp patches/Modules.socketmodule.c src/Modules/socketmodule.c
     103if [ $? -ne 0 ]; then
     104    echo >&2 "Error copying patched socketmodule.c"
     105    exit 1
     106fi
     107
     108# The following patch for fixing broken readline behavior on Itanium
     109# Linux definitely does *not* work on anything else.
     110if [ "`uname -m`" = "ia64" -a "`uname`" = "Linux" ]; then
     111    echo "Updating readline.c for Linux/Itanium"
     112    cp patches/readline.c-Itanium-fix src/Modules/readline.c
     113else
     114    # Readline patch: http://bugs.python.org/file14599/python-2.6-readline.patch
     115    # Associated bug: http://bugs.python.org/issue5833
     116    #
     117    # Committed to Python as r75747 in the py26-maint branch, but not
     118    # in time for 2.6.4 -- so we can remove this patch the next time
     119    # we update Python in Sage.
     120    cp patches/readline.c-spacebug src/Modules/readline.c
     121fi
     122
     123if [ $? -ne 0 ]; then
     124    echo >&2 "Error copying patched readline.c"
     125    exit 1
     126fi
     127
     128
    52129# We are setting LDFLAGS and CPPFLAGS so that we pick up Sage's readline
    53130LDFLAGS="-L$SAGE_LOCAL/lib $LDFLAGS"
    54131export LDFLAGS
    … …  
    62139    EXTRAFLAGS="--without-pymalloc"; export EXTRAFLAGS
    63140fi
    64141
     142# Program around weird bug in build process:
     143#      Apparently if you have this:
     144#         export DISTUTILS_DEBUG=1
     145#      in your environment variables, the build craps out.  No idea why this
     146#       is.
     147#       -- Yi Qiang
     148#
     149# This bug was fixed in Python, but not yet in Python 2.6.4. So this fix
     150# can be removed the next time we upgrade our version of Python. See
     151#
     152#   http://bugs.python.org/issue6954
     153#
     154# for the fix.
     155#
     156unset DISTUTILS_DEBUG
     157
     158
    65159cd src
    66160
    67161touch Include/*
    … …  
    93187            --enable-unicode=ucs4
    94188        fi
    95189    else
    96         ./configure $EXTRAFLAGS --prefix="$SAGE_LOCAL"  --enable-unicode=ucs4 CC="$CC $CFLAGS" CXX="$CXX $CXXFLAGS"
     190        ./configure $EXTRAFLAGS --prefix="$SAGE_LOCAL"  --enable-unicode=ucs4 \
     191            CC="$CC $CFLAGS" CXX="$CXX $CXXFLAGS"
    97192    fi
    98193
    99194
    100195    if [ $? -ne 0 ]; then
    101         echo "Error configuring Python."
     196        echo >&2 "Error configuring Python."
    102197        exit 1
    103198    fi
    104199
    105200    $MAKE
    106201    if [ $? -ne 0 ]; then
    107         echo "Error building Python."
     202        echo >&2 "Error building Python."
    108203        exit 1
    109204    fi
    110205
    111206    # running 'make install' in parallel is a bad, bad idea
     207    # FIXME: The proper way to disable parallel make is to use
     208    #        $MAKE -j1 ... (We rely on GNU make anyway).
    112209    MAKE=make; export MAKE
    113210    # the "-i" is crucial, esp. in the case of a major upgrade
    114211    make -i install
    115212    if [ $? -ne 0 ]; then
    116         echo "Error installing Python."
     213        echo >&2 "Error installing Python."
    117214        exit 1
    118215    fi
    119216}
    … …  
    122219# informative error message. This is helps in determining why the
    123220# configuration, compilation or installation failed. So put this before the
    124221# build() function.
    125 set +e
     222set +e # This is redundant here, but doesn't hurt to keep it... ;-)
    126223
    127224build
    128225
    129 cd $SAGE_LOCAL/lib
     226cd "$SAGE_LOCAL/lib"
    130227
    131228# move the python directory if we're upgrading from a version
    132 # of sage with python 2.6
    133 if [ -d python2.6/site-packages ]; then
    134    mv python2.6/site-packages python2.7/site-packages-old
     229# of sage with python 2.5
     230if [ -d python2.5/site-packages ]; then
     231   mv python2.5/site-packages python2.6/site-packages-old
    135232fi
    136233
    137 rm -rf python/python2.7 python/python2.6 python/python2.5 python/python python python2.4 python2.5
    138 ln -s python2.7 python
     234rm -rf python/python2.6 python/python2.5 python/python python python2.4 python2.5
     235ln -s python2.6 python
    139236if [ $? -ne 0 ]; then
    140     echo "Error creating symbolic link"
     237    echo >&2 "Error creating symbolic link"
    141238    exit 1
    142239fi
    143240
    144241# Sleeping for three seconds so that parallel 'make install' catches up
    145242# with the following test.
     243# XXX We have disabled parallel 'make install' above, but never mind.
    146244echo "Sleeping for three seconds before testing python"
    147245sleep 3
    148246
    … …  
    150248# All these modules are important and if any one
    151249# fails to build, Sage will not work.
    152250
     251import_errors=false
    153252for module in math hashlib crypt ; do
    154253   "$SAGE_LOCAL/bin/python" -c "import $module"
    155254   if [ $? -eq 0 ] ; then
    156255      echo "$module module imported OK"
    157256   else
    158       echo "$module module failed to import"
    159       exit 1
     257      echo >&2 "$module module failed to import"
     258      import_errors=true
     259      # exit 1 # not yet
    160260   fi
    161261done
     262
     263if $import_errors; then
     264    echo >&2 "Error: One or more modules failed to import."
     265    # Check if we are on Debian or one of its derivatives:
     266    if command -v dpkg && ! command -v dpkg-architecture >/dev/null; then
     267        echo >&2 "You may have to install 'dpkg-architecture'"
     268        echo >&2 "which is part of the Debian package 'dpkg-dev'."
     269        echo >&2 "Try installing it by typing"
     270        echo >&2 "    sudo apt-get install dpkg-dev"
     271        echo >&2 "and rerun 'make'."
     272    fi
     273    exit 1
     274fi
     275

comment:138 follow-up: Changed 8 years ago by fbissey

OK that's not a biggy while I had a look at #11447 I made my spkg before it was closed and possibly before it had reached its final shape. I wanted to wait for 2.7.3 but I will probably cut a 2.7.2 in which I will add this then.

I am keen to have this landing ASAP but I would understand if the release manager only wants to do it in a alpha0. That's both a big change and a big patch set. My concern is that my last patch may not be adequate. Steve tested a number of configurations and the order this comes out are all over the place. I may have to make it random which is not nice.

The other concern is #11986 which looks like a blast from the past and I have no clues on either of them.

comment:139 in reply to: ↑ 138 Changed 8 years ago by jdemeyer

Replying to fbissey:

OK that's not a biggy while I had a look at #11447 I made my spkg before it was closed and possibly before it had reached its final shape. I wanted to wait for 2.7.3 but I will probably cut a 2.7.2 in which I will add this then.

I am keen to have this landing ASAP but I would understand if the release manager only wants to do it in a alpha0. That's both a big change and a big patch set.

If you can finish it, I'd be happy to merge it. But to be realistic, I don't think this ticket is close to being finished. There is a huge amount of patches which needs to be reviewed, there is still #11986, there is the rebasing to #11447, maybe testing will reveal breakage on some systems...

comment:140 Changed 8 years ago by jdemeyer

There is a warning when building the reference manual: 

docstring of sage.crypto.boolean_function:3: (WARNING/2) Block quote ends without a blank line; unexpected unindent.

comment:141 Changed 8 years ago by jdemeyer

Another minor comment: please use UTF8 in SPKG.txt. Currently, "François" is encoded as latin1.

comment:142 Changed 8 years ago by jdemeyer

  • Authors set to François Bissey

comment:143 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-4.8 to sage-5.0

comment:144 follow-up: Changed 8 years ago by jdemeyer

  • Description modified (diff)
  • Summary changed from Upgrade python to 2.7 to Upgrade python to 2.7.x

By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.

comment:145 in reply to: ↑ 144 ; follow-up: Changed 8 years ago by fbissey

Replying to jdemeyer:

By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.

sage-5.0 is probably more realistic. Thanks for the useful comments, I hadn't seen the warnings in the documentation, this probably could be spun out of this ticket as it is probably a problem in the existing doc - just silent with the current python. I really would like to go to 2.7.3 when it comes out but I'll make a stop 2.7.2. I have a heavy workload this week but I'll try to put the spkg together. I know from sage-on-gentto that there will be some more patching to be done unfortunately although in some case I think that would be just a band aid. So more dependencies may appear :(

I am surprised at the "latin-1" encoding I am defaulting to UTF-8 and even my Changelogs now come correct everywhere.

We need someone with the know how to look at #11986.

comment:146 in reply to: ↑ 145 Changed 8 years ago by jdemeyer

Replying to fbissey:

Replying to jdemeyer:

By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.

sage-5.0 is probably more realistic.

Especially since William essentially defined sage-5.0 as the release which will contain this ticket.

I hadn't seen the warnings in the documentation, this probably could be spun out of this ticket as it is probably a problem in the existing doc - just silent with the current python.

True, but I have no idea how Python 2.7 has changed this particular doctest. It could also be that something changed inside Sphinx or Cython because of Python 2.7.

I am surprised at the "latin-1" encoding I am defaulting to UTF-8 and even my Changelogs now come correct everywhere.

And now you're punished for the rest of your live as a developer because your name is not ASCII (and please don't write "Francois", that just looks very wrong).

We need someone with the know how to look at #11986.

I might have a look, but I don't want to promise anything. In any case, once #11986 is really the only remaining issue, it will certainly get fixed.

comment:147 Changed 8 years ago by fbissey

  • Description modified (diff)

All right. Very busy week so far. I am removing the last patch and will spin it in a separate ticket because Steve did more testing and it appears that the test is somewhat random and was only working everywhere the same with python 2.6 by sheer luck.

Steve had a few interesting suggestions for replacing the test but that's clearly another ticket on which we'll forward people who touched that file for comments.

comment:148 follow-up: Changed 8 years ago by fbissey

  • Description modified (diff)

I finished updating the spkg to 2.7.2 in a moment of freedom. I called it 2.7.2.p0 as it has some patches in it. Hope all the permissions are right this time. And SPKG.txt is UTF-8 now.

I will need to update one patch and I think add a new one. At least relative to 4.7.2, I am so bogged down in work that I haven't been working at all on 4.8.0 which may require new patches.

Changed 8 years ago by fbissey

fix pure assertErrors failures - updated for python-2.7.2

Changed 8 years ago by fbissey

fix sage_unittest.py failures - updated for python-2.7.2

comment:149 Changed 8 years ago by fbissey

Patch set updated for python-2.7.2.

comment:150 Changed 8 years ago by jdemeyer

  • Description modified (diff)

comment:151 in reply to: ↑ 148 ; follow-up: Changed 8 years ago by jdemeyer

  • Description modified (diff)

Replying to fbissey:

I finished updating the spkg to 2.7.2 in a moment of freedom. I called it 2.7.2.p0 as it has some patches in it. Hope all the permissions are right this time.

Unfortunately, they are not. I fixed this in ​http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p0.spkg

comment:152 Changed 8 years ago by jdemeyer

  • Description modified (diff)

comment:153 in reply to: ↑ 151 ; follow-up: Changed 8 years ago by fbissey

Replying to jdemeyer:

Replying to fbissey:

I finished updating the spkg to 2.7.2 in a moment of freedom. I called it 2.7.2.p0 as it has some patches in it. Hope all the permissions are right this time.

Unfortunately, they are not. I fixed this in ​http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p0.spkg

OK. Was it the src folder and its content? Or did I forget something else in the spkg root folder?

comment:154 in reply to: ↑ 153 ; follow-up: Changed 8 years ago by jdemeyer

Replying to fbissey:

OK. Was it the src folder and its content?

Yes. I solved this by doing chmod ugo+rX -R .

Or did I forget something else in the spkg root folder?

Also, yes. spkg-check should be executable.

comment:155 Changed 8 years ago by jdemeyer

This fails to build from scratch with sage-4.8.alpha2 on sage.math: 

gcc  -pthread -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes -L/scratch/jdemeyer/merger/sage-5.0/local/lib   Parser/acceler.o Parser/grammar1.
o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar
.o Parser/pgen.o Objects/obmalloc.o Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o Parser/printgrammar.o Parser/pgenmain.o -
lpthread -ldl  -lutil -o Parser/pgen
Parser/pgen ./Grammar/Grammar ./Include/graminit.h ./Python/graminit.c
touch Parser/pgen.stamp
gcc  -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include  -fPIC -DPy_BUILD_CORE -o Python/ast.o Python/ast.c
gcc  -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include  -fPIC -DPy_BUILD_CORE -o Python/compile.o Python/compile.c
gcc  -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include  -fPIC -DPy_BUILD_CORE -o Python/graminit.o Python/graminit.c
gcc  -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include  -fPIC -DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c
gcc  -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include  -fPIC -DPy_BUILD_CORE \
              -DSVNVERSION="\"`LC_ALL=C svnversion .`\"" \
              -DHGVERSION="\"`LC_ALL=C hg id -i .`\"" \
              -DHGTAG="\"`LC_ALL=C hg id -t .`\"" \
              -DHGBRANCH="\"`LC_ALL=C hg id -b .`\"" \
              -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
You must compile Sage first using 'make' in the Sage root directory.
You must compile Sage first using 'make' in the Sage root directory.
You must compile Sage first using 'make' in the Sage root directory.
<command-line>: warning: missing terminating " character
<command-line>:1:7: warning: missing terminating " character
<command-line>:2:10: warning: missing terminating " character
./Modules/getbuildinfo.c: In function 'Py_GetBuildInfo':
./Modules/getbuildinfo.c:45: error: missing terminating " character
./Modules/getbuildinfo.c:45: error: expected expression before ')' token
./Modules/getbuildinfo.c:46: error: missing terminating " character
./Modules/getbuildinfo.c:46: error: missing terminating " character
./Modules/getbuildinfo.c:47: error: missing terminating " character
./Modules/getbuildinfo.c:47: error: missing terminating " character
./Modules/getbuildinfo.c:53: error: 'buildinfo' undeclared (first use in this function)
./Modules/getbuildinfo.c:53: error: (Each undeclared identifier is reported only once
./Modules/getbuildinfo.c:53: error: for each function it appears in.)
./Modules/getbuildinfo.c: In function '_Py_hgversion':
./Modules/getbuildinfo.c:72: error: missing terminating " character
./Modules/getbuildinfo.c:72: warning: 'return' with no value, in function returning non-void
./Modules/getbuildinfo.c: In function '_Py_hgidentifier':
./Modules/getbuildinfo.c:79: error: missing terminating " character
./Modules/getbuildinfo.c:79: error: expected expression before ';' token
./Modules/getbuildinfo.c:83: error: missing terminating " character
./Modules/getbuildinfo.c:83: error: expected expression before ';' token
make[2]: *** [Modules/getbuildinfo.o] Error 1
make[2]: Leaving directory `/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/spkg/build/python-2.7.2.p0/src'
Error building Python.

Mhy guess would be that spkg-install is trying to run Mercurial or something, which obviously fails because Mercurial has not been compiled yet (and it can't since it depends on Python).

comment:156 Changed 8 years ago by jdemeyer

The error above might be fixed by #5852...

comment:157 Changed 8 years ago by jhpalmieri

The current spkg installs fine on OS X 10.6 and 10.7, at least when I do "sage -f ..." in an existing Sage installation. I haven't tried building from scratch, and I haven't tried doctests or the patches yet. Would it be possible to provide a single large patch in addition to the individual small ones? That would be easier to apply for testing, while the small ones are easier to review.

comment:158 in reply to: ↑ 154 Changed 8 years ago by fbissey

Replying to jdemeyer:

Replying to fbissey:

OK. Was it the src folder and its content?

Yes. I solved this by doing chmod ugo+rX -R .

Or did I forget something else in the spkg root folder?

Also, yes. spkg-check should be executable.

Never used symbolic mode of chmod, always octal. But this looks like it is more effective to get the directory permissions right. Darn spkg-check.

Thanks for the tips. Sorry for being a pest at times.

I haven't tested the spkg from scratch on OSX I did on linux x86_64 and 4.8.alpha2. I will see what happens on my 10.5 machine later. 2.7.1 worked from scratch there i am sure. That looks very strange, can you link the full python building log?

comment:159 Changed 8 years ago by jhpalmieri

A build from scratch on OS X 10.6 worked just fine, by the way.

comment:160 Changed 8 years ago by jdemeyer

There is still the docbuild warning: 

docstring of sage.crypto.boolean_function:3: WARNING: Block quote ends without a blank line; unexpected unindent.

I am investigating.

comment:161 Changed 8 years ago by jdemeyer

I absolutely do not understand the docbuild warning but created a ticket at #12085.

comment:162 Changed 8 years ago by jdemeyer

  • Dependencies changed from #11986 to #5852, #11986, #12085

comment:163 follow-up: Changed 8 years ago by fbissey

Just an idea! This is because it wants to execute some mercurial commands to find if there are tags in the source. If we were to set 

DHGVERSION
DHGTAG
DHGBRANCH

To something or even nothing we may not have this problem. A normal run with a system mercurial installed looks like: 

x86_64-pc-linux-gnu-gcc -pthread -c -fno-strict-aliasing -O2 -pipe -march=native -fwrapv -DNDEBUG   -I. -IInclude -I./Include  -fPIC -DPy_BUILD_CORE \
      -DSVNVERSION="\"`LC_ALL=C svnversion .`\"" \
      -DHGVERSION="\"`LC_ALL=C hg id -i .`\"" \
      -DHGTAG="\"`LC_ALL=C hg id -t .`\"" \
      -DHGBRANCH="\"`LC_ALL=C hg id -b .`\"" \
      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
abort: repository . not found!
abort: repository . not found!
abort: repository . not found!

I will try to see if I can fit that in the spkg.

comment:164 in reply to: ↑ 163 Changed 8 years ago by jdemeyer

Replying to fbissey:

I will try to see if I can fit that in the spkg.

No, I would leave that alone. With #5852, it will build properly. Since you get errors anyway with the system Mercurial, apparently it's not a problem that hg fails.

comment:165 follow-up: Changed 8 years ago by fbissey

If you insist but it would be easy to path Makefile.pre.in to prevent this from happening. It is mainly used to get pre-release version info if you checked out python source from hg. It is supposed to fail if you are using a properly released tarball.

comment:166 in reply to: ↑ 165 Changed 8 years ago by jdemeyer

Replying to fbissey:

If you insist

I will not insist: if you want to make the patch, go ahead.

comment:167 Changed 8 years ago by fbissey

I may try something more subtle first. Makefile.pre.in has 

SVNVERSION=	@SVNVERSION@
HGVERSION=	@HGVERSION@
HGTAG=		@HGTAG@
HGBRANCH=	@HGBRANCH@

and 

	$(CC) -c $(PY_CFLAGS) \
	      -DSVNVERSION="\"`LC_ALL=C $(SVNVERSION)`\"" \
	      -DHGVERSION="\"`LC_ALL=C $(HGVERSION)`\"" \
	      -DHGTAG="\"`LC_ALL=C $(HGTAG)`\"" \
	      -DHGBRANCH="\"`LC_ALL=C $(HGBRANCH)`\"" \
	      -o $@ $(srcdir)/Modules/getbuildinfo.c

later so exporting HG* to something like /bin/true on non-windows OS would presumably solve the problem. But just patching to pass empty strings in the later would probably be safe everywhere.

comment:168 Changed 8 years ago by fbissey

The first patch doesn't apply in 4.8.alpha3 because of the changes in #12047. New patches will come some. There may be other surprises.

comment:169 Changed 8 years ago by fbissey

  • Description modified (diff)

Updated the first patch. Haven't been able to test things to find any new problems yet.

comment:170 Changed 8 years ago by jdemeyer

There are a few test failures on a Pentium 4 32-bit Linux system with sage-4.8.alpha2 + #9958 + #11986: 

sage -t  --long -force_lib devel/sage/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/suffix_trees.py", line 1323:
    sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
    True
Got:
    False
**********************************************************************
sage -t  --long -force_lib devel/sage/sage/misc/randstate.pyx
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/misc/randstate.pyx", line 61:
    sage: rtest()
Expected:
    (978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.83350776541997362)
Got:
    (978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.8335077654199736)
**********************************************************************
sage -t  --long -force_lib devel/sage/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 20:
    sage: it.next()
Expected:
    word: 6456
Got:
    word: 4564
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 23:
    sage: it.next()
Expected:
    word: 5645
Got:
    word: 6456
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 26:
    sage: it.next()
Expected:
    word: 4564
Got:
    word: 5645
**********************************************************************

comment:171 Changed 8 years ago by jdemeyer

I have made a patch for the Integer hashing, see #11986.

comment:172 Changed 8 years ago by fbissey

I have seen your patch in #11986 that's a good rewrite of the hashing in question, the -1 bit is particularly tricky. I mentioned it before but we probably need a different approach for combinat/words/nfactor_enumerable_word.py, the sequence is pretty much random depending on arch and other factors it seems. The randstate.pyx must have escaped ​http://trac.sagemath.org/sage_trac/attachment/ticket/9958/trac_9958-fixing_numericalnoise-part4.patch because the test results are identical to some performed later in the same file. I'll correct that shortly.

The suffix_trees.pyx rings a bell but I cannot remember where and when.

comment:173 Changed 8 years ago by fbissey

ha ​http://trac.sagemath.org/sage_trac/ticket/9958#comment:18 is where I had suffix_trees.pyx before. I don't have a 32 bit linux box anymore so it is more difficult. I note they were two you only see one now.

comment:174 Changed 8 years ago by jdemeyer

  • Description modified (diff)

comment:175 Changed 8 years ago by jdemeyer

  • Dependencies changed from #5852, #11986, #12085 to #5852, #11986, #12085, #12096

comment:176 Changed 8 years ago by fbissey

I need a kind soul with 32 bit linux system to explore the suffix tree issue, the doctest is as follow: 

            sage: from sage.combinat.words.suffix_trees import ImplicitSuffixTree, SuffixTrie
            sage: W = Words("ab")
            sage: t = ImplicitSuffixTree(W("aba"))
            sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
            True

It would be helpful to get the values of "W", "t" and "t.trie_type_dict()" on python-2.6 and 2.7. Then we can make a decision on whether the answer we get with python-2.7 is equivalent to the current output or not.

comment:177 Changed 8 years ago by strogdon

Here (x86) with sage-4.8.alpha3 + #9958 + #11986 the suffix_trees test passes: 

./sage -t  -long -force_lib devel/sage/sage/combinat/words/suffix_trees.py
sage -t -long -force_lib "devel/sage/sage/combinat/words/suffix_trees.py"
         [5.1 s]
 
----------------------------------------------------------------------
All tests passed!

The requested values are: 

sage: W
Words over Ordered Alphabet ['a', 'b']
sage: t
Implicit Suffix Tree of the word: aba
sage: t.trie_type_dict()
{(4, word: b): 5, (0, word: a): 4, (0, word: b): 3, (5, word: a): 1, (3, word: a): 2}

FWIW, on amd64 the test fails with the same sage-4.8.alpha3 + #9958 + #11986

comment:178 Changed 8 years ago by jdemeyer

On sage.math (x86_64 Linux), sage-4.8.alpha3 + #12106 (unrelated) + #9958 + #11986 gives errors in two files: 

sage -t  -force_lib devel/sage/sage/matrix/matrix_mod2e_dense.pyx
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 790:
    sage: float(A.density())
Expected:
    0.099738...
Got:
    0.099739
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 798:
    sage: float(A.density())
Expected:
    0.499759...
Got:
    0.49976
**********************************************************************

(related ticket: #9562) and 

sage -t  -force_lib devel/sage/sage/gsl/integration.pyx
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 110:
    sage: numerical_integral(x^2, 0, 1, max_points=100)
Expected:
    (0.33333333333333331, 3.7007434154171879e-15)
Got:
    (0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 115:
    sage: numerical_integral(sin(x)^3 + sin(x),  0, pi)
Expected:
    (3.333333333333333, 3.7007434154171883e-14)
Got:
    (3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 120:
    sage: numerical_integral(lambda x: sin(x)^3 + sin(x),  0, pi)
Expected:
    (3.333333333333333, 3.7007434154171883e-14)
Got:
    (3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 131:
    sage: numerical_integral(f, 0, 1, max_points=200, eps_abs=1e-7, eps_rel=1e-7, rule=4)
Expected:
    (0.33333333333333331, 3.7007434154171879e-15)
Got:
    (0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 172:
    sage: exp(-1/x).nintegral(x, 1, 2)  # via maxima
Expected:
    (0.50479221787318396, 5.6043194293440752e-15, 21, 0)
Got:
    (0.504792217873184, 5.604319429344075e-15, 21, 0)
**********************************************************************

(related ticket: #12047)

Also, the spkg should be rebased to #12096.

comment:179 Changed 8 years ago by jdemeyer

I get on 32-bit i386 Linux with sage-4.8.alpha2 + various patches including #9958: 

sage: from sage.combinat.words.suffix_trees import ImplicitSuffixTree, SuffixTrie
sage: W = Words("ab")
sage: t = ImplicitSuffixTree(W("aba"))
sage: W
Words over Ordered Alphabet ['a', 'b']
sage: t
Implicit Suffix Tree of the word: aba
sage: t.trie_type_dict()
{(4, word: a): 1, (0, word: b): 5, (0, word: a): 3, (5, word: a): 2, (3, word: b): 4}

comment:180 follow-up: Changed 8 years ago by fbissey

thanks for all that feedback. I knew there would be new stuff. I unfortunately (like all of us) not enough time to do things in a speedy fashion. I am bogged down as well with the sage-on-gentoo version of alpha3 not wanting to play nice with #4260 ​https://github.com/cschwan/sage-on-gentoo/issues/108 and it annoys me.

comment:181 in reply to: ↑ 180 Changed 8 years ago by jdemeyer

Replying to fbissey:

I am bogged down as well with the sage-on-gentoo version of alpha3 not wanting to play nice with #4260 ​https://github.com/cschwan/sage-on-gentoo/issues/108 and it annoys me.

Does vanilla Sage build and run on that same machine with the same compiler? Be sure to try a different gcc version, Linbox seems to be sensitive to that.

comment:182 Changed 8 years ago by strogdon

On x86 with sage-4.8.alpha3 + #9958 + #11986 I have the following failures: 

sage -t -long  -force_lib devel/sage-main/sage/gsl/integration.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 110:
    sage: numerical_integral(x^2, 0, 1, max_points=100)
Expected:
    (0.33333333333333331, 3.7007434154171879e-15)
Got:
    (0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 115:
    sage: numerical_integral(sin(x)^3 + sin(x),  0, pi)
Expected:
    (3.333333333333333, 3.7007434154171883e-14)
Got:
    (3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 120:
    sage: numerical_integral(lambda x: sin(x)^3 + sin(x),  0, pi)
Expected:
    (3.333333333333333, 3.7007434154171883e-14)
Got:
    (3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 131:
    sage: numerical_integral(f, 0, 1, max_points=200, eps_abs=1e-7, eps_rel=1e-7, rule=4)
Expected:
    (0.33333333333333331, 3.7007434154171879e-15)
Got:
    (0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 172:
    sage: exp(-1/x).nintegral(x, 1, 2)  # via maxima
Expected:
    (0.50479221787318396, 5.6043194293440752e-15, 21, 0)
Got:
    (0.504792217873184, 5.604319429344075e-15, 21, 0)
**********************************************************************

sage -t -long  -force_lib devel/sage-main/sage/misc/randstate.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/misc/randstate.pyx", line 61:
    sage: rtest()
Expected:
    (978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.83350776541997362)
Got:
    (978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.8335077654199736)
**********************************************************************

sage -t -long  -force_lib devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 790:
    sage: float(A.density())
Expected:
    0.099738...
Got:
    0.099739
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 798:
    sage: float(A.density())
Expected:
    0.499759...
Got:
    0.49976
**********************************************************************

sage -t -long  -force_lib devel/sage-main/sage/functions/transcendental.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/functions/transcendental.py", line 80:
    sage: w = exponential_integral_1(2,4); w
Expected:
    [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843924751e-05]
Got:
    [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.766562284392475e-05]
**********************************************************************

comment:183 Changed 8 years ago by fbissey

I now understand why you removed the transcendental patch Jereon some bits ended in my revised trac_9958-fixing_numericalnoise-part1_p2.patch when they shouldn't have. That's what is happening to Steve. I will rectify that.

I removed the gsl bits because of #12047 so that has to be redone.

And I haven't had time to try vanilla yet but it is on my TODO list and I suspect g++ too.

Changed 8 years ago by fbissey

fixing numerical noise part 1 - updated for 4.8.alpha3

comment:184 Changed 8 years ago by fbissey

  • Description modified (diff)

updated the first patch and readded the transcendental patch to the list.

comment:185 Changed 8 years ago by fbissey

For suffix_tree it looks like your two values don't agree: 

{(4, word: a): 1, (0, word: b): 5, (0, word: a): 3, (5, word: a): 2, (3, word: b): 4}
{(4, word: b): 5, (0, word: a): 4, (0, word: b): 3, (5, word: a): 1, (3, word: a): 2}

If we add that I don't see it on OS X x86 that looks bad. I also have an x86_64 machine where the value is the same as Jeroen and the test fails. Steve get the right value on the other hand so there is something going there.

Changed 8 years ago by fbissey

fix numerical noise in gsl/integration.pyx

Changed 8 years ago by fbissey

fix numerical noise introduced by m4rie

comment:186 Changed 8 years ago by fbissey

  • Description modified (diff)

Added two new patches for gsl integration and m4rie tickets merged in 4.8.alpha3.

comment:187 Changed 8 years ago by fbissey

  • Description modified (diff)

Rebased the spkg against #12096.

Changed 8 years ago by jdemeyer

Diff for the Python spkg, for review only

comment:188 Changed 8 years ago by jdemeyer

Everything passes against sage-4.8.alpha3 on 64bit. Yay!

comment:189 follow-up: Changed 8 years ago by fbissey

I forgot to add a new bit for randstate.pyx on 32 bit. I'll do that later but I know it is missing. Should we involve someone from combinatronics to get to the bottom of suffix_tree and nfactor_enumerate_word?

comment:190 in reply to: ↑ 189 Changed 8 years ago by jdemeyer

Replying to fbissey:

Should we involve someone from combinatronics to get to the bottom of suffix_tree and nfactor_enumerate_word?

Sounds good, try to send an email to sage-devel.

comment:191 follow-up: Changed 8 years ago by strogdon

For the sake of completeness, I now have

32bit: one failure 

sage -t -long  -force_lib devel/sage-main/sage/misc/randstate.pyx

which was mentioned above.

64bit: two failures 

sage -t -long  -force_lib devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 20:
    sage: it.next()
Expected:
    word: 5645
Got:
    word: 4564
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 23:
    sage: it.next()
Expected:
    word: 6456
Got:
    word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 26:
    sage: it.next()
Expected:
    word: 4564
Got:
    word: 6456
**********************************************************************


sage -t -long  -force_lib devel/sage-main/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/suffix_trees.py",
line 1323:
    sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2]
, [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
    True
Got:
    False
**********************************************************************

which have been mentioned previously.

comment:192 Changed 8 years ago by jdemeyer

This ticket made me notice a bug with the L-series of Eisenstein series: #12124. With that ticket applied, the patch to sage/modular/modform/eis_series.py here can be removed.

comment:193 in reply to: ↑ 191 Changed 8 years ago by hivert

First of all, I'm not an expert of combinatorics on word. So I'd rather Sebastien Labbe to jump in the discussion. I just trying to help.

64bit: two failures 

sage -t -long  -force_lib devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 20:
    sage: it.next()
Expected:
    word: 5645
Got:
    word: 4564
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 23:
    sage: it.next()
Expected:
    word: 6456
Got:
    word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 26:
    sage: it.next()
Expected:
    word: 4564
Got:
    word: 6456
**********************************************************************

This first failure is not a problem. The iterator here is extracted from a dict of dict so it is not unexpected to have a random order. Fixing the test of marking it at random should be ok. This test is an explanation for the user, the real feature is tested further in the file.

which have been mentioned previously. 

{(4, word: a): 1, (0, word: b): 5, (0, word: a): 3, (5, word: a): 2, (3, word: b): 4}
{(4, word: b): 5, (0, word: a): 4, (0, word: b): 3, (5, word: a): 1, (3, word: a): 2}

This one is more tricky: As far as I understand, both result are perfectly legible: the second is the same as the first after applying the following cyclic permutation (3,4,5). The numbering is random. So (again as far as I understand), the problem in only created by the cross-platform non-determinism of the set/dict data structure. I'm not sure here what is the best fix.

Cheers,

Florent

comment:194 Changed 8 years ago by fbissey

Thank you Florent, that is helpful. We kind of worked out that word was OK but we thought we could have a better test. If you think that's ok to mark it random I think we'll do that with some explanation.

The fact that both output in suffix_tree are legit is good to know. We can probably work around that. We may even have a complete patch set by Monday, cross fingers.

comment:195 Changed 8 years ago by jdemeyer

  • Description modified (diff)

Upgraded succesfully from sage-4.6.2, all tests pass except for the known problematic tests.

comment:196 follow-up: Changed 8 years ago by fbissey

I'll work on the remaining tests tomorrow, I hope. I will remove the eis_series.py and see if I can review #12124 as well. I looks like the python spkg will need rebasing again with #12131.

comment:197 in reply to: ↑ 196 Changed 8 years ago by jdemeyer

Replying to fbissey:

I'll work on the remaining tests tomorrow, I hope.

That would be great! I would love to see this finished in time, such that the next release can really be sage-5.0.

Changed 8 years ago by fbissey

test against more valid dictionaries in suffix_trees

Changed 8 years ago by fbissey

fix all numerical noise in randstate

Changed 8 years ago by fbissey

fixing numerical noise part 1 - updated 20111211: separate randstate and eis_modn_series

comment:198 Changed 8 years ago by fbissey

  • Description modified (diff)
  • Status changed from needs_work to needs_review

updating the patch set with a tentative solution for suffix_trees. It may not be exhaustive of all possible answers though. If we are going for it for the next release I can redo the python spkg to fix the lib/lib64 from #12131.

Putting to need review :)

comment:199 Changed 8 years ago by slabbe

Hi,

I just read randomly sage-devel (I was away from sage development for several weeks) and saw that the discussion about upgrading to 2.7 was having problem with some files I am related with (suffix_tree.py nfactor_enumerable_word.py). I still don't understand the dict problem for suffix tree (one minute was not enough). I will look into that to avoid similar problem in the future. If something else pops up in those file, I can do the fix next time.

Sébastien

comment:200 Changed 8 years ago by fbissey

Your input on the matter is welcome. According to Florent, the problem in suffix_trees is that there are several equivalent answers and some platforms now give a different one than the one you originally tested for. Note that we don't know what the trigger is for having a different answer it is not a 32/64 bit separation as we have seen both answers on the two kinds of systems.

comment:201 follow-up: Changed 8 years ago by fbissey

Actually, you should have a look at what I did for both of these in trac_9958-nfactor_enumerable_word-randomness.patch​ and trac_9958-suffix_trees-variations.patch​. If you could give your opinion on the patches or if you want to submit patches of your own it would be appreciated.

comment:202 Changed 8 years ago by strogdon

  • Description modified (diff)

With the current update I have the following failures: 

32bit
sage -t -long  -force_lib devel/sage-9958/sage/combinat/words/nfactor_enumerable_word
.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 23:
    sage: it.next()
Expected:
    word: 6456     # random
Got:
    word: 6456
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 25:
    sage: it.next()
Expected:
    word: 5645     # random
Got:
    word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 27:
    sage: it.next()
Expected:
    word: 4564     # random
Got:
    word: 4564
**********************************************************************

64bit

sage -t -long  -force_lib devel/sage-9958/sage/combinat/words/nfactor_enumerable_word
.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 23:
    sage: it.next()
Expected:
    word: 6456     # random
Got:
    word: 4564
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 25:
    sage: it.next()
Expected:
    word: 5645     # random
Got:
    word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 27:
    sage: it.next()
Expected:
    word: 4564     # random
Got:
    word: 6456
**********************************************************************

and on 32bit and 64bit 

sage -t -long  -force_lib devel/sage-9958/sage/misc/randstate.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/misc/randstate.pyx", line 15
3:
    sage: random(), getrandbits(20), uniform(5.0, 10.0), normalvariate(0, 1)
Expected:
    (0.82940228518742587, 624859L, 5.77894484361117, -0.42013668263087578)
Got:
    (0.8294022851874259, 624859L, 5.77894484361117, -0.4201366826308758)
**********************************************************************

I don't recall previously seeing this particular randstate.pyx failure. Perhaps I've missed something.

comment:203 Changed 8 years ago by jdemeyer

  • Status changed from needs_review to needs_work

Yeah, the #random should be on the line with the command to execute, not the result.

comment:204 Changed 8 years ago by fbissey

I had to make that mistake :(

The last randstate.pyx one is definitely new, I am sure it didn't happen before.

Changed 8 years ago by fbissey

take into account the randomness of it.next sequence of call - put random in the right place this time.

comment:205 Changed 8 years ago by fbissey

OK I replaced the trac_9958-nfactor_enumerable_word-randomness.patch in place. So random should be in the right place now. I am looking to see if I can reproduce this particular randstate.pyx problem.

comment:206 Changed 8 years ago by fbissey

I missed a line in randstate! And you cut the last digit of the line number Steve, now that caused me some confusion. Updating shortly.

Changed 8 years ago by fbissey

fix all numerical noise in randstate

comment:207 Changed 8 years ago by fbissey

  • Description modified (diff)
  • Status changed from needs_work to needs_review

OK new randstate patch, this time it is even a .patch instead of .pyx. Thanks for pointing out those mistakes.

comment:208 Changed 8 years ago by strogdon

OK, #9958 + #11986 + #12124, all tests pass here on 32bit and 64bit machines. Is it too early for champagne?

comment:209 follow-up: Changed 8 years ago by fbissey

Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order. I removed a patch for ia64 that didn't obviously port to python-2.7 so a look at that platform is also in order. In the mean time we can review #11986 and #12124 which I think can be done independently.

We could even split the combinat patches as they should work with python-2.6. It just there is nothing obvious to fix there.

comment:210 Changed 8 years ago by jdemeyer

As I have said before, testing is easy. I am currently running tests for the upcoming sage-4.8.alpha4, but after that, I could test the new Python on various systems.

But still, somebody needs to actually look at the patches and verify that they make sense.

comment:211 Changed 8 years ago by fbissey

OK Jeroen, that's a boring task for someone most of them are of the form: 

diff --git a/sage/misc/sage_unittest.py b/sage/misc/sage_unittest.py
--- a/sage/misc/sage_unittest.py
+++ b/sage/misc/sage_unittest.py
@@ -201,12 +201,12 @@
             Failure in _test_b:
             Traceback (most recent call last):
               ...
-            AssertionError
+            AssertionError: None
             ------------------------------------------------------------
             Failure in _test_d:
             Traceback (most recent call last):
               ...
-            AssertionError
+            AssertionError: None
             ------------------------------------------------------------
             Failure in _test_pickling:
             Traceback (most recent call last):
@@ -220,14 +220,14 @@
             running ._test_b() . . . fail
             Traceback (most recent call last):
               ...
-            AssertionError
+            AssertionError: None
             ------------------------------------------------------------
             running ._test_c() . . . pass
             running ._test_category() . . . pass
             running ._test_d() . . . fail
             Traceback (most recent call last):
               ...
-            AssertionError
+            AssertionError: None
             ------------------------------------------------------------
             running ._test_not_implemented_methods() . . . pass
             running ._test_pickling() . . . fail
@@ -249,7 +249,7 @@
               File ..., in _test_b
                 def _test_b(self, tester): tester.fail()
               ...
-            AssertionError
+            AssertionError: None
 
         In conjonction with ``%pdb on``, this allows for the debbuger
         to jump directly to the first failure location.
@@ -311,7 +311,7 @@
         sage: tester.assert_(1 == 0)
         Traceback (most recent call last):
         ...
-        AssertionError
+        AssertionError: False is not true
         sage: tester.assert_(1 == 0, "this is expected to fail")
         Traceback (most recent call last):
         ...

and other numerical noise niceties cutting one digit and sometimes the rounding makes a lot of digits disappear (and it looks better actually), for example in plot/colors.py 

@@ -290,23 +290,23 @@
 
         sage: from sage.plot.colors import rgbcolor
         sage: rgbcolor(Color(0.25, 0.4, 0.9))
-        (0.25, 0.40000000000000002, 0.90000000000000002)
+        (0.25, 0.4, 0.9)
         sage: rgbcolor('purple')

Only the two combinat patches are a bit more subtle.

comment:212 in reply to: ↑ 209 ; follow-up: Changed 8 years ago by kcrisman

Replying to fbissey:

Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order.

Is this all based on 4.8.alpha3? I can try to do this starting today. Jeroen might be done before me, though :) at least by the time I finish applying ALL of these patches.

comment:213 Changed 8 years ago by fbissey

It should apply to 4.8.alpha3, apply #11986 + #12124 also.

comment:214 in reply to: ↑ 212 ; follow-up: Changed 8 years ago by jdemeyer

Replying to kcrisman:

Replying to fbissey:

Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order.

Is this all based on 4.8.alpha3? I can try to do this starting today. Jeroen might be done before me, though :)

Go ahead, I'm currently tackling other beasts...

comment:215 Changed 8 years ago by jdemeyer

  • Dependencies changed from #5852, #11986, #12085, #12096 to #5852, #11986, #12085, #12096, #12124

comment:216 in reply to: ↑ 201 ; follow-up: Changed 8 years ago by slabbe

Replying to fbissey:

Actually, you should have a look at what I did for both of these in trac_9958-nfactor_enumerable_word-randomness.patch​ and trac_9958-suffix_trees-variations.patch​. If you could give your opinion on the patches or if you want to submit patches of your own it would be appreciated.

Hi,

I just look at the suffix tree code (written not by me but by Franco Saliola). The method trie_type_dict is not used anywhere in the file neither in sage/combinat/words neither in all Sage I would guess. So, it's kind of difficult to guess what the result should be (input and output blocks are absent in documentation). I think that method was used by Franco to compare the implementation of ImplicitSuffixTree with that of SuffixTrie but not seriously because the output is not practical (one can't create a DiGraph out of it).

Anyway, there are two loops in that method that both go through .iteritems() of a dictionary... Integer indexes present in the result depends on the ordering of those two loops. So, it is possible that a third different result is obtained in some other machine or system today or later. I just made a patch which should be more durable in the long term.

For the nfactor enumerable file, the fix is perfect.

Changed 8 years ago by slabbe

Fix machine dependant doctest in suffix tree

comment:217 in reply to: ↑ 214 Changed 8 years ago by kcrisman

Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order.

Is this all based on 4.8.alpha3? I can try to do this starting today. Jeroen might be done before me, though :)

Go ahead, I'm currently tackling other beasts...

Okay, at least everything applies and builds from this and the two other tickets. (Note that #11986 is listed as a dependency of and for #9958.)

So presumably sometime tomorrow I can report back. So far six files having tested well...

comment:218 in reply to: ↑ 216 Changed 8 years ago by fbissey

Replying to slabbe:

Anyway, there are two loops in that method that both go through .iteritems() of a dictionary... Integer indexes present in the result depends on the ordering of those two loops. So, it is possible that a third different result is obtained in some other machine or system today or later. I just made a patch which should be more durable in the long term.

For the nfactor enumerable file, the fix is perfect.

Thanks. I thought there could be a third possible results from Florent's comments (permutation (3,4,5)) but couldn't figure what it would look like. Happy to have some serious review for that bit.

comment:219 Changed 8 years ago by fbissey

OK the upgrade of R in #12057 shipped with 4.8.alpha4 makes a small bit of patch for r.py obsolete. Updating it shortly.

Changed 8 years ago by fbissey

fixing numerical noise part 1 - updated 20111214 for 4.8.alpha4

comment:220 Changed 8 years ago by fbissey

  • Description modified (diff)

First patch updated. I don't think there will be any more changes because of 4.8.alpha4 but I could be wrong.

comment:221 Changed 8 years ago by strogdon

It looks like a slight modification is still needed in interfaces/r.py. On both x86 and amd64 I get 

sage -t -long  -force_lib devel/sage-main/sage/interfaces/r.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha4/devel/sage-main/sage/interfaces/r.py", line 173:
    sage: sum(rr._sage_())
Expected:
    9.97721251689810...
Got:
    9.97721251689811
**********************************************************************

attachment:trac_9958-suffix_trees-variations-sl.patch:ticket:9958​ was used in the tests.

comment:222 Changed 8 years ago by fbissey

I have seen it on my sage-on-gentoo test run but I hadn't tried yet on a vanilla to check if it was caused by other gentoo stuff. I will add a small patch ASAP.

Changed 8 years ago by fbissey

numerical noise in interface/r.py updated for R-2.14.0

comment:223 Changed 8 years ago by fbissey

  • Description modified (diff)

New patch included for interface/r.py in 4.8.alpha4.

comment:224 follow-up: Changed 8 years ago by jason

How do we test this ticket? Is it enough to untar a fresh tarball of the 4.8.alpha4 source, replace the existing python spkg with the one from this ticket, and do a normal make? Then apply the patches to the sage library and run doctests?

On OSX 10.6.8, I tried downloading a fresh copy of 4.8.alpha4, putting this python spkg in the spkg/standard directory and deleting the old python spkg. I ended up getting a build error, seemingly right after I finished building scipy (even though scipy said it was built succesfully, but I was also building spkgs in parallel). I typed "make" again to restart the process and the same thing happened after rpy built---it said rpy was built successfully, but then it stopped the build, saying "Error building Sage".

comment:225 in reply to: ↑ 224 Changed 8 years ago by fbissey

Replying to jason:

How do we test this ticket? Is it enough to untar a fresh tarball of the 4.8.alpha4 source, replace the existing python spkg with the one from this ticket, and do a normal make? Then apply the patches to the sage library and run doctests?

Yes that's how you should do it.

On OSX 10.6.8, I tried downloading a fresh copy of 4.8.alpha4, putting this python spkg in the spkg/standard directory and deleting the old python spkg. I ended up getting a build error, seemingly right after I finished building scipy (even though scipy said it was built succesfully, but I was also building spkgs in parallel). I typed "make" again to restart the process and the same thing happened after rpy built---it said rpy was built successfully, but then it stopped the build, saying "Error building Sage".

Could you post the log for the build of python somewhere? scipy and rpy are built after sage (or at least they can be since they are runtime and not buildtime dependencies) so it is possible you were building sage itself at the same time. Is there a log for the sage spkg (in spkg/log)?

comment:226 Changed 8 years ago by jason

I posted the install.log here: ​http://sage.math.washington.edu/home/jason/sage-4.8.alpha4-python2.7-install.log.bz2

If you search for "Error building Sage", you'll see the multiple times I restarted the build (that message appeared after each time I restarted the build).

Here's the list of spkg logs I have: 

sage-4.8.alpha4newpython/spkg/logs% ls
atlas-3.8.4.log                gdmodule-0.56.p7.log           moin-1.9.1.p2.log              ratpoints-2.1.3.p1.log
blas-20070724.log              genus2reduction-0.3.p8.log     mpfi-1.3.4-cvs20071125.p8.log  readline-6.2.p1.log
boehm_gc-7.2.alpha6.p1.log     gfan-0.4plus.p1.log            mpfr-2.4.2.log                 rpy2-2.0.8.log
boost-cropped-1.34.1.log       givaro-3.2.13rc2.p2.log        mpir-2.1.3.p8.log              rubiks-20070912.p17.log
bzip2-1.0.5.log                glpk-4.44.log                  mpmath-0.17.log                sage_root-4.8.alpha4.log
cddlib-094f.p8.log             gnutls-2.2.1.p5.log            networkx-1.2.p1.log            sage_scripts-4.8.alpha4.log
cephes-2.8.log                 graphs-20070722.p1.log         ntl-5.5.2.log                  sagenb-0.8.25.log
cliquer-1.2.p10.log            gsl-1.15.log                   numpy-1.5.1.log                scipy-0.9.p1.log
conway_polynomials-0.2.log     iconv-1.13.1.p3.log            opencdk-0.6.6.p5.log           scons-1.2.0.log
cvxopt-1.1.3.log               iml-1.0.1.p13.log              palp-1.1.p3.log                setuptools-0.6.16.log
cython-0.15.1.log              ipython-0.10.2.p0.log          pari-2.5.0.p2.log              singular-3-1-3-3.p2.log
dir-0.1.log                    jinja2-2.5.5.log               patch-2.5.9.p2.log             sphinx-1.1.2.p0.log
docutils-0.7.p0.log            lapack-20071123.p2.log         pexpect-2.0.p4.log             sqlalchemy-0.5.8.log
ecl-11.1.2.cvs20111120.p0.log  lcalc-1.23.p9.log              pil-1.1.6.p4.log               sqlite-3.7.5.log
eclib-20100711.p0.log          libfplll-3.0.12.p1.log         polybori-0.7.1.p6.log          symmetrica-2.0.p7.log
ecm-6.3.p2.log                 libgcrypt-1.4.4.p3.log         polytopes_db-20100210.log      sympow-1.018.1.p9.log
elliptic_curves-0.3.log        libgpg_error-1.6.p3.log        ppl-0.11.2.log                 sympy-0.7.1.log
extcode-4.8.alpha4.log         libm4ri-20111004.log           prereq-0.9.log                 tachyon-0.98.9.p5.log
f2c-20070816.p2.log            libm4rie-20111004.log          pycrypto-2.1.0.log             termcap-1.3.1.p1.log
flint-1.5.0.p10.log            libpng-1.2.35.p3.log           pygments-1.3.1.p0.log          twisted-9.0.p2.log
flintqs-20070817.p6.log        linbox-1.1.6.p5.log            pynac-0.2.3.log                zlib-1.2.5.p0.log
fortran-20100629.log           matplotlib-1.0.1.p0.log        python-2.7.2.p0.log            zn_poly-0.9.p5.log
freetype-2.3.5.p3.log          maxima-5.23.2.p2.log           python_gnutls-1.1.4.p7.log     zodb3-3.7.0.p4.log
gd-2.0.35.p5.log               mercurial-1.8.4.log            r-2.14.0.p0.log

comment:227 Changed 8 years ago by kcrisman

All is well on the PPC OS X front, even with the r.py, amazingly; it failed the first time but passed on a rerun.

comment:228 Changed 8 years ago by fbissey

Jason the log you sent is not very useful, at least I cannot easily find something interesting. Could you provide the following:

  • python-2.7.2.p0.log
  • polybori-0.7.1.p6.log
  • ppl-0.11.2.log
  • pynac-0.2.3.log

That's a shot in the dark so far. But the building of sage never started as there is no log for sage itself.

I digged a little bit more before posting it looks like R fails to build so I want to see r-2.14.0.p0.log but that may be a separate issue from python. matplotlib seems to fail to build too so I want matplotlib-1.0.1.p0.log and now we are talking real potential python trouble.

comment:229 Changed 8 years ago by kcrisman

I'm going to try this with 

export MAKE="make -j2"

on the same platform and report back.

comment:230 Changed 8 years ago by jason

First, here are my standard build environment settings: 

export SAGE_MATPLOTLIB_GUI=yes
export SAGE_PARALLEL_SPKG_BUILD=yes     
export MAKE="make -j6"    
export SAGE_SPKG_INSTALL_DOCS=yes

All the requested logs are tarred up at ​http://sage.math.washington.edu/home/jason/newpython-spkg-logs.tar.gz

comment:231 Changed 8 years ago by jason

Tonight, I'll try building again without setting any of those environment variables to see if it succeeds with a "stock" build.

comment:232 Changed 8 years ago by kcrisman

Replying to kcrisman:

I'm going to try this with 

> export MAKE="make -j2"

on the same platform and report back.

Interestingly, Scipy and R are definitely (right now) building before Sage. And they usually do not, for sure not Scipy, on my ancient older Mac.

Scipy has successfully installed, as has matplotlib. R takes forever but so far is fine. Maybe it was "just a race condition"?

comment:233 Changed 8 years ago by jason

I bet the matplotlib error shows up for me because I define SAGE_MATPLOTLIB_GUI (so that matplotlib tries to build the GUI backends).

comment:234 Changed 8 years ago by fbissey

It is quite possibly something broken with the GUI from what I can see of your original log. scipy can build whenever after numpy independently of sage so I don't see a problem there.

comment:235 Changed 8 years ago by fbissey

Yes I think matplotlib cannot find tk. We may want to look at this in depth on OS X.

comment:236 Changed 8 years ago by jason

The matplotlib error is fixed upstream: ​https://github.com/matplotlib/matplotlib/issues/139

comment:237 Changed 8 years ago by fbissey

Very good, we may want to bump the spkg to 1.1.0 then. From my sage-on-gentoo experience there is just one minor doctest failure from matplotli-1.1.0 

sage -t -long -force_lib "devel/sage/sage/plot/colors.py"   
**********************************************************************
File "/usr/share/sage/devel/sage/sage/plot/colors.py", line 1241:
    sage: len(maps.maps)
Expected:
    134
Got:
    138
**********************************************************************
File "/usr/share/sage/devel/sage/sage/plot/colors.py", line 1285:
    sage: len(maps)
Expected:
    134
Got:
    138
**********************************************************************

Fairly standard each time we bump matplotlib or almost.

You R package definitely broke the first time around. I am fairly sure this is a parallel make issue. I have seen at least one instance that I suspect is parallel make on Gentoo. Of course given that it stop because of 

make[6]: gcc: Resource temporarily unavailable

I think, we may have something else altogether. It should go into a separate ticket.

comment:238 follow-up: Changed 8 years ago by jason

Do you already have a 1.1.0 spkg? I'm working on one right now, but you already have one...

comment:239 Changed 8 years ago by kcrisman

Scipy has successfully installed, as has matplotlib. R takes forever but so far is fine. Maybe it was "just a race condition"?

Everything built before the Sage library except Gap and SageTeX. No problems. I say this shouldn't keep the Python upgrade out, or?

comment:240 in reply to: ↑ 238 ; follow-up: Changed 8 years ago by fbissey

Replying to jason:

Do you already have a 1.1.0 spkg? I'm working on one right now, but you already have one...

I don't have one. I don't need one for sage-on-gentoo as I am using system packages. You go ahead and create a ticket to upgrade matplotlib. I say it is a show stopper because it would happen on other platforms. Hopefully we can review it quickly.

comment:241 in reply to: ↑ 240 Changed 8 years ago by kcrisman

Replying to fbissey:

Replying to jason:

Do you already have a 1.1.0 spkg? I'm working on one right now, but you already have one...

I don't have one. I don't need one for sage-on-gentoo as I am using system packages. You go ahead and create a ticket to upgrade matplotlib. I say it is a show stopper because it would happen on other platforms. Hopefully we can review it quickly.

What about just making a patch upgrade with ​the 1.5-liner rather than making a whole new package which would require even MORE testing? Plus, then you don't have to worry about getting those four extra colormaps :) Anyway, I feel this is easier and more reasonable just to get this in 5.0.alpha0 (or whatever release is next for such a big spkg change as this ticket).

comment:242 Changed 8 years ago by kcrisman

  • Authors changed from François Bissey to François Bissey, Steven Trogdon
  • Reviewers set to John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé

comment:243 Changed 8 years ago by fbissey

I have no objections to that. Faster is better in that case because we really want to land this quickly because every other moves of the tree makes me rewrite stuff. It start to be hard to manage.

comment:244 Changed 8 years ago by fbissey

  • Authors changed from François Bissey, Steven Trogdon to François Bissey, Steven Trogdon, Sébastien Labbé

comment:245 Changed 8 years ago by jason

I added a link to a patched version of the current matplotlib spkg. I also added the diff as an information-only patch above. Independent from this fix, I've also made a matplotlib 1.1.0 spkg, which also fixes the problem, but may need more testing than this very slight change to the existing matplotlib spkg.

comment:246 Changed 8 years ago by jason

  • Description modified (diff)

comment:247 Changed 8 years ago by jason

  • Description modified (diff)

comment:248 Changed 8 years ago by jason

By the way, the matplotlib 1.1.0 spkg is at #11915, but the spkg here should be sufficient.

comment:249 Changed 8 years ago by fbissey

#12131 has been merged, I'll rebase the python spkg by next Monday hopefully.

comment:250 Changed 8 years ago by jason

Using the new matplotlib-1.0.1.p1.spkg above, and my standard build environment variables: 

export SAGE_MATPLOTLIB_GUI=yes
export SAGE_PARALLEL_SPKG_BUILD=yes     
export MAKE="make -j6"    
export SAGE_SPKG_INSTALL_DOCS=yes

Everything except R builds nicely. R still has that message about resources not being available (something about fork). I then changed MAKE to "make -j2" and restarted the make and everything finished just fine. Again, on OSX 10.6.8 (4G of memory).

That's a bummer that make -j6 doesn't work. Regular old 4.8.alpha4 built fine, so I don't think it's just the new R spkg. I'll try building fresh one more time, but with MAKE=make -j4 just to check things out.

comment:251 Changed 8 years ago by jason

  • Description modified (diff)

The matplotlib spkg at #11915 is ready for review (and depends on the #11976, also ready for review). Instead of using the matplotlib-1.0.1.p1.spkg above, it would be preferable to review and merge #11976 and #11915.

comment:252 Changed 8 years ago by jhpalmieri

Okay, I started with 4.8.alpha4, added the new python spkg from this ticket, the new matplotlib spkg from #11915, applied all of the patches from this ticket, #11976, #11986, and #12124. I have a tarball with all of these:

​http://sage.math.washington.edu/home/palmieri/misc/9958/sage-4.8.alpha4-9958.tar

along with an upgrade path:

​http://sage.math.washington.edu/home/palmieri/misc/9958/sage-4.8.alpha4-9958/

for testing.

comment:253 Changed 8 years ago by jason

Did you also apply the patch from #11915?

comment:254 Changed 8 years ago by jhpalmieri

No, looks like I missed that one.

comment:256 Changed 8 years ago by jdemeyer

  • Description modified (diff)

comment:257 Changed 8 years ago by jason

I took jhpalmieri's original tarball, applied the patch from #11915 myself, built it with these settings: 

SAGE_MATPLOTLIB_GUI=yes
SAGE_PARALLEL_SPKG_BUILD=yes
MAKE=make -j12
SAGE_SPKG_INSTALL_DOCS=yes

on Ubuntu 10.04. Everything built fine, and make ptestlong passed all tests. Yeah!

comment:258 follow-up: Changed 8 years ago by kcrisman

What remains to be done here for positive review?

comment:259 in reply to: ↑ 258 Changed 8 years ago by fbissey

Replying to kcrisman:

What remains to be done here for positive review?

First I have to rebase the python spkg to #12131. Second there is a new matplotlib spkg currently merged in 4.8.alpha5 (unreleased) that needs review in #11915. Then someone has to look at the patches to make sure they are not completely stupid (long tiedous job).

Then we can move this to a positive review. I was hoping to be able to to rebase the spkg on Sunday but it didn't happen. I am busy doing a bit of code for my paid work but once it is done. I'll go for it, that may be tomorrow in my time zone.

comment:260 Changed 8 years ago by fbissey

python-2.7.2.p0 spkg rebased and available.

comment:261 Changed 8 years ago by fbissey

#11986 also needs review. It looks like it may need to go at the same time as python 2.7 as my own testing with python 2.6 on linux amd64 was not positive but everything is smooth with python 2.7.

comment:262 follow-up: Changed 8 years ago by strogdon

For sage-4.8.alpha4 with tickets #9958 + #11986 + #12124 + #11976 + #11915 all tests pass on x86 and on amd64 there is one TIME OUT 

sage -t -long  -force_lib devel/sage-matplotlib-1.1.0/sage/interfaces/ecm.py
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***

         [1800.2 s]

The test did pass when run individually. I'm sure this is due to running the tests in parallel. I've seen it before, not with this ticket, but with sage-on-gentoo builds.

comment:263 in reply to: ↑ 262 ; follow-up: Changed 8 years ago by fbissey

Replying to strogdon:

For sage-4.8.alpha4 with tickets #9958 + #11986 + #12124 + #11976 + #11915 all tests pass on x86 and on amd64 there is one TIME OUT 

sage -t -long  -force_lib devel/sage-matplotlib-1.1.0/sage/interfaces/ecm.py
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***

         [1800.2 s]

The test did pass when run individually. I'm sure this is due to running the tests in parallel. I've seen it before, not with this ticket, but with sage-on-gentoo builds.

I have seen it before too - in parallel tests as well. It seems that ecm can be a bad boy, not sure what to do about it but that would be outside of the scope of this ticket in my opinion.

Steve, did you do any test of #11986 with python-2.6?

comment:264 in reply to: ↑ 263 Changed 8 years ago by strogdon

Replying to fbissey:

Steve, did you do any test of #11986 with python-2.6?

No, I haven't done that with python-2.6. I will do it, but hopefully someone beats me to it.

comment:265 follow-up: Changed 8 years ago by jdemeyer

Upgrading from sage-4.5 fails with: 

Testing that Sage starts...
[2012-01-03 13:47:34]
Traceback (most recent call last):
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/bin/sage-eval", line 4, in <module>
    from sage.all import *
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/all.py", line 75, in <module>
    from sage.matrix.all     import *
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/all.py", line 1, in <module>
    from matrix_space import MatrixSpace, is_MatrixSpace
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 37, in <module>
    import matrix_modn_dense
  File "matrix_modn_dense.pyx", line 1, in init sage.matrix.matrix_modn_dense (sage/matrix/matrix_modn_dense.c:15026)
AttributeError: 'module' object has no attribute 'Matrix_dense'
Sage failed to start up.
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and send the log file
  /mnt/usb1/scratch/jdemeyer/sage-4.5-python27/start.log
Describe your computer, operating system, etc.

After sage -ba-force, it works again.

comment:266 in reply to: ↑ 265 ; follow-up: Changed 8 years ago by fbissey

Replying to jdemeyer:

Upgrading from sage-4.5 fails with: 

Testing that Sage starts...
[2012-01-03 13:47:34]
Traceback (most recent call last):
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/bin/sage-eval", line 4, in <module>
    from sage.all import *
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/all.py", line 75, in <module>
    from sage.matrix.all     import *
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/all.py", line 1, in <module>
    from matrix_space import MatrixSpace, is_MatrixSpace
  File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 37, in <module>
    import matrix_modn_dense
  File "matrix_modn_dense.pyx", line 1, in init sage.matrix.matrix_modn_dense (sage/matrix/matrix_modn_dense.c:15026)
AttributeError: 'module' object has no attribute 'Matrix_dense'
Sage failed to start up.
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and send the log file
  /mnt/usb1/scratch/jdemeyer/sage-4.5-python27/start.log
Describe your computer, operating system, etc.

After sage -ba-force, it works again.

Quite weird. You would think those particular libraries would have been rebuilt already after the linbox upgrade at least. Shouldn't a complete rebuild of sage occur after a python upgrade? In any case that seems mild for upgrading from 4.5 which was released on June 2010 many versions away.

comment:267 in reply to: ↑ 266 Changed 8 years ago by jdemeyer

  • Status changed from needs_review to needs_work
  • Work issues set to Fix upgrading

Replying to fbissey:

Shouldn't a complete rebuild of sage occur after a python upgrade?

I guess it should. Do you know how to make this happen?

comment:268 Changed 8 years ago by jdemeyer

  • Description modified (diff)

Changed 8 years ago by jdemeyer

Diff for the Python spkg (without deleted files), for review only

comment:269 Changed 8 years ago by jdemeyer

  • Description modified (diff)

comment:270 Changed 8 years ago by jdemeyer

  • Dependencies changed from #5852, #11986, #12085, #12096, #12124 to #5852, #11986, #12085, #12096, #12124, #11915
  • Description modified (diff)
  • Status changed from needs_work to needs_review
  • Work issues Fix upgrading deleted

Upgrading fixed by deleting 

$SAGE_ROOT/devel/sage-*/build

when an old version of Python is detected.

New spkg: ​http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p1.spkg

Changed 8 years ago by jdemeyer

Diff for the Python spkg 2.7.2.p0 -> 2.7.2.p1, for review only

comment:271 Changed 8 years ago by jdemeyer

  • Authors changed from François Bissey, Steven Trogdon, Sébastien Labbé to François Bissey, Steven Trogdon, Sébastien Labbé, Jeroen Demeyer

positive_review to the spkg, modulo my changes in the 2.7.2.p1 package.

comment:272 follow-up: Changed 8 years ago by fbissey

Looks good to me. I didn't notice I missed a patch in the stuff I removed. Anyway I'll be on low activity because my home computer has given up the ghost (writing from an old box cobbled together running puppy 4.00 from a _1_GB hard drive) and I am on leave.

Changed 8 years ago by jdemeyer

Changed 8 years ago by jdemeyer

Changed 8 years ago by jdemeyer

comment:273 Changed 8 years ago by jdemeyer

  • Description modified (diff)

Folded all the patches into only four patches.

comment:274 in reply to: ↑ 272 ; follow-up: Changed 8 years ago by jdemeyer

Replying to fbissey:

Looks good to me.

Meaning "positive review"? :-)

comment:275 in reply to: ↑ 274 Changed 8 years ago by fbissey

  • Reviewers changed from John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé to John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé, François Bissey
  • Status changed from needs_review to positive_review

Replying to jdemeyer:

Replying to fbissey:

Looks good to me.

Meaning "positive review"? :-)

Absolutely positive! Hopefully nothing will happen to make us break or add to those 4 megapatches.

comment:276 follow-up: Changed 8 years ago by jdemeyer

9958_combinat.patch​ is the only non-trivial change I guess, but has been discussed well in the comments on this ticket. Potentially, the trie_type_dict() test might yield other results, but we can still change the test in that case. Anyway, that method is nowhere called in the Sage library.

comment:277 in reply to: ↑ 276 ; follow-up: Changed 8 years ago by fbissey

Replying to jdemeyer:

9958_combinat.patch​ is the only non-trivial change I guess, but has been discussed well in the comments on this ticket. Potentially, the trie_type_dict() test might yield other results, but we can still change the test in that case. Anyway, that method is nowhere called in the Sage library.

I didn't notice you had listed this particular patch. Why not use trac_9958-suffix_trees-variations-sl.patch​ from Sébastien Labbé instead? It is more foolproof.

comment:278 Changed 8 years ago by fbissey

I obviously meant to use Sébastien Labbé's patch for suffix_tree, we can still use the other patch for nfactor_enumerable_word.py, Sébastien think that patch is perfect which I count as a positive review of it.

comment:279 in reply to: ↑ 277 ; follow-up: Changed 8 years ago by jdemeyer

Replying to fbissey:

I didn't notice you had listed this particular patch. Why not use trac_9958-suffix_trees-variations-sl.patch​ from Sébastien Labbé instead? It is more foolproof.

I overlooked that patch, it was not in the list of patches to be applied. I agree it is better.

Changed 8 years ago by jdemeyer

comment:280 in reply to: ↑ 279 ; follow-up: Changed 8 years ago by fbissey

Replying to jdemeyer:

Replying to fbissey:

I didn't notice you had listed this particular patch. Why not use trac_9958-suffix_trees-variations-sl.patch​ from Sébastien Labbé instead? It is more foolproof.

I overlooked that patch, it was not in the list of patches to be applied. I agree it is better.

It was ​http://trac.sagemath.org/sage_trac/ticket/9958?action=diff&version=273 but never mind we have it now.

comment:281 in reply to: ↑ 280 Changed 8 years ago by jdemeyer

Replying to fbissey:

It was ​http://trac.sagemath.org/sage_trac/ticket/9958?action=diff&version=273 but never mind we have it now.

Look more carefully, it wasn't :-)

comment:282 Changed 8 years ago by fbissey

You are right I put the wrong patch in the list actually I never updated the list to include the right one, complete oversight. It's good that we are getting this merged now before more confusion arise from the number of patches attached.

comment:283 Changed 8 years ago by jhpalmieri

François and others: thank you for all of the work you did on this ticket.

comment:284 Changed 8 years ago by slabbe

  • Authors changed from François Bissey, Steven Trogdon, Sébastien Labbé, Jeroen Demeyer to François Bissey, Steven Trogdon, Jeroen Demeyer

Hi,

I am removing my name from the authors list. For three lines of code I wrote, I don't feel as an author at all for such an important ticket. Being a reviewer of the small combinat part is enough and more appropriate.

Sébastien

comment:285 Changed 8 years ago by jdemeyer

  • Merged in set to sage-5.0.beta0
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:286 follow-up: Changed 8 years ago by kcrisman

Just as FYI, though I doubt it matters since the patch still applies, I get 

patching file Lib/distutils/command/sdist.py
Hunk #1 succeeded at 327 (offset 3 lines).
Hunk #2 succeeded at 342 (offset 3 lines).

before the other patches and configuration.

comment:287 Changed 8 years ago by fbissey

Doesn't matter anymore indeed, we have it merged in 5.0.beta0 finally. I am personally moving to #11705 and #11334 unless something very bad happen.

comment:288 in reply to: ↑ 286 Changed 8 years ago by jdemeyer

Replying to kcrisman:

Just as FYI, though I doubt it matters since the patch still applies, I get 

patching file Lib/distutils/command/sdist.py
Hunk #1 succeeded at 327 (offset 3 lines).
Hunk #2 succeeded at 342 (offset 3 lines).

Not a big deal. The patch still applies perfectly.

Note: See TracTickets for help on using tickets.