Opened 11 years ago
Last modified 9 years ago
#9958 closed enhancement
Upgrade python to 2.7.x — at Version 144
Reported by:  mhampton  Owned by:  tbd 

Priority:  major  Milestone:  sage5.0 
Component:  packages: standard  Keywords:  
Cc:  jhpalmieri, leif, jason, kcrisman, kini  Merged in:  
Authors:  François Bissey  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  Commit:  
Dependencies:  #11986  Stopgaps: 
Description (last modified by )
From the release notes:
Python 2.7.2 was released on June 11th, 2011.
The Python 2.7 series is scheduled to be the last major version in the 2.x series before 2.x moves into an extended maintenance period. The 2.7 series contains many of the features that were first released in Python 3.1. Improvements in this release include:
 An ordered dictionary type
 New unittest features including test skipping, new assert methods, and test discovery
 A much faster io module
 Automatic numbering of fields in the str.format() method
 Float repr improvements backported from 3.x
 Tile support for Tkinter
 A backport of the memoryview object from 3.x
 Set literals
 Set and dictionary comprehensions
 Dictionary views
 New syntax for nested with statements
 The sysconfig module
List of spkg that will need an artificial bump (or a real one) for upgrade to go smoothly (feel free to amend if I forgot something):
 cvxopt
 cython
 docutils
 gdmodule
 ipython
 jinja2
 matplotlib
 mercurial
 moin
 mpmath
 networkx
 numpy
 pexpect
 pil
 polybori?
 pycrypto
 pygments
 pynac
 python_gnutls
 r (because rpy is contained inside) (until #9906 gets merged)
 sagenb
 sagetex
 scipy
 setuptools
 sphinx
 sqlalchemy
 sympy
 twisted
 zodb3
[scons will work with either python2.6 or 2.7 without needing a rebuild]
Fixes #1159
New spkg: http://spkgupload.googlecode.com/files/python2.7.1.spkg
Apply:
 trac_9958fixing_numericalnoisepart1_p1.patch
 trac_9958fixing_numericalnoisepart2.patch
 trac_9958fixing_numericalnoisepart3.patch
 trac_9958fixing_numericalnoisepart4.patch
 trac_9958fix_transcendental.patch
 trac_9958fixing_colorspy.patch
 trac_9958fixlist_index.patch
 trac_9958fixpureAssertError.patch
 trac_9958sage_unittest.patch
 trac_9958mixedfix_p1.patch
 trac_9958fixmisc_cythonpy.patch
 trac_9958e_one_star.patch
 trac_9958symbolic_callable.patch
 trac_9958finite_crystals.patch
 trac_9958fixreal_mpfr.patch
 trac_995832_64bit_messages.patch
 trac_9958_junk_valueerror.patch
 trac_9958_enumerate64bit.patch
Change History (170)
comment:1 followup: ↓ 5 Changed 11 years ago by
comment:2 Changed 11 years ago by
 Cc jhpalmieri added
 Component changed from PLEASE CHANGE to packages
comment:3 Changed 11 years ago by
 Cc leif added
comment:4 Changed 11 years ago by
 Type changed from defect to enhancement
comment:5 in reply to: ↑ 1 Changed 11 years ago by
Replying to mhampton:
First attempt is at: http://sage.math.washington.edu/home/palmieri/SPKG/python2.7.p0.spkg
John (jhpalmieri) reports that this builds but Sage does not on sage.math, and "packages for twisted, zodb, pygments, and numpy don't build correctly."
Apparently the fix for http://bugs.python.org/issue7491 causes some of these problems.
This fix has been included in python2.6.5 onwards. We are using 2.6.5 which includes this fix and everything builds (haven't tried 2.7 yet). The only problem we have is with http://github.com/cschwan/sageongentoo/issues#issue/1 so I think it is unfair to single out this issue as the source of problems.
comment:6 Changed 10 years ago by
Is there still any interest in this upgrade? I wanted to use ordered dictionaries, which require 2.7. Unfortunately, active participation in this ticket is beyond my current capabilities...
comment:7 Changed 10 years ago by
It will happen at least in sageongentoo as we are following the system python. I have a couple of things in our tree to fix problems with python2.6.5 and 2.6.6 we are looking into 2.7 now. I should post any patch to have it working here.
We are not talking about using the new capabilities just porting, I imagine my counterpart in Mandriva does the same.
comment:8 Changed 10 years ago by
 Status changed from new to needs_work
I just attached a patch that is needed for python 2.6.5 and later.
comment:9 Changed 10 years ago by
 Cc jason added
comment:10 Changed 10 years ago by
Build sage4.6.2 (and dependency) against python2.7.1 on OS X. I get a lot of the following
Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/sitepackages/sage/misc/sage_unittest.py", line 275, in run test_method(tester = tester) File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/sitepackages/sage/misc/sage_unittest.py", line 498, in _test_pickling tester.assertEqual(loads(dumps(self._instance)), self._instance) File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/unittest/case.py", line 493, in assertEqual assertion_func = self._getAssertEqualityFunc(first, second) File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/unittest/case.py", line 476, in _getAssertEqualityFunc asserter = self._type_equality_funcs.get(type(first)) AttributeError: 'InstanceTester' object has no attribute '_type_equality_funcs' 
I patched python with the cpickle patch. Any ideas?
comment:11 Changed 10 years ago by
A lot of "doctest... DeprecationWarning?: ..." lines that were expected are just gone. Which fails the test.
comment:12 Changed 10 years ago by
Lots of numerical and a little bit of formatting noise.
comment:13 Changed 10 years ago by
Ok so now I understand the differences between unittest and unittest2 which is shipped with python2.7. This will require a massive number of nonbackward compatible changes. I am starting to experiment with a few sage components but that promise to be long and boring.
comment:14 Changed 10 years ago by
 Cc kcrisman added
comment:15 Changed 10 years ago by
I have attached a log of sage testall. This is a sageongentoo install a few tests are expected to fail https://github.com/cschwan/sageongentoo/wiki/Knowntestfailures but it should give you an idea of the problems we face.
comment:16 Changed 10 years ago by
 Description modified (diff)
comment:17 Changed 10 years ago by
I attached a log of test failures with 4.7.alpha4 (+ #7377). It is mostly number of decimals and messages. I added PYTHONWARNINGS=default to sageenv so I collected extra messages. The most important one being the deprecations of "sets".
There are 3 tests killed reason currently unknown. And a few that may need special attention.
comment:18 Changed 10 years ago by
A little bit more details, these are curious:
sage t long force_lib devel/sagemain/sage/rings/padics/padic_capped_relative_element.pyx ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/padics/padic_capped_relative_element.pyx", line 2297: sage: hash(R(1)) Expected: 95367431640624 Got: 1977800240
sage t long force_lib devel/sagemain/sage/combinat/words/suffix_trees.py ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/suffix_trees.py", line 1345: sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]]) Expected: True Got: False ********************************************************************** sage t long force_lib devel/sagemain/sage/combinat/words/suffix_trees.py ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/suffix_trees.py", line 1345: sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]]) Expected: True Got: False **********************************************************************
This one doesn't worry me as much but should be looked at
sage t long force_lib devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py", line 22: sage: it.next() Expected: word: 5645 Got: word: 4564 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py", line 24: sage: it.next() Expected: word: 4564 Got: word: 5645 **********************************************************************
sage t long force_lib devel/sagemain/sage/structure/parent.pyx ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/structure/parent.pyx", line 634: sage: CCls()._test_eq() Expected: Traceback (most recent call last): ... AssertionError: <class '__main__.CCls'> == None Got nothing
The following were killed:
sage t long force_lib devel/sagemain/sage/rings/homset.py # Killed/crashed sage t long force_lib devel/sagemain/sage/schemes/generic/scheme.py # Killed/crashed
In my original run
sage t long force_lib "devel/sagemain/sage/rings/morphism.pyx"
also got killed but not in a subsequent run after I adopted a small change in sagedoctest to get rid of PYTHONWARNINGS=default in sageenv.
Example of killed test:
sage t long verbose force_lib "devel/sagemain/sage/rings/homset.py" ... Trying: phi = S.hom([b,a])###line 141:_sage_ >>> phi = S.hom([b,a]) Expecting nothing ok Trying: phi == loads(dumps(phi))###line 142:_sage_ >>> phi == loads(dumps(phi)) Expecting: True /usr/lib64/libcsage.so(print_backtrace+0x24)[0x7f83de940534] /usr/lib64/libcsage.so(sigdie+0x1d)[0x7f83de9405cd] /usr/lib64/libcsage.so(sage_signal_handler+0x131)[0x7f83de940741] /lib64/libpthread.so.0(+0xfae0)[0x7f83e2385ae0] /usr/lib64/libsingular.so.3(_Z9id_DeletePP10sip_sidealP9sip_sring+0x53)[0x7f83c5c48383] /usr/lib64/python2.7/sitepackages/sage/libs/singular/groebner_strategy.so(+0x388d)[0x7f83ad27488d] /usr/lib64/libpython2.7.so.1.0(PyDict_Clear+0xfc)[0x7f83e261597c] /usr/lib64/libpython2.7.so.1.0(+0x82a09)[0x7f83e2615a09] /usr/lib64/libpython2.7.so.1.0(+0x112e7e)[0x7f83e26a5e7e] /usr/lib64/libpython2.7.so.1.0(_PyObject_GC_Malloc+0x11c)[0x7f83e26a684c] /usr/lib64/libpython2.7.so.1.0(_PyObject_GC_New+0xd)[0x7f83e26a685d] /usr/lib64/libpython2.7.so.1.0(+0x7fe11)[0x7f83e2612e11] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/python2.7/sitepackages/sage/rings/polynomial/polydict.so(+0x168c3)[0x7f83c8f378c3] /usr/lib64/libpython2.7.so.1.0(+0xa0468)[0x7f83e2633468] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/python2.7/sitepackages/sage/rings/polynomial/polydict.so(+0xa451)[0x7f83c8f2b451] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47)[0x7f83e2670ca7] /usr/lib64/python2.7/libdynload/cPickle.so(+0x5f06)[0x7f83dcb2df06] /usr/lib64/python2.7/libdynload/cPickle.so(+0xaf47)[0x7f83dcb32f47] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/python2.7/sitepackages/sage/structure/sage_object.so(+0x152fc)[0x7f83dc7012fc] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x56bc)[0x7f83e267692c] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f83e26782f2] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5a78)[0x7f83e2676ce8] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(+0x71282)[0x7f83e2604282] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(+0x71282)[0x7f83e2604282] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(+0x7138b)[0x7f83e260438b] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f] /usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd] /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f83e26782f2] /usr/lib64/libpython2.7.so.1.0(+0xff25c)[0x7f83e269225c] /usr/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x90)[0x7f83e2693090] /usr/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0x1ff)[0x7f83e2693c6f] /usr/lib64/libpython2.7.so.1.0(Py_Main+0xb53)[0x7f83e26a4fc3] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7f83e201cb6d] /usr/bin/python2.7[0x4008a9]  Unhandled SIGSEGV: A segmentation fault occurred in Sage. This probably occurred because a *compiled* component of Sage has a bug in it and is not properly wrapped with sig_on(), sig_off(). You might want to run Sage under gdb with 'sage gdb' to debug this. Sage will now terminate.  The doctested process was killed by signal 11 [2.5 s]
Changed 10 years ago by
small patch to sagedoctest in sage_script to reenable sage's deprecation warnings. Idea by my friend Steve Trogdon.
Changed 10 years ago by
this form of comparison is removed in python 2.6.5 and later, refreshed using git.
comment:19 Changed 10 years ago by
It looks like the crash problems originates somewhere in libsingular
sage: f = Zmod(6).cover() sage: f.kernel() Principal ideal (6) of Integer Ring sage: R.<x,y> = PolynomialRing(QQ, 2) sage: S.<a,b> = R.quo(x^2 + y^2) sage: phi = S.cover() sage: phi == loads(dumps(phi)) True sage: phi == R.quo(x^2 + y^3).cover() Program received signal SIGSEGV, Segmentation fault. id_Delete (h=0x50687d8, r=0x0) at ideals.cc:127 127 ideals.cc: No such file or directory. in ideals.cc (gdb) bt #0 id_Delete (h=0x50687d8, r=0x0) at ideals.cc:127 #1 0x00007fffb9670b49 in __pyx_pf_4sage_4libs_8singular_17groebner_strategy_16GroebnerStrategy_1__dealloc__ ( o=<value optimized out>) at sage/libs/singular/groebner_strategy.cpp:2570 #2 __pyx_tp_dealloc_4sage_4libs_8singular_17groebner_strategy_GroebnerStrategy (o=<value optimized out>) at sage/libs/singular/groebner_strategy.cpp:3190 #3 0x00007ffff7aa98c7 in PyDict_Clear (op=<value optimized out>) at Objects/dictobject.c:891 #4 0x00007ffff7aa990f in dict_tp_clear (op=<value optimized out>) at Objects/dictobject.c:2088 #5 0x00007ffff7b32f27 in delete_garbage (generation=0) at Modules/gcmodule.c:769 #6 collect (generation=0) at Modules/gcmodule.c:930 #7 0x00007ffff7b33616 in collect_generations (basicsize=<value optimized out>) at Modules/gcmodule.c:996 #8 _PyObject_GC_Malloc (basicsize=<value optimized out>) at Modules/gcmodule.c:1457 #9 0x00007ffff7ac4a96 in PyType_GenericAlloc (type=0x7ffff7d9ad40, nitems=0) at Objects/typeobject.c:744 #10 0x00007ffff7a8d256 in BaseException_new (type=<value optimized out>, args=<value optimized out>, kwds=<value optimized out>) at Objects/exceptions.c:34 #11 0x00007ffff7ac57b5 in type_call (type=0x7ffff7d9ad40, args=0x4ae46d0, kwds=0x0) at Objects/typeobject.c:712 #12 0x00007ffff7a77539 in PyObject_Call (func=0x7ffff7d9ad40, arg=0x4ae46d0, kw=0x0) at Objects/abstract.c:2529 #13 0x00007ffff7b00b45 in PyEval_CallObjectWithKeywords (func=0x7ffff7d9ad40, arg=0x4ae46d0, kw=0x0) at Python/ceval.c:3881 #14 0x00007ffff7b10443 in PyErr_NormalizeException (exc=0x7fffffffa778, val=0x7fffffffa770, tb=0x7fffffffa768) at Python/errors.c:190 #15 0x00007fffe433021c in __Pyx_GetException (type=0x7fffffffa7c0, value=0x7fffffffa7b8, tb=0x7fffffffa7c8) at sage/structure/parent.c:20909 #16 0x00007fffe433e4d1 in __pyx_pf_4sage_9structure_6parent_6Parent_2element_class (__pyx_v_self=0x4aea500, unused=<value optimized out>) at sage/structure/parent.c:4251 #17 0x00007ffff7aac1e2 in PyCFunction_Call (func=0x504ebd8, arg=0x7ffff7f81050, kw=<value optimized out>) at Objects/methodobject.c:90 #18 0x00007ffff7a77539 in PyObject_Call (func=0x504ebd8, arg=0x7ffff7f81050, kw=0x0) at Objects/abstract.c:2529 #19 0x00007ffff7b00b45 in PyEval_CallObjectWithKeywords (func=0x504ebd8, arg=0x7ffff7f81050, kw=0x0) at Python/ceval.c:3881 #20 0x00007ffff7a8b51f in methoddescr_call (descr=<value optimized out>, args=0x7ffff7f81050, kwds=0x0) at Objects/descrobject.c:246 #21 0x00007ffff7a77539 in PyObject_Call (func=0x138e950, arg=0x7ffff7e9c050, kw=0x0) at Objects/abstract.c:2529 #22 0x00007ffff7b05efe in do_call (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4230 #23 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4035 #24 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665 #25 0x00007ffff7b07b65 in PyEval_EvalCodeEx (co=0x1159e30, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=3, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3252 #26 0x00007ffff7a99a97 in function_call (func=0x115d5f0, arg=0xfadeb0, kw=0x0) at Objects/funcobject.c:526 #27 0x00007ffff7a77539 in PyObject_Call (func=0x115d5f0, arg=0xfadeb0, kw=0x0) at Objects/abstract.c:2529 #28 0x00007ffff7a77cca in PyObject_CallFunctionObjArgs (callable=0x115d5f0) at Objects/abstract.c:2760 #29 0x00007ffff7ac8a8e in slot_tp_descr_get (self=0x13179d0, obj=0x4aea500, type=0x19e91f0) at Objects/typeobject.c:5621 #30 0x00007ffff7aae433 in _PyObject_GenericGetAttrWithDict (obj=0x4aea500, name=0x13c5a78, dict=0x5068fb0) at Objects/object.c:1432 #31 0x00007ffff7aae4d1 in PyObject_GenericGetAttr (obj=<value optimized out>, name=<value optimized out>) at Objects/object.c:1454 #32 0x00007fffe434ed62 in __pyx_tp_getattro_4sage_9structure_6parent_Parent (o=0x4aea500, n=0x13c5a78) at sage/structure/parent.c:18702 #33 0x00007ffff7aad8f4 in PyObject_GetAttr (v=0x4aea500, name=<value optimized out>) at Objects/object.c:1189 #34 0x00007ffff7afdbab in builtin_hasattr (self=<value optimized out>, args=<value optimized out>) at Python/bltinmodule.c:885 #35 0x00007ffff7aac1a0 in PyCFunction_Call (func=0x7ffff7fae3b0, arg=0x503d680, kw=<value optimized out>) at Objects/methodobject.c:81 #36 0x00007ffff7b05ba3 in call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4012 #37 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665 #38 0x00007ffff7b05cfa in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4098 #39 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4033 #40 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665 #41 0x00007ffff7b07b65 in PyEval_EvalCodeEx (co=0x1159e30, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=3, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3252 #42 0x00007ffff7a99a97 in function_call (func=0x115d5f0, arg=0xfadbe0, kw=0x0) at Objects/funcobject.c:526
comment:20 Changed 10 years ago by
Same two doctests killed after upgrade to alpha5.
comment:21 Changed 10 years ago by
Overeager garbage collection in python2.7.1! Preceding the sequence by
sage: import gc sage: gc.disable()
Makes everything go smoothly.
comment:22 Changed 10 years ago by
 Description modified (diff)
comment:23 followup: ↓ 26 Changed 10 years ago by
trac_9958fix_cmp.patch is perfectly reasonable, since this is the idiom used everywhere else in similar situations. I don't have enough background to judge on the warning patch.
comment:24 Changed 10 years ago by
Put an updated list of tests failing with 4.7.alpha5 and the latest patches from Nicolas M. Thiery. I think the next area do need some work is the problem of garbage collection in polynomial rings that leads to "random" failures in libsingular. See: http://groups.google.com/group/sagedevel/browse_thread/thread/8c165c887d6b9e54
comment:25 Changed 10 years ago by
 Description modified (diff)
Split the deprecation issue in its own ticket in #11244  this is a different patch.
comment:26 in reply to: ↑ 23 Changed 10 years ago by
Replying to nthiery:
trac_9958fix_cmp.patch is perfectly reasonable, since this is the idiom used everywhere else in similar situations. I don't have enough background to judge on the warning patch.
I will put the cmp patch in its own ticket would you review it?
comment:27 Changed 10 years ago by
I have attached some patches to fix all the noise for numerics and formatting changes. What's left should be looked at before patching.
There are several failures because hashes have changed but I cannot say it is allright to just change the doctest:
sage t long force_lib "devel/sagemain/sage/rings/padics/padic_capped_relative_element.pyx" ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/padics/padic_capped_relative_element.pyx", line 2339: sage: hash(R(1)) Expected: 95367431640624 Got: 1977800240 **********************************************************************
sage t long force_lib "devel/sagemain/sage/rings/integer.pyx" ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/integer.pyx", line 2995: sage: n = 920390823904823094890238490238484; n.__hash__() Expected: 6874330978542788722 Got: 2623069716 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/integer.pyx", line 3010: sage: hash(n) Expected: 9223372036854767616 Got: 8192 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/integer.pyx", line 3013: sage: hash(n) == hash(int(n)) Expected: True Got: False **********************************************************************
That hash(n) == hash(int(n)) is now false should be looked into carefully.
sage t long force_lib "devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py" ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py", line 20: sage: it.next() Expected: word: 6456 Got: word: 5645 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py", line 22: sage: it.next() Expected: word: 5645 Got: word: 6456 **********************************************************************
I think that one is OK, just a change of order but a second opinion would be nice.
Finally there are still three tests killed in libsingular's id_delete because a ring has been garbage collected even so there are some precautions in place for it: http://groups.google.com/group/sagedevel/browse_thread/thread/8c165c887d6b9e54
Next we need an updated python spkg so that more people can play and even try the patchbot on it.
comment:28 Changed 10 years ago by
 Description modified (diff)
The replacement of type.cmp by cmp in strata.py is now in its own ticket #11264 as it can be applied independently of a python upgrade.
comment:29 Changed 10 years ago by
I forgot because I build on sageongentoo but I'll post some patch to build against python2.7 a little bit later.
comment:30 Changed 10 years ago by
Unless someone steps in I'll make a new python2.7.1 spkg by the end of the week. I'll have to figure out if the latest change going in for sage4.7 will be needed for that one.
comment:31 Changed 10 years ago by
I won't be able to do that this week.
comment:32 Changed 10 years ago by
 Description modified (diff)
Our lead sageongentoo debugger Martin tracked down the segfault problem to cython, this is now #11339
comment:33 Changed 10 years ago by
I'd like to try this but don't have time to figure out exactly what patches are needed in what order  can someone update the description with this? Also, is it the same spkg that Marshall originally posted that one should try?
comment:34 followup: ↓ 35 Changed 10 years ago by
Also, is it possible to test this by ./sage f something, or would one have to take a fresh tarball and replace the Python spkg?
comment:35 in reply to: ↑ 34 Changed 10 years ago by
Replying to kcrisman:
Also, is it possible to test this by ./sage f something, or would one have to take a fresh tarball and replace the Python spkg?
Hi Karl,
I am mainly developing from sageongentoo at the moment. I need to make a current python2.7.1 spkg to match. I will update the description and give the patch order when there is a spkg (feel free to make one). The patch have initially been made against 4.7.1.alpha0 and the current version works against 4.7.1.alpha1. You would get failures against a 4.7 because #7377 finally landed.
I think the best course of action will be to start from scratch, it may even be useful to host an alternative tarball somewhere because the change in python version will be extremely messy. I don't know if we can do it without breaking sage upgrade or being overly complicated.
In the meantime you could have a look at #11244 (which I need to update for 4.7.1.alpha1 because of #10334).
Francois
comment:36 Changed 10 years ago by
OK if you want to look at #11244 it can be tested against either the latest 4.7 or 4.7.1.alpha1 now.
comment:37 followup: ↓ 38 Changed 10 years ago by
I see. Well, I just want to make sure that it all works fine on older Macs, of course. I definitely don't have time to make an spkg now, though I do have time to do ./sage f and slap a few patches on it :) Just let me know when you get around to it; I don't think there is a huge rush.
I should probably test 4.7.1.alpha1 on my box. I'm amazed at Jeroen's stamina on this, especially with several different versions simultaneously existing.
comment:38 in reply to: ↑ 37 Changed 10 years ago by
Replying to kcrisman:
I see. Well, I just want to make sure that it all works fine on older Macs, of course. I definitely don't have time to make an spkg now, though I do have time to do ./sage f and slap a few patches on it :) Just let me know when you get around to it; I don't think there is a huge rush.
I should probably test 4.7.1.alpha1 on my box. I'm amazed at Jeroen's stamina on this, especially with several different versions simultaneously existing.
That's why I asked about #11244 it is an issue I split from this ticket because it should apply on a sage running python2.6 without side effects. So it could be in place when we come around to the upgrade. Any ideas on #11339 which was recently split from this ticket is also welcome.
comment:39 Changed 10 years ago by
 Description modified (diff)
My friend Steven Trogdon made some quick and dirty spkgs. His early tests indicate that setuptools needs to be updated. This is now #11363 it also revealed that I missed something in misc/cython.py watch out for an updated patch sometimes this weekend. There were also a few things that needed to be done in two other patches which I refreshed yesterday. I have a python2.7.1 spkg ready to be uploaded probably sometimes today. If you want fun before that you can always use Steve's spkg (they are not in a reviewable state) at: http://www.d.umn.edu/~strogdon/sage/
comment:40 Changed 10 years ago by
 Description modified (diff)
Didn't take long. I have been naughty and committed two changes to patches that needed touching after my first commit of python2.7.1 so the log will show 3 entries for just the one SPKG.txt entry. They were minor changes. I dropped a number of patches because they were applied upstream. However in the case of two patches I dropped them because the upstream has changed in a way they don't apply anymore but I don't know if patching is still necessary for the underlying issues. If so new patches will need to be developed. The issues are
 readline on itanium
 multiprocessing on solaris
As promised I will give a patch order. The next to last one needs updating as stated earlier and the last one is really optional. The patches do not have to be taken n that order as they don't overlap.
comment:41 Changed 10 years ago by
I forgot apply the patches on 4.7.1.alpha1, there may be rejection on 4.7.rcX.
comment:42 Changed 10 years ago by
misc/cython.py patch updated for people who want to test it.
comment:43 Changed 10 years ago by
corrected a syntax error in the last patch.
Changed 10 years ago by
Not strictly necessary but this file needed a scrub. (refreshed for syntax errors)
comment:44 Changed 10 years ago by
Had to redo trac_9958build_module_listpy.patch again as I messed up the syntax of one bit.
comment:45 followup: ↓ 50 Changed 10 years ago by
My friend Steve Trogdon made a prepatched sage4.7.1.alpha1 spkg (4.7.1.alpha1.p0 we named it) available to make testing easier, find it with hos own test.log here http://www.d.umn.edu/~strogdon/sage/ It has all the necessary patches from #11244 and this this ticket. We are a bit puzzled by the cause of failure in sage/misc/cython.py, white spaces?
comment:46 followup: ↓ 47 Changed 10 years ago by
I am getting the twisted install fail that jhpalmieri refers to. So my guess is that the Gentoo system has some updated things along those lines that vanilla 4.7.1.alpha1 does not. Is that possible?
comment:47 in reply to: ↑ 46 Changed 10 years ago by
Replying to kcrisman:
I am getting the twisted install fail that jhpalmieri refers to. So my guess is that the Gentoo system has some updated things along those lines that vanilla 4.7.1.alpha1 does not. Is that possible?
Well sageongentoo uses a newer twisted. On the other I and Steve have build twisted in vanilla sage with python2.7.1. Did you get the new setuptools spkg from #11363? It is necessary to build twisted with python2.7.1.
comment:48 Changed 10 years ago by
No, I didn't realize there was an spkg involved with that. I installed that first, and it seems to be working now.
comment:49 Changed 10 years ago by
 Description modified (diff)
comment:50 in reply to: ↑ 45 Changed 10 years ago by
Replying to fbissey:
My friend Steve Trogdon made a prepatched sage4.7.1.alpha1 spkg (4.7.1.alpha1.p0 we named it) available to make testing easier, find it with hos own test.log here http://www.d.umn.edu/~strogdon/sage/ It has all the necessary patches from #11244 and this this ticket.
Is one of these patches needed before doing the Sage spkg itself? I get an immediate failure
gcc o src/convert.os c fPIC I/Users/student/Desktop/sage4.7.1.alpha1/local/include I/Users/student/Desktop/sage4.7.1.alpha1/local/include/python2.6 I/Users/student/Desktop/sage4.7.1.alpha1/local/include/NTL Iinclude src/convert.c In file included from include/memory.h:23, from /Users/student/Desktop/sage4.7.1.alpha1/local/include/pari/pari.h:42, from include/convert.h:12, from src/convert.c:14: include/interrupt.h:32:20: error: Python.h: No such file or directory scons: *** [src/convert.os] Error 1 *** TOUCHING ALL CYTHON (.pyx) FILES *** gcc o src/convert.os c fPIC I/Users/student/Desktop/sage4.7.1.alpha1/local/include I/Users/student/Desktop/sage4.7.1.alpha1/local/include/python2.6 I/Users/student/Desktop/sage4.7.1.alpha1/local/include/NTL Iinclude src/convert.c In file included from include/memory.h:23, from /Users/student/Desktop/sage4.7.1.alpha1/local/include/pari/pari.h:42, from include/convert.h:12, from src/convert.c:14: include/interrupt.h:32:20: error: Python.h: No such file or directory scons: *** [src/convert.os] Error 1  sage: Building and installing modified Sage library files. Installing c_lib gcc o src/convert.os c fPIC I/Users/student/Desktop/sage4.7.1.alpha1/local/include I/Users/student/Desktop/sage4.7.1.alpha1/local/include/python2.6 I/Users/student/Desktop/sage4.7.1.alpha1/local/include/NTL Iinclude src/convert.c In file included from include/memory.h:23, from /Users/student/Desktop/sage4.7.1.alpha1/local/include/pari/pari.h:42, from include/convert.h:12, from src/convert.c:14: include/interrupt.h:32:20: error: Python.h: No such file or directory scons: *** [src/convert.os] Error 1 ERROR: There was an error building c_lib.
which I assume is related to one of the patches involved here, though I couldn't say which one...
comment:51 Changed 10 years ago by
Did you rebuild cython after the python upgrade? In fact I have now added a list of package that needs to be rebuild after the python upgrade. Not all of them are necessary for building sage (scipy could be done after) but cython, numpy, matplotlib and pynac really need to be rebuilt.
Once you have done that apply all the patches with build in their names:
 trac_9958build_setuppy.patch
 trac_9958buildsage_clib.patch
 trac_9958build_misc_cythonpy.patch
 trac_9958build_module_listpy.patch (actually optional)
That should help sage build but will not correct many of the doctests. To fix most of the doctests apply the patches in #11244 as well as the one in the description of this bug. I have made a list now, have you seen it?
comment:52 followup: ↓ 53 Changed 10 years ago by
Well, I'm building from scratch, so that shouldn't affect it from that side.
Are you saying that before I build the Sage library, I have to apply these patches? That's a little tricky in a scratch build (for me, at least).
Yes, I saw the lists, but I thought they were only relevant for an upgrade/after actually building Sage. My apologies. I may not be able to finish testing this properly for a while now.
comment:53 in reply to: ↑ 52 Changed 10 years ago by
Replying to kcrisman:
Well, I'm building from scratch, so that shouldn't affect it from that side.
Are you saying that before I build the Sage library, I have to apply these patches? That's a little tricky in a scratch build (for me, at least).
Yes, I saw the lists, but I thought they were only relevant for an upgrade/after actually building Sage. My apologies. I may not be able to finish testing this properly for a while now.
That's all right there are still things to sort out. But you could use the sage spkg prepared by Steve with all patches included rather than patch yourself: http://www.d.umn.edu/~strogdon/sage/sage4.7.1.alpha1.p0.spkg
And yes it is tricky from scratch unless you have a premade spkg.
comment:54 followup: ↓ 55 Changed 10 years ago by
I may have made mistakes, but I tried to do this, starting with sage4.7.rc4:
 install the new python spkg
 install the new setuptools spkg (from #11363)
 apply the patches from #11244 to the main Sage library
 apply the patches from this ticket with "build" in their name  some of these didn't apply cleanly, so I had to work around that.
I used that to build a new version of Sage (using "sage sdist ..."). The tar file is here:
On sage.math, I've been able to untar this and build Sage, although tests haven't run yet. You can also try upgrading, although I haven't tested this at all: run
$ sage upgrade http://sage.math.washington.edu/home/palmieri/misc/python2.7/sage4.7.rc4python2.7/
comment:55 in reply to: ↑ 54 Changed 10 years ago by
Replying to jhpalmieri:
I may have made mistakes, but I tried to do this, starting with sage4.7.rc4:
 install the new python spkg
 install the new setuptools spkg (from #11363)
 apply the patches from #11244 to the main Sage library
 apply the patches from this ticket with "build" in their name  some of these didn't apply cleanly, so I had to work around that.
I used that to build a new version of Sage (using "sage sdist ..."). The tar file is here:
On sage.math, I've been able to untar this and build Sage, although tests haven't run yet. You can also try upgrading, although I haven't tested this at all: run
$ sage upgrade http://sage.math.washington.edu/home/palmieri/misc/python2.7/sage4.7.rc4python2.7/
I have tried to keep abreast of everything so the patches are based on 4.7.1.alpha1 there are enough differences that a lot of patches for 4.7 would need rebasing.
So I decided to go straight for the bleeding edge. I am doing some more work on the "build" patches, I am trying to do something smarter with the python version number so some change could be applied independently of this ticket. The module_list,py patch definitely won't apply cleanly on 4.7.rc mainly because of #10334. What's more #11264 and #11236 which are needed here are merged in 4.7.1.alpha0.
That's all a bit of hassle to start from 4.7.1.alpha, but I would have to start again if I was starting from 4.7.rc.
Changed 10 years ago by
let's not forget the hardcoded version of python in SConstruct for sage_clib
comment:56 Changed 10 years ago by
I have updated trac_9958buildsage_clib.patch it could now be splited in its own ticket. It enables sageclib to be smarter about the python version used. It would figure it on its own. I am looking at doing the same for setup.py, anyone would review that in its own ticket? Once in, we could at least build sage against python2.6 or 2.7 without changes.
Changed 10 years ago by
change needed in setup.py made it python version smart. I also scrubbed the debian stuff
comment:57 Changed 10 years ago by
OK I also made setup.py python version smart any volunteer to review a separate ticket?
comment:58 Changed 10 years ago by
g++ o libcsage.dylib single_module flat_namespace undefined dynamic_lookup dynamiclib src/convert.os src/interrupt.os src/memory.os src/mpn_pylong.os src/mpz_pylong.os src/mpz_longlong.os src/stdsage.os src/gmp_globals.os src/ZZ_pylong.os src/ntl_wrap.os L/Users/student/Desktop/sage4.7.1.alpha1/local/lib L/Users/student/Desktop/sage4.7.1.alpha1/local/lib/python/config lntl lpari lgmp lpython2.7 Traceback (most recent call last): File "setup.py", line 18, in <module> from module_list import ext_modules File "/Users/student/Desktop/sage4.7.1.alpha1/devel/sagemain/module_list.py", line 1529, in <module> flint_depends], TypeError: cannot concatenate 'str' and 'list' objects sage: There was an error installing modified sage library code. real 0m34.577s user 0m19.519s sys 0m6.901s Error building new version of SAGE. You might try typing 'sage ba' or write to sagesupport with as much information as possible. real 2m42.066s user 0m52.903s sys 0m25.133s sage: An error occurred while installing sage4.7.1.alpha1.p0
Note this is the p0 package referred to above.
comment:59 followup: ↓ 61 Changed 10 years ago by
Well, there was a problem with this but it was corrected. I've checked the present sage4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage4.7.1.alpha1.p0.spkg) should be
b54da7bee966eaf15942964a05a8b6b1 sage4.7.1.alpha1.p0.spkg
If the spkg you have is not this could you download again and see if that fixed things.
Changed 10 years ago by
Some minor changes. The biggest change is to the doctest which explicitly refereed to python2.6 also we have extra include folders in a doctest apparently. [updated 20110525]
comment:60 Changed 10 years ago by
 Description modified (diff)
I am splitting the ticket again: in #11376 we make sage building system smart and work out the python version by itself. If we get this in the next alpha that will make testing easier as you won't need a patched sage spkg to build against python2.7.1. One will still need to patch the doctests afterwards and any other issues but it will build.
In #11377 I put the clean up of module_list.py this is not necessary for python2.7.1 per see but it would be a huge favour to me if it was going in (it makes work on sageongentoo and sageprefix much easier).
comment:61 in reply to: ↑ 59 ; followup: ↓ 62 Changed 10 years ago by
Replying to strogdon:
Well, there was a problem with this but it was corrected. I've checked the present sage4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage4.7.1.alpha1.p0.spkg) should be
b54da7bee966eaf15942964a05a8b6b1 sage4.7.1.alpha1.p0.spkg
If the spkg you have is not this could you download again and see if that fixed things.
It didn't seem to fix things (and I did check the md5 sum). I'm now starting a build from scratch, having replaced the python and sage spkgs.
By the way, now that 4.7.1.alpha1 is "official", one may have to upgrade the dropin spkg.
comment:62 in reply to: ↑ 61 Changed 10 years ago by
Replying to kcrisman:
Replying to strogdon:
Well, there was a problem with this but it was corrected. I've checked the present sage4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage4.7.1.alpha1.p0.spkg) should be b54da7bee966eaf15942964a05a8b6b1 sage4.7.1.alpha1.p0.spkg If the spkg you have is not this could you download again and see if that fixed things.
It didn't seem to fix things (and I did check the md5 sum). I'm now starting a build from scratch, having replaced the python and sage spkgs. By the way, now that 4.7.1.alpha1 is "official", one may have to upgrade the dropin spkg.
OK, for those interested I have a new dropin spkg based on the released 4.7.1.alpha1. This spkg includes the patches from this ticket as well as the patches from tickets #11244, #11373, #11376 and #11377 . I'm presently building Sage to make sure everything is as it should be. If successful, I'll post links to the dropin spkg later today.
comment:63 Changed 10 years ago by
comment:64 Changed 10 years ago by
The following have been uploaded:
http://www.d.umn.edu/~strogdon/sage/install.log4.7.1.alpha1
http://www.d.umn.edu/~strogdon/sage/test.log4.7.1.alpha1
http://www.d.umn.edu/~strogdon/sage/sage4.7.1.alpha1.p0.spkg
http://www.d.umn.edu/~strogdon/sage/sage4.7.1.alpha1.p0.spkg.md5
for comparisons, perhaps just under the wire for alpha2. Note, the dropin sage spkg has the same revision .p0 extension, so I've provided the md5 sum for the spkg.
comment:65 Changed 10 years ago by
 Description modified (diff)
Changed the cython.py patch in the list to accommodate new changes made in #11376.
comment:66 followup: ↓ 67 Changed 10 years ago by
End of a looong traceback while building from scratch on PPC 10.4 with the spkgs dropped in (though not the absolute most recent alpha1.p0 spkg, but that hasn't even been gotten to yet, so it shouldn't matter). Just FYI in case that helps.
File "/Users/student/Desktop/sage4.7.1.alpha1python/local/lib/python/os.py", line 284, in walk if isdir(join(top, name)): File "/Users/student/Desktop/sage4.7.1.alpha1python/local/lib/python/genericpath.py", line 44, in isdir return stat.S_ISDIR(st.st_mode) File "/Users/student/Desktop/sage4.7.1.alpha1python/local/lib/python/stat.py", line 41, in S_ISDIR return S_IFMT(mode) == S_IFDIR RuntimeError: maximum recursion depth exceeded Error installing Zope/Twisted interface. real 0m16.055s user 0m3.349s sys 0m4.671s sage: An error occurred while installing twisted9.0.p2
comment:67 in reply to: ↑ 66 Changed 10 years ago by
Replying to kcrisman:
End of a looong traceback while building from scratch on PPC 10.4 with the spkgs dropped in (though not the absolute most recent alpha1.p0 spkg, but that hasn't even been gotten to yet, so it shouldn't matter). Just FYI in case that helps.
File "/Users/student/Desktop/sage4.7.1.alpha1python/local/lib/python/os.py", line 284, in walk if isdir(join(top, name)): File "/Users/student/Desktop/sage4.7.1.alpha1python/local/lib/python/genericpath.py", line 44, in isdir return stat.S_ISDIR(st.st_mode) File "/Users/student/Desktop/sage4.7.1.alpha1python/local/lib/python/stat.py", line 41, in S_ISDIR return S_IFMT(mode) == S_IFDIR RuntimeError: maximum recursion depth exceeded Error installing Zope/Twisted interface. real 0m16.055s user 0m3.349s sys 0m4.671s sage: An error occurred while installing twisted9.0.p2
Were you using the new setuptools from #11363? If so it may be worth trying a newer twisted from #8741.
comment:68 Changed 10 years ago by
 Dependencies set to #11156 #11236 #11244 #11264 #11339 #11363 #11376
comment:69 Changed 10 years ago by
Updated trac_9958fixmisc_cythonpy.patch to take into account changes in #11376. For some reason the numpy include line appears after the application of the other patches over there. Why it didn't appear before is a mistery.
comment:70 Changed 10 years ago by
By the way, I don't know how feasible this is, but it would be very nice if the new python spkg passed selftests: set SAGE_CHECK to "yes" and install the spkg. Previous versions of Python have failed the selftests, but it's perhaps the only standard spkg for which this is true. With the spkg here, on an OS X box:
344 tests OK. 4 tests failed: test__locale test_ctypes test_distutils test_locale 38 tests skipped: test_aepack test_al test_applesingle test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dl test_epoll test_gdb test_gdbm test_gl test_imageop test_imgfile test_largefile test_linuxaudiodev test_macos test_macostools test_ossaudiodev test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 5 skips unexpected on darwin: test_aepack test_applesingle test_dl test_macos test_scriptpackages make: *** [test] Error 1 An error occurred while testing Python
On sage.math:
346 tests OK. 1 test failed: test_distutils 39 tests skipped: test_aepack test_al test_applesingle test_bsddb test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dbm test_dl test_gdb test_gdbm test_gl test_imageop test_imgfile test_kqueue test_linuxaudiodev test_macos test_macostools test_ossaudiodev test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 7 skips unexpected on linux2: test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_tk test_ttk_guionly make: *** [test] Error 1 An error occurred while testing Python
(I didn't upgrade the distutils package or anything else, just ran "sage f python2.7.1.spkg", so some of these may go away.)
comment:71 Changed 10 years ago by
 Cc kini added
comment:72 Changed 10 years ago by
python testing is difficult. Even in gentoo the procedure is complicated. I'll have a look at it but I cannot make promise. There may be modules that we shouldn't build in the first place (nothing useful to sage, don't worry).
We have this for example in the ebuild:
if use berkdb; then ewarn "\"bsddb\" module is outofdate and no longer maintained inside devlang/python. It has" ewarn "been additionally removed in Python 3. You should use external, still maintained \"bsddb3\"" ewarn "module provided by devpython/bsddb3 which supports both Python 2 and Python 3." fi
gdb, gdm, dbm and tk require extra things on the host system so they may very well fail without. Actually according to the ebuild distutils and gdm are known to fail.
comment:73 followup: ↓ 76 Changed 10 years ago by
Just for the record:
http://trac.sagemath.org/sage_trac/ticket/8664#comment:33 gives an example on how to automatically rebuild packages that depend on a newly added spkg (here: Python).
(Artificial version bumps aren't necessary, and simply touch
ing spkg/installed/<spkgthatothersdependon>
alone doesn't work.)
In general:
 Copy the new spkg(s) to
$SAGE_ROOT/spkg/standard/
. export SAGE_UPGRADING=yes
 (Optionally set
MAKE
andSAGE_PARALLEL_SPKG_BUILD
, perhapsSAGE_CHECK
.)  Run
make
(or bettermake build
) rather than doing./sage f ...
 Apply patches to the Sage library.
 Run
./sage b
(or./sage baforce
in case not all dependencies are covered bymodule_list.py
, which in general I'd consider a bug, but this might be necessary upon a Python upgrade)  Run tests or whatever.
Note that Sage switches to the "main" branch whenever the Sage spkg gets reinstalled.
Of course patches to the build system are usually a bit trickier, but as far as I can see Francois already sorted these out.
comment:74 Changed 10 years ago by
Thanks for the pointer, it will make testing easier for everyone. The bad news is that all the bits that makes testing easier won't be in 4.7.1. The good news is that everything is merged in 4.7.2.alpha0 at the moment. So once the next release cycle begins things will get easier. No patch will be needed to the build system. All that will remain will be doctest fix and issues like #11339.
I will update trac_9958fixing_numericalnoisepart2.patch shortly as #11255 broke it. Which means an extra patch may appear later once I have put all the pieces back together.
comment:75 Changed 10 years ago by
OK. There are a few more patches needed for numerical and formatting noise in 4.7.2.alpha1 but now anyone should be able to upgrade python without a premade sage spkg. All fixes to be able to build sage against python2.7 out of the box are in!
comment:76 in reply to: ↑ 73 ; followup: ↓ 77 Changed 10 years ago by
(Artificial version bumps aren't necessary, and simply
touch
ingspkg/installed/<spkgthatothersdependon>
alone doesn't work.)In general:
 Copy the new spkg(s) to
$SAGE_ROOT/spkg/standard/
.export SAGE_UPGRADING=yes
Huh, I didn't know about that, and it would be quite useful. I can't find it in either the installation guide or the developer guide, though  is this documented yet? Just curious.
Francois, I'll try to give this a test run on PPC OS X and/or Cygwin if you like.
comment:77 in reply to: ↑ 76 Changed 10 years ago by
Replying to kcrisman:
Francois, I'll try to give this a test run on PPC OS X and/or Cygwin if you like.
Be my guest, I thought that the fact things just became easier warranted an announcement.
comment:78 Changed 10 years ago by
Wonderful, this is great news! :) Thanks!
comment:79 Changed 10 years ago by
So far Mac PPC is doing fine (I just swapped out the spkgs in a source tar). Numpy, mercurial, various other pythonrelated spkgs apparently built ok, though it isn't at Scipy or Sage yet. I did NOT check with SAGE_CHECK=yes
since I didn't want to have to watch the build and turn that off during the Python build... Anyway, looks good so far. If it finishes I'll apply all the patches :)
comment:80 Changed 10 years ago by
I tried this (new python spkg plus all of the patches, then built from scratch) and I'm seeing a bunch of doctest failures. Are some of these due to #11339?
The following tests failed: sage t long force_lib devel/sage/sage/categories/finite_crystals.py # 1 doctests failed sage t long force_lib devel/sage/sage/rings/morphism.pyx # 0 doctests failed sage t long force_lib devel/sage/sage/rings/integer.pyx # 3 doctests failed sage t long force_lib devel/sage/sage/rings/homset.py # 0 doctests failed sage t long force_lib devel/sage/sage/rings/padics/padic_capped_relative_element.pyx # 1 doctests failed sage t long force_lib devel/sage/sage/combinat/e_one_star.py # 1 doctests failed sage t long force_lib devel/sage/sage/combinat/words/nfactor_enumerable_word.py # 2 doctests failed sage t long force_lib devel/sage/sage/geometry/polyhedra.py # 1 doctests failed sage t long force_lib devel/sage/sage/schemes/generic/scheme.py # 0 doctests failed sage t long force_lib devel/sage/sage/symbolic/callable.py # 5 doctests failed
I think I did everything right. I put together a Sage distribution with the python spkg and the patches, based on 4.7.2.alpha1:
Upgrade path:
comment:81 Changed 10 years ago by
Any failures from #11339 gives you a killed doctest, so
sage t long force_lib devel/sage/sage/rings/morphism.pyx sage t long force_lib devel/sage/sage/rings/homset.py sage t long force_lib devel/sage/sage/schemes/generic/scheme.py
could all be, a verbose run would help figure it out. By nature #11339 is somewhat random, not everyone see the same failures but all these are familiar.
Just checking the others with my current list.
 sage/categories/finite_crystals.py is a new one introduced in 4.7.2 where the error message has changed slightly when using python2.7.
 sage/rings/integer.pyx is definitely python2.7 related but not patched yet because I am not sure what should be done there.
 sage/rings/padics/padic_capped_relative_element.pyx same thing
 sage/combinat/e_one_star.py I had a patch but it became to fuzzy haven't done a new one.
 sage/combinat/words/nfactor_enumerable_word.py python2.7 two successive results are inverted, I want someone else to look at that one to make sure we can just do the obvious thing.
 sage/symbolic/callable.py new in 4.7.2 change in the error message because we are running python2.7
 sage/geometry/polyhedra.py never seen that one before, can you rerun it and give more info on the failure?
As you can see a few more patches are needed and apart from #11339 which is a big show stopper because of the crashes at least two other tests point to some potential problems that should be looked at.
comment:82 followup: ↓ 88 Changed 10 years ago by
Here are the failures I get on x86:
sage t long force_lib devel/sagemain/sage/libs/cremona/newforms.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/misc/sageinspect.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/misc/randstate.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/rings/real_mpfr.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/stats/intlist.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/combinat/e_one_star.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/geometry/polyhedra.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed sage t long force_lib devel/sagemain/sage/symbolic/callable.py # 5 doctests failed sage t long force_lib devel/sagemain/sage/functions/transcendental.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/categories/finite_crystals.py # 1 doctests failed
The failures newforms.pyx, real_mpfr.pyx and intlist.pyx are all of the form:
File "/storage/sage/sage4.7.2.alpha1/devel/sagemain/sage/libs/cremona/newforms.pyx", line 53: sage: ECModularSymbol(E) Expected: Traceback (most recent call last): ... OverflowError: long int too large to convert to int Got: Traceback (most recent call last): File "/storage/sage/sage4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/storage/sage/sage4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/storage/sage/sage4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_1[11]>", line 1, in <module> ECModularSymbol(E)###line 53: sage: ECModularSymbol(E) File "newforms.pyx", line 69, in sage.libs.cremona.newforms.ECModularSymbol.__init__ (sage/libs/cremona/newforms.cpp:1820) a6 = new_bigint(int(E.a6())) OverflowError: Python int too large to convert to C long
These tests do not fail on amd64 because "long int too large to convert to int" is returned as the OverflowError? instead of "Python int too large to convert to C long".
comment:83 Changed 10 years ago by
I had forgotten about these. We need different test results for 32 and 64 bits for these.
comment:84 Changed 10 years ago by
Here's the failure for sage/geometry/polyhedra.py
:
sage t long force_lib "devel/sage/sage/geometry/polyhedra.py" ********************************************************************** File "/mnt/usb1/scratch/palmieri/9958/sage4.7.2.9958/devel/sage/sage/geometry/polyhedra.py", line 4948: sage: ppoints[0] Expected: (1.92296268638e15, 1.92296268638e15) Got: (0.0, 0.0) **********************************************************************
It looks like numerical noise, but is it an acceptable level of accuracy?
comment:85 Changed 10 years ago by
Now I remember that test, it is even in the log sage4.7.alpha5test.log.bz2 I produced a while ago. Strangely enough it doesn't happen anymore on my sageongentoo install. I also have pari2.5.0 and maxima5.25.0 on this install, I suspect one of them may be responsible for the disparition of that failure
comment:86 Changed 10 years ago by
I am wrong! Not maxima or pari, there is a patch for it in trac_9958fixing_numericalnoisepart1.patch. So it looks like that bit may not be applied properly.
comment:87 Changed 10 years ago by
 Description modified (diff)
Added three new patches. I will rework the patches that need adjusting between 32 and 64 bits next.
comment:88 in reply to: ↑ 82 ; followup: ↓ 89 Changed 10 years ago by
Replying to strogdon:
Here are the failures I get on x86:
sage t long force_lib devel/sagemain/sage/libs/cremona/newforms.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/misc/sageinspect.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/misc/randstate.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/rings/real_mpfr.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/stats/intlist.pyx # 1 doctests failed sage t long force_lib devel/sagemain/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/combinat/e_one_star.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/geometry/polyhedra.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed sage t long force_lib devel/sagemain/sage/symbolic/callable.py # 5 doctests failed sage t long force_lib devel/sagemain/sage/functions/transcendental.py # 1 doctests failed sage t long force_lib devel/sagemain/sage/categories/finite_crystals.py # 1 doctests failedThe failures newforms.pyx, real_mpfr.pyx and intlist.pyx are all of the form:
File "/storage/sage/sage4.7.2.alpha1/devel/sagemain/sage/libs/cremona/newforms.pyx", line 53: sage: ECModularSymbol(E) Expected: Traceback (most recent call last): ... OverflowError: long int too large to convert to int Got: Traceback (most recent call last): File "/storage/sage/sage4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/storage/sage/sage4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/storage/sage/sage4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_1[11]>", line 1, in <module> ECModularSymbol(E)###line 53: sage: ECModularSymbol(E) File "newforms.pyx", line 69, in sage.libs.cremona.newforms.ECModularSymbol.__init__ (sage/libs/cremona/newforms.cpp:1820) a6 = new_bigint(int(E.a6())) OverflowError: Python int too large to convert to C longThese tests do not fail on amd64 because "long int too large to convert to int" is returned as the OverflowError? instead of "Python int too large to convert to C long".
Steve, what about
sage t long force_lib devel/sagemain/sage/rings/polynomial/polynomial_rational_flint.pyx
comment:89 in reply to: ↑ 88 ; followup: ↓ 90 Changed 10 years ago by
comment:90 in reply to: ↑ 89 Changed 10 years ago by
Replying to strogdon:
Replying to fbissey:
Replying to strogdon:
Steve, what about
sage t long force_lib devel/sagemain/sage/rings/polynomial/polynomial_rational_flint.pyxHere on x86 and amd64 the test passes. So, there seems to be some sort of inconsistency, doesn't it?
Yes! I'll fix it but it should be investigated. Gut feeling is on variation in cython definitions.
comment:91 Changed 10 years ago by
 Description modified (diff)
comment:92 Changed 10 years ago by
I've tried to remove the failures known above here. Note that I didn't get the last few patches in time, but I removed those failures as well, assuming they were the same. The timeouts/killed are probably unrelated  my computer is < 1 GHz, so I get a lot of timeouts and I forgot to set SAGE_TIMEOUT_LONG
before I ran the tests  but at any rate I hope this will be useful. After the hurricane I'll start this up again and run the remaining tests.
sage t long "devel/sage/sage/crypto/mq/mpolynomialsystem.py" # Killed/crashed sage t long "devel/sage/sage/functions/transcendental.py" sage t long "devel/sage/sage/groups/generic.py" # Killed/crashed sage t long "devel/sage/sage/interfaces/maxima.py" # Time out sage t long "devel/sage/sage/interfaces/singular.py" # Killed/crashed sage t long "devel/sage/sage/libs/cremona/newforms.pyx" sage t long "devel/sage/sage/libs/ppl.pyx" # Time out sage t long "devel/sage/sage/libs/singular/ring.pyx" # Killed/crashed sage t long "devel/sage/sage/misc/randstate.pyx" sage t long "devel/sage/sage/misc/sageinspect.py" sage t long "devel/sage/sage/rings/multi_power_series_ring_element.py" # Killed/crashed sage t long "devel/sage/sage/rings/polynomial/multi_polynomial_ideal.py" # Killed/crashed sage t long "devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx" # Killed/crashed sage t long "devel/sage/sage/rings/polynomial/polynomial_singular_interface.py" # Killed/crashed sage t long "devel/sage/sage/rings/real_mpfr.pyx" sage t long "devel/sage/sage/rings/tests.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/BSD.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_curve_isogeny.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_field.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_finite_field.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_generic.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_local_data.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_number_field.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_point.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py" # Time out sage t long "devel/sage/sage/schemes/elliptic_curves/gal_reps.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/heegner.py" # Killed/crashed sage t long "devel/sage/sage/schemes/elliptic_curves/period_lattice.py" # Killed/crashed sage t long "devel/sage/sage/schemes/generic/scheme.py" # Killed/crashed sage t long "devel/sage/sage/schemes/plane_conics/con_finite_field.py" # Killed/crashed sage t long "devel/sage/sage/schemes/plane_curves/projective_curve.py" # Killed/crashed sage t long "devel/sage/sage/stats/intlist.pyx" sage t long "devel/sage/sage/structure/sage_object.pyx" # Killed/crashed
comment:93 followup: ↓ 95 Changed 10 years ago by
Here is one example.
sage t long "devel/sage/sage/functions/transcendental.py" ********************************************************************** File "/Users/student/Desktop/sage4.7.2.alpha1/devel/sage/sage/functions/transcendental.py", line 83: sage: w = exponential_integral_1(2,4); w Expected: [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843924751e05] Got: [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.766562284392475e05] **********************************************************************
Here is one of the killed ones (not a timeout, I'm surprised):
sage t long "devel/sage/sage/crypto/mq/mpolynomialsystem.py" The doctested process was killed by signal 10 [352.0 s]
Could this be an unrelated 4.7.2.alpha1 issue on my platform?
comment:94 Changed 10 years ago by
I am particularly interested in the killed/crashed tests. And also those for which there is already a patch.
sage t long "devel/sage/sage/libs/cremona/newforms.pyx" sage t long "devel/sage/sage/rings/real_mpfr.pyx" sage t long "devel/sage/sage/stats/intlist.pyx"
are the 32/64 bits message difference already mentioned by Steve.
The timeouts are the regular suspect on ppc if I am not mistaken.
The transcendental.py doctest you are quoting is worrying me. It is patched in trac_9958fixing_numericalnoisepart1.patch but you have a slightly different result again (one less digit on the last number). We may have to consider a different patch.
All killed/crashed doctest are candidates for #11339 but you'll have to run sage gdb on them to be sure.
comment:95 in reply to: ↑ 93 Changed 10 years ago by
Replying to kcrisman:
Here is one of the killed ones (not a timeout, I'm surprised):
sage t long "devel/sage/sage/crypto/mq/mpolynomialsystem.py" The doctested process was killed by signal 10 [352.0 s]
Could this be an unrelated 4.7.2.alpha1 issue on my platform?
Does SIGUSR1
(#10) have a special meaning on Darwin?
comment:96 Changed 10 years ago by
Almost anything with "polynomial" in its name is likely to use singular (and crypto/mq/mpolynomialsystem.py is no exception), signal 10 is apparently a user defined signal so it could be anything.
We'll have to wait for the hurricane to move on before Karl can answer us on any of this points.
Looking back at you list Karl I am also very curious about
sage t long "devel/sage/sage/structure/sage_object.pyx sage t long "devel/sage/sage/misc/sageinspect.py" sage t long "devel/sage/sage/misc/randstate.pyx"
two of these have already been patched (sage_object and randstate) in this ticket so I am quite curious about what's wrong with them.
comment:97 Changed 10 years ago by
Yeah, I'm sorry I couldn't provide more information. The tests only just finished after classes, and I had one hour to copy/paste the above, restart just doing the failed tests, write a lecture, check email, and then turn off and unplug my PPC machine. I just couldn't do any more. I certainly wasn't taking an eMac home on the bike/train with me!
But we are just due for heavy rain and wind, so hopefully Monday I can get back on it and tell you more, save an extended power outage. Much better than an earthquake, I think fbissey would heartily agree. I wish I'd had a chance to run tests after #11339, but I just knew it would take eons. I'd do it on one of my home PPC machines (believe it or not, I have two more) but one I can no longer get !XCode for, and the other is also slow enough it wouldn't be ready by the time we have to unplug it... sigh.
Anyway, I'll keep this ticket on the priority list for the machine I did these tests on.
comment:98 Changed 10 years ago by
So, I did test this with Sage 4.7.2.alpha2 on my "last" 32bit machine, a Pentium4 Prescott (which has SSE3) running Ubuntu 9.04; my other Pentium4 (Northwood, without SSE3 / PNI) recently died, and I won't revive it in the near future.
The good news are: The patches all still apply to 4.7.2.alpha2, though many hunks with partially large offsets, but no fuzz.
The bad news: I expected some numerical noise because of 32bit architecture with (there rather rare) SSE3 (mfpmath=sse
), but also experienced at least one segfault (in sage/rings/homset.py
). I'll have to investigate the rest, for now just:
... sage t long force_lib "devel/sage/sage/geometry/polyhedra.py" ********************************************************************** File "/media/H1TBP6linux2/Sage/sage4.7.2.alpha2gcc4.5.1/devel/sage/sage/geometry/polyhedra.py", line 4948: sage: ppoints[0] Expected: (1.92296268638e15, 1.92296268638e15) Got: (0.0, 0.0) ********************************************************************** 1 items had failures: 1 of 8 in __main__.example_171 ***Test Failed*** 1 failures. For whitespace errors, see the file /home/leif/.sage//tmp/.doctest_polyhedra.py [166.0 s]  The following tests failed: sage t long force_lib "devel/sage/sage/libs/cremona/newforms.pyx" sage t long force_lib "devel/sage/sage/functions/transcendental.py" sage t long force_lib "devel/sage/sage/misc/randstate.pyx" sage t long force_lib "devel/sage/sage/combinat/words/suffix_trees.py" sage t long force_lib "devel/sage/sage/combinat/words/nfactor_enumerable_word.py" sage t long force_lib "devel/sage/sage/rings/morphism.pyx" sage t long force_lib "devel/sage/sage/rings/real_mpfr.pyx" sage t long force_lib "devel/sage/sage/rings/homset.py" sage t long force_lib "devel/sage/sage/stats/intlist.pyx" sage t long force_lib "devel/sage/sage/schemes/generic/scheme.py" sage t long force_lib "devel/sage/sage/geometry/polyhedra.py" Total time for all tests: 26263.8 seconds make: *** [testlong] Error 128 real 439m23.357s user 393m40.644s sys 31m7.485s leif@californication:~/Sage/sage4.7.2.alpha2gcc4.5.1$
(The tail of the log. More to come later.)
Note that I didn't test in parallel, although with some other CPUgreedy process running, but timeouts are unlikely. Vanilla 4.7.2.alpha2 passed all tests in exactly the same setting (and I did rebuild all dependent packages when installing the Python 2.7 spkg, modulo missing extension module dependencies in module_list.py
).
comment:99 followup: ↓ 100 Changed 10 years ago by
Ok, three segfaults (two of them without backtrace / Sage message, for whatever reason), some Python ints too large ("as usual" I guess), some numerical noise, the rest apparently related to dictionaries / hashing(?).
Attaching short logs of rerun tests (all reproducable)...
Changed 10 years ago by
Pentium4 Prescott (SSE3 / PNI, 32bit), GCC 4.5.1, native code, mfpmath=sse
, Ubuntu 9.04 x86
Changed 10 years ago by
Pentium4 Prescott (SSE3 / PNI, 32bit), GCC 4.5.1, native code, mfpmath=sse, Ubuntu 9.04 x86 (The three segfaulting files tested in verbose mode.)
comment:100 in reply to: ↑ 99 Changed 10 years ago by
Replying to leif:
Attaching short logs of rerun tests (all reproducable)...
Done.
Haha, the first segfault is an especially nice one:
... 10 tests in __main__.example_7 6 tests in __main__.example_8 7 tests in __main__.example_9 511 tests in 55 items. 511 passed and 0 failed. Test passed. Segmentation fault [9.8 s]  The following tests failed: sage t long verbose "devel/sage/sage/rings/morphism.pyx" Total time for all tests: 9.8 seconds real 0m10.013s user 0m6.980s sys 0m1.068s
comment:101 followups: ↓ 102 ↓ 106 Changed 10 years ago by
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
sage t long force_lib "devel/sage/sage/combinat/words/nfactor_enumerable_word.py"
is known to fail but is probably a minor adjustment.
sage t long force_lib "devel/sage/sage/combinat/words/suffix_trees.py"
is new. The rest look like #11339 material.
comment:102 in reply to: ↑ 101 Changed 10 years ago by
Replying to fbissey:
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
The failing doctests are all in Doctest_failures_Sage4.7.2.alpha2_Linux_x86_SSE3individual_tests_rerun.log.
[...] The rest look like #11339 material.
I'm just to apply the patches from there; the machine is dead slow, doing a ./sage baforce
atm to exclude missing extension module dependencies, as I've not built from scratch.
comment:103 Changed 10 years ago by
I was too fast. Some of the failures are already reported by Steve. I may try to update the patch set a bit tonight.
comment:104 Changed 10 years ago by
Just experienced some new(?) error when cloning the Sage library for #11339:
... bytecompiling /home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python2.7/sitepackages/sage/misc/copying.py to copying.pyc bytecompiling /home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python2.7/sitepackages/sage/version.py to version.pyc running install_egg_info Removing /home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python2.7/sitepackages/sage0.0.0py2.7.egginfo Writing /home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python2.7/sitepackages/sage0.0.0py2.7.egginfo Exception in thread Thread4 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python/threading.py", line 530, in __bootstrap_inner File "/home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python/threading.py", line 483, in run File "/home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python/multiprocessing/pool.py", line 272, in _handle_workers <type 'exceptions.TypeError'>: 'NoneType' object is not callable Exception in thread Thread1 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python/threading.py", line 530, in __bootstrap_inner File "/home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python/threading.py", line 483, in run File "/home/leif/Sage/sage4.7.2.alpha2gcc4.5.1/local/lib/python/multiprocessing/pool.py", line 272, in _handle_workers <type 'exceptions.TypeError'>: 'NoneType' object is not callable real 1m35.816s user 1m35.794s sys 0m5.648s After cloning, if you change any Sage library files and want to rebuild the html version of the reference manual, use the command sage docbuild reference html (after running 'sage b'). ...
comment:105 Changed 10 years ago by
Waaaaaah!
leif@californication:~/Sage/sage4.7.2.alpha2gcc4.5.1/devel/sage995811339$ ../../sage hg import v ~/Sage/patches/trac_11339_refcount_singular_polynomials.patch applying /home/leif/Sage/patches/trac_11339_refcount_singular_polynomials.patch patching file sage/rings/polynomial/multi_polynomial_libsingular.pxd patching file sage/rings/polynomial/multi_polynomial_libsingular.pyx Hunk #13 FAILED at 727 Hunk #14 FAILED at 741 Hunk #15 FAILED at 763 Hunk #16 FAILED at 773 Hunk #17 FAILED at 825 Hunk #18 succeeded at 910 with fuzz 2 (offset 72 lines). Hunk #19 FAILED at 875 Hunk #20 succeeded at 961 (offset 75 lines). Hunk #21 succeeded at 980 (offset 75 lines). Hunk #22 succeeded at 1006 (offset 75 lines). ...
(The first patch still applies, with offsets.)
I'm not going to fix that...
comment:106 in reply to: ↑ 101 ; followup: ↓ 107 Changed 10 years ago by
Replying to fbissey:
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
I'm sure the polyhedra.py patch was needed at one time but it now (?) appears that
@@ 4946,7 +4946,7 @@ sage: proj = ProjectionFuncStereographic([1.1,1.1,1.1]) sage: ppoints = [proj(vector(x)) for x in cube] sage: ppoints[0]  (0.0, 0.0) + (1.92296268638e15, 1.92296268638e15) """ def __init__(self, projection_point): """
from trac_9958fixing_numericalnoisepart1.patch is no longer needed.
comment:107 in reply to: ↑ 106 Changed 10 years ago by
Replying to strogdon:
Replying to fbissey:
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
I'm sure the polyhedra.py patch was needed at one time but it now (?) appears that
@@ 4946,7 +4946,7 @@ sage: proj = ProjectionFuncStereographic([1.1,1.1,1.1]) sage: ppoints = [proj(vector(x)) for x in cube] sage: ppoints[0]  (0.0, 0.0) + (1.92296268638e15, 1.92296268638e15) """ def __init__(self, projection_point): """from trac_9958fixing_numericalnoisepart1.patch is no longer needed.
You are right I am reading the test results backwards. It now seems to go to 0 properly. One less patch in the patch and one less worry.
comment:108 followup: ↓ 110 Changed 10 years ago by
Okay, you can get a full list of current failure and what they looked like at this link. Some are definitely known above, but others may be unknown. The timeouts are probably just due to this machine (and the Maxima one), but others I don't know whether they are here or #11339.
comment:109 followup: ↓ 115 Changed 10 years ago by
 Description modified (diff)
Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.
comment:110 in reply to: ↑ 108 Changed 10 years ago by
Replying to kcrisman:
Okay, you can get a full list of current failure and what they looked like at this link. Some are definitely known above, but others may be unknown. The timeouts are probably just due to this machine (and the Maxima one), but others I don't know whether they are here or #11339.
You have some nasty stuff in polyhedra.py
File "/Users/student/Desktop/sage4.7.2.alpha1/devel/sage/sage/geometry/polyhedra.py", line 5250: sage: _.show() Exception raised: Traceback (most recent call last): File "/Users/student/Desktop/sage4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/Users/student/Desktop/sage4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/Users/student/Desktop/sage4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_183[5]>", line 1, in <module> _.show()###line 5250: sage: _.show() File "/Users/student/Desktop/sage4.7.2.alpha1/local/lib/python/sitepackages/sage/misc/displayhook.py", line 174, in displayhook print_obj(sys.stdout, obj) File "/Users/student/Desktop/sage4.7.2.alpha1/local/lib/python/sitepackages/sage/misc/displayhook.py", line 142, in print_obj print >>out_stream, `obj` File "base.pyx", line 80, in sage.plot.plot3d.base.Graphics3d.__repr__ (sage/plot/plot3d/base.c:2526) File "base.pyx", line 1124, in sage.plot.plot3d.base.Graphics3d.show (sage/plot/plot3d/base.c:10344) File "base.pyx", line 663, in sage.plot.plot3d.base.Graphics3d.export_jmol (sage/plot/plot3d/base.c:5867) File "/Users/student/Desktop/sage4.7.2.alpha1/local/lib/python2.7/zipfile.py", line 1138, in writestr self.fp.write(zinfo.FileHeader()) File "/Users/student/Desktop/sage4.7.2.alpha1/local/lib/python2.7/zipfile.py", line 348, in FileHeader len(filename), len(extra)) error: integer out of range for 'H' format code
You get an extra digit in randstate compared to x86 :( [note to self it is in numerical_noise_part4].
sageinspect looks bad :( but not hopeless. The message is just overly long. I will have to postpone the rest for later.
comment:111 Changed 10 years ago by
comment:112 followup: ↓ 114 Changed 10 years ago by
In polyhedra.py it looks like you have a problem with jmol printing 3D images to file. It involves zip, looking at the pyx there is an instance of testing zip. Could you paste the cython code that produced sage/plot/plot3d/base.c:5867 ?
comment:113 Changed 10 years ago by
 Description modified (diff)
Updated the patch set again. split real_mpfr.pyx from the mixedfix patch. The 32/64 bits issues reported is separate from the message that was already patched. Added a patch to fix 32/64 bit messages.
comment:114 in reply to: ↑ 112 Changed 10 years ago by
Replying to fbissey:
In polyhedra.py it looks like you have a problem with jmol printing 3D images to file. It involves zip, looking at the pyx there is an instance of testing zip. Could you paste the cython code that produced sage/plot/plot3d/base.c:5867 ?
/* "sage/plot/plot3d/base.pyx":663 * f.write("isosurface fullylit; pmesh o* fullylit; set antialiasdisplay on;\n") * * render_params.output_archive.writestr('SCRIPT', f.getvalue()) # <<<<<<<<<<<<<< * render_params.output_archive.close() * */ __pyx_t_3 = PyObject_GetAttr(__pyx_v_render_params, __pyx_n_s__output_archive); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_3); __pyx_t_2 = PyObject_GetAttr(__pyx_t_3, __pyx_n_s__writestr); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = PyObject_GetAttr(__pyx_v_f, __pyx_n_s__getvalue); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = PyObject_Call(__pyx_t_3, ((PyObject *)__pyx_empty_tuple), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(((PyObject *)__pyx_t_3)); __Pyx_INCREF(((PyObject *)__pyx_n_s__SCRIPT)); PyTuple_SET_ITEM(__pyx_t_3, 0, ((PyObject *)__pyx_n_s__SCRIPT)); __Pyx_GIVEREF(((PyObject *)__pyx_n_s__SCRIPT)); PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_4 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_t_3), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(((PyObject *)__pyx_t_3)); __pyx_t_3 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
5867 is
__pyx_t_4 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_t_3), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
I was also surprised by the polyhedra.py additional weirdness. Anything I can do to help identify it? Warning: I'm planning on building and testing alpha2 without Python 2.7 now, which will tie up most resources on this machine for a couple days.
comment:115 in reply to: ↑ 109 ; followup: ↓ 116 Changed 10 years ago by
Replying to fbissey:
Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.
Here the fix_transcendental patch fixed the associated test on x86 but now the test fails on amd64:
sage t long force_lib devel/sagemain/sage/functions/transcendental.py ********************************************************************** File "/storage/sage/sage4.7.2.alpha2/devel/sagemain/sage/functions/transcendental.p y", line 83: sage: w = exponential_integral_1(2,4); w Expected: [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843 924...e05] Got: [0.04890051070806112, 0.0037793524098489063, 0.00036008245216265873, 3.7665622843 924534e05] **********************************************************************
This is odd! I wonder what changed? Previously this test must have returned ...67 and not ...63 for it to pass.
comment:116 in reply to: ↑ 115 Changed 10 years ago by
Replying to strogdon:
Replying to fbissey:
Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.
Here the fix_transcendental patch fixed the associated test on x86 but now the test fails on amd64:
sage t long force_lib devel/sagemain/sage/functions/transcendental.py ********************************************************************** File "/storage/sage/sage4.7.2.alpha2/devel/sagemain/sage/functions/transcendental.p y", line 83: sage: w = exponential_integral_1(2,4); w Expected: [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843 924...e05] Got: [0.04890051070806112, 0.0037793524098489063, 0.00036008245216265873, 3.7665622843 924534e05] **********************************************************************This is odd! I wonder what changed? Previously this test must have returned ...67 and not ...63 for it to pass.
It's my fault for not noticing this difference between 32 and 64 bit results. It was there before but I didn't notice it. I thought the last number was the only thing different so when I truncated the last number to the common digits I dropped the difference between 32 and 64 bit. So it is entirely my fault.
Changed 10 years ago by
fixing numerical noise in transcendental.py split from numerical noise pt1  updated 2011092
comment:117 followup: ↓ 118 Changed 10 years ago by
transcendental patch updated.
comment:118 in reply to: ↑ 117 ; followup: ↓ 119 Changed 10 years ago by
comment:119 in reply to: ↑ 118 Changed 10 years ago by
comment:120 followup: ↓ 121 Changed 10 years ago by
Just for reference, on the PPC machine all tests pass with alpha2 except three timeouts which are longstanding (I didn't bother to set SAGE_TIMEOUT_LONG
but I've seen these many times).
So all the rest are due to this ticket or #11339.
Would it be helpful for me to do #11339 on this new build (without Python 2.7), or is that in limbo right now anyway? Alternately, is there any way to decouple this ticket from that one?
comment:121 in reply to: ↑ 120 Changed 10 years ago by
Replying to kcrisman:
Just for reference, on the PPC machine all tests pass with alpha2 except three timeouts which are longstanding (I didn't bother to set
SAGE_TIMEOUT_LONG
but I've seen these many times).So all the rest are due to this ticket or #11339.
That's a very useful point of reference. Thank you for providing it.
Would it be helpful for me to do #11339 on this new build (without Python 2.7), or is that in limbo right now anyway? Alternately, is there any way to decouple this ticket from that one?
Technically you can apply #11339 without python 2.7  the behavior we want to correct is only revealed when we use python 2.7 but I guess it would be useful to know if there are any side effects if you apply the patch on a python 2.6 install. Bear in mind that the current patch are not necessarily the best solution to the problem, what's more they are a whack a mole game. Fix something here and some new fault appears in a completely unexpected location.
comment:122 Changed 10 years ago by
I will update the python spkg to 2.7.3 when it will be released. 2.7.3 will include the patch to python that we have been carrying in sage for the longest time. See: http://bugs.python.org/issue7689
Does anyone know if the apocalypse is about to happen?
comment:123 Changed 10 years ago by
 Description modified (diff)
Fixes #1159, according to that ticket.
comment:124 Changed 10 years ago by
That's really weird. Some of the hashing problems I encounter with python 2.7 are almost the same as reported in http://trac.sagemath.org/sage_trac/ticket/4957#comment:3
sage t long force_lib "devel/sagemain/sage/rings/integer.pyx" ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/integer.pyx", line 3046: sage: n = 920390823904823094890238490238484; n.__hash__() Expected: 6874330978542788722 Got: 2623069716 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/integer.pyx", line 3061: sage: hash(n) Expected: 9223372036854767616 Got: 8192 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/rings/integer.pyx", line 3064: sage: hash(n) == hash(int(n)) Expected: True Got: False **********************************************************************
comment:125 Changed 10 years ago by
With python2.6:
sage: n=2^63+2^13 sage: n 9223372036854784000 sage: int(n) 9223372036854784000L sage: hash(n) 9223372036854767616 sage: hash(int(n)) 9223372036854767616
with python2.7
sage: n=2^63+2^13 sage: n 9223372036854784000 sage: hash(n) 8192 sage: int(n) 9223372036854784000L sage: hash(int(n)) 9223372036854767616
Furthermore in both case:
sage: type(n) <type 'sage.rings.integer.Integer'> sage: type(int(n)) <type 'long'>
So I would expect different hashes naively.
comment:126 Changed 10 years ago by
Regarding
sage t long force_lib "devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py" ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py", line 20: sage: it.next() Expected: word: 6456 Got: word: 5645 ********************************************************************** File "/usr/share/sage/devel/sagemain/sage/combinat/words/nfactor_enumerable_word.py", line 22: sage: it.next() Expected: word: 5645 Got: word: 6456 **********************************************************************
With python2.7
sage: w = Word([4,5,6])^7 sage: w word: 456456456456456456456 sage: it = w.factor_iterator(4) sage: it.next() word: 5645 sage: it.next() word: 6456 sage: it.next() word: 4564
with python2.6
sage: w = Word([4,5,6])^7 sage: w word: 456456456456456456456 sage: it = w.factor_iterator(4) sage: it.next() word: 6456 sage: it.next() word: 5645 sage: it.next() word: 4564
So it looks like with python 2.6 we where moving in the series of number with a stride 2 while python 2.7 works through the sequence with a stride 1.
comment:127 Changed 10 years ago by
Here on x86 I get only two failures:
sage t long force_lib devel/sagemain/sage/misc/randstate.pyx ********************************************************************** File "/storage/sage/sage4.7.2.rc0/devel/sagemain/sage/misc/randstate.pyx", line 61: sage: rtest() Expected: (978, 0.184109262667515, 3*x^2  1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 603 59, 0.83350776541997362) Got: (978, 0.184109262667515, 3*x^2  1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 603 59, 0.8335077654199736) **********************************************************************
and
sage t long force_lib devel/sagemain/sage/matrix/matrix_double_dense.pyx ********************************************************************** File "/storage/sage/sage4.7.2.rc0/devel/sagemain/sage/matrix/matrix_double_dense.py x", line 1046: sage: A.singular_values(eps='junk') Expected: Traceback (most recent call last): ... ValueError: invalid literal for float(): junk Got: Traceback (most recent call last): File "/storage/sage/sage4.7.2.rc0/local/bin/ncadoctest.py", line 1231, in run_ one_test self.run_one_example(test, example, filename, compileflags) File "/storage/sage/sage4.7.2.rc0/local/bin/sagedoctest.py", line 38, in run_o ne_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags ) File "/storage/sage/sage4.7.2.rc0/local/bin/ncadoctest.py", line 1172, in run_ one_example compileflags, 1) in test.globs File "<doctest __main__.example_19[37]>", line 1, in <module> A.singular_values(eps='junk')###line 1046: sage: A.singular_values(eps='junk') File "matrix_double_dense.pyx", line 1075, in sage.matrix.matrix_double_dense.M atrix_double_dense.singular_values (sage/matrix/matrix_double_dense.c:6338) eps = RDF(eps) File "parent.pyx", line 988, in sage.structure.parent.Parent.__call__ (sage/str ucture/parent.c:7326) File "coerce_maps.pyx", line 82, in sage.structure.coerce_maps.DefaultConvertMa p_unique._call_ (sage/structure/coerce_maps.c:3268) File "coerce_maps.pyx", line 77, in sage.structure.coerce_maps.DefaultConvertMa p_unique._call_ (sage/structure/coerce_maps.c:3171) File "real_double.pyx", line 548, in sage.rings.real_double.RealDoubleElement._ _init__ (sage/rings/real_double.c:5567) ValueError: could not convert string to float: junk **********************************************************************
The latter also fails here the same way on amd64 and is documented on sagedevel
http://groups.google.com/group/sagedevel/browse_thread/thread/8e1d7f665ac5be37
but with some slight differences.
comment:128 Changed 10 years ago by
OK Steve, I got the matrix_double_dense but you don't get the numerical noise. I believe the noise may come from python2.7.2. Interesting that the hashing and enumeration problem are 64bit only. I'll check on the mac.
comment:129 Changed 10 years ago by
 Description modified (diff)
New patch to fix the message about junk.
comment:130 Changed 10 years ago by
 Dependencies changed from #11156 #11236 #11244 #11264 #11339 #11363 #11376 to #11156 #11236 #11244 #11264 #11339 #11363 #11376 #11986
I have now split the hashing issue in #11986
comment:131 Changed 10 years ago by
 Description modified (diff)
One more patch added to fix combinat/words/nfactor_enumerable_word.py on 64 bit system. I believe is output is equivalent in both case so it is not important.
comment:132 Changed 10 years ago by
 Dependencies changed from #11156 #11236 #11244 #11264 #11339 #11363 #11376 #11986 to #11986
 Description modified (diff)
Removed old dependencies.
comment:133 followup: ↓ 135 Changed 10 years ago by
 Milestone set to sage4.8
The spkg needs to be rebased to #11447.
comment:134 Changed 10 years ago by
Permissions is the spkg should be 0644 (or 0755 for executables), currently many files are 0640 or 0750.
comment:135 in reply to: ↑ 133 ; followup: ↓ 136 Changed 10 years ago by
comment:136 in reply to: ↑ 135 Changed 10 years ago by
Replying to fbissey:
Replying to jdemeyer:
The spkg needs to be rebased to #11447.
I looked at that before making my spkg. I believe the strategy used in #11447 is in python2.7. Someone with a debian system proves me wrong but I think that works out of the box.
Well, we did take the patch from Python upstream, but IIRC there are further changes to spkginstall
, and the new spkg needs the previous one to be mentioned in SPKG.txt
anyway (to pass the merger's sanity check).
comment:137 Changed 10 years ago by
Indeed, there've been a lot of changes inbetween:

spkginstall
diff Nu python2.7.1/spkginstall python2.6.4.p11/spkginstall
old new 5 5 6 6 CUR=`pwd` 7 7 8 if [ "$SAGE_LOCAL" = "" ]; then9 echo "SAGE_LOCAL undefined ... exiting";10 echo"Maybe run 'sage sh'?"11 exit 18 if [ z "$SAGE_LOCAL" ]; then 9 echo >&2 "SAGE_LOCAL undefined ... exiting" 10 echo >&2 "Maybe run 'sage sh'?" 11 exit 1 12 12 fi 13 13 14 14 # PATCH 15 15 16 # Due to issues building Python on Ubuntu 11.04, a file is patched 17 # on these systems. 16 # When building Python 2.6 on Debian and derivatives with multiarch, 17 # system library directories are not correctly setup. This patch 18 # fixes this, with noop on other systems. 19 # The fix requires 'dpkgarchitecture' to be installed there though 20 # (see below). 21 echo "Patching src/setup.py for multiarch." 22 patch p0 < patches/setup.py.multiarch.patch 23 if [ $? ne 0 ]; then 24 echo >&2 "Error patching src/setup.py" 25 exit 1 26 fi 18 27 19 if [ f /etc/issue ] && grep "Ubuntu 11.04" /etc/issue ; then 20 echo "Patching src/Modules/Setup.dist as this is Ubuntu 11.04" 21 patch p0 < patches/Modules.Setup.dist.patch 28 # Due to a python bug with Solaris 29 # see http://bugs.python.org/issue1759169 30 # it is necessary to apply a patch to configure.in 31 # then run autoconf. This not only generates a 32 # new 'configure' script, but some subdirectories too 33 # so these will be copied. 34 35 if [ "x`uname`" = xSunOS ] ; then 36 echo "Applying a revised 'configure' script for Solaris" 37 echo "See http://bugs.python.org/issue1759169" 38 echo "http://trac.sagemath.org/sage_trac/ticket/7867" 39 cp r patches/autom4te.cache patches/configure patches/configure.in src 22 40 if [ $? ne 0 ]; then 23 echo "Error patching src/Modules/Setup.dist" 41 echo >&2 "Failed to apply the Solaris patches needed for" 42 echo >&2 "http://bugs.python.org/issue1759169" 43 echo >&2 "http://trac.sagemath.org/sage_trac/ticket/7867" 24 44 exit 1 25 45 fi 46 echo "Setting HAVE_FD_TRANSFER=0 for Solaris to allow" 47 echo "the python module '_multiprocessing' to build" 48 echo "See: http://trac.sagemath.org/sage_trac/ticket/8440" 49 cp patches/setup.py src 50 if [ $? ne 0 ]; then 51 echo >&2 "Failed to apply the Solaris patch needed for" 52 echo >&2 "http://trac.sagemath.org/sage_trac/ticket/8440" 53 exit 1 54 fi 55 fi 56 57 58 cp patches/locale.py src/Lib/locale.py 59 if [ $? ne 0 ]; then 60 echo >&2 "Error copying patched locale.py" 61 exit 1 26 62 fi 27 63 28 patch p0 < patches/Lib.distutils.command.sdist.patch 64 cp patches/Makefile.pre.in src/Makefile.pre.in 29 65 if [ $? ne 0 ]; then 30 echo "Error copying patched sdist.py"66 echo >&2 "Error copying patched Makefile.pre.in" 31 67 exit 1 32 68 fi 33 69 34 patch p0 < patches/Lib.socket.patch 70 cp patches/sdist.py src/Lib/distutils/command/sdist.py 35 71 if [ $? ne 0 ]; then 36 echo "Error copying patched socket.py"72 echo >&2 "Error copying patched sdist.py" 37 73 exit 1 38 74 fi 39 75 40 patch p0 < patches/dynamic_class_copyreg_py.patch 76 cp patches/socket.py src/Lib/socket.py 41 77 if [ $? ne 0 ]; then 42 echo "Error copying patched pickle.py"78 echo >&2 "Error copying patched socket.py" 43 79 exit 1 44 80 fi 45 81 46 patch p0 < patches/dynamic_class_copyreg_c.patch 82 cp patches/pickle.py src/Lib/pickle.py 47 83 if [ $? ne 0 ]; then 48 echo "Error copying patched cPickle.c"84 echo >&2 "Error copying patched pickle.py" 49 85 exit 1 50 86 fi 51 87 88 cp patches/cPickle.c src/Modules/cPickle.c 89 if [ $? ne 0 ]; then 90 echo >&2 "Error copying patched cPickle.c" 91 exit 1 92 fi 93 94 # Due to a problem with _socket not building on OpenSolaris on x64 95 # see http://bugs.python.org/issue8852 96 # http://trac.sagemath.org/sage_trac/ticket/9041 97 # http://trac.sagemath.org/sage_trac/ticket/9022 98 # Modules/socketmodule.c needs patching. The patch consists of 99 # only checking if things are defined before trying to build with them 100 # so it is safe (and desirable) on any platform. 101 102 cp patches/Modules.socketmodule.c src/Modules/socketmodule.c 103 if [ $? ne 0 ]; then 104 echo >&2 "Error copying patched socketmodule.c" 105 exit 1 106 fi 107 108 # The following patch for fixing broken readline behavior on Itanium 109 # Linux definitely does *not* work on anything else. 110 if [ "`uname m`" = "ia64" a "`uname`" = "Linux" ]; then 111 echo "Updating readline.c for Linux/Itanium" 112 cp patches/readline.cItaniumfix src/Modules/readline.c 113 else 114 # Readline patch: http://bugs.python.org/file14599/python2.6readline.patch 115 # Associated bug: http://bugs.python.org/issue5833 116 # 117 # Committed to Python as r75747 in the py26maint branch, but not 118 # in time for 2.6.4  so we can remove this patch the next time 119 # we update Python in Sage. 120 cp patches/readline.cspacebug src/Modules/readline.c 121 fi 122 123 if [ $? ne 0 ]; then 124 echo >&2 "Error copying patched readline.c" 125 exit 1 126 fi 127 128 52 129 # We are setting LDFLAGS and CPPFLAGS so that we pick up Sage's readline 53 130 LDFLAGS="L$SAGE_LOCAL/lib $LDFLAGS" 54 131 export LDFLAGS … … 62 139 EXTRAFLAGS="withoutpymalloc"; export EXTRAFLAGS 63 140 fi 64 141 142 # Program around weird bug in build process: 143 # Apparently if you have this: 144 # export DISTUTILS_DEBUG=1 145 # in your environment variables, the build craps out. No idea why this 146 # is. 147 #  Yi Qiang 148 # 149 # This bug was fixed in Python, but not yet in Python 2.6.4. So this fix 150 # can be removed the next time we upgrade our version of Python. See 151 # 152 # http://bugs.python.org/issue6954 153 # 154 # for the fix. 155 # 156 unset DISTUTILS_DEBUG 157 158 65 159 cd src 66 160 67 161 touch Include/* … … 93 187 enableunicode=ucs4 94 188 fi 95 189 else 96 ./configure $EXTRAFLAGS prefix="$SAGE_LOCAL" enableunicode=ucs4 CC="$CC $CFLAGS" CXX="$CXX $CXXFLAGS" 190 ./configure $EXTRAFLAGS prefix="$SAGE_LOCAL" enableunicode=ucs4 \ 191 CC="$CC $CFLAGS" CXX="$CXX $CXXFLAGS" 97 192 fi 98 193 99 194 100 195 if [ $? ne 0 ]; then 101 echo "Error configuring Python."196 echo >&2 "Error configuring Python." 102 197 exit 1 103 198 fi 104 199 105 200 $MAKE 106 201 if [ $? ne 0 ]; then 107 echo "Error building Python."202 echo >&2 "Error building Python." 108 203 exit 1 109 204 fi 110 205 111 206 # running 'make install' in parallel is a bad, bad idea 207 # FIXME: The proper way to disable parallel make is to use 208 # $MAKE j1 ... (We rely on GNU make anyway). 112 209 MAKE=make; export MAKE 113 210 # the "i" is crucial, esp. in the case of a major upgrade 114 211 make i install 115 212 if [ $? ne 0 ]; then 116 echo "Error installing Python."213 echo >&2 "Error installing Python." 117 214 exit 1 118 215 fi 119 216 } … … 122 219 # informative error message. This is helps in determining why the 123 220 # configuration, compilation or installation failed. So put this before the 124 221 # build() function. 125 set +e 222 set +e # This is redundant here, but doesn't hurt to keep it... ;) 126 223 127 224 build 128 225 129 cd $SAGE_LOCAL/lib226 cd "$SAGE_LOCAL/lib" 130 227 131 228 # move the python directory if we're upgrading from a version 132 # of sage with python 2. 6133 if [ d python2. 6/sitepackages ]; then134 mv python2. 6/sitepackages python2.7/sitepackagesold229 # of sage with python 2.5 230 if [ d python2.5/sitepackages ]; then 231 mv python2.5/sitepackages python2.6/sitepackagesold 135 232 fi 136 233 137 rm rf python/python2. 7 python/python2.6 python/python2.5 python/python python python2.4 python2.5138 ln s python2. 7python234 rm rf python/python2.6 python/python2.5 python/python python python2.4 python2.5 235 ln s python2.6 python 139 236 if [ $? ne 0 ]; then 140 echo "Error creating symbolic link"237 echo >&2 "Error creating symbolic link" 141 238 exit 1 142 239 fi 143 240 144 241 # Sleeping for three seconds so that parallel 'make install' catches up 145 242 # with the following test. 243 # XXX We have disabled parallel 'make install' above, but never mind. 146 244 echo "Sleeping for three seconds before testing python" 147 245 sleep 3 148 246 … … 150 248 # All these modules are important and if any one 151 249 # fails to build, Sage will not work. 152 250 251 import_errors=false 153 252 for module in math hashlib crypt ; do 154 253 "$SAGE_LOCAL/bin/python" c "import $module" 155 254 if [ $? eq 0 ] ; then 156 255 echo "$module module imported OK" 157 256 else 158 echo "$module module failed to import" 159 exit 1 257 echo >&2 "$module module failed to import" 258 import_errors=true 259 # exit 1 # not yet 160 260 fi 161 261 done 262 263 if $import_errors; then 264 echo >&2 "Error: One or more modules failed to import." 265 # Check if we are on Debian or one of its derivatives: 266 if command v dpkg && ! command v dpkgarchitecture >/dev/null; then 267 echo >&2 "You may have to install 'dpkgarchitecture'" 268 echo >&2 "which is part of the Debian package 'dpkgdev'." 269 echo >&2 "Try installing it by typing" 270 echo >&2 " sudo aptget install dpkgdev" 271 echo >&2 "and rerun 'make'." 272 fi 273 exit 1 274 fi 275
comment:138 followup: ↓ 139 Changed 10 years ago by
OK that's not a biggy while I had a look at #11447 I made my spkg before it was closed and possibly before it had reached its final shape. I wanted to wait for 2.7.3 but I will probably cut a 2.7.2 in which I will add this then.
I am keen to have this landing ASAP but I would understand if the release manager only wants to do it in a alpha0. That's both a big change and a big patch set. My concern is that my last patch may not be adequate. Steve tested a number of configurations and the order this comes out are all over the place. I may have to make it random which is not nice.
The other concern is #11986 which looks like a blast from the past and I have no clues on either of them.
comment:139 in reply to: ↑ 138 Changed 10 years ago by
Replying to fbissey:
OK that's not a biggy while I had a look at #11447 I made my spkg before it was closed and possibly before it had reached its final shape. I wanted to wait for 2.7.3 but I will probably cut a 2.7.2 in which I will add this then.
I am keen to have this landing ASAP but I would understand if the release manager only wants to do it in a alpha0. That's both a big change and a big patch set.
If you can finish it, I'd be happy to merge it. But to be realistic, I don't think this ticket is close to being finished. There is a huge amount of patches which needs to be reviewed, there is still #11986, there is the rebasing to #11447, maybe testing will reveal breakage on some systems...
comment:140 Changed 10 years ago by
There is a warning when building the reference manual:
docstring of sage.crypto.boolean_function:3: (WARNING/2) Block quote ends without a blank line; unexpected unindent.
comment:141 Changed 10 years ago by
Another minor comment: please use UTF8 in SPKG.txt
. Currently, "François" is encoded as latin1.
comment:142 Changed 10 years ago by
comment:143 Changed 10 years ago by
 Milestone changed from sage4.8 to sage5.0
comment:144 Changed 10 years ago by
 Description modified (diff)
 Summary changed from Upgrade python to 2.7 to Upgrade python to 2.7.x
By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.
First attempt is at: http://sage.math.washington.edu/home/palmieri/SPKG/python2.7.p0.spkg
John (jhpalmieri) reports that this builds but Sage does not on sage.math, and "packages for twisted, zodb, pygments, and numpy don't build correctly."
Apparently the fix for http://bugs.python.org/issue7491 causes some of these problems.
On the numpy/scipy lists Ralf Gommers says "Numpy 1.5 should work with Python 2.7 and 3.1 and not be too far off. In August hopefully".
So it looks like http://trac.sagemath.org/sage_trac/ticket/9808 should fix the numpy issues.