#9958 closed enhancement (fixed)
Upgrade python to 2.7.x
| Reported by: | mhampton | Owned by: | tbd |
|---|---|---|---|
| Priority: | major | Milestone: | sage-5.0 |
| Component: | packages: standard | Keywords: | |
| Cc: | jhpalmieri, leif, jason, kcrisman, kini | Merged in: | sage-5.0.beta0 |
| Authors: | François Bissey, Steven Trogdon, Jeroen Demeyer | Reviewers: | John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé, François Bissey |
| Report Upstream: | N/A | Work issues: | |
| Branch: | Commit: | ||
| Dependencies: | #5852, #11986, #12085, #12096, #12124, #11915 | Stopgaps: |
Description (last modified by )
From the release notes:
Python 2.7.2 was released on June 11th, 2011.
The Python 2.7 series is scheduled to be the last major version in the 2.x series before 2.x moves into an extended maintenance period. The 2.7 series contains many of the features that were first released in Python 3.1. Improvements in this release include:
- An ordered dictionary type
- New unittest features including test skipping, new assert methods, and test discovery
- A much faster io module
- Automatic numbering of fields in the str.format() method
- Float repr improvements backported from 3.x
- Tile support for Tkinter
- A backport of the memoryview object from 3.x
- Set literals
- Set and dictionary comprehensions
- Dictionary views
- New syntax for nested with statements
- The sysconfig module
Fixes #1159
New spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p1.spkg (diffs for reviewing (without deleted files): python-2.7.2.p0.diff and python-2.7.2.p0-p1.diff)
Apply:
Attachments (46)
Change History (334)
comment:1 follow-up: ↓ 5 Changed 10 years ago by
comment:2 Changed 10 years ago by
- Cc jhpalmieri added
- Component changed from PLEASE CHANGE to packages
comment:3 Changed 10 years ago by
- Cc leif added
comment:4 Changed 10 years ago by
- Type changed from defect to enhancement
comment:5 in reply to: ↑ 1 Changed 10 years ago by
Replying to mhampton:
First attempt is at: http://sage.math.washington.edu/home/palmieri/SPKG/python-2.7.p0.spkg
John (jhpalmieri) reports that this builds but Sage does not on sage.math, and "packages for twisted, zodb, pygments, and numpy don't build correctly."
Apparently the fix for http://bugs.python.org/issue7491 causes some of these problems.
This fix has been included in python-2.6.5 onwards. We are using 2.6.5 which includes this fix and everything builds (haven't tried 2.7 yet). The only problem we have is with http://github.com/cschwan/sage-on-gentoo/issues#issue/1 so I think it is unfair to single out this issue as the source of problems.
comment:6 Changed 10 years ago by
Is there still any interest in this upgrade? I wanted to use ordered dictionaries, which require 2.7. Unfortunately, active participation in this ticket is beyond my current capabilities...
comment:7 Changed 10 years ago by
It will happen at least in sage-on-gentoo as we are following the system python. I have a couple of things in our tree to fix problems with python-2.6.5 and 2.6.6 we are looking into 2.7 now. I should post any patch to have it working here.
We are not talking about using the new capabilities just porting, I imagine my counterpart in Mandriva does the same.
comment:8 Changed 10 years ago by
- Status changed from new to needs_work
I just attached a patch that is needed for python 2.6.5 and later.
comment:9 Changed 10 years ago by
- Cc jason added
comment:10 Changed 10 years ago by
Build sage-4.6.2 (and dependency) against python-2.7.1 on OS X. I get a lot of the following
Expected nothing
Got:
Failure in _test_pickling:
Traceback (most recent call last):
File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/site-packages/sage/misc/sage_unittest.py", line 275, in run
test_method(tester = tester)
File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/site-packages/sage/misc/sage_unittest.py", line 498, in _test_pickling
tester.assertEqual(loads(dumps(self._instance)), self._instance)
File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/unittest/case.py", line 493, in assertEqual
assertion_func = self._getAssertEqualityFunc(first, second)
File "/Users/frb15/Desktop/Gentoo/usr/lib/python2.7/unittest/case.py", line 476, in _getAssertEqualityFunc
asserter = self._type_equality_funcs.get(type(first))
AttributeError: 'InstanceTester' object has no attribute '_type_equality_funcs'
------------------------------------------------------------
I patched python with the cpickle patch. Any ideas?
comment:11 Changed 10 years ago by
A lot of "doctest... DeprecationWarning?: ..." lines that were expected are just gone. Which fails the test.
comment:12 Changed 10 years ago by
Lots of numerical and a little bit of formatting noise.
comment:13 Changed 10 years ago by
Ok so now I understand the differences between unittest and unittest2 which is shipped with python-2.7. This will require a massive number of non-backward compatible changes. I am starting to experiment with a few sage components but that promise to be long and boring.
comment:14 Changed 10 years ago by
- Cc kcrisman added
comment:15 Changed 10 years ago by
I have attached a log of sage -testall. This is a sage-on-gentoo install a few tests are expected to fail https://github.com/cschwan/sage-on-gentoo/wiki/Known-test-failures but it should give you an idea of the problems we face.
comment:16 Changed 10 years ago by
- Description modified (diff)
comment:17 Changed 10 years ago by
I attached a log of test failures with 4.7.alpha4 (+ #7377). It is mostly number of decimals and messages. I added PYTHONWARNINGS=default to sage-env so I collected extra messages. The most important one being the deprecations of "sets".
There are 3 tests killed reason currently unknown. And a few that may need special attention.
comment:18 Changed 10 years ago by
A little bit more details, these are curious:
sage -t -long -force_lib devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx", line 2297:
sage: hash(R(-1))
Expected:
95367431640624
Got:
1977800240
sage -t -long -force_lib devel/sage-main/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/suffix_trees.py", line 1345:
sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
True
Got:
False
**********************************************************************
sage -t -long -force_lib devel/sage-main/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/suffix_trees.py", line 1345:
sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
True
Got:
False
**********************************************************************
This one doesn't worry me as much but should be looked at
sage -t -long -force_lib devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 22:
sage: it.next()
Expected:
word: 5645
Got:
word: 4564
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 24:
sage: it.next()
Expected:
word: 4564
Got:
word: 5645
**********************************************************************
sage -t -long -force_lib devel/sage-main/sage/structure/parent.pyx
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/structure/parent.pyx", line 634:
sage: CCls()._test_eq()
Expected:
Traceback (most recent call last):
...
AssertionError: <class '__main__.CCls'> == None
Got nothing
The following were killed:
sage -t -long -force_lib devel/sage-main/sage/rings/homset.py # Killed/crashed sage -t -long -force_lib devel/sage-main/sage/schemes/generic/scheme.py # Killed/crashed
In my original run
sage -t -long -force_lib "devel/sage-main/sage/rings/morphism.pyx"
also got killed but not in a subsequent run after I adopted a small change in sage-doctest to get rid of PYTHONWARNINGS=default in sage-env.
Example of killed test:
sage -t -long -verbose -force_lib "devel/sage-main/sage/rings/homset.py"
...
Trying:
phi = S.hom([b,a])###line 141:_sage_ >>> phi = S.hom([b,a])
Expecting nothing
ok
Trying:
phi == loads(dumps(phi))###line 142:_sage_ >>> phi == loads(dumps(phi))
Expecting:
True
/usr/lib64/libcsage.so(print_backtrace+0x24)[0x7f83de940534]
/usr/lib64/libcsage.so(sigdie+0x1d)[0x7f83de9405cd]
/usr/lib64/libcsage.so(sage_signal_handler+0x131)[0x7f83de940741]
/lib64/libpthread.so.0(+0xfae0)[0x7f83e2385ae0]
/usr/lib64/libsingular.so.3(_Z9id_DeletePP10sip_sidealP9sip_sring+0x53)[0x7f83c5c48383]
/usr/lib64/python2.7/site-packages/sage/libs/singular/groebner_strategy.so(+0x388d)[0x7f83ad27488d]
/usr/lib64/libpython2.7.so.1.0(PyDict_Clear+0xfc)[0x7f83e261597c]
/usr/lib64/libpython2.7.so.1.0(+0x82a09)[0x7f83e2615a09]
/usr/lib64/libpython2.7.so.1.0(+0x112e7e)[0x7f83e26a5e7e]
/usr/lib64/libpython2.7.so.1.0(_PyObject_GC_Malloc+0x11c)[0x7f83e26a684c]
/usr/lib64/libpython2.7.so.1.0(_PyObject_GC_New+0xd)[0x7f83e26a685d]
/usr/lib64/libpython2.7.so.1.0(+0x7fe11)[0x7f83e2612e11]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/python2.7/site-packages/sage/rings/polynomial/polydict.so(+0x168c3)[0x7f83c8f378c3]
/usr/lib64/libpython2.7.so.1.0(+0xa0468)[0x7f83e2633468]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/python2.7/site-packages/sage/rings/polynomial/polydict.so(+0xa451)[0x7f83c8f2b451]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47)[0x7f83e2670ca7]
/usr/lib64/python2.7/lib-dynload/cPickle.so(+0x5f06)[0x7f83dcb2df06]
/usr/lib64/python2.7/lib-dynload/cPickle.so(+0xaf47)[0x7f83dcb32f47]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/python2.7/site-packages/sage/structure/sage_object.so(+0x152fc)[0x7f83dc7012fc]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x56bc)[0x7f83e267692c]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f83e26782f2]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5a78)[0x7f83e2676ce8]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(+0x71282)[0x7f83e2604282]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(+0x71282)[0x7f83e2604282]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(+0x7138b)[0x7f83e260438b]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(+0x5a24f)[0x7f83e25ed24f]
/usr/lib64/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7f83e25de453]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x491d)[0x7f83e2675b8d]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x522e)[0x7f83e267649e]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88d)[0x7f83e26781dd]
/usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f83e26782f2]
/usr/lib64/libpython2.7.so.1.0(+0xff25c)[0x7f83e269225c]
/usr/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x90)[0x7f83e2693090]
/usr/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0x1ff)[0x7f83e2693c6f]
/usr/lib64/libpython2.7.so.1.0(Py_Main+0xb53)[0x7f83e26a4fc3]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f83e201cb6d]
/usr/bin/python2.7[0x4008a9]
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred in Sage.
This probably occurred because a *compiled* component of Sage has a bug
in it and is not properly wrapped with sig_on(), sig_off(). You might
want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate.
------------------------------------------------------------------------
The doctested process was killed by signal 11
[2.5 s]
Changed 10 years ago by
small patch to sage-doctest in sage_script to reenable sage's deprecation warnings. Idea by my friend Steve Trogdon.
Changed 10 years ago by
this form of comparison is removed in python 2.6.5 and later, refreshed using --git.
comment:19 Changed 10 years ago by
It looks like the crash problems originates somewhere in libsingular
sage: f = Zmod(6).cover()
sage: f.kernel()
Principal ideal (6) of Integer Ring
sage: R.<x,y> = PolynomialRing(QQ, 2)
sage: S.<a,b> = R.quo(x^2 + y^2)
sage: phi = S.cover()
sage: phi == loads(dumps(phi))
True
sage: phi == R.quo(x^2 + y^3).cover()
Program received signal SIGSEGV, Segmentation fault.
id_Delete (h=0x50687d8, r=0x0) at ideals.cc:127
127 ideals.cc: No such file or directory.
in ideals.cc
(gdb) bt
#0 id_Delete (h=0x50687d8, r=0x0) at ideals.cc:127
#1 0x00007fffb9670b49 in __pyx_pf_4sage_4libs_8singular_17groebner_strategy_16GroebnerStrategy_1__dealloc__ (
o=<value optimized out>) at sage/libs/singular/groebner_strategy.cpp:2570
#2 __pyx_tp_dealloc_4sage_4libs_8singular_17groebner_strategy_GroebnerStrategy (o=<value optimized out>)
at sage/libs/singular/groebner_strategy.cpp:3190
#3 0x00007ffff7aa98c7 in PyDict_Clear (op=<value optimized out>) at Objects/dictobject.c:891
#4 0x00007ffff7aa990f in dict_tp_clear (op=<value optimized out>) at Objects/dictobject.c:2088
#5 0x00007ffff7b32f27 in delete_garbage (generation=0) at Modules/gcmodule.c:769
#6 collect (generation=0) at Modules/gcmodule.c:930
#7 0x00007ffff7b33616 in collect_generations (basicsize=<value optimized out>) at Modules/gcmodule.c:996
#8 _PyObject_GC_Malloc (basicsize=<value optimized out>) at Modules/gcmodule.c:1457
#9 0x00007ffff7ac4a96 in PyType_GenericAlloc (type=0x7ffff7d9ad40, nitems=0) at Objects/typeobject.c:744
#10 0x00007ffff7a8d256 in BaseException_new (type=<value optimized out>, args=<value optimized out>, kwds=<value optimized out>)
at Objects/exceptions.c:34
#11 0x00007ffff7ac57b5 in type_call (type=0x7ffff7d9ad40, args=0x4ae46d0, kwds=0x0) at Objects/typeobject.c:712
#12 0x00007ffff7a77539 in PyObject_Call (func=0x7ffff7d9ad40, arg=0x4ae46d0, kw=0x0) at Objects/abstract.c:2529
#13 0x00007ffff7b00b45 in PyEval_CallObjectWithKeywords (func=0x7ffff7d9ad40, arg=0x4ae46d0, kw=0x0) at Python/ceval.c:3881
#14 0x00007ffff7b10443 in PyErr_NormalizeException (exc=0x7fffffffa778, val=0x7fffffffa770, tb=0x7fffffffa768)
at Python/errors.c:190
#15 0x00007fffe433021c in __Pyx_GetException (type=0x7fffffffa7c0, value=0x7fffffffa7b8, tb=0x7fffffffa7c8)
at sage/structure/parent.c:20909
#16 0x00007fffe433e4d1 in __pyx_pf_4sage_9structure_6parent_6Parent_2element_class (__pyx_v_self=0x4aea500,
unused=<value optimized out>) at sage/structure/parent.c:4251
#17 0x00007ffff7aac1e2 in PyCFunction_Call (func=0x504ebd8, arg=0x7ffff7f81050, kw=<value optimized out>)
at Objects/methodobject.c:90
#18 0x00007ffff7a77539 in PyObject_Call (func=0x504ebd8, arg=0x7ffff7f81050, kw=0x0) at Objects/abstract.c:2529
#19 0x00007ffff7b00b45 in PyEval_CallObjectWithKeywords (func=0x504ebd8, arg=0x7ffff7f81050, kw=0x0) at Python/ceval.c:3881
#20 0x00007ffff7a8b51f in methoddescr_call (descr=<value optimized out>, args=0x7ffff7f81050, kwds=0x0)
at Objects/descrobject.c:246
#21 0x00007ffff7a77539 in PyObject_Call (func=0x138e950, arg=0x7ffff7e9c050, kw=0x0) at Objects/abstract.c:2529
#22 0x00007ffff7b05efe in do_call (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4230
#23 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4035
#24 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665
#25 0x00007ffff7b07b65 in PyEval_EvalCodeEx (co=0x1159e30, globals=<value optimized out>, locals=<value optimized out>,
args=<value optimized out>, argcount=3, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3252
#26 0x00007ffff7a99a97 in function_call (func=0x115d5f0, arg=0xfadeb0, kw=0x0) at Objects/funcobject.c:526
#27 0x00007ffff7a77539 in PyObject_Call (func=0x115d5f0, arg=0xfadeb0, kw=0x0) at Objects/abstract.c:2529
#28 0x00007ffff7a77cca in PyObject_CallFunctionObjArgs (callable=0x115d5f0) at Objects/abstract.c:2760
#29 0x00007ffff7ac8a8e in slot_tp_descr_get (self=0x13179d0, obj=0x4aea500, type=0x19e91f0) at Objects/typeobject.c:5621
#30 0x00007ffff7aae433 in _PyObject_GenericGetAttrWithDict (obj=0x4aea500, name=0x13c5a78, dict=0x5068fb0) at Objects/object.c:1432
#31 0x00007ffff7aae4d1 in PyObject_GenericGetAttr (obj=<value optimized out>, name=<value optimized out>) at Objects/object.c:1454
#32 0x00007fffe434ed62 in __pyx_tp_getattro_4sage_9structure_6parent_Parent (o=0x4aea500, n=0x13c5a78)
at sage/structure/parent.c:18702
#33 0x00007ffff7aad8f4 in PyObject_GetAttr (v=0x4aea500, name=<value optimized out>) at Objects/object.c:1189
#34 0x00007ffff7afdbab in builtin_hasattr (self=<value optimized out>, args=<value optimized out>) at Python/bltinmodule.c:885
#35 0x00007ffff7aac1a0 in PyCFunction_Call (func=0x7ffff7fae3b0, arg=0x503d680, kw=<value optimized out>)
at Objects/methodobject.c:81
#36 0x00007ffff7b05ba3 in call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4012
#37 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665
#38 0x00007ffff7b05cfa in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4098
#39 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4033
#40 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2665
#41 0x00007ffff7b07b65 in PyEval_EvalCodeEx (co=0x1159e30, globals=<value optimized out>, locals=<value optimized out>,
args=<value optimized out>, argcount=3, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3252
#42 0x00007ffff7a99a97 in function_call (func=0x115d5f0, arg=0xfadbe0, kw=0x0) at Objects/funcobject.c:526
comment:20 Changed 10 years ago by
Same two doctests killed after upgrade to alpha5.
comment:21 Changed 10 years ago by
Over-eager garbage collection in python-2.7.1! Preceding the sequence by
sage: import gc sage: gc.disable()
Makes everything go smoothly.
comment:22 Changed 10 years ago by
- Description modified (diff)
comment:23 follow-up: ↓ 26 Changed 10 years ago by
trac_9958-fix_cmp.patch is perfectly reasonable, since this is the idiom used everywhere else in similar situations. I don't have enough background to judge on the warning patch.
comment:24 Changed 10 years ago by
Put an updated list of tests failing with 4.7.alpha5 and the latest patches from Nicolas M. Thiery. I think the next area do need some work is the problem of garbage collection in polynomial rings that leads to "random" failures in libsingular. See: http://groups.google.com/group/sage-devel/browse_thread/thread/8c165c887d6b9e54
comment:25 Changed 10 years ago by
- Description modified (diff)
Split the deprecation issue in its own ticket in #11244 - this is a different patch.
comment:26 in reply to: ↑ 23 Changed 10 years ago by
Replying to nthiery:
trac_9958-fix_cmp.patch is perfectly reasonable, since this is the idiom used everywhere else in similar situations. I don't have enough background to judge on the warning patch.
I will put the cmp patch in its own ticket would you review it?
comment:27 Changed 10 years ago by
I have attached some patches to fix all the noise for numerics and formatting changes. What's left should be looked at before patching.
There are several failures because hashes have changed but I cannot say it is allright to just change the doctest:
sage -t -long -force_lib "devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/padics/padic_capped_relative_element.pyx", line 2339:
sage: hash(R(-1))
Expected:
95367431640624
Got:
1977800240
**********************************************************************
sage -t -long -force_lib "devel/sage-main/sage/rings/integer.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 2995:
sage: n = -920390823904823094890238490238484; n.__hash__()
Expected:
6874330978542788722
Got:
-2623069716
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3010:
sage: hash(n)
Expected:
-9223372036854767616
Got:
8192
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3013:
sage: hash(n) == hash(int(n))
Expected:
True
Got:
False
**********************************************************************
That hash(n) == hash(int(n)) is now false should be looked into carefully.
sage -t -long -force_lib "devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 20:
sage: it.next()
Expected:
word: 6456
Got:
word: 5645
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 22:
sage: it.next()
Expected:
word: 5645
Got:
word: 6456
**********************************************************************
I think that one is OK, just a change of order but a second opinion would be nice.
Finally there are still three tests killed in libsingular's id_delete because a ring has been garbage collected even so there are some precautions in place for it: http://groups.google.com/group/sage-devel/browse_thread/thread/8c165c887d6b9e54
Next we need an updated python spkg so that more people can play and even try the patchbot on it.
comment:28 Changed 10 years ago by
- Description modified (diff)
The replacement of type.cmp by cmp in strata.py is now in its own ticket #11264 as it can be applied independently of a python upgrade.
comment:29 Changed 10 years ago by
I forgot because I build on sage-on-gentoo but I'll post some patch to build against python-2.7 a little bit later.
comment:30 Changed 10 years ago by
Unless someone steps in I'll make a new python-2.7.1 spkg by the end of the week. I'll have to figure out if the latest change going in for sage-4.7 will be needed for that one.
comment:31 Changed 9 years ago by
I won't be able to do that this week.
comment:32 Changed 9 years ago by
- Description modified (diff)
Our lead sage-on-gentoo debugger Martin tracked down the segfault problem to cython, this is now #11339
comment:33 Changed 9 years ago by
I'd like to try this but don't have time to figure out exactly what patches are needed in what order - can someone update the description with this? Also, is it the same spkg that Marshall originally posted that one should try?
comment:34 follow-up: ↓ 35 Changed 9 years ago by
Also, is it possible to test this by ./sage -f something, or would one have to take a fresh tarball and replace the Python spkg?
comment:35 in reply to: ↑ 34 Changed 9 years ago by
Replying to kcrisman:
Also, is it possible to test this by ./sage -f something, or would one have to take a fresh tarball and replace the Python spkg?
Hi Karl,
I am mainly developing from sage-on-gentoo at the moment. I need to make a current python-2.7.1 spkg to match. I will update the description and give the patch order when there is a spkg (feel free to make one). The patch have initially been made against 4.7.1.alpha0 and the current version works against 4.7.1.alpha1. You would get failures against a 4.7 because #7377 finally landed.
I think the best course of action will be to start from scratch, it may even be useful to host an alternative tarball somewhere because the change in python version will be extremely messy. I don't know if we can do it without breaking sage -upgrade or being overly complicated.
In the meantime you could have a look at #11244 (which I need to update for 4.7.1.alpha1 because of #10334).
Francois
comment:36 Changed 9 years ago by
OK if you want to look at #11244 it can be tested against either the latest 4.7 or 4.7.1.alpha1 now.
comment:37 follow-up: ↓ 38 Changed 9 years ago by
I see. Well, I just want to make sure that it all works fine on older Macs, of course. I definitely don't have time to make an spkg now, though I do have time to do ./sage -f and slap a few patches on it :) Just let me know when you get around to it; I don't think there is a huge rush.
I should probably test 4.7.1.alpha1 on my box. I'm amazed at Jeroen's stamina on this, especially with several different versions simultaneously existing.
comment:38 in reply to: ↑ 37 Changed 9 years ago by
Replying to kcrisman:
I see. Well, I just want to make sure that it all works fine on older Macs, of course. I definitely don't have time to make an spkg now, though I do have time to do ./sage -f and slap a few patches on it :) Just let me know when you get around to it; I don't think there is a huge rush.
I should probably test 4.7.1.alpha1 on my box. I'm amazed at Jeroen's stamina on this, especially with several different versions simultaneously existing.
That's why I asked about #11244 it is an issue I split from this ticket because it should apply on a sage running python-2.6 without side effects. So it could be in place when we come around to the upgrade. Any ideas on #11339 which was recently split from this ticket is also welcome.
comment:39 Changed 9 years ago by
- Description modified (diff)
My friend Steven Trogdon made some quick and dirty spkgs. His early tests indicate that setuptools needs to be updated. This is now #11363 it also revealed that I missed something in misc/cython.py watch out for an updated patch sometimes this weekend. There were also a few things that needed to be done in two other patches which I refreshed yesterday. I have a python-2.7.1 spkg ready to be uploaded probably sometimes today. If you want fun before that you can always use Steve's spkg (they are not in a reviewable state) at: http://www.d.umn.edu/~strogdon/sage/
comment:40 Changed 9 years ago by
- Description modified (diff)
Didn't take long. I have been naughty and committed two changes to patches that needed touching after my first commit of python-2.7.1 so the log will show 3 entries for just the one SPKG.txt entry. They were minor changes. I dropped a number of patches because they were applied upstream. However in the case of two patches I dropped them because the upstream has changed in a way they don't apply anymore but I don't know if patching is still necessary for the underlying issues. If so new patches will need to be developed. The issues are
- readline on itanium
- multiprocessing on solaris
As promised I will give a patch order. The next to last one needs updating as stated earlier and the last one is really optional. The patches do not have to be taken n that order as they don't overlap.
comment:41 Changed 9 years ago by
I forgot apply the patches on 4.7.1.alpha1, there may be rejection on 4.7.rcX.
comment:42 Changed 9 years ago by
misc/cython.py patch updated for people who want to test it.
comment:43 Changed 9 years ago by
corrected a syntax error in the last patch.
Changed 9 years ago by
Not strictly necessary but this file needed a scrub. (refreshed for syntax errors)
comment:44 Changed 9 years ago by
Had to redo trac_9958-build_module_listpy.patch again as I messed up the syntax of one bit.
comment:45 follow-up: ↓ 50 Changed 9 years ago by
My friend Steve Trogdon made a prepatched sage-4.7.1.alpha1 spkg (4.7.1.alpha1.p0 we named it) available to make testing easier, find it with hos own test.log here http://www.d.umn.edu/~strogdon/sage/ It has all the necessary patches from #11244 and this this ticket. We are a bit puzzled by the cause of failure in sage/misc/cython.py, white spaces?
comment:46 follow-up: ↓ 47 Changed 9 years ago by
I am getting the twisted install fail that jhpalmieri refers to. So my guess is that the Gentoo system has some updated things along those lines that vanilla 4.7.1.alpha1 does not. Is that possible?
comment:47 in reply to: ↑ 46 Changed 9 years ago by
Replying to kcrisman:
I am getting the twisted install fail that jhpalmieri refers to. So my guess is that the Gentoo system has some updated things along those lines that vanilla 4.7.1.alpha1 does not. Is that possible?
Well sage-on-gentoo uses a newer twisted. On the other I and Steve have build twisted in vanilla sage with python-2.7.1. Did you get the new setuptools spkg from #11363? It is necessary to build twisted with python-2.7.1.
comment:48 Changed 9 years ago by
No, I didn't realize there was an spkg involved with that. I installed that first, and it seems to be working now.
comment:49 Changed 9 years ago by
- Description modified (diff)
comment:50 in reply to: ↑ 45 Changed 9 years ago by
Replying to fbissey:
My friend Steve Trogdon made a prepatched sage-4.7.1.alpha1 spkg (4.7.1.alpha1.p0 we named it) available to make testing easier, find it with hos own test.log here http://www.d.umn.edu/~strogdon/sage/ It has all the necessary patches from #11244 and this this ticket.
Is one of these patches needed before doing the Sage spkg itself? I get an immediate failure
gcc -o src/convert.os -c -fPIC -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/python2.6 -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/NTL -Iinclude src/convert.c
In file included from include/memory.h:23,
from /Users/student/Desktop/sage-4.7.1.alpha1/local/include/pari/pari.h:42,
from include/convert.h:12,
from src/convert.c:14:
include/interrupt.h:32:20: error: Python.h: No such file or directory
scons: *** [src/convert.os] Error 1
*** TOUCHING ALL CYTHON (.pyx) FILES ***
gcc -o src/convert.os -c -fPIC -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/python2.6 -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/NTL -Iinclude src/convert.c
In file included from include/memory.h:23,
from /Users/student/Desktop/sage-4.7.1.alpha1/local/include/pari/pari.h:42,
from include/convert.h:12,
from src/convert.c:14:
include/interrupt.h:32:20: error: Python.h: No such file or directory
scons: *** [src/convert.os] Error 1
----------------------------------------------------------
sage: Building and installing modified Sage library files.
Installing c_lib
gcc -o src/convert.os -c -fPIC -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/python2.6 -I/Users/student/Desktop/sage-4.7.1.alpha1/local/include/NTL -Iinclude src/convert.c
In file included from include/memory.h:23,
from /Users/student/Desktop/sage-4.7.1.alpha1/local/include/pari/pari.h:42,
from include/convert.h:12,
from src/convert.c:14:
include/interrupt.h:32:20: error: Python.h: No such file or directory
scons: *** [src/convert.os] Error 1
ERROR: There was an error building c_lib.
which I assume is related to one of the patches involved here, though I couldn't say which one...
comment:51 Changed 9 years ago by
Did you rebuild cython after the python upgrade? In fact I have now added a list of package that needs to be rebuild after the python upgrade. Not all of them are necessary for building sage (scipy could be done after) but cython, numpy, matplotlib and pynac really need to be rebuilt.
Once you have done that apply all the patches with build in their names:
- trac_9958-build_setuppy.patch
- trac_9958-build-sage_clib.patch
- trac_9958-build_misc_cythonpy.patch
- trac_9958-build_module_listpy.patch (actually optional)
That should help sage build but will not correct many of the doctests. To fix most of the doctests apply the patches in #11244 as well as the one in the description of this bug. I have made a list now, have you seen it?
comment:52 follow-up: ↓ 53 Changed 9 years ago by
Well, I'm building from scratch, so that shouldn't affect it from that side.
Are you saying that before I build the Sage library, I have to apply these patches? That's a little tricky in a scratch build (for me, at least).
Yes, I saw the lists, but I thought they were only relevant for an upgrade/after actually building Sage. My apologies. I may not be able to finish testing this properly for a while now.
comment:53 in reply to: ↑ 52 Changed 9 years ago by
Replying to kcrisman:
Well, I'm building from scratch, so that shouldn't affect it from that side.
Are you saying that before I build the Sage library, I have to apply these patches? That's a little tricky in a scratch build (for me, at least).
Yes, I saw the lists, but I thought they were only relevant for an upgrade/after actually building Sage. My apologies. I may not be able to finish testing this properly for a while now.
That's all right there are still things to sort out. But you could use the sage spkg prepared by Steve with all patches included rather than patch yourself: http://www.d.umn.edu/~strogdon/sage/sage-4.7.1.alpha1.p0.spkg
And yes it is tricky from scratch unless you have a premade spkg.
comment:54 follow-up: ↓ 55 Changed 9 years ago by
I may have made mistakes, but I tried to do this, starting with sage-4.7.rc4:
- install the new python spkg
- install the new setuptools spkg (from #11363)
- apply the patches from #11244 to the main Sage library
- apply the patches from this ticket with "build" in their name -- some of these didn't apply cleanly, so I had to work around that.
I used that to build a new version of Sage (using "sage -sdist ..."). The tar file is here:
On sage.math, I've been able to untar this and build Sage, although tests haven't run yet. You can also try upgrading, although I haven't tested this at all: run
$ sage -upgrade http://sage.math.washington.edu/home/palmieri/misc/python2.7/sage-4.7.rc4-python2.7/
comment:55 in reply to: ↑ 54 Changed 9 years ago by
Replying to jhpalmieri:
I may have made mistakes, but I tried to do this, starting with sage-4.7.rc4:
- install the new python spkg
- install the new setuptools spkg (from #11363)
- apply the patches from #11244 to the main Sage library
- apply the patches from this ticket with "build" in their name -- some of these didn't apply cleanly, so I had to work around that.
I used that to build a new version of Sage (using "sage -sdist ..."). The tar file is here:
On sage.math, I've been able to untar this and build Sage, although tests haven't run yet. You can also try upgrading, although I haven't tested this at all: run
$ sage -upgrade http://sage.math.washington.edu/home/palmieri/misc/python2.7/sage-4.7.rc4-python2.7/
I have tried to keep abreast of everything so the patches are based on 4.7.1.alpha1 there are enough differences that a lot of patches for 4.7 would need rebasing.
So I decided to go straight for the bleeding edge. I am doing some more work on the "build" patches, I am trying to do something smarter with the python version number so some change could be applied independently of this ticket. The module_list,py patch definitely won't apply cleanly on 4.7.rc mainly because of #10334. What's more #11264 and #11236 which are needed here are merged in 4.7.1.alpha0.
That's all a bit of hassle to start from 4.7.1.alpha, but I would have to start again if I was starting from 4.7.rc.
comment:56 Changed 9 years ago by
I have updated trac_9958-build-sage_clib.patch it could now be splited in its own ticket. It enables sage-clib to be smarter about the python version used. It would figure it on its own. I am looking at doing the same for setup.py, anyone would review that in its own ticket? Once in, we could at least build sage against python2.6 or 2.7 without changes.
Changed 9 years ago by
change needed in setup.py made it python version smart. I also scrubbed the debian stuff
comment:57 Changed 9 years ago by
OK I also made setup.py python version smart any volunteer to review a separate ticket?
comment:58 Changed 9 years ago by
g++ -o libcsage.dylib -single_module -flat_namespace -undefined dynamic_lookup -dynamiclib src/convert.os src/interrupt.os src/memory.os src/mpn_pylong.os src/mpz_pylong.os src/mpz_longlong.os src/stdsage.os src/gmp_globals.os src/ZZ_pylong.os src/ntl_wrap.os -L/Users/student/Desktop/sage-4.7.1.alpha1/local/lib -L/Users/student/Desktop/sage-4.7.1.alpha1/local/lib/python/config -lntl -lpari -lgmp -lpython2.7
Traceback (most recent call last):
File "setup.py", line 18, in <module>
from module_list import ext_modules
File "/Users/student/Desktop/sage-4.7.1.alpha1/devel/sage-main/module_list.py", line 1529, in <module>
flint_depends],
TypeError: cannot concatenate 'str' and 'list' objects
sage: There was an error installing modified sage library code.
real 0m34.577s
user 0m19.519s
sys 0m6.901s
Error building new version of SAGE.
You might try typing 'sage -ba' or write to sage-support with as much information as possible.
real 2m42.066s
user 0m52.903s
sys 0m25.133s
sage: An error occurred while installing sage-4.7.1.alpha1.p0
Note this is the p0 package referred to above.
comment:59 follow-up: ↓ 61 Changed 9 years ago by
Well, there was a problem with this but it was corrected. I've checked the present sage-4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage-4.7.1.alpha1.p0.spkg) should be
b54da7bee966eaf15942964a05a8b6b1 sage-4.7.1.alpha1.p0.spkg
If the spkg you have is not this could you download again and see if that fixed things.
Changed 9 years ago by
Some minor changes. The biggest change is to the doctest which explicitly refereed to python2.6 also we have extra include folders in a doctest apparently. [updated 20110525]
comment:60 Changed 9 years ago by
- Description modified (diff)
I am splitting the ticket again: in #11376 we make sage building system smart and work out the python version by itself. If we get this in the next alpha that will make testing easier as you won't need a patched sage spkg to build against python-2.7.1. One will still need to patch the doctests afterwards and any other issues but it will build.
In #11377 I put the clean up of module_list.py this is not necessary for python-2.7.1 per see but it would be a huge favour to me if it was going in (it makes work on sage-on-gentoo and sage-prefix much easier).
comment:61 in reply to: ↑ 59 ; follow-up: ↓ 62 Changed 9 years ago by
Replying to strogdon:
Well, there was a problem with this but it was corrected. I've checked the present sage-4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage-4.7.1.alpha1.p0.spkg) should be
b54da7bee966eaf15942964a05a8b6b1 sage-4.7.1.alpha1.p0.spkg
If the spkg you have is not this could you download again and see if that fixed things.
It didn't seem to fix things (and I did check the md5 sum). I'm now starting a build from scratch, having replaced the python and sage spkgs.
By the way, now that 4.7.1.alpha1 is "official", one may have to upgrade the drop-in spkg.
comment:62 in reply to: ↑ 61 Changed 9 years ago by
Replying to kcrisman:
Replying to strogdon:
Well, there was a problem with this but it was corrected. I've checked the present sage-4.7.1.alpha.p0 and this problem should not be there. The md5sum (md5sum sage-4.7.1.alpha1.p0.spkg) should be b54da7bee966eaf15942964a05a8b6b1 sage-4.7.1.alpha1.p0.spkg If the spkg you have is not this could you download again and see if that fixed things.
It didn't seem to fix things (and I did check the md5 sum). I'm now starting a build from scratch, having replaced the python and sage spkgs. By the way, now that 4.7.1.alpha1 is "official", one may have to upgrade the drop-in spkg.
OK, for those interested I have a new drop-in spkg based on the released 4.7.1.alpha1. This spkg includes the patches from this ticket as well as the patches from tickets #11244, #11373, #11376 and #11377 . I'm presently building Sage to make sure everything is as it should be. If successful, I'll post links to the drop-in spkg later today.
comment:63 Changed 9 years ago by
comment:64 Changed 9 years ago by
The following have been uploaded:
http://www.d.umn.edu/~strogdon/sage/install.log-4.7.1.alpha1
http://www.d.umn.edu/~strogdon/sage/test.log-4.7.1.alpha1
http://www.d.umn.edu/~strogdon/sage/sage-4.7.1.alpha1.p0.spkg
http://www.d.umn.edu/~strogdon/sage/sage-4.7.1.alpha1.p0.spkg.md5
for comparisons, perhaps just under the wire for alpha2. Note, the drop-in sage spkg has the same revision .p0 extension, so I've provided the md5 sum for the spkg.
comment:65 Changed 9 years ago by
- Description modified (diff)
Changed the cython.py patch in the list to accommodate new changes made in #11376.
comment:66 follow-up: ↓ 67 Changed 9 years ago by
End of a looong traceback while building from scratch on PPC 10.4 with the spkgs dropped in (though not the absolute most recent alpha1.p0 spkg, but that hasn't even been gotten to yet, so it shouldn't matter). Just FYI in case that helps.
File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/os.py", line 284, in walk
if isdir(join(top, name)):
File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/genericpath.py", line 44, in isdir
return stat.S_ISDIR(st.st_mode)
File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/stat.py", line 41, in S_ISDIR
return S_IFMT(mode) == S_IFDIR
RuntimeError: maximum recursion depth exceeded
Error installing Zope/Twisted interface.
real 0m16.055s
user 0m3.349s
sys 0m4.671s
sage: An error occurred while installing twisted-9.0.p2
comment:67 in reply to: ↑ 66 Changed 9 years ago by
Replying to kcrisman:
End of a looong traceback while building from scratch on PPC 10.4 with the spkgs dropped in (though not the absolute most recent alpha1.p0 spkg, but that hasn't even been gotten to yet, so it shouldn't matter). Just FYI in case that helps.
File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/os.py", line 284, in walk if isdir(join(top, name)): File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/genericpath.py", line 44, in isdir return stat.S_ISDIR(st.st_mode) File "/Users/student/Desktop/sage-4.7.1.alpha1-python/local/lib/python/stat.py", line 41, in S_ISDIR return S_IFMT(mode) == S_IFDIR RuntimeError: maximum recursion depth exceeded Error installing Zope/Twisted interface. real 0m16.055s user 0m3.349s sys 0m4.671s sage: An error occurred while installing twisted-9.0.p2
Were you using the new setuptools from #11363? If so it may be worth trying a newer twisted from #8741.
comment:68 Changed 9 years ago by
- Dependencies set to #11156 #11236 #11244 #11264 #11339 #11363 #11376
comment:69 Changed 9 years ago by
Updated trac_9958-fix-misc_cythonpy.patch to take into account changes in #11376. For some reason the numpy include line appears after the application of the other patches over there. Why it didn't appear before is a mistery.
comment:70 Changed 9 years ago by
By the way, I don't know how feasible this is, but it would be very nice if the new python spkg passed self-tests: set SAGE_CHECK to "yes" and install the spkg. Previous versions of Python have failed the self-tests, but it's perhaps the only standard spkg for which this is true. With the spkg here, on an OS X box:
344 tests OK.
4 tests failed:
test__locale test_ctypes test_distutils test_locale
38 tests skipped:
test_aepack test_al test_applesingle test_bsddb test_bsddb3
test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_dl test_epoll test_gdb test_gdbm test_gl test_imageop
test_imgfile test_largefile test_linuxaudiodev test_macos
test_macostools test_ossaudiodev test_scriptpackages test_smtpnet
test_socketserver test_startfile test_sunaudiodev test_timeout
test_tk test_ttk_guionly test_urllib2net test_urllibnet
test_winreg test_winsound test_zipfile64
5 skips unexpected on darwin:
test_aepack test_applesingle test_dl test_macos
test_scriptpackages
make: *** [test] Error 1
An error occurred while testing Python
On sage.math:
346 tests OK.
1 test failed:
test_distutils
39 tests skipped:
test_aepack test_al test_applesingle test_bsddb test_bsddb185
test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_dbm test_dl test_gdb test_gdbm test_gl test_imageop
test_imgfile test_kqueue test_linuxaudiodev test_macos
test_macostools test_ossaudiodev test_scriptpackages test_smtpnet
test_socketserver test_startfile test_sunaudiodev test_timeout
test_tk test_ttk_guionly test_urllib2net test_urllibnet
test_winreg test_winsound test_zipfile64
7 skips unexpected on linux2:
test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_tk
test_ttk_guionly
make: *** [test] Error 1
An error occurred while testing Python
(I didn't upgrade the distutils package or anything else, just ran "sage -f python-2.7.1.spkg", so some of these may go away.)
comment:71 Changed 9 years ago by
- Cc kini added
comment:72 Changed 9 years ago by
python testing is difficult. Even in gentoo the procedure is complicated. I'll have a look at it but I cannot make promise. There may be modules that we shouldn't build in the first place (nothing useful to sage, don't worry).
We have this for example in the ebuild:
if use berkdb; then ewarn "\"bsddb\" module is out-of-date and no longer maintained inside dev-lang/python. It has" ewarn "been additionally removed in Python 3. You should use external, still maintained \"bsddb3\"" ewarn "module provided by dev-python/bsddb3 which supports both Python 2 and Python 3." fi
gdb, gdm, dbm and tk require extra things on the host system so they may very well fail without. Actually according to the ebuild distutils and gdm are known to fail.
comment:73 follow-up: ↓ 76 Changed 9 years ago by
Just for the record:
http://trac.sagemath.org/sage_trac/ticket/8664#comment:33 gives an example on how to automatically rebuild packages that depend on a newly added spkg (here: Python).
(Artificial version bumps aren't necessary, and simply touching spkg/installed/<spkg-that-others-depend-on> alone doesn't work.)
In general:
- Copy the new spkg(s) to
$SAGE_ROOT/spkg/standard/. export SAGE_UPGRADING=yes- (Optionally set
MAKEandSAGE_PARALLEL_SPKG_BUILD, perhapsSAGE_CHECK.) - Run
make(or bettermake build) rather than doing./sage -f ... - Apply patches to the Sage library.
- Run
./sage -b(or./sage -ba-forcein case not all dependencies are covered bymodule_list.py, which in general I'd consider a bug, but this might be necessary upon a Python upgrade) - Run tests or whatever.
Note that Sage switches to the "main" branch whenever the Sage spkg gets reinstalled.
Of course patches to the build system are usually a bit trickier, but as far as I can see Francois already sorted these out.
comment:74 Changed 9 years ago by
Thanks for the pointer, it will make testing easier for everyone. The bad news is that all the bits that makes testing easier won't be in 4.7.1. The good news is that everything is merged in 4.7.2.alpha0 at the moment. So once the next release cycle begins things will get easier. No patch will be needed to the build system. All that will remain will be doctest fix and issues like #11339.
I will update trac_9958-fixing_numericalnoise-part2.patch shortly as #11255 broke it. Which means an extra patch may appear later once I have put all the pieces back together.
comment:75 Changed 9 years ago by
OK. There are a few more patches needed for numerical and formatting noise in 4.7.2.alpha1 but now anyone should be able to upgrade python without a premade sage spkg. All fixes to be able to build sage against python-2.7 out of the box are in!
comment:76 in reply to: ↑ 73 ; follow-up: ↓ 77 Changed 9 years ago by
(Artificial version bumps aren't necessary, and simply
touchingspkg/installed/<spkg-that-others-depend-on>alone doesn't work.)In general:
- Copy the new spkg(s) to
$SAGE_ROOT/spkg/standard/.export SAGE_UPGRADING=yes
Huh, I didn't know about that, and it would be quite useful. I can't find it in either the installation guide or the developer guide, though - is this documented yet? Just curious.
Francois, I'll try to give this a test run on PPC OS X and/or Cygwin if you like.
comment:77 in reply to: ↑ 76 Changed 9 years ago by
Replying to kcrisman:
Francois, I'll try to give this a test run on PPC OS X and/or Cygwin if you like.
Be my guest, I thought that the fact things just became easier warranted an announcement.
comment:78 Changed 9 years ago by
Wonderful, this is great news! :) Thanks!
comment:79 Changed 9 years ago by
So far Mac PPC is doing fine (I just swapped out the spkgs in a source tar). Numpy, mercurial, various other python-related spkgs apparently built ok, though it isn't at Scipy or Sage yet. I did NOT check with SAGE_CHECK=yes since I didn't want to have to watch the build and turn that off during the Python build... Anyway, looks good so far. If it finishes I'll apply all the patches :)
comment:80 Changed 9 years ago by
I tried this (new python spkg plus all of the patches, then built from scratch) and I'm seeing a bunch of doctest failures. Are some of these due to #11339?
The following tests failed:
sage -t -long -force_lib devel/sage/sage/categories/finite_crystals.py # 1 doctests failed
sage -t -long -force_lib devel/sage/sage/rings/morphism.pyx # 0 doctests failed
sage -t -long -force_lib devel/sage/sage/rings/integer.pyx # 3 doctests failed
sage -t -long -force_lib devel/sage/sage/rings/homset.py # 0 doctests failed
sage -t -long -force_lib devel/sage/sage/rings/padics/padic_capped_relative_element.pyx # 1 doctests failed
sage -t -long -force_lib devel/sage/sage/combinat/e_one_star.py # 1 doctests failed
sage -t -long -force_lib devel/sage/sage/combinat/words/nfactor_enumerable_word.py # 2 doctests failed
sage -t -long -force_lib devel/sage/sage/geometry/polyhedra.py # 1 doctests failed
sage -t -long -force_lib devel/sage/sage/schemes/generic/scheme.py # 0 doctests failed
sage -t -long -force_lib devel/sage/sage/symbolic/callable.py # 5 doctests failed
I think I did everything right. I put together a Sage distribution with the python spkg and the patches, based on 4.7.2.alpha1:
Upgrade path:
comment:81 Changed 9 years ago by
Any failures from #11339 gives you a killed doctest, so
sage -t -long -force_lib devel/sage/sage/rings/morphism.pyx sage -t -long -force_lib devel/sage/sage/rings/homset.py sage -t -long -force_lib devel/sage/sage/schemes/generic/scheme.py
could all be, a verbose run would help figure it out. By nature #11339 is somewhat random, not everyone see the same failures but all these are familiar.
Just checking the others with my current list.
- sage/categories/finite_crystals.py is a new one introduced in 4.7.2 where the error message has changed slightly when using python-2.7.
- sage/rings/integer.pyx is definitely python-2.7 related but not patched yet because I am not sure what should be done there.
- sage/rings/padics/padic_capped_relative_element.pyx same thing
- sage/combinat/e_one_star.py I had a patch but it became to fuzzy haven't done a new one.
- sage/combinat/words/nfactor_enumerable_word.py python-2.7 two successive results are inverted, I want someone else to look at that one to make sure we can just do the obvious thing.
- sage/symbolic/callable.py new in 4.7.2 change in the error message because we are running python-2.7
- sage/geometry/polyhedra.py never seen that one before, can you re-run it and give more info on the failure?
As you can see a few more patches are needed and apart from #11339 which is a big show stopper because of the crashes at least two other tests point to some potential problems that should be looked at.
comment:82 follow-up: ↓ 88 Changed 9 years ago by
Here are the failures I get on x86:
sage -t -long -force_lib devel/sage-main/sage/libs/cremona/newforms.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/misc/sageinspect.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/misc/randstate.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/rings/real_mpfr.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/stats/intlist.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/combinat/e_one_star.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/geometry/polyhedra.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed sage -t -long -force_lib devel/sage-main/sage/symbolic/callable.py # 5 doctests failed sage -t -long -force_lib devel/sage-main/sage/functions/transcendental.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/categories/finite_crystals.py # 1 doctests failed
The failures newforms.pyx, real_mpfr.pyx and intlist.pyx are all of the form:
File "/storage/sage/sage-4.7.2.alpha1/devel/sage-main/sage/libs/cremona/newforms.pyx", line 53:
sage: ECModularSymbol(E)
Expected:
Traceback (most recent call last):
...
OverflowError: long int too large to convert to int
Got:
Traceback (most recent call last):
File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/storage/sage/sage-4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_1[11]>", line 1, in <module>
ECModularSymbol(E)###line 53:
sage: ECModularSymbol(E)
File "newforms.pyx", line 69, in sage.libs.cremona.newforms.ECModularSymbol.__init__ (sage/libs/cremona/newforms.cpp:1820)
a6 = new_bigint(int(E.a6()))
OverflowError: Python int too large to convert to C long
These tests do not fail on amd64 because "long int too large to convert to int" is returned as the OverflowError? instead of "Python int too large to convert to C long".
comment:83 Changed 9 years ago by
I had forgotten about these. We need different test results for 32 and 64 bits for these.
comment:84 Changed 9 years ago by
Here's the failure for sage/geometry/polyhedra.py:
sage -t -long -force_lib "devel/sage/sage/geometry/polyhedra.py"
**********************************************************************
File "/mnt/usb1/scratch/palmieri/9958/sage-4.7.2.9958/devel/sage/sage/geometry/polyhedra.py", line 4948:
sage: ppoints[0]
Expected:
(-1.92296268638e-15, -1.92296268638e-15)
Got:
(0.0, 0.0)
**********************************************************************
It looks like numerical noise, but is it an acceptable level of accuracy?
comment:85 Changed 9 years ago by
Now I remember that test, it is even in the log sage-4.7.alpha5-test.log.bz2 I produced a while ago. Strangely enough it doesn't happen anymore on my sage-on-gentoo install. I also have pari-2.5.0 and maxima-5.25.0 on this install, I suspect one of them may be responsible for the disparition of that failure
comment:86 Changed 9 years ago by
I am wrong! Not maxima or pari, there is a patch for it in trac_9958-fixing_numericalnoise-part1.patch. So it looks like that bit may not be applied properly.
comment:87 Changed 9 years ago by
- Description modified (diff)
Added three new patches. I will rework the patches that need adjusting between 32 and 64 bits next.
comment:88 in reply to: ↑ 82 ; follow-up: ↓ 89 Changed 9 years ago by
Replying to strogdon:
Here are the failures I get on x86:
sage -t -long -force_lib devel/sage-main/sage/libs/cremona/newforms.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/misc/sageinspect.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/misc/randstate.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/rings/real_mpfr.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/stats/intlist.pyx # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/combinat/e_one_star.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/geometry/polyhedra.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed sage -t -long -force_lib devel/sage-main/sage/symbolic/callable.py # 5 doctests failed sage -t -long -force_lib devel/sage-main/sage/functions/transcendental.py # 1 doctests failed sage -t -long -force_lib devel/sage-main/sage/categories/finite_crystals.py # 1 doctests failedThe failures newforms.pyx, real_mpfr.pyx and intlist.pyx are all of the form:
File "/storage/sage/sage-4.7.2.alpha1/devel/sage-main/sage/libs/cremona/newforms.pyx", line 53: sage: ECModularSymbol(E) Expected: Traceback (most recent call last): ... OverflowError: long int too large to convert to int Got: Traceback (most recent call last): File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/storage/sage/sage-4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/storage/sage/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_1[11]>", line 1, in <module> ECModularSymbol(E)###line 53: sage: ECModularSymbol(E) File "newforms.pyx", line 69, in sage.libs.cremona.newforms.ECModularSymbol.__init__ (sage/libs/cremona/newforms.cpp:1820) a6 = new_bigint(int(E.a6())) OverflowError: Python int too large to convert to C longThese tests do not fail on amd64 because "long int too large to convert to int" is returned as the OverflowError? instead of "Python int too large to convert to C long".
Steve, what about
sage -t -long -force_lib devel/sage-main/sage/rings/polynomial/polynomial_rational_flint.pyx
comment:89 in reply to: ↑ 88 ; follow-up: ↓ 90 Changed 9 years ago by
comment:90 in reply to: ↑ 89 Changed 9 years ago by
Replying to strogdon:
Replying to fbissey:
Replying to strogdon:
Steve, what about
sage -t -long -force_lib devel/sage-main/sage/rings/polynomial/polynomial_rational_flint.pyxHere on x86 and amd64 the test passes. So, there seems to be some sort of inconsistency, doesn't it?
Yes! I'll fix it but it should be investigated. Gut feeling is on variation in cython definitions.
comment:91 Changed 9 years ago by
- Description modified (diff)
comment:92 Changed 9 years ago by
I've tried to remove the failures known above here. Note that I didn't get the last few patches in time, but I removed those failures as well, assuming they were the same. The timeouts/killed are probably unrelated - my computer is < 1 GHz, so I get a lot of timeouts and I forgot to set SAGE_TIMEOUT_LONG before I ran the tests - but at any rate I hope this will be useful. After the hurricane I'll start this up again and run the remaining tests.
sage -t -long "devel/sage/sage/crypto/mq/mpolynomialsystem.py" # Killed/crashed
sage -t -long "devel/sage/sage/functions/transcendental.py"
sage -t -long "devel/sage/sage/groups/generic.py" # Killed/crashed
sage -t -long "devel/sage/sage/interfaces/maxima.py" # Time out
sage -t -long "devel/sage/sage/interfaces/singular.py" # Killed/crashed
sage -t -long "devel/sage/sage/libs/cremona/newforms.pyx"
sage -t -long "devel/sage/sage/libs/ppl.pyx" # Time out
sage -t -long "devel/sage/sage/libs/singular/ring.pyx" # Killed/crashed
sage -t -long "devel/sage/sage/misc/randstate.pyx"
sage -t -long "devel/sage/sage/misc/sageinspect.py"
sage -t -long "devel/sage/sage/rings/multi_power_series_ring_element.py" # Killed/crashed
sage -t -long "devel/sage/sage/rings/polynomial/multi_polynomial_ideal.py" # Killed/crashed
sage -t -long "devel/sage/sage/rings/polynomial/multi_polynomial_libsingular.pyx" # Killed/crashed
sage -t -long "devel/sage/sage/rings/polynomial/polynomial_singular_interface.py" # Killed/crashed
sage -t -long "devel/sage/sage/rings/real_mpfr.pyx"
sage -t -long "devel/sage/sage/rings/tests.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/BSD.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_curve_isogeny.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_field.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_finite_field.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_generic.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_local_data.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_number_field.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_point.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py" # Time out
sage -t -long "devel/sage/sage/schemes/elliptic_curves/gal_reps.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/heegner.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/elliptic_curves/period_lattice.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/generic/scheme.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/plane_conics/con_finite_field.py" # Killed/crashed
sage -t -long "devel/sage/sage/schemes/plane_curves/projective_curve.py" # Killed/crashed
sage -t -long "devel/sage/sage/stats/intlist.pyx"
sage -t -long "devel/sage/sage/structure/sage_object.pyx" # Killed/crashed
comment:93 follow-up: ↓ 95 Changed 9 years ago by
Here is one example.
sage -t -long "devel/sage/sage/functions/transcendental.py"
**********************************************************************
File "/Users/student/Desktop/sage-4.7.2.alpha1/devel/sage/sage/functions/transcendental.py", line 83:
sage: w = exponential_integral_1(2,4); w
Expected:
[0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843924751e-05]
Got:
[0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.766562284392475e-05]
**********************************************************************
Here is one of the killed ones (not a timeout, I'm surprised):
sage -t -long "devel/sage/sage/crypto/mq/mpolynomialsystem.py"
The doctested process was killed by signal 10
[352.0 s]
Could this be an unrelated 4.7.2.alpha1 issue on my platform?
comment:94 Changed 9 years ago by
I am particularly interested in the killed/crashed tests. And also those for which there is already a patch.
sage -t -long "devel/sage/sage/libs/cremona/newforms.pyx" sage -t -long "devel/sage/sage/rings/real_mpfr.pyx" sage -t -long "devel/sage/sage/stats/intlist.pyx"
are the 32/64 bits message difference already mentioned by Steve.
The timeouts are the regular suspect on ppc if I am not mistaken.
The transcendental.py doctest you are quoting is worrying me. It is patched in trac_9958-fixing_numericalnoise-part1.patch but you have a slightly different result again (one less digit on the last number). We may have to consider a different patch.
All killed/crashed doctest are candidates for #11339 but you'll have to run sage -gdb on them to be sure.
comment:95 in reply to: ↑ 93 Changed 9 years ago by
Replying to kcrisman:
Here is one of the killed ones (not a timeout, I'm surprised):
sage -t -long "devel/sage/sage/crypto/mq/mpolynomialsystem.py"
The doctested process was killed by signal 10
[352.0 s]
Could this be an unrelated 4.7.2.alpha1 issue on my platform?
Does SIGUSR1 (#10) have a special meaning on Darwin?
comment:96 Changed 9 years ago by
Almost anything with "polynomial" in its name is likely to use singular (and crypto/mq/mpolynomialsystem.py is no exception), signal 10 is apparently a user defined signal so it could be anything.
We'll have to wait for the hurricane to move on before Karl can answer us on any of this points.
Looking back at you list Karl I am also very curious about
sage -t -long "devel/sage/sage/structure/sage_object.pyx sage -t -long "devel/sage/sage/misc/sageinspect.py" sage -t -long "devel/sage/sage/misc/randstate.pyx"
two of these have already been patched (sage_object and randstate) in this ticket so I am quite curious about what's wrong with them.
comment:97 Changed 9 years ago by
Yeah, I'm sorry I couldn't provide more information. The tests only just finished after classes, and I had one hour to copy/paste the above, restart just doing the failed tests, write a lecture, check email, and then turn off and unplug my PPC machine. I just couldn't do any more. I certainly wasn't taking an eMac home on the bike/train with me!
But we are just due for heavy rain and wind, so hopefully Monday I can get back on it and tell you more, save an extended power outage. Much better than an earthquake, I think fbissey would heartily agree. I wish I'd had a chance to run tests after #11339, but I just knew it would take eons. I'd do it on one of my home PPC machines (believe it or not, I have two more) but one I can no longer get !XCode for, and the other is also slow enough it wouldn't be ready by the time we have to unplug it... sigh.
Anyway, I'll keep this ticket on the priority list for the machine I did these tests on.
comment:98 Changed 9 years ago by
So, I did test this with Sage 4.7.2.alpha2 on my "last" 32-bit machine, a Pentium4 Prescott (which has SSE3) running Ubuntu 9.04; my other Pentium4 (Northwood, without SSE3 / PNI) recently died, and I won't revive it in the near future.
The good news are: The patches all still apply to 4.7.2.alpha2, though many hunks with partially large offsets, but no fuzz.
The bad news: I expected some numerical noise because of 32-bit architecture with (there rather rare) SSE3 (-mfpmath=sse), but also experienced at least one segfault (in sage/rings/homset.py). I'll have to investigate the rest, for now just:
...
sage -t -long -force_lib "devel/sage/sage/geometry/polyhedra.py"
**********************************************************************
File "/media/H-1TB-P6-linux2/Sage/sage-4.7.2.alpha2-gcc-4.5.1/devel/sage/sage/geometry/polyhedra.py", line 4948:
sage: ppoints[0]
Expected:
(-1.92296268638e-15, -1.92296268638e-15)
Got:
(0.0, 0.0)
**********************************************************************
1 items had failures:
1 of 8 in __main__.example_171
***Test Failed*** 1 failures.
For whitespace errors, see the file /home/leif/.sage//tmp/.doctest_polyhedra.py
[166.0 s]
----------------------------------------------------------------------
The following tests failed:
sage -t -long -force_lib "devel/sage/sage/libs/cremona/newforms.pyx"
sage -t -long -force_lib "devel/sage/sage/functions/transcendental.py"
sage -t -long -force_lib "devel/sage/sage/misc/randstate.pyx"
sage -t -long -force_lib "devel/sage/sage/combinat/words/suffix_trees.py"
sage -t -long -force_lib "devel/sage/sage/combinat/words/nfactor_enumerable_word.py"
sage -t -long -force_lib "devel/sage/sage/rings/morphism.pyx"
sage -t -long -force_lib "devel/sage/sage/rings/real_mpfr.pyx"
sage -t -long -force_lib "devel/sage/sage/rings/homset.py"
sage -t -long -force_lib "devel/sage/sage/stats/intlist.pyx"
sage -t -long -force_lib "devel/sage/sage/schemes/generic/scheme.py"
sage -t -long -force_lib "devel/sage/sage/geometry/polyhedra.py"
Total time for all tests: 26263.8 seconds
make: *** [testlong] Error 128
real 439m23.357s
user 393m40.644s
sys 31m7.485s
leif@californication:~/Sage/sage-4.7.2.alpha2-gcc-4.5.1$
(The tail of the log. More to come later.)
Note that I didn't test in parallel, although with some other CPU-greedy process running, but time-outs are unlikely. Vanilla 4.7.2.alpha2 passed all tests in exactly the same setting (and I did rebuild all dependent packages when installing the Python 2.7 spkg, modulo missing extension module dependencies in module_list.py).
comment:99 follow-up: ↓ 100 Changed 9 years ago by
Ok, three segfaults (two of them without backtrace / Sage message, for whatever reason), some Python ints too large ("as usual" I guess), some numerical noise, the rest apparently related to dictionaries / hashing(?).
Attaching short logs of rerun tests (all reproducable)...
Changed 9 years ago by
Pentium4 Prescott (SSE3 / PNI, 32-bit), GCC 4.5.1, native code, -mfpmath=sse, Ubuntu 9.04 x86
Changed 9 years ago by
Pentium4 Prescott (SSE3 / PNI, 32-bit), GCC 4.5.1, native code, -mfpmath=sse, Ubuntu 9.04 x86 (The three segfaulting files tested in verbose mode.)
comment:100 in reply to: ↑ 99 Changed 9 years ago by
Replying to leif:
Attaching short logs of rerun tests (all reproducable)...
Done.
Haha, the first segfault is an especially nice one:
... 10 tests in __main__.example_7 6 tests in __main__.example_8 7 tests in __main__.example_9 511 tests in 55 items. 511 passed and 0 failed. Test passed. Segmentation fault [9.8 s] ---------------------------------------------------------------------- The following tests failed: sage -t -long -verbose "devel/sage/sage/rings/morphism.pyx" Total time for all tests: 9.8 seconds real 0m10.013s user 0m6.980s sys 0m1.068s
comment:101 follow-ups: ↓ 102 ↓ 106 Changed 9 years ago by
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
sage -t -long -force_lib "devel/sage/sage/combinat/words/nfactor_enumerable_word.py"
is known to fail but is probably a minor adjustment.
sage -t -long -force_lib "devel/sage/sage/combinat/words/suffix_trees.py"
is new. The rest look like #11339 material.
comment:102 in reply to: ↑ 101 Changed 9 years ago by
Replying to fbissey:
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
The failing doctests are all in Doctest_failures_Sage-4.7.2.alpha2_Linux_x86_SSE3-individual_tests_rerun.log.
[...] The rest look like #11339 material.
I'm just to apply the patches from there; the machine is dead slow, doing a ./sage -ba-force atm to exclude missing extension module dependencies, as I've not built from scratch.
comment:103 Changed 9 years ago by
I was too fast. Some of the failures are already reported by Steve. I may try to update the patch set a bit tonight.
comment:104 Changed 9 years ago by
Just experienced some new(?) error when cloning the Sage library for #11339:
...
byte-compiling /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage/misc/copying.py to copying.pyc
byte-compiling /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage/version.py to version.pyc
running install_egg_info
Removing /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage-0.0.0-py2.7.egg-info
Writing /home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python2.7/site-packages/sage-0.0.0-py2.7.egg-info
Exception in thread Thread-4 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 530, in __bootstrap_inner
File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 483, in run
File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/multiprocessing/pool.py", line 272, in _handle_workers
<type 'exceptions.TypeError'>: 'NoneType' object is not callable
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 530, in __bootstrap_inner
File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/threading.py", line 483, in run
File "/home/leif/Sage/sage-4.7.2.alpha2-gcc-4.5.1/local/lib/python/multiprocessing/pool.py", line 272, in _handle_workers
<type 'exceptions.TypeError'>: 'NoneType' object is not callable
real 1m35.816s
user 1m35.794s
sys 0m5.648s
After cloning, if you change any Sage library files and want to rebuild
the html version of the reference manual, use the command
sage -docbuild reference html
(after running 'sage -b').
...
comment:105 Changed 9 years ago by
Waaaaaah!
leif@californication:~/Sage/sage-4.7.2.alpha2-gcc-4.5.1/devel/sage-9958-11339$ ../../sage -hg import -v ~/Sage/patches/trac_11339_refcount_singular_polynomials.patch applying /home/leif/Sage/patches/trac_11339_refcount_singular_polynomials.patch patching file sage/rings/polynomial/multi_polynomial_libsingular.pxd patching file sage/rings/polynomial/multi_polynomial_libsingular.pyx Hunk #13 FAILED at 727 Hunk #14 FAILED at 741 Hunk #15 FAILED at 763 Hunk #16 FAILED at 773 Hunk #17 FAILED at 825 Hunk #18 succeeded at 910 with fuzz 2 (offset 72 lines). Hunk #19 FAILED at 875 Hunk #20 succeeded at 961 (offset 75 lines). Hunk #21 succeeded at 980 (offset 75 lines). Hunk #22 succeeded at 1006 (offset 75 lines). ...
(The first patch still applies, with offsets.)
I'm not going to fix that...
comment:106 in reply to: ↑ 101 ; follow-up: ↓ 107 Changed 9 years ago by
Replying to fbissey:
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
I'm sure the polyhedra.py patch was needed at one time but it now (?) appears that
@@ -4946,7 +4946,7 @@
sage: proj = ProjectionFuncStereographic([1.1,1.1,1.1])
sage: ppoints = [proj(vector(x)) for x in cube]
sage: ppoints[0]
- (0.0, 0.0)
+ (-1.92296268638e-15, -1.92296268638e-15)
"""
def __init__(self, projection_point):
"""
from trac_9958-fixing_numericalnoise-part1.patch is no longer needed.
comment:107 in reply to: ↑ 106 Changed 9 years ago by
Replying to strogdon:
Replying to fbissey:
Thanks leif. Nothing worrying there. There is a patch for sage/geometry/polyhedra.py but obviously it doesn't apply properly anymore. transcendantal.py, randstate.pyx and intlist.pyx are all already patched but may need more tweaking, I'd be grateful for the output of these.
I'm sure the polyhedra.py patch was needed at one time but it now (?) appears that
@@ -4946,7 +4946,7 @@ sage: proj = ProjectionFuncStereographic([1.1,1.1,1.1]) sage: ppoints = [proj(vector(x)) for x in cube] sage: ppoints[0] - (0.0, 0.0) + (-1.92296268638e-15, -1.92296268638e-15) """ def __init__(self, projection_point): """from trac_9958-fixing_numericalnoise-part1.patch is no longer needed.
You are right I am reading the test results backwards. It now seems to go to 0 properly. One less patch in the patch and one less worry.
comment:108 follow-up: ↓ 110 Changed 9 years ago by
Okay, you can get a full list of current failure and what they looked like at this link. Some are definitely known above, but others may be unknown. The timeouts are probably just due to this machine (and the Maxima one), but others I don't know whether they are here or #11339.
comment:109 follow-up: ↓ 115 Changed 9 years ago by
- Description modified (diff)
Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.
comment:110 in reply to: ↑ 108 Changed 9 years ago by
Replying to kcrisman:
Okay, you can get a full list of current failure and what they looked like at this link. Some are definitely known above, but others may be unknown. The timeouts are probably just due to this machine (and the Maxima one), but others I don't know whether they are here or #11339.
You have some nasty stuff in polyhedra.py
File "/Users/student/Desktop/sage-4.7.2.alpha1/devel/sage/sage/geometry/polyhedra.py", line 5250:
sage: _.show()
Exception raised:
Traceback (most recent call last):
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_183[5]>", line 1, in <module>
_.show()###line 5250:
sage: _.show()
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python/site-packages/sage/misc/displayhook.py", line 174, in displayhook
print_obj(sys.stdout, obj)
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python/site-packages/sage/misc/displayhook.py", line 142, in print_obj
print >>out_stream, `obj`
File "base.pyx", line 80, in sage.plot.plot3d.base.Graphics3d.__repr__ (sage/plot/plot3d/base.c:2526)
File "base.pyx", line 1124, in sage.plot.plot3d.base.Graphics3d.show (sage/plot/plot3d/base.c:10344)
File "base.pyx", line 663, in sage.plot.plot3d.base.Graphics3d.export_jmol (sage/plot/plot3d/base.c:5867)
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python2.7/zipfile.py", line 1138, in writestr
self.fp.write(zinfo.FileHeader())
File "/Users/student/Desktop/sage-4.7.2.alpha1/local/lib/python2.7/zipfile.py", line 348, in FileHeader
len(filename), len(extra))
error: integer out of range for 'H' format code
You get an extra digit in randstate compared to x86 :( [note to self it is in numerical_noise_part4].
sage-inspect looks bad :( but not hopeless. The message is just overly long. I will have to postpone the rest for later.
comment:111 Changed 9 years ago by
comment:112 follow-up: ↓ 114 Changed 9 years ago by
In polyhedra.py it looks like you have a problem with jmol printing 3D images to file. It involves zip, looking at the pyx there is an instance of testing zip. Could you paste the cython code that produced sage/plot/plot3d/base.c:5867 ?
comment:113 Changed 9 years ago by
- Description modified (diff)
Updated the patch set again. split real_mpfr.pyx from the mixedfix patch. The 32/64 bits issues reported is separate from the message that was already patched. Added a patch to fix 32/64 bit messages.
comment:114 in reply to: ↑ 112 Changed 9 years ago by
Replying to fbissey:
In polyhedra.py it looks like you have a problem with jmol printing 3D images to file. It involves zip, looking at the pyx there is an instance of testing zip. Could you paste the cython code that produced sage/plot/plot3d/base.c:5867 ?
/* "sage/plot/plot3d/base.pyx":663
* f.write("isosurface fullylit; pmesh o* fullylit; set antialiasdisplay on;\n")
*
* render_params.output_archive.writestr('SCRIPT', f.getvalue()) # <<<<<<<<<<<<<<
* render_params.output_archive.close()
*
*/
__pyx_t_3 = PyObject_GetAttr(__pyx_v_render_params, __pyx_n_s__output_archive); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_3);
__pyx_t_2 = PyObject_GetAttr(__pyx_t_3, __pyx_n_s__writestr); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_2);
__Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
__pyx_t_3 = PyObject_GetAttr(__pyx_v_f, __pyx_n_s__getvalue); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_3);
__pyx_t_4 = PyObject_Call(__pyx_t_3, ((PyObject *)__pyx_empty_tuple), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_4);
__Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
__pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(((PyObject *)__pyx_t_3));
__Pyx_INCREF(((PyObject *)__pyx_n_s__SCRIPT));
PyTuple_SET_ITEM(__pyx_t_3, 0, ((PyObject *)__pyx_n_s__SCRIPT));
__Pyx_GIVEREF(((PyObject *)__pyx_n_s__SCRIPT));
PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_4);
__Pyx_GIVEREF(__pyx_t_4);
__pyx_t_4 = 0;
__pyx_t_4 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_t_3), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_4);
__Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
__Pyx_DECREF(((PyObject *)__pyx_t_3)); __pyx_t_3 = 0;
__Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
5867 is
__pyx_t_4 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_t_3), NULL); if (unlikely(!__pyx_t_4)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 663; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
I was also surprised by the polyhedra.py additional weirdness. Anything I can do to help identify it? Warning: I'm planning on building and testing alpha2 without Python 2.7 now, which will tie up most resources on this machine for a couple days.
comment:115 in reply to: ↑ 109 ; follow-up: ↓ 116 Changed 9 years ago by
Replying to fbissey:
Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.
Here the fix_transcendental patch fixed the associated test on x86 but now the test fails on amd64:
sage -t -long -force_lib devel/sage-main/sage/functions/transcendental.py
**********************************************************************
File "/storage/sage/sage-4.7.2.alpha2/devel/sage-main/sage/functions/transcendental.p
y", line 83:
sage: w = exponential_integral_1(2,4); w
Expected:
[0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843
924...e-05]
Got:
[0.04890051070806112, 0.0037793524098489063, 0.00036008245216265873, 3.7665622843
924534e-05]
**********************************************************************
This is odd! I wonder what changed? Previously this test must have returned ...67 and not ...63 for it to pass.
comment:116 in reply to: ↑ 115 Changed 9 years ago by
Replying to strogdon:
Replying to fbissey:
Updated the first patch for polyhedra.py, split transcendental.py in its own patch. I lowered the precision tested in the last numbers so that results from 32 and 64 bits are not different.
Here the fix_transcendental patch fixed the associated test on x86 but now the test fails on amd64:
sage -t -long -force_lib devel/sage-main/sage/functions/transcendental.py ********************************************************************** File "/storage/sage/sage-4.7.2.alpha2/devel/sage-main/sage/functions/transcendental.p y", line 83: sage: w = exponential_integral_1(2,4); w Expected: [0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843 924...e-05] Got: [0.04890051070806112, 0.0037793524098489063, 0.00036008245216265873, 3.7665622843 924534e-05] **********************************************************************This is odd! I wonder what changed? Previously this test must have returned ...67 and not ...63 for it to pass.
It's my fault for not noticing this difference between 32 and 64 bit results. It was there before but I didn't notice it. I thought the last number was the only thing different so when I truncated the last number to the common digits I dropped the difference between 32 and 64 bit. So it is entirely my fault.
Changed 9 years ago by
fixing numerical noise in transcendental.py- split from numerical noise pt1 - updated 2011092
comment:117 follow-up: ↓ 118 Changed 9 years ago by
transcendental patch updated.
comment:118 in reply to: ↑ 117 ; follow-up: ↓ 119 Changed 9 years ago by
comment:119 in reply to: ↑ 118 Changed 9 years ago by
comment:120 follow-up: ↓ 121 Changed 9 years ago by
Just for reference, on the PPC machine all tests pass with alpha2 except three timeouts which are long-standing (I didn't bother to set SAGE_TIMEOUT_LONG but I've seen these many times).
So all the rest are due to this ticket or #11339.
Would it be helpful for me to do #11339 on this new build (without Python 2.7), or is that in limbo right now anyway? Alternately, is there any way to decouple this ticket from that one?
comment:121 in reply to: ↑ 120 Changed 9 years ago by
Replying to kcrisman:
Just for reference, on the PPC machine all tests pass with alpha2 except three timeouts which are long-standing (I didn't bother to set
SAGE_TIMEOUT_LONGbut I've seen these many times).So all the rest are due to this ticket or #11339.
That's a very useful point of reference. Thank you for providing it.
Would it be helpful for me to do #11339 on this new build (without Python 2.7), or is that in limbo right now anyway? Alternately, is there any way to decouple this ticket from that one?
Technically you can apply #11339 without python 2.7 - the behavior we want to correct is only revealed when we use python 2.7 but I guess it would be useful to know if there are any side effects if you apply the patch on a python 2.6 install. Bear in mind that the current patch are not necessarily the best solution to the problem, what's more they are a whack a mole game. Fix something here and some new fault appears in a completely unexpected location.
comment:122 Changed 9 years ago by
I will update the python spkg to 2.7.3 when it will be released. 2.7.3 will include the patch to python that we have been carrying in sage for the longest time. See: http://bugs.python.org/issue7689
Does anyone know if the apocalypse is about to happen?
comment:123 Changed 9 years ago by
- Description modified (diff)
Fixes #1159, according to that ticket.
comment:124 Changed 9 years ago by
That's really weird. Some of the hashing problems I encounter with python 2.7 are almost the same as reported in http://trac.sagemath.org/sage_trac/ticket/4957#comment:3
sage -t -long -force_lib "devel/sage-main/sage/rings/integer.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3046:
sage: n = -920390823904823094890238490238484; n.__hash__()
Expected:
6874330978542788722
Got:
-2623069716
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3061:
sage: hash(n)
Expected:
-9223372036854767616
Got:
8192
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/rings/integer.pyx", line 3064:
sage: hash(n) == hash(int(n))
Expected:
True
Got:
False
**********************************************************************
comment:125 Changed 9 years ago by
With python-2.6:
sage: n=2^63+2^13 sage: n 9223372036854784000 sage: int(n) 9223372036854784000L sage: hash(n) -9223372036854767616 sage: hash(int(n)) -9223372036854767616
with python-2.7
sage: n=2^63+2^13 sage: n 9223372036854784000 sage: hash(n) 8192 sage: int(n) 9223372036854784000L sage: hash(int(n)) -9223372036854767616
Furthermore in both case:
sage: type(n) <type 'sage.rings.integer.Integer'> sage: type(int(n)) <type 'long'>
So I would expect different hashes naively.
comment:126 Changed 9 years ago by
Regarding
sage -t -long -force_lib "devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 20:
sage: it.next()
Expected:
word: 6456
Got:
word: 5645
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 22:
sage: it.next()
Expected:
word: 5645
Got:
word: 6456
**********************************************************************
With python-2.7
sage: w = Word([4,5,6])^7 sage: w word: 456456456456456456456 sage: it = w.factor_iterator(4) sage: it.next() word: 5645 sage: it.next() word: 6456 sage: it.next() word: 4564
with python-2.6
sage: w = Word([4,5,6])^7 sage: w word: 456456456456456456456 sage: it = w.factor_iterator(4) sage: it.next() word: 6456 sage: it.next() word: 5645 sage: it.next() word: 4564
So it looks like with python 2.6 we where moving in the series of number with a stride 2 while python 2.7 works through the sequence with a stride 1.
comment:127 Changed 9 years ago by
Here on x86 I get only two failures:
sage -t -long -force_lib devel/sage-main/sage/misc/randstate.pyx
**********************************************************************
File "/storage/sage/sage-4.7.2.rc0/devel/sage-main/sage/misc/randstate.pyx", line 61:
sage: rtest()
Expected:
(978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 603
59, 0.83350776541997362)
Got:
(978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 603
59, 0.8335077654199736)
**********************************************************************
and
sage -t -long -force_lib devel/sage-main/sage/matrix/matrix_double_dense.pyx
**********************************************************************
File "/storage/sage/sage-4.7.2.rc0/devel/sage-main/sage/matrix/matrix_double_dense.py
x", line 1046:
sage: A.singular_values(eps='junk')
Expected:
Traceback (most recent call last):
...
ValueError: invalid literal for float(): junk
Got:
Traceback (most recent call last):
File "/storage/sage/sage-4.7.2.rc0/local/bin/ncadoctest.py", line 1231, in run_
one_test
self.run_one_example(test, example, filename, compileflags)
File "/storage/sage/sage-4.7.2.rc0/local/bin/sagedoctest.py", line 38, in run_o
ne_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags
)
File "/storage/sage/sage-4.7.2.rc0/local/bin/ncadoctest.py", line 1172, in run_
one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_19[37]>", line 1, in <module>
A.singular_values(eps='junk')###line 1046:
sage: A.singular_values(eps='junk')
File "matrix_double_dense.pyx", line 1075, in sage.matrix.matrix_double_dense.M
atrix_double_dense.singular_values (sage/matrix/matrix_double_dense.c:6338)
eps = RDF(eps)
File "parent.pyx", line 988, in sage.structure.parent.Parent.__call__ (sage/str
ucture/parent.c:7326)
File "coerce_maps.pyx", line 82, in sage.structure.coerce_maps.DefaultConvertMa
p_unique._call_ (sage/structure/coerce_maps.c:3268)
File "coerce_maps.pyx", line 77, in sage.structure.coerce_maps.DefaultConvertMa
p_unique._call_ (sage/structure/coerce_maps.c:3171)
File "real_double.pyx", line 548, in sage.rings.real_double.RealDoubleElement._
_init__ (sage/rings/real_double.c:5567)
ValueError: could not convert string to float: junk
**********************************************************************
The latter also fails here the same way on amd64 and is documented on sage-devel
http://groups.google.com/group/sage-devel/browse_thread/thread/8e1d7f665ac5be37
but with some slight differences.
comment:128 Changed 9 years ago by
OK Steve, I got the matrix_double_dense but you don't get the numerical noise. I believe the noise may come from python-2.7.2. Interesting that the hashing and enumeration problem are 64bit only. I'll check on the mac.
comment:129 Changed 9 years ago by
- Description modified (diff)
New patch to fix the message about junk.
comment:130 Changed 9 years ago by
- Dependencies changed from #11156 #11236 #11244 #11264 #11339 #11363 #11376 to #11156 #11236 #11244 #11264 #11339 #11363 #11376 #11986
I have now split the hashing issue in #11986
comment:131 Changed 9 years ago by
- Description modified (diff)
One more patch added to fix combinat/words/nfactor_enumerable_word.py on 64 bit system. I believe is output is equivalent in both case so it is not important.
comment:132 Changed 9 years ago by
- Dependencies changed from #11156 #11236 #11244 #11264 #11339 #11363 #11376 #11986 to #11986
- Description modified (diff)
Removed old dependencies.
comment:133 follow-up: ↓ 135 Changed 9 years ago by
- Milestone set to sage-4.8
The spkg needs to be rebased to #11447.
comment:134 Changed 9 years ago by
Permissions is the spkg should be 0644 (or 0755 for executables), currently many files are 0640 or 0750.
comment:135 in reply to: ↑ 133 ; follow-up: ↓ 136 Changed 9 years ago by
comment:136 in reply to: ↑ 135 Changed 9 years ago by
Replying to fbissey:
Replying to jdemeyer:
The spkg needs to be rebased to #11447.
I looked at that before making my spkg. I believe the strategy used in #11447 is in python-2.7. Someone with a debian system proves me wrong but I think that works out of the box.
Well, we did take the patch from Python upstream, but IIRC there are further changes to spkg-install, and the new spkg needs the previous one to be mentioned in SPKG.txt anyway (to pass the merger's sanity check).
comment:137 Changed 9 years ago by
Indeed, there've been a lot of changes inbetween:
-
spkg-install
diff -Nu python-2.7.1/spkg-install python-2.6.4.p11/spkg-install
old new 5 5 6 6 CUR=`pwd` 7 7 8 if [ "$SAGE_LOCAL" = "" ]; then9 echo "SAGE_LOCAL undefined ... exiting";10 echo"Maybe run 'sage -sh'?"11 exit 18 if [ -z "$SAGE_LOCAL" ]; then 9 echo >&2 "SAGE_LOCAL undefined ... exiting" 10 echo >&2 "Maybe run 'sage -sh'?" 11 exit 1 12 12 fi 13 13 14 14 # PATCH 15 15 16 # Due to issues building Python on Ubuntu 11.04, a file is patched 17 # on these systems. 16 # When building Python 2.6 on Debian and derivatives with multiarch, 17 # system library directories are not correctly setup. This patch 18 # fixes this, with no-op on other systems. 19 # The fix requires 'dpkg-architecture' to be installed there though 20 # (see below). 21 echo "Patching src/setup.py for multiarch." 22 patch -p0 < patches/setup.py.multiarch.patch 23 if [ $? -ne 0 ]; then 24 echo >&2 "Error patching src/setup.py" 25 exit 1 26 fi 18 27 19 if [ -f /etc/issue ] && grep "Ubuntu 11.04" /etc/issue ; then 20 echo "Patching src/Modules/Setup.dist as this is Ubuntu 11.04" 21 patch -p0 < patches/Modules.Setup.dist.patch 28 # Due to a python bug with Solaris 29 # see http://bugs.python.org/issue1759169 30 # it is necessary to apply a patch to configure.in 31 # then run autoconf. This not only generates a 32 # new 'configure' script, but some subdirectories too 33 # so these will be copied. 34 35 if [ "x`uname`" = xSunOS ] ; then 36 echo "Applying a revised 'configure' script for Solaris" 37 echo "See http://bugs.python.org/issue1759169" 38 echo "http://trac.sagemath.org/sage_trac/ticket/7867" 39 cp -r patches/autom4te.cache patches/configure patches/configure.in src 22 40 if [ $? -ne 0 ]; then 23 echo "Error patching src/Modules/Setup.dist" 41 echo >&2 "Failed to apply the Solaris patches needed for" 42 echo >&2 "http://bugs.python.org/issue1759169" 43 echo >&2 "http://trac.sagemath.org/sage_trac/ticket/7867" 24 44 exit 1 25 45 fi 46 echo "Setting HAVE_FD_TRANSFER=0 for Solaris to allow" 47 echo "the python module '_multiprocessing' to build" 48 echo "See: http://trac.sagemath.org/sage_trac/ticket/8440" 49 cp patches/setup.py src 50 if [ $? -ne 0 ]; then 51 echo >&2 "Failed to apply the Solaris patch needed for" 52 echo >&2 "http://trac.sagemath.org/sage_trac/ticket/8440" 53 exit 1 54 fi 55 fi 56 57 58 cp patches/locale.py src/Lib/locale.py 59 if [ $? -ne 0 ]; then 60 echo >&2 "Error copying patched locale.py" 61 exit 1 26 62 fi 27 63 28 patch -p0 < patches/Lib.distutils.command.sdist.patch 64 cp patches/Makefile.pre.in src/Makefile.pre.in 29 65 if [ $? -ne 0 ]; then 30 echo "Error copying patched sdist.py"66 echo >&2 "Error copying patched Makefile.pre.in" 31 67 exit 1 32 68 fi 33 69 34 patch -p0 < patches/Lib.socket.patch 70 cp patches/sdist.py src/Lib/distutils/command/sdist.py 35 71 if [ $? -ne 0 ]; then 36 echo "Error copying patched socket.py"72 echo >&2 "Error copying patched sdist.py" 37 73 exit 1 38 74 fi 39 75 40 patch -p0 < patches/dynamic_class_copyreg_py.patch 76 cp patches/socket.py src/Lib/socket.py 41 77 if [ $? -ne 0 ]; then 42 echo "Error copying patched pickle.py"78 echo >&2 "Error copying patched socket.py" 43 79 exit 1 44 80 fi 45 81 46 patch -p0 < patches/dynamic_class_copyreg_c.patch 82 cp patches/pickle.py src/Lib/pickle.py 47 83 if [ $? -ne 0 ]; then 48 echo "Error copying patched cPickle.c"84 echo >&2 "Error copying patched pickle.py" 49 85 exit 1 50 86 fi 51 87 88 cp patches/cPickle.c src/Modules/cPickle.c 89 if [ $? -ne 0 ]; then 90 echo >&2 "Error copying patched cPickle.c" 91 exit 1 92 fi 93 94 # Due to a problem with _socket not building on OpenSolaris on x64 95 # see http://bugs.python.org/issue8852 96 # http://trac.sagemath.org/sage_trac/ticket/9041 97 # http://trac.sagemath.org/sage_trac/ticket/9022 98 # Modules/socketmodule.c needs patching. The patch consists of 99 # only checking if things are defined before trying to build with them 100 # so it is safe (and desirable) on any platform. 101 102 cp patches/Modules.socketmodule.c src/Modules/socketmodule.c 103 if [ $? -ne 0 ]; then 104 echo >&2 "Error copying patched socketmodule.c" 105 exit 1 106 fi 107 108 # The following patch for fixing broken readline behavior on Itanium 109 # Linux definitely does *not* work on anything else. 110 if [ "`uname -m`" = "ia64" -a "`uname`" = "Linux" ]; then 111 echo "Updating readline.c for Linux/Itanium" 112 cp patches/readline.c-Itanium-fix src/Modules/readline.c 113 else 114 # Readline patch: http://bugs.python.org/file14599/python-2.6-readline.patch 115 # Associated bug: http://bugs.python.org/issue5833 116 # 117 # Committed to Python as r75747 in the py26-maint branch, but not 118 # in time for 2.6.4 -- so we can remove this patch the next time 119 # we update Python in Sage. 120 cp patches/readline.c-spacebug src/Modules/readline.c 121 fi 122 123 if [ $? -ne 0 ]; then 124 echo >&2 "Error copying patched readline.c" 125 exit 1 126 fi 127 128 52 129 # We are setting LDFLAGS and CPPFLAGS so that we pick up Sage's readline 53 130 LDFLAGS="-L$SAGE_LOCAL/lib $LDFLAGS" 54 131 export LDFLAGS … … 62 139 EXTRAFLAGS="--without-pymalloc"; export EXTRAFLAGS 63 140 fi 64 141 142 # Program around weird bug in build process: 143 # Apparently if you have this: 144 # export DISTUTILS_DEBUG=1 145 # in your environment variables, the build craps out. No idea why this 146 # is. 147 # -- Yi Qiang 148 # 149 # This bug was fixed in Python, but not yet in Python 2.6.4. So this fix 150 # can be removed the next time we upgrade our version of Python. See 151 # 152 # http://bugs.python.org/issue6954 153 # 154 # for the fix. 155 # 156 unset DISTUTILS_DEBUG 157 158 65 159 cd src 66 160 67 161 touch Include/* … … 93 187 --enable-unicode=ucs4 94 188 fi 95 189 else 96 ./configure $EXTRAFLAGS --prefix="$SAGE_LOCAL" --enable-unicode=ucs4 CC="$CC $CFLAGS" CXX="$CXX $CXXFLAGS" 190 ./configure $EXTRAFLAGS --prefix="$SAGE_LOCAL" --enable-unicode=ucs4 \ 191 CC="$CC $CFLAGS" CXX="$CXX $CXXFLAGS" 97 192 fi 98 193 99 194 100 195 if [ $? -ne 0 ]; then 101 echo "Error configuring Python."196 echo >&2 "Error configuring Python." 102 197 exit 1 103 198 fi 104 199 105 200 $MAKE 106 201 if [ $? -ne 0 ]; then 107 echo "Error building Python."202 echo >&2 "Error building Python." 108 203 exit 1 109 204 fi 110 205 111 206 # running 'make install' in parallel is a bad, bad idea 207 # FIXME: The proper way to disable parallel make is to use 208 # $MAKE -j1 ... (We rely on GNU make anyway). 112 209 MAKE=make; export MAKE 113 210 # the "-i" is crucial, esp. in the case of a major upgrade 114 211 make -i install 115 212 if [ $? -ne 0 ]; then 116 echo "Error installing Python."213 echo >&2 "Error installing Python." 117 214 exit 1 118 215 fi 119 216 } … … 122 219 # informative error message. This is helps in determining why the 123 220 # configuration, compilation or installation failed. So put this before the 124 221 # build() function. 125 set +e 222 set +e # This is redundant here, but doesn't hurt to keep it... ;-) 126 223 127 224 build 128 225 129 cd $SAGE_LOCAL/lib226 cd "$SAGE_LOCAL/lib" 130 227 131 228 # move the python directory if we're upgrading from a version 132 # of sage with python 2. 6133 if [ -d python2. 6/site-packages ]; then134 mv python2. 6/site-packages python2.7/site-packages-old229 # of sage with python 2.5 230 if [ -d python2.5/site-packages ]; then 231 mv python2.5/site-packages python2.6/site-packages-old 135 232 fi 136 233 137 rm -rf python/python2. 7 python/python2.6 python/python2.5 python/python python python2.4 python2.5138 ln -s python2. 7python234 rm -rf python/python2.6 python/python2.5 python/python python python2.4 python2.5 235 ln -s python2.6 python 139 236 if [ $? -ne 0 ]; then 140 echo "Error creating symbolic link"237 echo >&2 "Error creating symbolic link" 141 238 exit 1 142 239 fi 143 240 144 241 # Sleeping for three seconds so that parallel 'make install' catches up 145 242 # with the following test. 243 # XXX We have disabled parallel 'make install' above, but never mind. 146 244 echo "Sleeping for three seconds before testing python" 147 245 sleep 3 148 246 … … 150 248 # All these modules are important and if any one 151 249 # fails to build, Sage will not work. 152 250 251 import_errors=false 153 252 for module in math hashlib crypt ; do 154 253 "$SAGE_LOCAL/bin/python" -c "import $module" 155 254 if [ $? -eq 0 ] ; then 156 255 echo "$module module imported OK" 157 256 else 158 echo "$module module failed to import" 159 exit 1 257 echo >&2 "$module module failed to import" 258 import_errors=true 259 # exit 1 # not yet 160 260 fi 161 261 done 262 263 if $import_errors; then 264 echo >&2 "Error: One or more modules failed to import." 265 # Check if we are on Debian or one of its derivatives: 266 if command -v dpkg && ! command -v dpkg-architecture >/dev/null; then 267 echo >&2 "You may have to install 'dpkg-architecture'" 268 echo >&2 "which is part of the Debian package 'dpkg-dev'." 269 echo >&2 "Try installing it by typing" 270 echo >&2 " sudo apt-get install dpkg-dev" 271 echo >&2 "and rerun 'make'." 272 fi 273 exit 1 274 fi 275
comment:138 follow-up: ↓ 139 Changed 9 years ago by
OK that's not a biggy while I had a look at #11447 I made my spkg before it was closed and possibly before it had reached its final shape. I wanted to wait for 2.7.3 but I will probably cut a 2.7.2 in which I will add this then.
I am keen to have this landing ASAP but I would understand if the release manager only wants to do it in a alpha0. That's both a big change and a big patch set. My concern is that my last patch may not be adequate. Steve tested a number of configurations and the order this comes out are all over the place. I may have to make it random which is not nice.
The other concern is #11986 which looks like a blast from the past and I have no clues on either of them.
comment:139 in reply to: ↑ 138 Changed 9 years ago by
Replying to fbissey:
OK that's not a biggy while I had a look at #11447 I made my spkg before it was closed and possibly before it had reached its final shape. I wanted to wait for 2.7.3 but I will probably cut a 2.7.2 in which I will add this then.
I am keen to have this landing ASAP but I would understand if the release manager only wants to do it in a alpha0. That's both a big change and a big patch set.
If you can finish it, I'd be happy to merge it. But to be realistic, I don't think this ticket is close to being finished. There is a huge amount of patches which needs to be reviewed, there is still #11986, there is the rebasing to #11447, maybe testing will reveal breakage on some systems...
comment:140 Changed 9 years ago by
There is a warning when building the reference manual:
docstring of sage.crypto.boolean_function:3: (WARNING/2) Block quote ends without a blank line; unexpected unindent.
comment:141 Changed 9 years ago by
Another minor comment: please use UTF8 in SPKG.txt. Currently, "François" is encoded as latin1.
comment:142 Changed 9 years ago by
comment:143 Changed 9 years ago by
- Milestone changed from sage-4.8 to sage-5.0
comment:144 follow-up: ↓ 145 Changed 9 years ago by
- Description modified (diff)
- Summary changed from Upgrade python to 2.7 to Upgrade python to 2.7.x
By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.
comment:145 in reply to: ↑ 144 ; follow-up: ↓ 146 Changed 9 years ago by
Replying to jdemeyer:
By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.
sage-5.0 is probably more realistic. Thanks for the useful comments, I hadn't seen the warnings in the documentation, this probably could be spun out of this ticket as it is probably a problem in the existing doc - just silent with the current python. I really would like to go to 2.7.3 when it comes out but I'll make a stop 2.7.2. I have a heavy workload this week but I'll try to put the spkg together. I know from sage-on-gentto that there will be some more patching to be done unfortunately although in some case I think that would be just a band aid. So more dependencies may appear :(
I am surprised at the "latin-1" encoding I am defaulting to UTF-8 and even my Changelogs now come correct everywhere.
We need someone with the know how to look at #11986.
comment:146 in reply to: ↑ 145 Changed 9 years ago by
Replying to fbissey:
Replying to jdemeyer:
By the way: Python 2.7.2 was released on June 11th, 2011 so you might as well go for that.
sage-5.0 is probably more realistic.
Especially since William essentially defined sage-5.0 as the release which will contain this ticket.
I hadn't seen the warnings in the documentation, this probably could be spun out of this ticket as it is probably a problem in the existing doc - just silent with the current python.
True, but I have no idea how Python 2.7 has changed this particular doctest. It could also be that something changed inside Sphinx or Cython because of Python 2.7.
I am surprised at the "latin-1" encoding I am defaulting to UTF-8 and even my Changelogs now come correct everywhere.
And now you're punished for the rest of your live as a developer because your name is not ASCII (and please don't write "Francois", that just looks very wrong).
We need someone with the know how to look at #11986.
I might have a look, but I don't want to promise anything. In any case, once #11986 is really the only remaining issue, it will certainly get fixed.
comment:147 Changed 9 years ago by
- Description modified (diff)
All right. Very busy week so far. I am removing the last patch and will spin it in a separate ticket because Steve did more testing and it appears that the test is somewhat random and was only working everywhere the same with python 2.6 by sheer luck.
Steve had a few interesting suggestions for replacing the test but that's clearly another ticket on which we'll forward people who touched that file for comments.
comment:148 follow-up: ↓ 151 Changed 9 years ago by
- Description modified (diff)
I finished updating the spkg to 2.7.2 in a moment of freedom. I called it 2.7.2.p0 as it has some patches in it. Hope all the permissions are right this time. And SPKG.txt is UTF-8 now.
I will need to update one patch and I think add a new one. At least relative to 4.7.2, I am so bogged down in work that I haven't been working at all on 4.8.0 which may require new patches.
comment:149 Changed 9 years ago by
Patch set updated for python-2.7.2.
comment:150 Changed 9 years ago by
- Description modified (diff)
comment:151 in reply to: ↑ 148 ; follow-up: ↓ 153 Changed 9 years ago by
- Description modified (diff)
Replying to fbissey:
I finished updating the spkg to 2.7.2 in a moment of freedom. I called it 2.7.2.p0 as it has some patches in it. Hope all the permissions are right this time.
Unfortunately, they are not. I fixed this in http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p0.spkg
comment:152 Changed 9 years ago by
- Description modified (diff)
comment:153 in reply to: ↑ 151 ; follow-up: ↓ 154 Changed 9 years ago by
Replying to jdemeyer:
Replying to fbissey:
I finished updating the spkg to 2.7.2 in a moment of freedom. I called it 2.7.2.p0 as it has some patches in it. Hope all the permissions are right this time.
Unfortunately, they are not. I fixed this in http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p0.spkg
OK. Was it the src folder and its content? Or did I forget something else in the spkg root folder?
comment:154 in reply to: ↑ 153 ; follow-up: ↓ 158 Changed 9 years ago by
Replying to fbissey:
OK. Was it the src folder and its content?
Yes. I solved this by doing chmod ugo+rX -R .
Or did I forget something else in the spkg root folder?
Also, yes. spkg-check should be executable.
comment:155 Changed 9 years ago by
This fails to build from scratch with sage-4.8.alpha2 on sage.math:
gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -L/scratch/jdemeyer/merger/sage-5.0/local/lib Parser/acceler.o Parser/grammar1.
o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar
.o Parser/pgen.o Objects/obmalloc.o Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o Parser/printgrammar.o Parser/pgenmain.o -
lpthread -ldl -lutil -o Parser/pgen
Parser/pgen ./Grammar/Grammar ./Include/graminit.h ./Python/graminit.c
touch Parser/pgen.stamp
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include -fPIC -DPy_BUILD_CORE -o Python/ast.o Python/ast.c
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include -fPIC -DPy_BUILD_CORE -o Python/compile.o Python/compile.c
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include -fPIC -DPy_BUILD_CORE -o Python/graminit.o Python/graminit.c
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include -fPIC -DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/scratch/jdemeyer/mer
ger/sage-5.0/local/include -fPIC -DPy_BUILD_CORE \
-DSVNVERSION="\"`LC_ALL=C svnversion .`\"" \
-DHGVERSION="\"`LC_ALL=C hg id -i .`\"" \
-DHGTAG="\"`LC_ALL=C hg id -t .`\"" \
-DHGBRANCH="\"`LC_ALL=C hg id -b .`\"" \
-o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
You must compile Sage first using 'make' in the Sage root directory.
You must compile Sage first using 'make' in the Sage root directory.
You must compile Sage first using 'make' in the Sage root directory.
<command-line>: warning: missing terminating " character
<command-line>:1:7: warning: missing terminating " character
<command-line>:2:10: warning: missing terminating " character
./Modules/getbuildinfo.c: In function 'Py_GetBuildInfo':
./Modules/getbuildinfo.c:45: error: missing terminating " character
./Modules/getbuildinfo.c:45: error: expected expression before ')' token
./Modules/getbuildinfo.c:46: error: missing terminating " character
./Modules/getbuildinfo.c:46: error: missing terminating " character
./Modules/getbuildinfo.c:47: error: missing terminating " character
./Modules/getbuildinfo.c:47: error: missing terminating " character
./Modules/getbuildinfo.c:53: error: 'buildinfo' undeclared (first use in this function)
./Modules/getbuildinfo.c:53: error: (Each undeclared identifier is reported only once
./Modules/getbuildinfo.c:53: error: for each function it appears in.)
./Modules/getbuildinfo.c: In function '_Py_hgversion':
./Modules/getbuildinfo.c:72: error: missing terminating " character
./Modules/getbuildinfo.c:72: warning: 'return' with no value, in function returning non-void
./Modules/getbuildinfo.c: In function '_Py_hgidentifier':
./Modules/getbuildinfo.c:79: error: missing terminating " character
./Modules/getbuildinfo.c:79: error: expected expression before ';' token
./Modules/getbuildinfo.c:83: error: missing terminating " character
./Modules/getbuildinfo.c:83: error: expected expression before ';' token
make[2]: *** [Modules/getbuildinfo.o] Error 1
make[2]: Leaving directory `/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/spkg/build/python-2.7.2.p0/src'
Error building Python.
Mhy guess would be that spkg-install is trying to run Mercurial or something, which obviously fails because Mercurial has not been compiled yet (and it can't since it depends on Python).
comment:156 Changed 9 years ago by
The error above might be fixed by #5852...
comment:157 Changed 9 years ago by
The current spkg installs fine on OS X 10.6 and 10.7, at least when I do "sage -f ..." in an existing Sage installation. I haven't tried building from scratch, and I haven't tried doctests or the patches yet. Would it be possible to provide a single large patch in addition to the individual small ones? That would be easier to apply for testing, while the small ones are easier to review.
comment:158 in reply to: ↑ 154 Changed 9 years ago by
Replying to jdemeyer:
Replying to fbissey:
OK. Was it the src folder and its content?
Yes. I solved this by doing
chmod ugo+rX -R .Or did I forget something else in the spkg root folder?
Also, yes. spkg-check should be executable.
Never used symbolic mode of chmod, always octal. But this looks like it is more effective to get the directory permissions right. Darn spkg-check.
Thanks for the tips. Sorry for being a pest at times.
I haven't tested the spkg from scratch on OSX I did on linux x86_64 and 4.8.alpha2. I will see what happens on my 10.5 machine later. 2.7.1 worked from scratch there i am sure. That looks very strange, can you link the full python building log?
comment:159 Changed 9 years ago by
A build from scratch on OS X 10.6 worked just fine, by the way.
comment:160 Changed 9 years ago by
There is still the docbuild warning:
docstring of sage.crypto.boolean_function:3: WARNING: Block quote ends without a blank line; unexpected unindent.
I am investigating.
comment:161 Changed 9 years ago by
I absolutely do not understand the docbuild warning but created a ticket at #12085.
comment:162 Changed 9 years ago by
- Dependencies changed from #11986 to #5852, #11986, #12085
comment:163 follow-up: ↓ 164 Changed 9 years ago by
Just an idea! This is because it wants to execute some mercurial commands to find if there are tags in the source. If we were to set
DHGVERSION DHGTAG DHGBRANCH
To something or even nothing we may not have this problem. A normal run with a system mercurial installed looks like:
x86_64-pc-linux-gnu-gcc -pthread -c -fno-strict-aliasing -O2 -pipe -march=native -fwrapv -DNDEBUG -I. -IInclude -I./Include -fPIC -DPy_BUILD_CORE \
-DSVNVERSION="\"`LC_ALL=C svnversion .`\"" \
-DHGVERSION="\"`LC_ALL=C hg id -i .`\"" \
-DHGTAG="\"`LC_ALL=C hg id -t .`\"" \
-DHGBRANCH="\"`LC_ALL=C hg id -b .`\"" \
-o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
abort: repository . not found!
abort: repository . not found!
abort: repository . not found!
I will try to see if I can fit that in the spkg.
comment:164 in reply to: ↑ 163 Changed 9 years ago by
comment:165 follow-up: ↓ 166 Changed 9 years ago by
If you insist but it would be easy to path Makefile.pre.in to prevent this from happening. It is mainly used to get pre-release version info if you checked out python source from hg. It is supposed to fail if you are using a properly released tarball.
comment:166 in reply to: ↑ 165 Changed 9 years ago by
comment:167 Changed 9 years ago by
I may try something more subtle first. Makefile.pre.in has
SVNVERSION= @SVNVERSION@ HGVERSION= @HGVERSION@ HGTAG= @HGTAG@ HGBRANCH= @HGBRANCH@
and
$(CC) -c $(PY_CFLAGS) \ -DSVNVERSION="\"`LC_ALL=C $(SVNVERSION)`\"" \ -DHGVERSION="\"`LC_ALL=C $(HGVERSION)`\"" \ -DHGTAG="\"`LC_ALL=C $(HGTAG)`\"" \ -DHGBRANCH="\"`LC_ALL=C $(HGBRANCH)`\"" \ -o $@ $(srcdir)/Modules/getbuildinfo.c
later so exporting HG* to something like /bin/true on non-windows OS would presumably solve the problem. But just patching to pass empty strings in the later would probably be safe everywhere.
comment:168 Changed 9 years ago by
The first patch doesn't apply in 4.8.alpha3 because of the changes in #12047. New patches will come some. There may be other surprises.
comment:169 Changed 9 years ago by
- Description modified (diff)
Updated the first patch. Haven't been able to test things to find any new problems yet.
comment:170 Changed 9 years ago by
There are a few test failures on a Pentium 4 32-bit Linux system with sage-4.8.alpha2 + #9958 + #11986:
sage -t --long -force_lib devel/sage/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/suffix_trees.py", line 1323:
sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
True
Got:
False
**********************************************************************
sage -t --long -force_lib devel/sage/sage/misc/randstate.pyx
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/misc/randstate.pyx", line 61:
sage: rtest()
Expected:
(978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.83350776541997362)
Got:
(978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.8335077654199736)
**********************************************************************
sage -t --long -force_lib devel/sage/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 20:
sage: it.next()
Expected:
word: 6456
Got:
word: 4564
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 23:
sage: it.next()
Expected:
word: 5645
Got:
word: 6456
**********************************************************************
File "/home/jdemeyer/sage-5.0/devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py", line 26:
sage: it.next()
Expected:
word: 4564
Got:
word: 5645
**********************************************************************
comment:171 Changed 9 years ago by
I have made a patch for the Integer hashing, see #11986.
comment:172 Changed 9 years ago by
I have seen your patch in #11986 that's a good rewrite of the hashing in question, the -1 bit is particularly tricky. I mentioned it before but we probably need a different approach for combinat/words/nfactor_enumerable_word.py, the sequence is pretty much random depending on arch and other factors it seems. The randstate.pyx must have escaped http://trac.sagemath.org/sage_trac/attachment/ticket/9958/trac_9958-fixing_numericalnoise-part4.patch because the test results are identical to some performed later in the same file. I'll correct that shortly.
The suffix_trees.pyx rings a bell but I cannot remember where and when.
comment:173 Changed 9 years ago by
ha http://trac.sagemath.org/sage_trac/ticket/9958#comment:18 is where I had suffix_trees.pyx before. I don't have a 32 bit linux box anymore so it is more difficult. I note they were two you only see one now.
comment:174 Changed 9 years ago by
- Description modified (diff)
comment:175 Changed 9 years ago by
- Dependencies changed from #5852, #11986, #12085 to #5852, #11986, #12085, #12096
comment:176 Changed 9 years ago by
I need a kind soul with 32 bit linux system to explore the suffix tree issue, the doctest is as follow:
sage: from sage.combinat.words.suffix_trees import ImplicitSuffixTree, SuffixTrie
sage: W = Words("ab")
sage: t = ImplicitSuffixTree(W("aba"))
sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2], [(4, W("b")), 5], [(5, W("a")), 1]])
True
It would be helpful to get the values of "W", "t" and "t.trie_type_dict()" on python-2.6 and 2.7. Then we can make a decision on whether the answer we get with python-2.7 is equivalent to the current output or not.
comment:177 Changed 9 years ago by
Here (x86) with sage-4.8.alpha3 + #9958 + #11986 the suffix_trees test passes:
./sage -t -long -force_lib devel/sage/sage/combinat/words/suffix_trees.py
sage -t -long -force_lib "devel/sage/sage/combinat/words/suffix_trees.py"
[5.1 s]
----------------------------------------------------------------------
All tests passed!
The requested values are:
sage: W
Words over Ordered Alphabet ['a', 'b']
sage: t
Implicit Suffix Tree of the word: aba
sage: t.trie_type_dict()
{(4, word: b): 5, (0, word: a): 4, (0, word: b): 3, (5, word: a): 1, (3, word: a): 2}
FWIW, on amd64 the test fails with the same sage-4.8.alpha3 + #9958 + #11986
comment:178 Changed 9 years ago by
On sage.math (x86_64 Linux), sage-4.8.alpha3 + #12106 (unrelated) + #9958 + #11986 gives errors in two files:
sage -t -force_lib devel/sage/sage/matrix/matrix_mod2e_dense.pyx
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 790:
sage: float(A.density())
Expected:
0.099738...
Got:
0.099739
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 798:
sage: float(A.density())
Expected:
0.499759...
Got:
0.49976
**********************************************************************
(related ticket: #9562) and
sage -t -force_lib devel/sage/sage/gsl/integration.pyx
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 110:
sage: numerical_integral(x^2, 0, 1, max_points=100)
Expected:
(0.33333333333333331, 3.7007434154171879e-15)
Got:
(0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 115:
sage: numerical_integral(sin(x)^3 + sin(x), 0, pi)
Expected:
(3.333333333333333, 3.7007434154171883e-14)
Got:
(3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 120:
sage: numerical_integral(lambda x: sin(x)^3 + sin(x), 0, pi)
Expected:
(3.333333333333333, 3.7007434154171883e-14)
Got:
(3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 131:
sage: numerical_integral(f, 0, 1, max_points=200, eps_abs=1e-7, eps_rel=1e-7, rule=4)
Expected:
(0.33333333333333331, 3.7007434154171879e-15)
Got:
(0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/mnt/usb1/scratch/jdemeyer/merger/sage-5.0/devel/sage-main/sage/gsl/integration.pyx", line 172:
sage: exp(-1/x).nintegral(x, 1, 2) # via maxima
Expected:
(0.50479221787318396, 5.6043194293440752e-15, 21, 0)
Got:
(0.504792217873184, 5.604319429344075e-15, 21, 0)
**********************************************************************
(related ticket: #12047)
Also, the spkg should be rebased to #12096.
comment:179 Changed 9 years ago by
I get on 32-bit i386 Linux with sage-4.8.alpha2 + various patches including #9958:
sage: from sage.combinat.words.suffix_trees import ImplicitSuffixTree, SuffixTrie
sage: W = Words("ab")
sage: t = ImplicitSuffixTree(W("aba"))
sage: W
Words over Ordered Alphabet ['a', 'b']
sage: t
Implicit Suffix Tree of the word: aba
sage: t.trie_type_dict()
{(4, word: a): 1, (0, word: b): 5, (0, word: a): 3, (5, word: a): 2, (3, word: b): 4}
comment:180 follow-up: ↓ 181 Changed 9 years ago by
thanks for all that feedback. I knew there would be new stuff. I unfortunately (like all of us) not enough time to do things in a speedy fashion. I am bogged down as well with the sage-on-gentoo version of alpha3 not wanting to play nice with #4260 https://github.com/cschwan/sage-on-gentoo/issues/108 and it annoys me.
comment:181 in reply to: ↑ 180 Changed 9 years ago by
Replying to fbissey:
I am bogged down as well with the sage-on-gentoo version of alpha3 not wanting to play nice with #4260 https://github.com/cschwan/sage-on-gentoo/issues/108 and it annoys me.
Does vanilla Sage build and run on that same machine with the same compiler? Be sure to try a different gcc version, Linbox seems to be sensitive to that.
comment:182 Changed 9 years ago by
On x86 with sage-4.8.alpha3 + #9958 + #11986 I have the following failures:
sage -t -long -force_lib devel/sage-main/sage/gsl/integration.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 110:
sage: numerical_integral(x^2, 0, 1, max_points=100)
Expected:
(0.33333333333333331, 3.7007434154171879e-15)
Got:
(0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 115:
sage: numerical_integral(sin(x)^3 + sin(x), 0, pi)
Expected:
(3.333333333333333, 3.7007434154171883e-14)
Got:
(3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 120:
sage: numerical_integral(lambda x: sin(x)^3 + sin(x), 0, pi)
Expected:
(3.333333333333333, 3.7007434154171883e-14)
Got:
(3.333333333333333, 3.700743415417188e-14)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 131:
sage: numerical_integral(f, 0, 1, max_points=200, eps_abs=1e-7, eps_rel=1e-7, rule=4)
Expected:
(0.33333333333333331, 3.7007434154171879e-15)
Got:
(0.3333333333333333, 3.700743415417188e-15)
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/gsl/integration.pyx", line 172:
sage: exp(-1/x).nintegral(x, 1, 2) # via maxima
Expected:
(0.50479221787318396, 5.6043194293440752e-15, 21, 0)
Got:
(0.504792217873184, 5.604319429344075e-15, 21, 0)
**********************************************************************
sage -t -long -force_lib devel/sage-main/sage/misc/randstate.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/misc/randstate.pyx", line 61:
sage: rtest()
Expected:
(978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.83350776541997362)
Got:
(978, 0.184109262667515, -3*x^2 - 1/12, (4,5), [ 0, 1, 1, 0, 0 ], 1161603091, 60359, 0.8335077654199736)
**********************************************************************
sage -t -long -force_lib devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 790:
sage: float(A.density())
Expected:
0.099738...
Got:
0.099739
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/matrix/matrix_mod2e_dense.pyx", line 798:
sage: float(A.density())
Expected:
0.499759...
Got:
0.49976
**********************************************************************
sage -t -long -force_lib devel/sage-main/sage/functions/transcendental.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/functions/transcendental.py", line 80:
sage: w = exponential_integral_1(2,4); w
Expected:
[0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.7665622843924751e-05]
Got:
[0.04890051070806112, 0.0037793524098489067, 0.00036008245216265873, 3.766562284392475e-05]
**********************************************************************
comment:183 Changed 9 years ago by
I now understand why you removed the transcendental patch Jereon some bits ended in my revised trac_9958-fixing_numericalnoise-part1_p2.patch when they shouldn't have. That's what is happening to Steve. I will rectify that.
I removed the gsl bits because of #12047 so that has to be redone.
And I haven't had time to try vanilla yet but it is on my TODO list and I suspect g++ too.
comment:184 Changed 9 years ago by
- Description modified (diff)
updated the first patch and readded the transcendental patch to the list.
comment:185 Changed 9 years ago by
For suffix_tree it looks like your two values don't agree:
{(4, word: a): 1, (0, word: b): 5, (0, word: a): 3, (5, word: a): 2, (3, word: b): 4}
{(4, word: b): 5, (0, word: a): 4, (0, word: b): 3, (5, word: a): 1, (3, word: a): 2}
If we add that I don't see it on OS X x86 that looks bad. I also have an x86_64 machine where the value is the same as Jeroen and the test fails. Steve get the right value on the other hand so there is something going there.
comment:186 Changed 9 years ago by
- Description modified (diff)
Added two new patches for gsl integration and m4rie tickets merged in 4.8.alpha3.
comment:188 Changed 9 years ago by
Everything passes against sage-4.8.alpha3 on 64bit. Yay!
comment:189 follow-up: ↓ 190 Changed 9 years ago by
I forgot to add a new bit for randstate.pyx on 32 bit. I'll do that later but I know it is missing. Should we involve someone from combinatronics to get to the bottom of suffix_tree and nfactor_enumerate_word?
comment:190 in reply to: ↑ 189 Changed 9 years ago by
Replying to fbissey:
Should we involve someone from combinatronics to get to the bottom of suffix_tree and nfactor_enumerate_word?
Sounds good, try to send an email to sage-devel.
comment:191 follow-up: ↓ 193 Changed 9 years ago by
For the sake of completeness, I now have
32bit: one failure
sage -t -long -force_lib devel/sage-main/sage/misc/randstate.pyx
which was mentioned above.
64bit: two failures
sage -t -long -force_lib devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 20:
sage: it.next()
Expected:
word: 5645
Got:
word: 4564
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 23:
sage: it.next()
Expected:
word: 6456
Got:
word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 26:
sage: it.next()
Expected:
word: 4564
Got:
word: 6456
**********************************************************************
sage -t -long -force_lib devel/sage-main/sage/combinat/words/suffix_trees.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/suffix_trees.py",
line 1323:
sage: t.trie_type_dict() == dict([[(0, W("a")), 4], [(0, W("b")), 3], [(3, W("a")), 2]
, [(4, W("b")), 5], [(5, W("a")), 1]])
Expected:
True
Got:
False
**********************************************************************
which have been mentioned previously.
comment:192 Changed 9 years ago by
This ticket made me notice a bug with the L-series of Eisenstein series: #12124. With that ticket applied, the patch to sage/modular/modform/eis_series.py here can be removed.
comment:193 in reply to: ↑ 191 Changed 9 years ago by
First of all, I'm not an expert of combinatorics on word. So I'd rather Sebastien Labbe to jump in the discussion. I just trying to help.
64bit: two failures
sage -t -long -force_lib devel/sage-main/sage/combinat/words/nfactor_enumerable_word.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 20:
sage: it.next()
Expected:
word: 5645
Got:
word: 4564
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 23:
sage: it.next()
Expected:
word: 6456
Got:
word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-main/sage/combinat/words/nfactor_enumerable
_word.py", line 26:
sage: it.next()
Expected:
word: 4564
Got:
word: 6456
**********************************************************************
This first failure is not a problem. The iterator here is extracted from a dict of dict so it is not unexpected to have a random order. Fixing the test of marking it at random should be ok. This test is an explanation for the user, the real feature is tested further in the file.
which have been mentioned previously.
{(4, word: a): 1, (0, word: b): 5, (0, word: a): 3, (5, word: a): 2, (3, word: b): 4}
{(4, word: b): 5, (0, word: a): 4, (0, word: b): 3, (5, word: a): 1, (3, word: a): 2}
This one is more tricky: As far as I understand, both result are perfectly legible: the second is the same as the first after applying the following cyclic permutation (3,4,5). The numbering is random. So (again as far as I understand), the problem in only created by the cross-platform non-determinism of the set/dict data structure. I'm not sure here what is the best fix.
Cheers,
Florent
comment:194 Changed 9 years ago by
Thank you Florent, that is helpful. We kind of worked out that word was OK but we thought we could have a better test. If you think that's ok to mark it random I think we'll do that with some explanation.
The fact that both output in suffix_tree are legit is good to know. We can probably work around that. We may even have a complete patch set by Monday, cross fingers.
comment:195 Changed 9 years ago by
- Description modified (diff)
Upgraded succesfully from sage-4.6.2, all tests pass except for the known problematic tests.
comment:196 follow-up: ↓ 197 Changed 9 years ago by
comment:197 in reply to: ↑ 196 Changed 9 years ago by
Replying to fbissey:
I'll work on the remaining tests tomorrow, I hope.
That would be great! I would love to see this finished in time, such that the next release can really be sage-5.0.
Changed 9 years ago by
fixing numerical noise part 1 - updated 20111211: separate randstate and eis_modn_series
comment:198 Changed 9 years ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
updating the patch set with a tentative solution for suffix_trees. It may not be exhaustive of all possible answers though. If we are going for it for the next release I can redo the python spkg to fix the lib/lib64 from #12131.
Putting to need review :)
comment:199 Changed 9 years ago by
Hi,
I just read randomly sage-devel (I was away from sage development for several weeks) and saw that the discussion about upgrading to 2.7 was having problem with some files I am related with (suffix_tree.py nfactor_enumerable_word.py). I still don't understand the dict problem for suffix tree (one minute was not enough). I will look into that to avoid similar problem in the future. If something else pops up in those file, I can do the fix next time.
Sébastien
comment:200 Changed 9 years ago by
Your input on the matter is welcome. According to Florent, the problem in suffix_trees is that there are several equivalent answers and some platforms now give a different one than the one you originally tested for. Note that we don't know what the trigger is for having a different answer it is not a 32/64 bit separation as we have seen both answers on the two kinds of systems.
comment:201 follow-up: ↓ 216 Changed 9 years ago by
Actually, you should have a look at what I did for both of these in trac_9958-nfactor_enumerable_word-randomness.patch and trac_9958-suffix_trees-variations.patch. If you could give your opinion on the patches or if you want to submit patches of your own it would be appreciated.
comment:202 Changed 9 years ago by
- Description modified (diff)
With the current update I have the following failures:
32bit
sage -t -long -force_lib devel/sage-9958/sage/combinat/words/nfactor_enumerable_word
.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 23:
sage: it.next()
Expected:
word: 6456 # random
Got:
word: 6456
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 25:
sage: it.next()
Expected:
word: 5645 # random
Got:
word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 27:
sage: it.next()
Expected:
word: 4564 # random
Got:
word: 4564
**********************************************************************
64bit
sage -t -long -force_lib devel/sage-9958/sage/combinat/words/nfactor_enumerable_word
.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 23:
sage: it.next()
Expected:
word: 6456 # random
Got:
word: 4564
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 25:
sage: it.next()
Expected:
word: 5645 # random
Got:
word: 5645
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/combinat/words/nfactor_enume
rable_word.py", line 27:
sage: it.next()
Expected:
word: 4564 # random
Got:
word: 6456
**********************************************************************
and on 32bit and 64bit
sage -t -long -force_lib devel/sage-9958/sage/misc/randstate.pyx
**********************************************************************
File "/storage/sage/sage-4.8.alpha3/devel/sage-9958/sage/misc/randstate.pyx", line 15
3:
sage: random(), getrandbits(20), uniform(5.0, 10.0), normalvariate(0, 1)
Expected:
(0.82940228518742587, 624859L, 5.77894484361117, -0.42013668263087578)
Got:
(0.8294022851874259, 624859L, 5.77894484361117, -0.4201366826308758)
**********************************************************************
I don't recall previously seeing this particular randstate.pyx failure. Perhaps I've missed something.
comment:203 Changed 9 years ago by
- Status changed from needs_review to needs_work
Yeah, the #random should be on the line with the command to execute, not the result.
comment:204 Changed 9 years ago by
I had to make that mistake :(
The last randstate.pyx one is definitely new, I am sure it didn't happen before.
Changed 9 years ago by
take into account the randomness of it.next sequence of call - put random in the right place this time.
comment:205 Changed 9 years ago by
OK I replaced the trac_9958-nfactor_enumerable_word-randomness.patch in place. So random should be in the right place now. I am looking to see if I can reproduce this particular randstate.pyx problem.
comment:206 Changed 9 years ago by
I missed a line in randstate! And you cut the last digit of the line number Steve, now that caused me some confusion. Updating shortly.
comment:207 Changed 9 years ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
OK new randstate patch, this time it is even a .patch instead of .pyx. Thanks for pointing out those mistakes.
comment:208 Changed 9 years ago by
comment:209 follow-up: ↓ 212 Changed 9 years ago by
Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order. I removed a patch for ia64 that didn't obviously port to python-2.7 so a look at that platform is also in order. In the mean time we can review #11986 and #12124 which I think can be done independently.
We could even split the combinat patches as they should work with python-2.6. It just there is nothing obvious to fix there.
comment:210 Changed 9 years ago by
As I have said before, testing is easy. I am currently running tests for the upcoming sage-4.8.alpha4, but after that, I could test the new Python on various systems.
But still, somebody needs to actually look at the patches and verify that they make sense.
comment:211 Changed 9 years ago by
OK Jeroen, that's a boring task for someone most of them are of the form:
diff --git a/sage/misc/sage_unittest.py b/sage/misc/sage_unittest.py
--- a/sage/misc/sage_unittest.py
+++ b/sage/misc/sage_unittest.py
@@ -201,12 +201,12 @@
Failure in _test_b:
Traceback (most recent call last):
...
- AssertionError
+ AssertionError: None
------------------------------------------------------------
Failure in _test_d:
Traceback (most recent call last):
...
- AssertionError
+ AssertionError: None
------------------------------------------------------------
Failure in _test_pickling:
Traceback (most recent call last):
@@ -220,14 +220,14 @@
running ._test_b() . . . fail
Traceback (most recent call last):
...
- AssertionError
+ AssertionError: None
------------------------------------------------------------
running ._test_c() . . . pass
running ._test_category() . . . pass
running ._test_d() . . . fail
Traceback (most recent call last):
...
- AssertionError
+ AssertionError: None
------------------------------------------------------------
running ._test_not_implemented_methods() . . . pass
running ._test_pickling() . . . fail
@@ -249,7 +249,7 @@
File ..., in _test_b
def _test_b(self, tester): tester.fail()
...
- AssertionError
+ AssertionError: None
In conjonction with ``%pdb on``, this allows for the debbuger
to jump directly to the first failure location.
@@ -311,7 +311,7 @@
sage: tester.assert_(1 == 0)
Traceback (most recent call last):
...
- AssertionError
+ AssertionError: False is not true
sage: tester.assert_(1 == 0, "this is expected to fail")
Traceback (most recent call last):
...
and other numerical noise niceties cutting one digit and sometimes the rounding makes a lot of digits disappear (and it looks better actually), for example in plot/colors.py
@@ -290,23 +290,23 @@
sage: from sage.plot.colors import rgbcolor
sage: rgbcolor(Color(0.25, 0.4, 0.9))
- (0.25, 0.40000000000000002, 0.90000000000000002)
+ (0.25, 0.4, 0.9)
sage: rgbcolor('purple')
Only the two combinat patches are a bit more subtle.
comment:212 in reply to: ↑ 209 ; follow-up: ↓ 214 Changed 9 years ago by
Replying to fbissey:
Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order.
Is this all based on 4.8.alpha3? I can try to do this starting today. Jeroen might be done before me, though :) at least by the time I finish applying ALL of these patches.
comment:213 Changed 9 years ago by
comment:214 in reply to: ↑ 212 ; follow-up: ↓ 217 Changed 9 years ago by
Replying to kcrisman:
Replying to fbissey:
Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order.
Is this all based on 4.8.alpha3? I can try to do this starting today. Jeroen might be done before me, though :)
Go ahead, I'm currently tackling other beasts...
comment:215 Changed 9 years ago by
- Dependencies changed from #5852, #11986, #12085, #12096 to #5852, #11986, #12085, #12096, #12124
comment:216 in reply to: ↑ 201 ; follow-up: ↓ 218 Changed 9 years ago by
Replying to fbissey:
Actually, you should have a look at what I did for both of these in trac_9958-nfactor_enumerable_word-randomness.patch and trac_9958-suffix_trees-variations.patch. If you could give your opinion on the patches or if you want to submit patches of your own it would be appreciated.
Hi,
I just look at the suffix tree code (written not by me but by Franco Saliola). The method trie_type_dict is not used anywhere in the file neither in sage/combinat/words neither in all Sage I would guess. So, it's kind of difficult to guess what the result should be (input and output blocks are absent in documentation). I think that method was used by Franco to compare the implementation of ImplicitSuffixTree with that of SuffixTrie but not seriously because the output is not practical (one can't create a DiGraph out of it).
Anyway, there are two loops in that method that both go through .iteritems() of a dictionary... Integer indexes present in the result depends on the ordering of those two loops. So, it is possible that a third different result is obtained in some other machine or system today or later. I just made a patch which should be more durable in the long term.
For the nfactor enumerable file, the fix is perfect.
comment:217 in reply to: ↑ 214 Changed 9 years ago by
Unfortunately, I'd say it is too early. We need Karl to test it on his old ppc OS X 10.4 mac and a run on solaris is also in order.
Is this all based on 4.8.alpha3? I can try to do this starting today. Jeroen might be done before me, though :)
Go ahead, I'm currently tackling other beasts...
Okay, at least everything applies and builds from this and the two other tickets. (Note that #11986 is listed as a dependency of and for #9958.)
So presumably sometime tomorrow I can report back. So far six files having tested well...
comment:218 in reply to: ↑ 216 Changed 9 years ago by
Replying to slabbe:
Anyway, there are two loops in that method that both go through
.iteritems()of a dictionary... Integer indexes present in the result depends on the ordering of those two loops. So, it is possible that a third different result is obtained in some other machine or system today or later. I just made a patch which should be more durable in the long term.For the nfactor enumerable file, the fix is perfect.
Thanks. I thought there could be a third possible results from Florent's comments (permutation (3,4,5)) but couldn't figure what it would look like. Happy to have some serious review for that bit.
comment:219 Changed 9 years ago by
OK the upgrade of R in #12057 shipped with 4.8.alpha4 makes a small bit of patch for r.py obsolete. Updating it shortly.
comment:220 Changed 9 years ago by
- Description modified (diff)
First patch updated. I don't think there will be any more changes because of 4.8.alpha4 but I could be wrong.
comment:221 Changed 9 years ago by
It looks like a slight modification is still needed in interfaces/r.py. On both x86 and amd64 I get
sage -t -long -force_lib devel/sage-main/sage/interfaces/r.py
**********************************************************************
File "/storage/sage/sage-4.8.alpha4/devel/sage-main/sage/interfaces/r.py", line 173:
sage: sum(rr._sage_())
Expected:
9.97721251689810...
Got:
9.97721251689811
**********************************************************************
attachment:trac_9958-suffix_trees-variations-sl.patch:ticket:9958 was used in the tests.
comment:222 Changed 9 years ago by
I have seen it on my sage-on-gentoo test run but I hadn't tried yet on a vanilla to check if it was caused by other gentoo stuff. I will add a small patch ASAP.
comment:223 Changed 9 years ago by
- Description modified (diff)
New patch included for interface/r.py in 4.8.alpha4.
comment:224 follow-up: ↓ 225 Changed 9 years ago by
How do we test this ticket? Is it enough to untar a fresh tarball of the 4.8.alpha4 source, replace the existing python spkg with the one from this ticket, and do a normal make? Then apply the patches to the sage library and run doctests?
On OSX 10.6.8, I tried downloading a fresh copy of 4.8.alpha4, putting this python spkg in the spkg/standard directory and deleting the old python spkg. I ended up getting a build error, seemingly right after I finished building scipy (even though scipy said it was built succesfully, but I was also building spkgs in parallel). I typed "make" again to restart the process and the same thing happened after rpy built---it said rpy was built successfully, but then it stopped the build, saying "Error building Sage".
comment:225 in reply to: ↑ 224 Changed 9 years ago by
Replying to jason:
How do we test this ticket? Is it enough to untar a fresh tarball of the 4.8.alpha4 source, replace the existing python spkg with the one from this ticket, and do a normal make? Then apply the patches to the sage library and run doctests?
Yes that's how you should do it.
On OSX 10.6.8, I tried downloading a fresh copy of 4.8.alpha4, putting this python spkg in the spkg/standard directory and deleting the old python spkg. I ended up getting a build error, seemingly right after I finished building scipy (even though scipy said it was built succesfully, but I was also building spkgs in parallel). I typed "make" again to restart the process and the same thing happened after rpy built---it said rpy was built successfully, but then it stopped the build, saying "Error building Sage".
Could you post the log for the build of python somewhere? scipy and rpy are built after sage (or at least they can be since they are runtime and not buildtime dependencies) so it is possible you were building sage itself at the same time. Is there a log for the sage spkg (in spkg/log)?
comment:226 Changed 9 years ago by
I posted the install.log here: http://sage.math.washington.edu/home/jason/sage-4.8.alpha4-python2.7-install.log.bz2
If you search for "Error building Sage", you'll see the multiple times I restarted the build (that message appeared after each time I restarted the build).
Here's the list of spkg logs I have:
sage-4.8.alpha4newpython/spkg/logs% ls atlas-3.8.4.log gdmodule-0.56.p7.log moin-1.9.1.p2.log ratpoints-2.1.3.p1.log blas-20070724.log genus2reduction-0.3.p8.log mpfi-1.3.4-cvs20071125.p8.log readline-6.2.p1.log boehm_gc-7.2.alpha6.p1.log gfan-0.4plus.p1.log mpfr-2.4.2.log rpy2-2.0.8.log boost-cropped-1.34.1.log givaro-3.2.13rc2.p2.log mpir-2.1.3.p8.log rubiks-20070912.p17.log bzip2-1.0.5.log glpk-4.44.log mpmath-0.17.log sage_root-4.8.alpha4.log cddlib-094f.p8.log gnutls-2.2.1.p5.log networkx-1.2.p1.log sage_scripts-4.8.alpha4.log cephes-2.8.log graphs-20070722.p1.log ntl-5.5.2.log sagenb-0.8.25.log cliquer-1.2.p10.log gsl-1.15.log numpy-1.5.1.log scipy-0.9.p1.log conway_polynomials-0.2.log iconv-1.13.1.p3.log opencdk-0.6.6.p5.log scons-1.2.0.log cvxopt-1.1.3.log iml-1.0.1.p13.log palp-1.1.p3.log setuptools-0.6.16.log cython-0.15.1.log ipython-0.10.2.p0.log pari-2.5.0.p2.log singular-3-1-3-3.p2.log dir-0.1.log jinja2-2.5.5.log patch-2.5.9.p2.log sphinx-1.1.2.p0.log docutils-0.7.p0.log lapack-20071123.p2.log pexpect-2.0.p4.log sqlalchemy-0.5.8.log ecl-11.1.2.cvs20111120.p0.log lcalc-1.23.p9.log pil-1.1.6.p4.log sqlite-3.7.5.log eclib-20100711.p0.log libfplll-3.0.12.p1.log polybori-0.7.1.p6.log symmetrica-2.0.p7.log ecm-6.3.p2.log libgcrypt-1.4.4.p3.log polytopes_db-20100210.log sympow-1.018.1.p9.log elliptic_curves-0.3.log libgpg_error-1.6.p3.log ppl-0.11.2.log sympy-0.7.1.log extcode-4.8.alpha4.log libm4ri-20111004.log prereq-0.9.log tachyon-0.98.9.p5.log f2c-20070816.p2.log libm4rie-20111004.log pycrypto-2.1.0.log termcap-1.3.1.p1.log flint-1.5.0.p10.log libpng-1.2.35.p3.log pygments-1.3.1.p0.log twisted-9.0.p2.log flintqs-20070817.p6.log linbox-1.1.6.p5.log pynac-0.2.3.log zlib-1.2.5.p0.log fortran-20100629.log matplotlib-1.0.1.p0.log python-2.7.2.p0.log zn_poly-0.9.p5.log freetype-2.3.5.p3.log maxima-5.23.2.p2.log python_gnutls-1.1.4.p7.log zodb3-3.7.0.p4.log gd-2.0.35.p5.log mercurial-1.8.4.log r-2.14.0.p0.log
comment:227 Changed 9 years ago by
All is well on the PPC OS X front, even with the r.py, amazingly; it failed the first time but passed on a rerun.
comment:228 Changed 9 years ago by
Jason the log you sent is not very useful, at least I cannot easily find something interesting. Could you provide the following:
- python-2.7.2.p0.log
- polybori-0.7.1.p6.log
- ppl-0.11.2.log
- pynac-0.2.3.log
That's a shot in the dark so far. But the building of sage never started as there is no log for sage itself.
I digged a little bit more before posting it looks like R fails to build so I want to see r-2.14.0.p0.log but that may be a separate issue from python. matplotlib seems to fail to build too so I want matplotlib-1.0.1.p0.log and now we are talking real potential python trouble.
comment:229 Changed 9 years ago by
I'm going to try this with
export MAKE="make -j2"
on the same platform and report back.
comment:230 Changed 9 years ago by
First, here are my standard build environment settings:
export SAGE_MATPLOTLIB_GUI=yes export SAGE_PARALLEL_SPKG_BUILD=yes export MAKE="make -j6" export SAGE_SPKG_INSTALL_DOCS=yes
All the requested logs are tarred up at http://sage.math.washington.edu/home/jason/newpython-spkg-logs.tar.gz
comment:231 Changed 9 years ago by
Tonight, I'll try building again without setting any of those environment variables to see if it succeeds with a "stock" build.
comment:232 Changed 9 years ago by
Replying to kcrisman:
I'm going to try this with
> export MAKE="make -j2"
on the same platform and report back.
Interestingly, Scipy and R are definitely (right now) building before Sage. And they usually do not, for sure not Scipy, on my ancient older Mac.
Scipy has successfully installed, as has matplotlib. R takes forever but so far is fine. Maybe it was "just a race condition"?
comment:233 Changed 9 years ago by
I bet the matplotlib error shows up for me because I define SAGE_MATPLOTLIB_GUI (so that matplotlib tries to build the GUI backends).
comment:234 Changed 9 years ago by
It is quite possibly something broken with the GUI from what I can see of your original log. scipy can build whenever after numpy independently of sage so I don't see a problem there.
comment:235 Changed 9 years ago by
Yes I think matplotlib cannot find tk. We may want to look at this in depth on OS X.
comment:236 Changed 9 years ago by
The matplotlib error is fixed upstream: https://github.com/matplotlib/matplotlib/issues/139
comment:237 Changed 9 years ago by
Very good, we may want to bump the spkg to 1.1.0 then. From my sage-on-gentoo experience there is just one minor doctest failure from matplotli-1.1.0
sage -t -long -force_lib "devel/sage/sage/plot/colors.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/plot/colors.py", line 1241:
sage: len(maps.maps)
Expected:
134
Got:
138
**********************************************************************
File "/usr/share/sage/devel/sage/sage/plot/colors.py", line 1285:
sage: len(maps)
Expected:
134
Got:
138
**********************************************************************
Fairly standard each time we bump matplotlib or almost.
You R package definitely broke the first time around. I am fairly sure this is a parallel make issue. I have seen at least one instance that I suspect is parallel make on Gentoo. Of course given that it stop because of
make[6]: gcc: Resource temporarily unavailable
I think, we may have something else altogether. It should go into a separate ticket.
comment:238 follow-up: ↓ 240 Changed 9 years ago by
Do you already have a 1.1.0 spkg? I'm working on one right now, but you already have one...
comment:239 Changed 9 years ago by
Scipy has successfully installed, as has matplotlib. R takes forever but so far is fine. Maybe it was "just a race condition"?
Everything built before the Sage library except Gap and SageTeX. No problems. I say this shouldn't keep the Python upgrade out, or?
comment:240 in reply to: ↑ 238 ; follow-up: ↓ 241 Changed 9 years ago by
Replying to jason:
Do you already have a 1.1.0 spkg? I'm working on one right now, but you already have one...
I don't have one. I don't need one for sage-on-gentoo as I am using system packages. You go ahead and create a ticket to upgrade matplotlib. I say it is a show stopper because it would happen on other platforms. Hopefully we can review it quickly.
comment:241 in reply to: ↑ 240 Changed 9 years ago by
Replying to fbissey:
Replying to jason:
Do you already have a 1.1.0 spkg? I'm working on one right now, but you already have one...
I don't have one. I don't need one for sage-on-gentoo as I am using system packages. You go ahead and create a ticket to upgrade matplotlib. I say it is a show stopper because it would happen on other platforms. Hopefully we can review it quickly.
What about just making a patch upgrade with the 1.5-liner rather than making a whole new package which would require even MORE testing? Plus, then you don't have to worry about getting those four extra colormaps :) Anyway, I feel this is easier and more reasonable just to get this in 5.0.alpha0 (or whatever release is next for such a big spkg change as this ticket).
comment:242 Changed 9 years ago by
- Reviewers set to John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé
comment:243 Changed 9 years ago by
I have no objections to that. Faster is better in that case because we really want to land this quickly because every other moves of the tree makes me rewrite stuff. It start to be hard to manage.
comment:244 Changed 9 years ago by
comment:245 Changed 9 years ago by
I added a link to a patched version of the current matplotlib spkg. I also added the diff as an information-only patch above. Independent from this fix, I've also made a matplotlib 1.1.0 spkg, which also fixes the problem, but may need more testing than this very slight change to the existing matplotlib spkg.
comment:246 Changed 9 years ago by
- Description modified (diff)
comment:247 Changed 9 years ago by
- Description modified (diff)
comment:248 Changed 9 years ago by
By the way, the matplotlib 1.1.0 spkg is at #11915, but the spkg here should be sufficient.
comment:249 Changed 9 years ago by
#12131 has been merged, I'll rebase the python spkg by next Monday hopefully.
comment:250 Changed 9 years ago by
Using the new matplotlib-1.0.1.p1.spkg above, and my standard build environment variables:
export SAGE_MATPLOTLIB_GUI=yes export SAGE_PARALLEL_SPKG_BUILD=yes export MAKE="make -j6" export SAGE_SPKG_INSTALL_DOCS=yes
Everything except R builds nicely. R still has that message about resources not being available (something about fork). I then changed MAKE to "make -j2" and restarted the make and everything finished just fine. Again, on OSX 10.6.8 (4G of memory).
That's a bummer that make -j6 doesn't work. Regular old 4.8.alpha4 built fine, so I don't think it's just the new R spkg. I'll try building fresh one more time, but with MAKE=make -j4 just to check things out.
comment:251 Changed 9 years ago by
- Description modified (diff)
comment:252 Changed 9 years ago by
Okay, I started with 4.8.alpha4, added the new python spkg from this ticket, the new matplotlib spkg from #11915, applied all of the patches from this ticket, #11976, #11986, and #12124. I have a tarball with all of these:
http://sage.math.washington.edu/home/palmieri/misc/9958/sage-4.8.alpha4-9958.tar
along with an upgrade path:
http://sage.math.washington.edu/home/palmieri/misc/9958/sage-4.8.alpha4-9958/
for testing.
comment:253 Changed 9 years ago by
Did you also apply the patch from #11915?
comment:254 Changed 9 years ago by
No, looks like I missed that one.
comment:255 Changed 9 years ago by
Okay, here are versions including the patch from #11915:
comment:256 Changed 9 years ago by
- Description modified (diff)
comment:257 Changed 9 years ago by
I took jhpalmieri's original tarball, applied the patch from #11915 myself, built it with these settings:
SAGE_MATPLOTLIB_GUI=yes SAGE_PARALLEL_SPKG_BUILD=yes MAKE=make -j12 SAGE_SPKG_INSTALL_DOCS=yes
on Ubuntu 10.04. Everything built fine, and make ptestlong passed all tests. Yeah!
comment:258 follow-up: ↓ 259 Changed 9 years ago by
What remains to be done here for positive review?
comment:259 in reply to: ↑ 258 Changed 9 years ago by
Replying to kcrisman:
What remains to be done here for positive review?
First I have to rebase the python spkg to #12131. Second there is a new matplotlib spkg currently merged in 4.8.alpha5 (unreleased) that needs review in #11915. Then someone has to look at the patches to make sure they are not completely stupid (long tiedous job).
Then we can move this to a positive review. I was hoping to be able to to rebase the spkg on Sunday but it didn't happen. I am busy doing a bit of code for my paid work but once it is done. I'll go for it, that may be tomorrow in my time zone.
comment:260 Changed 9 years ago by
python-2.7.2.p0 spkg rebased and available.
comment:261 Changed 9 years ago by
#11986 also needs review. It looks like it may need to go at the same time as python 2.7 as my own testing with python 2.6 on linux amd64 was not positive but everything is smooth with python 2.7.
comment:262 follow-up: ↓ 263 Changed 9 years ago by
For sage-4.8.alpha4 with tickets #9958 + #11986 + #12124 + #11976 + #11915 all tests pass on x86 and on amd64 there is one TIME OUT
sage -t -long -force_lib devel/sage-matplotlib-1.1.0/sage/interfaces/ecm.py
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***
[1800.2 s]
The test did pass when run individually. I'm sure this is due to running the tests in parallel. I've seen it before, not with this ticket, but with sage-on-gentoo builds.
comment:263 in reply to: ↑ 262 ; follow-up: ↓ 264 Changed 9 years ago by
Replying to strogdon:
For sage-4.8.alpha4 with tickets #9958 + #11986 + #12124 + #11976 + #11915 all tests pass on x86 and on amd64 there is one TIME OUT
sage -t -long -force_lib devel/sage-matplotlib-1.1.0/sage/interfaces/ecm.py *** *** Error: TIMED OUT! PROCESS KILLED! *** *** [1800.2 s]The test did pass when run individually. I'm sure this is due to running the tests in parallel. I've seen it before, not with this ticket, but with sage-on-gentoo builds.
I have seen it before too - in parallel tests as well. It seems that ecm can be a bad boy, not sure what to do about it but that would be outside of the scope of this ticket in my opinion.
Steve, did you do any test of #11986 with python-2.6?
comment:264 in reply to: ↑ 263 Changed 9 years ago by
comment:265 follow-up: ↓ 266 Changed 9 years ago by
Upgrading from sage-4.5 fails with:
Testing that Sage starts...
[2012-01-03 13:47:34]
Traceback (most recent call last):
File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/bin/sage-eval", line 4, in <module>
from sage.all import *
File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/all.py", line 75, in <module>
from sage.matrix.all import *
File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/all.py", line 1, in <module>
from matrix_space import MatrixSpace, is_MatrixSpace
File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 37, in <module>
import matrix_modn_dense
File "matrix_modn_dense.pyx", line 1, in init sage.matrix.matrix_modn_dense (sage/matrix/matrix_modn_dense.c:15026)
AttributeError: 'module' object has no attribute 'Matrix_dense'
Sage failed to start up.
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and send the log file
/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/start.log
Describe your computer, operating system, etc.
After sage -ba-force, it works again.
comment:266 in reply to: ↑ 265 ; follow-up: ↓ 267 Changed 9 years ago by
Replying to jdemeyer:
Upgrading from sage-4.5 fails with:
Testing that Sage starts... [2012-01-03 13:47:34] Traceback (most recent call last): File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/bin/sage-eval", line 4, in <module> from sage.all import * File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/all.py", line 75, in <module> from sage.matrix.all import * File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/all.py", line 1, in <module> from matrix_space import MatrixSpace, is_MatrixSpace File "/mnt/usb1/scratch/jdemeyer/sage-4.5-python27/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 37, in <module> import matrix_modn_dense File "matrix_modn_dense.pyx", line 1, in init sage.matrix.matrix_modn_dense (sage/matrix/matrix_modn_dense.c:15026) AttributeError: 'module' object has no attribute 'Matrix_dense' Sage failed to start up. Please email sage-devel (http://groups.google.com/group/sage-devel) explaining the problem and send the log file /mnt/usb1/scratch/jdemeyer/sage-4.5-python27/start.log Describe your computer, operating system, etc.After
sage -ba-force, it works again.
Quite weird. You would think those particular libraries would have been rebuilt already after the linbox upgrade at least. Shouldn't a complete rebuild of sage occur after a python upgrade? In any case that seems mild for upgrading from 4.5 which was released on June 2010 many versions away.
comment:267 in reply to: ↑ 266 Changed 9 years ago by
- Status changed from needs_review to needs_work
- Work issues set to Fix upgrading
Replying to fbissey:
Shouldn't a complete rebuild of sage occur after a python upgrade?
I guess it should. Do you know how to make this happen?
comment:268 Changed 9 years ago by
- Description modified (diff)
comment:269 Changed 9 years ago by
- Description modified (diff)
comment:270 Changed 9 years ago by
- Dependencies changed from #5852, #11986, #12085, #12096, #12124 to #5852, #11986, #12085, #12096, #12124, #11915
- Description modified (diff)
- Status changed from needs_work to needs_review
- Work issues Fix upgrading deleted
Upgrading fixed by deleting
$SAGE_ROOT/devel/sage-*/build
when an old version of Python is detected.
New spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/python-2.7.2.p1.spkg
comment:271 Changed 9 years ago by
positive_review to the spkg, modulo my changes in the 2.7.2.p1 package.
comment:272 follow-up: ↓ 274 Changed 9 years ago by
Looks good to me. I didn't notice I missed a patch in the stuff I removed. Anyway I'll be on low activity because my home computer has given up the ghost (writing from an old box cobbled together running puppy 4.00 from a _1_GB hard drive) and I am on leave.
Changed 9 years ago by
Changed 9 years ago by
Changed 9 years ago by
comment:273 Changed 9 years ago by
- Description modified (diff)
Folded all the patches into only four patches.
comment:274 in reply to: ↑ 272 ; follow-up: ↓ 275 Changed 9 years ago by
comment:275 in reply to: ↑ 274 Changed 9 years ago by
- Reviewers changed from John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé to John Palmieri, Karl-Dieter Crisman, Jason Grout, Jeroen Demeyer, Steven Trogdon, Leif Leonhardy, Sébastien Labbé, François Bissey
- Status changed from needs_review to positive_review
comment:276 follow-up: ↓ 277 Changed 9 years ago by
9958_combinat.patch is the only non-trivial change I guess, but has been discussed well in the comments on this ticket. Potentially, the trie_type_dict() test might yield other results, but we can still change the test in that case. Anyway, that method is nowhere called in the Sage library.
comment:277 in reply to: ↑ 276 ; follow-up: ↓ 279 Changed 9 years ago by
Replying to jdemeyer:
9958_combinat.patch is the only non-trivial change I guess, but has been discussed well in the comments on this ticket. Potentially, the
trie_type_dict()test might yield other results, but we can still change the test in that case. Anyway, that method is nowhere called in the Sage library.
I didn't notice you had listed this particular patch. Why not use trac_9958-suffix_trees-variations-sl.patch from Sébastien Labbé instead? It is more foolproof.
comment:278 Changed 9 years ago by
I obviously meant to use Sébastien Labbé's patch for suffix_tree, we can still use the other patch for nfactor_enumerable_word.py, Sébastien think that patch is perfect which I count as a positive review of it.
comment:279 in reply to: ↑ 277 ; follow-up: ↓ 280 Changed 9 years ago by
Replying to fbissey:
I didn't notice you had listed this particular patch. Why not use trac_9958-suffix_trees-variations-sl.patch from Sébastien Labbé instead? It is more foolproof.
I overlooked that patch, it was not in the list of patches to be applied. I agree it is better.
Changed 9 years ago by
comment:280 in reply to: ↑ 279 ; follow-up: ↓ 281 Changed 9 years ago by
Replying to jdemeyer:
Replying to fbissey:
I didn't notice you had listed this particular patch. Why not use trac_9958-suffix_trees-variations-sl.patch from Sébastien Labbé instead? It is more foolproof.
I overlooked that patch, it was not in the list of patches to be applied. I agree it is better.
It was http://trac.sagemath.org/sage_trac/ticket/9958?action=diff&version=273 but never mind we have it now.
comment:281 in reply to: ↑ 280 Changed 9 years ago by
Replying to fbissey:
It was http://trac.sagemath.org/sage_trac/ticket/9958?action=diff&version=273 but never mind we have it now.
Look more carefully, it wasn't :-)
comment:282 Changed 9 years ago by
You are right I put the wrong patch in the list actually I never updated the list to include the right one, complete oversight. It's good that we are getting this merged now before more confusion arise from the number of patches attached.
comment:283 Changed 9 years ago by
François and others: thank you for all of the work you did on this ticket.
comment:284 Changed 9 years ago by
Hi,
I am removing my name from the authors list. For three lines of code I wrote, I don't feel as an author at all for such an important ticket. Being a reviewer of the small combinat part is enough and more appropriate.
Sébastien
comment:285 Changed 9 years ago by
- Merged in set to sage-5.0.beta0
- Resolution set to fixed
- Status changed from positive_review to closed
comment:286 follow-up: ↓ 288 Changed 9 years ago by
Just as FYI, though I doubt it matters since the patch still applies, I get
patching file Lib/distutils/command/sdist.py Hunk #1 succeeded at 327 (offset 3 lines). Hunk #2 succeeded at 342 (offset 3 lines).
before the other patches and configuration.
comment:287 Changed 9 years ago by
comment:288 in reply to: ↑ 286 Changed 9 years ago by
Replying to kcrisman:
Just as FYI, though I doubt it matters since the patch still applies, I get
patching file Lib/distutils/command/sdist.py Hunk #1 succeeded at 327 (offset 3 lines). Hunk #2 succeeded at 342 (offset 3 lines).
Not a big deal. The patch still applies perfectly.
First attempt is at: http://sage.math.washington.edu/home/palmieri/SPKG/python-2.7.p0.spkg
John (jhpalmieri) reports that this builds but Sage does not on sage.math, and "packages for twisted, zodb, pygments, and numpy don't build correctly."
Apparently the fix for http://bugs.python.org/issue7491 causes some of these problems.
On the numpy/scipy lists Ralf Gommers says "Numpy 1.5 should work with Python 2.7 and 3.1 and not be too far off. In August hopefully".
So it looks like http://trac.sagemath.org/sage_trac/ticket/9808 should fix the numpy issues.