#27754 closed enhancement (fixed)
Upgrade: Python 3.8.5
Reported by:  slelievre  Owned by:  

Priority:  blocker  Milestone:  sage9.2 
Component:  packages: standard  Keywords:  days101 
Cc:  chapoton, embray, fbissey, jdemeyer, mkoeppe, slelievre, ghtimokau, isuruf, ghkliem, tscrim, heluani, ghsheerluck  Merged in:  
Authors:  Jeroen Demeyer, John Palmieri, Matthias Koeppe, Antonio Rojas  Reviewers:  Matthias Koeppe, François Bissey 
Report Upstream:  Reported upstream. Developers acknowledge bug.  Work issues:  
Branch:  f1cda12 (Commits)  Commit:  
Dependencies:  #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140, #30351  Stopgaps: 
Description (last modified by )
This is to upgrade to Python 3.8.x.
Tarball: see checksums.ini
Previous and next Python upgrades:
See also:
 #30384 Adopt the “time windowbased” policy for support of Python versions from NEP 29
Attachments (1)
Change History (222)
comment:1 Changed 18 months ago by
comment:2 Changed 18 months ago by
I think it's a bit early for testing. I'd rather wait for the first beta, which is supposed to be released later this month.
comment:3 Changed 17 months ago by
Python 3.8.0b1 is out:
comment:4 Changed 17 months ago by
Note: this requires a Cython upgrade.
comment:5 Changed 17 months ago by
Cython upgrade needs review at #27886.
comment:6 Changed 17 months ago by
 Dependencies set to #28012
 Keywords days101 added
comment:7 Changed 17 months ago by
I'll have a look at this today.
comment:8 Changed 17 months ago by
 Dependencies changed from #28012 to #27886
comment:9 Changed 17 months ago by
 Description modified (diff)
 Milestone changed from sagewishlist to sage8.9
 Type changed from task to enhancement
comment:10 Changed 17 months ago by
 Branch set to u/jdemeyer/upgrade__python_3_8_x
comment:11 Changed 17 months ago by
 Commit set to 46837b74871a41b8f79af5c9d477428b183dc2d7
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
46837b7  Python 3.8.0

comment:12 Changed 17 months ago by
 Dependencies #27886 deleted
 Description modified (diff)
Actually, thanks to two patches which are likely going to be accepted upstream, it no longer depends on the Cython upgrade.
comment:13 Changed 17 months ago by
 Report Upstream changed from N/A to Reported upstream. Developers acknowledge bug.
comment:14 Changed 17 months ago by
There is a mysterious build failure of matplotlib:
Processing /home/jdemeyer/sagepython3/local/var/tmp/sage/build/matplotlib2.2.3.p0/src Created temporary directory: /tmp/pipreqbuildvq4okaxf Added file:///home/jdemeyer/sagepython3/local/var/tmp/sage/build/matplotlib2.2.3.p0/src to build tracker '/tmp/pipreqtrackerxcv165b_' Running setup.py (path:/tmp/pipreqbuildvq4okaxf/setup.py) egg_info for package from file:///home/jdemeyer/sagepython3/local/var/tmp/sage/build/matplotlib2.2.3.p0/src Running command python setup.py egg_info Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pipreqbuildvq4okaxf/setup.py", line 172, in <module> result = package.check() File "/tmp/pipreqbuildvq4okaxf/setupext.py", line 963, in check import numpy File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/__init__.py", line 142, in <module> from . import core File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/core/__init__.py", line 16, in <module> from . import multiarray File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/core/multiarray.py", line 12, in <module> from . import overrides File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/core/overrides.py", line 8, in <module> from numpy.compat._inspect import getargspec File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/compat/__init__.py", line 14, in <module> from . import py3k File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/compat/py3k.py", line 15, in <module> from pathlib import Path, PurePath File "/home/jdemeyer/sagepython3/local/lib/python3.8/pathlib.py", line 391, in <module> class _NormalAccessor(_Accessor): File "/home/jdemeyer/sagepython3/local/lib/python3.8/pathlib.py", line 415, in _NormalAccessor link_to = os.link AttributeError: module 'os' has no attribute 'link'
But when running Python normally, os.link
does exist:
jdemeyer@sage4:~/sagepython3$ ./sage python3 Python 3.8.0b1 (default, Jun 19 2019, 16:54:26) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.link <builtin function link>
I have no clue how this could happen.
comment:15 followups: ↓ 29 ↓ 64 Changed 17 months ago by
I tried this with OS X and ran into a problem with Pillow:
File "setup.py", line 316, in build_extensions _add_directory(library_dirs, sysroot+"/usr/lib") TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")
Python 3.7.3 does this:
Python 3.7.3 (default, Jun 17 2019, 15:38:22) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' >>>
The new Python does this:
Python 3.8.0b1 (default, Jun 19 2019, 11:09:54) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') >>>
comment:16 followup: ↓ 17 Changed 17 months ago by
I see the same problem with matplotlib. For laughs, I put a print(dir(os))
command right before the line link_to = os.link
. The output is identical to what I get when I do import os; dir(os)
from sage python3
, except it is missing link
.
If I run make k
, I also see problems building the Sage library, by the way: lots of errors of the form
RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable.
comment:17 in reply to: ↑ 16 ; followup: ↓ 18 Changed 17 months ago by
Replying to jhpalmieri:
I see the same problem with matplotlib. For laughs, I put a
print(dir(os))
command right before the linelink_to = os.link
. The output is identical to what I get when I doimport os; dir(os)
fromsage python3
, except it is missinglink
.
That makes no sense. Somebody is fooling with us...
Could you try also print(os)
? Maybe there is a different os
module shadowing the standard one or something...
Just in case, I checked that the problem unfortunately persists after make distclean
.
And a Google search for this error doesn't find anything.
comment:18 in reply to: ↑ 17 Changed 17 months ago by
I'm guessing the problem is coming from matplotlib
's setup.py:
import os try: del os.link except AttributeError: pass
comment:19 followup: ↓ 20 Changed 17 months ago by
My question is why it worked with Python 3.7.3 but not 3.8.0.
comment:20 in reply to: ↑ 19 Changed 17 months ago by
Replying to jhpalmieri:
My question is why it worked with Python 3.7.3 but not 3.8.0.
A different implementation of pathlib
: https://bugs.python.org/issue26978
comment:21 Changed 17 months ago by
 Description modified (diff)
I reported this to matplotlib: https://github.com/matplotlib/matplotlib/issues/14580
comment:22 Changed 17 months ago by
 Dependencies set to #28023
 Description modified (diff)
comment:23 Changed 17 months ago by
 Commit changed from 46837b74871a41b8f79af5c9d477428b183dc2d7 to cc78db22810ca605ed28235b45eeafe72418b269
comment:24 Changed 17 months ago by
With matplotlib fixes, I'm getting these errors at runtime:
Exception ignored in: <sage.misc.weak_dict.WeakValueDictEraser object at 0x7f1bf60b5460> Traceback (most recent call last): File "sage/misc/weak_dict.pyx", line 219, in sage.misc.weak_dict.WeakValueDictEraser.__call__ (build/cythonized/sage/misc/weak_dict.c:1968) File "sage/cpython/dict_del_by_value.pyx", line 375, in sage.cpython.dict_del_by_value.del_dictitem_by_exact_value (build/cythonized/sage/cpython/dict_del_by_value.c:2311) File "sage/cpython/dict_del_by_value.pyx", line 294, in sage.cpython.dict_del_by_value.ensure_allows_deletions (build/cythonized/sage/cpython/dict_del_by_value.c:2025) AssertionError:
comment:25 Changed 17 months ago by
 Dependencies changed from #28023 to #28023, #28025
comment:26 Changed 17 months ago by
Also getargspec is completely broken, causing docbuild errors.
comment:27 followup: ↓ 28 Changed 17 months ago by
see #27971 for getargspec ?
comment:28 in reply to: ↑ 27 Changed 17 months ago by
comment:29 in reply to: ↑ 15 ; followup: ↓ 33 Changed 17 months ago by
Replying to jhpalmieri:
I tried this with OS X and ran into a problem with Pillow: Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")
This is actually added in a Sage patch. It's failing because of the removal of macos_no_include.patch
, which was added in #27631. I'll have to see if that patch is necessary anymore. If not, we need to unpatch Pillow.
comment:30 Changed 17 months ago by
I still gets lots of errors as in comment:16. The build process for sagelib
seems to go into an infinite loop. I just hit ctrlC, and there are about 2400 instances of
************************************************************************ Error building the Sage library ************************************************************************ Please email sagedevel (http://groups.google.com/group/sagedevel) explaining the problem and including the relevant part of the log file /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/logs/pkgs/sagelib8.8.rc2.log Describe your computer, operating system, etc.
in the log file, all connected with the same error message.
comment:31 followup: ↓ 32 Changed 17 months ago by
On which system are you seeing that? The build of the Sage library worked for me on Linux.
comment:32 in reply to: ↑ 31 Changed 17 months ago by
Replying to jdemeyer:
On which system are you seeing that? The build of the Sage library worked for me on Linux.
OS X, both 10.13 and 10.14.
comment:33 in reply to: ↑ 29 Changed 17 months ago by
Replying to jhpalmieri:
Replying to jhpalmieri:
I tried this with OS X and ran into a problem with Pillow: Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")This is actually added in a Sage patch. It's failing because of the removal of
macos_no_include.patch
, which was added in #27631. I'll have to see if that patch is necessary anymore. If not, we need to unpatch Pillow.
On an OS X 10.14 machine, one affected by #27631 (no /usr/include
), pip
builds just fine, so I think Python and pip are finding zlib, but Pillow does not.
If I keep the Pillow patch, it fails to build at all, with the error mentioned above. If I delete the Pillow patch and build, I see in its log file "The headers and library files could not be found for zlib".
Changed 17 months ago by
comment:34 Changed 17 months ago by
Here is the start of the log file. I've deleted more than 2000 repeated tracebacks.
comment:35 Changed 16 months ago by
I think the problem is this (from the 3.8 changelog):
on macOS, the spawn start method is now used by default in multiprocessing
The documentation for multiprocessing
then points to a bug report which discusses our old friend OBJC_DISABLE_INITIALIZE_FORK_SAFETY
(#25921). So if we get multiprocessing to work properly on OS X, maybe we can remove the line
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
from sageenv.
In the meantime, I don't know how to get it working properly; using fork
instead of spawn
in src/setup.py
is the only way I've found to proceed so far. I don't understand the distinctions in the multiprocessing documentation between fork
and spawn
to know what should be set differently to use spawn
correctly.
comment:36 Changed 16 months ago by
Using fork
instead of spawn
lets me build the Sage library:

src/setup.py
diff git a/src/setup.py b/src/setup.py index 42ba3a63da..5e2d65f9cc 100755
a b def execute_list_of_commands_in_parallel(command_list, nthreads): 493 493 progress = progress_fmt.format(i+1, N) 494 494 command_list[i] = command_list[i] + (progress,) 495 495 496 from multiprocessing import Pool 496 import multiprocessing as mp 497 mp.set_start_method('fork') 497 498 # map_async handles KeyboardInterrupt correctly if an argument is 498 499 # given to get(). Plain map() and apply_async() do not work 499 500 # correctly, see Trac #16113. 500 pool = Pool(nthreads)501 pool = mp.Pool(nthreads) 501 502 result = pool.map_async(apply_func_progress, command_list, 1).get(99999) 502 503 pool.close() 503 504 pool.join()
I made a similar change to allow docbuilding to work. Now I get failures with docbuilding. First, lots of warnings:
..../local/lib/python3.8/sitepackages/matplotlib/backends/backend_agg.py:531: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
Then an error:
Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__main__.py", line 2, in <module> main() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 1731, in main builder() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 352, in _wrapper getattr(get_builder(document), 'inventory')(*args, **kwds) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 548, in _wrapper build_many(build_ref_doc, L) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 289, in build_many ret = x.get(99999) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/multiprocessing/pool.py", line 768, in get raise self._value OSError: WARNING: error while formatting arguments for sage.calculus.riemann.Riemann_Map.get_szego: bad operand type for unary : 'NoneType'
comment:37 Changed 16 months ago by
And of course I can't run any doctests because they use multiprocessing
, and I haven't figured out how to tell doctesting (forker.py in particular) to use fork
instead of spawn
. Sage starts and some basic things work. The Jupyter notebook is giving me problems.
comment:38 Changed 16 months ago by
I also didn't manage to build the docs because the same error. I know it's a problem with sage_getargspec
but I haven't checked further.
comment:39 Changed 16 months ago by
I don't have a good solution for the problem with Pillow (comment:33). I have been deleting the patch and force installing zlib
, which works for now.
comment:40 Changed 16 months ago by
For the record: for the spawn
vs. fork
issue with OS X, I have had to make changes in three files: src/setup.py
, src/sage_setup/docbuild/__init__.py
, and Cython/Build/Dependencies.py
. The changes are all pretty much the same; for example in docbuild/__init__.py
:

src/sage_setup/docbuild/__init__.py
diff git a/src/sage_setup/docbuild/__init__.py b/src/sage_setup/docbuild/__init__.py index 0f2700168a..6d1ff4e377 100644
a b if not (CYGWIN_VERSION and CYGWIN_VERSION[0] < 3): 279 279 # serialize this by taking the pool (and thus the call to fork()) out 280 280 # completely: The call to Sphinx leaks memory, so we need to build each 281 281 # document in its own process to control the RAM usage. 282 from multiprocessing import Pool 283 pool = Pool(NUM_THREADS, maxtasksperchild=1) 282 import multiprocessing 283 ctx = multiprocessing.get_context('fork') 284 pool = ctx.Pool(NUM_THREADS, maxtasksperchild=1) 284 285 # map_async handles KeyboardInterrupt correctly. Plain map and 285 286 # apply_async does not, so don't use it. 286 287 x = pool.map_async(target, args, 1)
comment:41 followup: ↓ 42 Changed 16 months ago by
I am also seeing the error "ImportError?: cannot import name 'clock' from 'time' (unknown location)". I don't think this is coming from the Sage library, but rather from other packages, and in particular brial and sympy may need to be changed.
$ grep R "from time import clock" local/lib/python3.8/ local/lib/python3.8//sitepackages/sympy/plotting/pygletplot/plot_window.py:from time import clock local/lib/python3.8//sitepackages/brial/interpolate.py:from time import clock local/lib/python3.8//sitepackages/future/backports/test/pystone.py:from time import clock local/lib/python3.8//sitepackages/zmq/backend/cffi/_poll.py: from time import clock as monotonic local/lib/python3.8//sitepackages/sage/coding/information_set_decoder.py: from time import clock as process_time
Changing the import to something like
try: from time import perf_counter as clock # or process_time instead of perf_counter? except ImportError: from time import clock
might work.
comment:42 in reply to: ↑ 41 Changed 16 months ago by
Replying to jhpalmieri:
I am also seeing the error "ImportError?: cannot import name 'clock' from 'time' (unknown location)". I don't think this is coming from the Sage library, but rather from other packages, and in particular brial and sympy may need to be changed.
$ grep R "from time import clock" local/lib/python3.8/ local/lib/python3.8//sitepackages/sympy/plotting/pygletplot/plot_window.py:from time import clock local/lib/python3.8//sitepackages/brial/interpolate.py:from time import clock local/lib/python3.8//sitepackages/future/backports/test/pystone.py:from time import clock local/lib/python3.8//sitepackages/zmq/backend/cffi/_poll.py: from time import clock as monotonic local/lib/python3.8//sitepackages/sage/coding/information_set_decoder.py: from time import clock as process_timeChanging the import to something like
try: from time import perf_counter as clock # or process_time instead of perf_counter? except ImportError: from time import clockmight work.
 The sympy issue has been fixed 16 days ago.
 I have just made a pull request for brial
 The zmq issue is already fixed in https://github.com/zeromq/pyzmq/blob/master/zmq/backend/cffi/_poll.py
 The last line (our own information_set_decoder.py) is already correctly using a try/except for this import.
comment:43 Changed 16 months ago by
 Dependencies changed from #28023, #28025 to #28023, #28025, #28116
 Description modified (diff)
comment:44 Changed 14 months ago by
 Dependencies changed from #28023, #28025, #28116 to #28025
 Description modified (diff)
comment:45 Changed 14 months ago by
Python 3.8 is getting closer, it would be great if somebody could review the dependency #28025.
comment:46 Changed 14 months ago by
 Dependencies changed from #28025 to #28025, #28428
comment:47 Changed 13 months ago by
 Description modified (diff)
Python 3.8.0 is out. Added tarball link to ticket description.
comment:48 Changed 12 months ago by
 Branch changed from u/jdemeyer/upgrade__python_3_8_x to public/python3.8
 Commit changed from cc78db22810ca605ed28235b45eeafe72418b269 to 193e9f49517a95e483af785d9879948468e9ed98
 Dependencies #28025, #28428 deleted
 Milestone changed from sage8.9 to sage9.0
New commits:
193e9f4  Python 3.8.0

comment:49 Changed 12 months ago by
apparently some patches do no longer apply..
comment:50 Changed 12 months ago by
It looks to me as though the two new patches were actually merged into the release, and so they are no longer needed. I'm not positive, though.
comment:51 Changed 12 months ago by
On OS X, the Pillow problem is still there. I tried upgrading to the most recent version of Pillow, but it did not help. I'm going to force installation of zlib
and proceed from there.
comment:52 Changed 12 months ago by
There is still a problem with a from time import clock
statement. I don't know which one is causing the problem, but I'm guessing either brial or sympy. Neither package has fixed it in a release, as far as I can tell, but I think it's been fixed upstream in both.
comment:53 Changed 12 months ago by
I am also, not surprisingly, having the same multiprocessing
problems on OS X as before. I can get the Sage library and the documentation to build, but I don't know how to get doctesting to work.
comment:54 Changed 12 months ago by
FWIW after updating to python 3.8 on Arch I'm getting test failures on 223 modules, so things don't look good. Most of them seem to stem from the same 34 issues though.
comment:55 followups: ↓ 72 ↓ 77 Changed 11 months ago by
After taking a closer look, besides some string changes, most of the failures fall into one of the following categories:
(1) Hash changes  all hashes have changed in 3.8. This requires updating many tests, making them conditional on the python version. Not sure if it makes sense to fork so many tests at this point rather than just drop python2 support or at least not run those tests.
(2) Sorting changes  many lists are now output in a different order. Same comment as in (1) about the solution.
(3) "DeprecationWarning?: In future, it will be an error for 'np.bool_' scalars to be interpreted as an index" from numpy. These are fixed with https://github.com/matplotlib/matplotlib/commit/4651a125
(4) "DeprecationWarning?: PY_SSIZE_T_CLEAN will be required for '#' formats"  this happens mostly when pickling/unpickling functions. No idea where the actual code that throws the warning is.
(5) "DeprecationWarning?: an integer is required (got type sage.rings.real_mpfr.RealLiteral?). Implicit conversion to integers using int is deprecated, and may be removed in a future version of Python." when calling time.sleep with a Sage (sage.rings.real_mpfr.RealLiteral?) float. Coercing to float first fixes the issue.
(6) "SyntaxWarning?: "is" with a literal. Did you mean "=="?" when comparing strings with 'is'
(7) Use of the removed cgi.escape in sage/interacts/debugger.py. Should be changed to html.escape.
(8) The signature of code() (used in sage/misc/fpickle.pyx) has changed and now requires a 'posonlyargcount' argument.
(9) Mismatch after pickling/unpickling for some objects:
sage: A = Poset()._hasse_diagram.antichains() sage: A == loads(dumps(A)) False
For packagers that don't care about python2 compatibility (or 32 bit hashes) a WIP patch that fixes or works around most of these issues is at https://aur.archlinux.org/cgit/aur.git/tree/sagemathpython3.8.patch?h=sagemathgit. A backport of https://github.com/BRiAl/BRiAl/commit/74d86170 is also needed.
comment:56 Changed 11 months ago by
 Cc ghtimokau added
comment:57 Changed 11 months ago by
Python 3.8.1rc1 was released. See pythonannounce post:
comment:58 followup: ↓ 63 Changed 11 months ago by
It would be great if someone with access to OS X could look at this. I don't know how to solve all of the problems mentioned above. In particular:
 multiprocessing on OS X has a different default behavior than on linux (comment:35). As a result, the Sage library doesn't build and the docs don't build without some modification, and I don't know how to get doctesting to work.
 Pillow does not find the system
zlib
. Is there a better solution than forceinstalling Sage'szlib
?
comment:59 Changed 10 months ago by
I have access to macOS but I would need to be told what to try.
comment:60 followup: ↓ 61 Changed 10 months ago by
Reports of issues with SageMath and Python 3.8:
 Sage error from module
typing
and python 3.8 https://ask.sagemath.org/question/49326
 Sage crash report: Manjaro, Python 3.8.1 https://trac.sagemath.org/ticket/28946
comment:61 in reply to: ↑ 60 Changed 10 months ago by
Replying to slelievre:
Reports of issues with SageMath and Python 3.8:
 Sage error from module
typing
and python 3.8 https://ask.sagemath.org/question/49326
 Sage crash report: Manjaro, Python 3.8.1 https://trac.sagemath.org/ticket/28946
None of these is a Sage issue: the first one is caused by the user mixing up distro and pip packages, the second one is a ntl soversion mismatch in Manjaro. The errors that still need to be figured out (besides the macOS problems) are (4), (5) and (9) in comment:55 above.
comment:62 Changed 10 months ago by
 Milestone changed from sage9.0 to sage9.2
Sage 9.0 is out. I think this should wait until after 9.1 to give Sage with Python 3.7 a little more time to breathe, but of course this is debatable.
comment:63 in reply to: ↑ 58 Changed 10 months ago by
Replying to jhpalmieri:
 Pillow does not find the system
zlib
. Is there a better solution than forceinstalling Sage'szlib
?
I have created #29019 for the problems with Pillow because they also affect #27824.
comment:64 in reply to: ↑ 15 Changed 10 months ago by
Replying to jhpalmieri:
I tried this with OS X and ran into a problem with Pillow:
File "setup.py", line 316, in build_extensions _add_directory(library_dirs, sysroot+"/usr/lib") TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")Python 3.7.3 does this:
Python 3.7.3 (default, Jun 17 2019, 15:38:22) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' >>>The new Python does this:
Python 3.8.0b1 (default, Jun 19 2019, 11:09:54) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') >>>
As far as I can see, this config variable was invented in #27631 and is not available on any other Python than our patched Python.
comment:65 Changed 10 months ago by
indeed, Py_MACOS_SYSROOT
comes from #27631.
Note that #27631 followed a cpython patch we proposed on
https://bugs.python.org/issue36231
which was rejected in favour of
https://github.com/python/cpython/commit/c421c66a58a6caae30f0679d7e61411418e67cec
and so in 3.7.x with x>3 and in 3.y with y>7 the problem apparently is fixed, but I don't quite know how.
I don't have any access to a modern MacOS box with Xcode without sane /usr/incude
etc, so it's hard to experiment for me (I guess the patch for pillow in #27631 needs to be replaced with whatever hack they came up with for MacOS.
comment:66 followup: ↓ 67 Changed 10 months ago by
See comment:33 — I could not build Pillow (after removing Sage's patch) with Python 3.8. So I don't know if I believe that the problem has been fixed.
comment:67 in reply to: ↑ 66 ; followup: ↓ 69 Changed 10 months ago by
Replying to jhpalmieri:
See comment:33 — I could not build Pillow (after removing Sage's patch) with Python 3.8. So I don't know if I believe that the problem has been fixed.
Please try again with the latest commit in #29019.
comment:68 Changed 10 months ago by
 Commit changed from 193e9f49517a95e483af785d9879948468e9ed98 to 9cf6f0ebf83c57384161fdc1594bc8156e1c49c3
comment:69 in reply to: ↑ 67 Changed 10 months ago by
Replying to mkoeppe:
Replying to jhpalmieri:
See comment:33 — I could not build Pillow (after removing Sage's patch) with Python 3.8. So I don't know if I believe that the problem has been fixed.
Please try again with the latest commit in #29019.
That works for me. With this branch plus #29019, the build succeeds until I get to the Sage library, at which point the changes to multiprocessing start causing problems (as in comment:16, comment:35, comment:36).
comment:70 Changed 10 months ago by
See https://github.com/cython/cython/issues/3262 and https://github.com/condaforge/cythonfeedstock/pull/59 (?) for multiprocessing issues with Cython + Python 3.8 + OS X. One proposed solution there is to disable parallel processing on OS X (or more generally if multiprocessing.get_start_method() == 'spawn'
). Do we want to do that, or are we willing to use a patch like the proposal in comment:40, applied to Cython/Build/Dependencies.py
?
comment:71 Changed 10 months ago by
And by the way, even with the patches as in comment:40, I still can't get the documentation to build. It ends with
[dochtml] ForkingPickler(file, protocol).dump(obj) [dochtml] AttributeError: Can't pickle local object 'build_many.<locals>.run_worker'
comment:72 in reply to: ↑ 55 ; followup: ↓ 73 Changed 7 months ago by
Replying to arojas: [...]
For packagers that don't care about python2 compatibility (or 32 bit hashes) a WIP patch that fixes or works around most of these issues is at https://aur.archlinux.org/cgit/aur.git/tree/sagemathpython3.8.patch?h=sagemathgit. A backport of https://github.com/BRiAl/BRiAl/commit/74d86170 is also needed.
@arojas, I'm trying to update sage on nix to 9.0 with python 3.8. I'm using your 3.8 patch in the 9.0 version [1]. I'm still seeing some test failures caused by the python update though:
sage t long src/sage/categories/homset.py # 3 doctests failed sage t long src/sage/combinat/cluster_algebra_quiver/quiver_mutation_type.py # 1 doctest failed sage t long src/sage/combinat/finite_state_machine.py # 1 doctest failed sage t long src/sage/combinat/posets/hasse_diagram.py # 2 doctests failed sage t long src/sage/groups/cubic_braid.py # 1 doctest failed sage t long src/sage/homology/simplicial_complex_morphism.py # 1 doctest failed sage t long src/sage/structure/sequence.py # 2 doctests failed
Here[2] is the full log. You said that your patch fixes "most" failures, so is that to be expected? Do you have the same failures on arch?
[1] https://aur.archlinux.org/cgit/aur.git/plain/sagemathpython3.8.patch?h=sagemathgit&id=138b3db82d8d13f9c66bf8d7515aa37671710e0a [2] https://gist.github.com/timokau/13c97bf8f45162f3d4155999fc5bec13
comment:73 in reply to: ↑ 72 Changed 7 months ago by
Replying to ghtimokau:
Here[2] is the full log. You said that your patch fixes "most" failures, so is that to be expected? Do you have the same failures on arch?
Yes, I have the same failures (the one in cubic_braid no longer happens in the latest beta). They are mostly related to pickling, which I don't understand.
comment:74 Changed 7 months ago by
Thanks!
I had a look at the python 3.8 changelog, the most obvious change seemed to be the change of the default pickle protocol (3 > 4). But I have patched python to revert that change and that did not fix the issue.
I'll look into gitbisect'ing the cpython update, though that is of course not exactly trivial. Maybe it will help.
comment:75 Changed 7 months ago by
there is a change of API in the ast, so this breaks old versions of ipython with python 3.8. In particular, it breaks ipython 5.8 so you cannot run sage under jupyter in python 3.8 so far.
comment:76 Changed 7 months ago by
 Dependencies set to #28197
I've added as a dependency: #28197  upgrade to ipython 7
comment:77 in reply to: ↑ 55 Changed 7 months ago by
Replying to arojas:
For packagers that don't care about python2 compatibility (or 32 bit hashes) a WIP patch that fixes or works around most of these issues is at https://aur.archlinux.org/cgit/aur.git/tree/sagemathpython3.8.patch?h=sagemathgit.
Thanks for the patch. A version of this patch including the 32 bit hashes can now be found at https://salsa.debian.org/scienceteam/sagemath//blob/master/debian/patches/u0versionpython3.8.patch.
Unfortunately many of the ordering issues addressed in this patch also depend on 32/64 bit now. See 32bit build log at https://buildd.debian.org/status/fetch.php?pkg=sagemath&arch=i386&ver=9.03&stamp=1586083972&raw=0.
comment:78 Changed 5 months ago by
 Dependencies changed from #28197 to #28197, #29042
comment:79 Changed 5 months ago by
 Cc isuruf added
comment:80 Changed 4 months ago by
Can we get this ticket unstuck please?
comment:81 Changed 4 months ago by
I am seeing negative progress: I can't even get the Sage library to build on OS X anymore: I just see errors as in comment:16. I assume the problem is parallel building with spawn
instead of fork
, but I can't figure out where to change things to get it to work.
For the record, I tried this with Python 3.8.4, not 3.8.0. I'll try with 3.8.0, but I don't expect any difference. I tried this but it didn't help:

src/sage_setup/run_parallel.py
diff git a/src/sage_setup/run_parallel.py b/src/sage_setup/run_parallel.py index 7a9b787b4e..a4cf73801d 100644
a b def execute_list_of_commands_in_parallel(command_list, nthreads): 73 73 progress = progress_fmt.format(i+1, N) 74 74 command_list[i] = command_list[i] + (progress,) 75 75 76 from multiprocessing import Pool 76 import multiprocessing 77 ctx = multiprocessing.get_context('fork') 77 78 # map_async handles KeyboardInterrupt correctly if an argument is 78 79 # given to get(). Plain map() and apply_async() do not work 79 80 # correctly, see Trac #16113. 80 pool = Pool(nthreads)81 pool = ctx.Pool(nthreads, maxtasksperchild=1) 81 82 result = pool.map_async(apply_func_progress, command_list, 1).get(99999) 82 83 pool.close() 83 84 pool.join()
comment:82 Changed 4 months ago by
 Commit changed from 9cf6f0ebf83c57384161fdc1594bc8156e1c49c3 to 9688a9c6064603725c6aca4278f7098b758743c9
comment:83 Changed 4 months ago by
 Description modified (diff)
 Summary changed from Upgrade: Python 3.8.x to Upgrade: Python 3.8.4
comment:84 Changed 4 months ago by
I'll try to fix / work around the build problem
comment:86 Changed 4 months ago by
 Commit changed from 9688a9c6064603725c6aca4278f7098b758743c9 to c44c448acfb4152e981d91ec02b4eb6ae34c57a7
comment:87 Changed 4 months ago by
Not sure if the Cython update (from #29861) was necessary. But 0.29.21 does have a change specifically for 3.8.
comment:88 Changed 4 months ago by
sagelib builds now on my Mac. Haven't tested anything else yet.
comment:89 Changed 4 months ago by
Right, thanks, I forgot to deal with Cython. I can now build sagelib. Also the docs, with the following changes:

src/sage_setup/docbuild/utils.py
diff git a/src/sage_setup/docbuild/utils.py b/src/sage_setup/docbuild/utils.py index 272f7e2d0f..2691339ea6 100644
a b def build_many(target, args, processes=None): 112 112 ... 113 113 WorkerDiedException: worker for 4 died with nonzero exit code 9 114 114 """ 115 from multiprocessing import Process, Queue, cpu_count 115 import multiprocessing 116 ctx = multiprocessing.get_context('fork') 117 Process = ctx.Process 118 Queue = ctx.Queue 116 119 from six.moves.queue import Empty 117 120 118 121 if processes is None: 119 processes = cpu_count()122 processes = multiprocessing.cpu_count() 120 123 121 124 workers = [None] * processes 122 125 tasks = enumerate(args)
Doctesting doesn't work, though; first, it failed completely, and then I tried to patch a few files in src/sage/doctest
, but I was not successful. (I got some tests to run, but there were doctest failures, and then the process just quit.)
comment:90 Changed 4 months ago by
According to https://docs.python.org/3/library/multiprocessing.html#multiprocessing.get_context,
get_context
is available for python >= 3.4. We support python >= 3.6, so I would guess it is safe to make this change to utils.py
. Could you push it to the branch please?
comment:91 Changed 3 months ago by
 Commit changed from c44c448acfb4152e981d91ec02b4eb6ae34c57a7 to 6a8c543c70ee90b5ad332ace5aebf5e498fb1444
Branch pushed to git repo; I updated commit sha1. New commits:
6a8c543  trac 27754: docbuilding on OS X

comment:92 Changed 3 months ago by
This builds sagelib
plus the documentation for me on, either using the system Python or Python 3.8.4.
comment:93 Changed 3 months ago by
 Dependencies changed from #28197, #29042 to #28197, #29042, #29861
comment:94 Changed 3 months ago by
Thanks. Do we know if the problem with doctesting is specific to macOS?
comment:95 Changed 3 months ago by
With this branch, everything builds on an Ubuntu virtual machine. I didn't run doctests.
I also didn't know about the
multiprocessing.set_start_method('fork', force=True)
command. With this change, doctests on OS X run, although there are a lot of failures:

src/sage/doctest/external.py
diff git a/src/sage/doctest/external.py b/src/sage/doctest/external.py index b0db07b47a..af49ca29e5 100644
a b AUTHORS: 25 25 # http://www.gnu.org/licenses/ 26 26 #***************************************************************************** 27 27 28 from multiprocessing import Array 28 import multiprocessing 29 import os 30 if os.uname().sysname == 'Darwin': 31 multiprocessing.set_start_method('fork', force=True) 32 Array = multiprocessing.Array 29 33 30 34 import urllib.error 31 35 from urllib.request import Request, urlopen 
src/sage/doctest/forker.py
diff git a/src/sage/doctest/forker.py b/src/sage/doctest/forker.py index 75559667d1..8f5f2bff2b 100644
a b from sage.repl.user_globals import set_globals 63 63 from sage.cpython.atexit import restore_atexit 64 64 from sage.cpython.string import bytes_to_str, str_to_bytes 65 65 66 if os.uname().sysname == 'Darwin': 67 multiprocessing.set_start_method('fork', force=True) 66 68 67 69 # All doctests run as if the following future imports are present 68 70 import __future__
comment:96 Changed 3 months ago by
Could you push it on the branch?
comment:97 Changed 3 months ago by
Without the changes from comment 95, I get the following error:
$ ./sage t src/sage/geometry/polyhedron/base.py too many failed tests, not using stored timings Running doctests with ID 202007151942599c236f91. Git branch: bigupdate Using optional=build,cryptominisat,dochtml,sage Doctesting 1 file.  Doctests interrupted: 0/1 files tested  Total time for all tests: 0.0 seconds cpu time: 0.0 seconds cumulative wall time: 0.0 seconds Traceback (most recent call last): File "/Users/mkoeppe/s/sage/sagerebasing/src/bin/sageruntests", line 177, in <module> err = DC.run() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 1207, in run self.run_doctests() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 908, in run_doctests self.dispatcher.dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2039, in dispatch self.parallel_dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 1934, in parallel_dispatch w.start() # This might take some time File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2206, in start super(DocTestWorker, self).start() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'DocTestDispatcher.parallel_dispatch.<locals>.sel_exit'
Is this the same error that you were getting?
comment:98 Changed 3 months ago by
 Commit changed from 6a8c543c70ee90b5ad332ace5aebf5e498fb1444 to 5a5fa82344129e34f03b62ec2c3bb0a72de0d808
Branch pushed to git repo; I updated commit sha1. New commits:
5a5fa82  trac 27754: on OS X, use 'fork' with multiprocessing when doctesting

comment:99 Changed 3 months ago by
Yes, that's the error I was getting. Now I'm getting the same doctest failures on OS X as on the Ubuntu virtual machine (approximately: I think I get 113 files with failures on OS X, 114 with failures on Ubuntu).
comment:100 Changed 3 months ago by
I tried make ptest
, and it ended with:
sage t src/sage/calculus/riemann.pyx # 18 doctests failed sage t src/sage/functions/log.py # 2 doctests failed sage t src/sage/graphs/graph_plot.py # 4 doctests failed Doctests interrupted: 344/4127 files tested File "/Users/mkoeppe/s/sage/sagerebasing/src/bin/sageruntests", line 177, in <module> err = DC.run() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 1207, in run self.run_doctests() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 908, in run_doctests self.dispatcher.dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2044, in dispatch self.parallel_dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 1939, in parallel_dispatch w.start() # This might take some time File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2211, in start super(DocTestWorker, self).start() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_fork.py", line 69, in _launch child_r, parent_w = os.pipe() OSError: [Errno 24] Too many open files make: *** [ptest] Error 1
comment:101 Changed 3 months ago by
make ptestlong
completed for me (with make j6
). I see several types of failures:
 sorting issues:
File "src/sage/dynamics/arithmetic_dynamics/projective_ds.py", line 3148, in sage.dynamics.arithmetic_dynamics.projective_ds.?.automorphism_group Failed example: D6.automorphism_group() Expected: [ [1 0] [0 w] [0 1] [w 0] [w  1 0] [ 0 w  1] [0 1], [1 0], [1 0], [0 1], [ 0 1], [ 1 0] ] Got: [ [1 0] [0 1] [ 0 w  1] [w 0] [w  1 0] [0 w] [0 1], [1 0], [ 1 0], [0 1], [ 0 1], [1 0] ]
 hashes have changed:
File "src/sage/combinat/similarity_class_type.py", line 411, in sage.combinat.similarity_class_type.?.__hash__ Failed example: hash(PT1) Expected: 5050909583595644741 Got: 925386691174542829
 pickling?
File "src/sage/manifolds/differentiable/affine_connection.py", line 1785, in sage.manifolds.differentiable.affine_connection.AffineConnection.riemann Failed example: for i in M.irange(): for j in M.irange(): for k in M.irange(): nab.add_coef(eV)[i,j,k] = nab.coef(eVW)[i,j,k,c_uvW].expr() Exception raised: multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "sage/misc/fpickle.pyx", line 99, in sage.misc.fpickle.call_pickled_function (build/cythonized/sage/misc/fpickle.c:2279) f = eval("unpickle_function(fp)", sage.all.__dict__, {'fp':fp}) File "<string>", line 1, in <module> File "sage/misc/fpickle.pyx", line 90, in sage.misc.fpickle.unpickle_function (build/cythonized/sage/misc/fpickle.c:1975) recovered = pickle.loads(pickled) File "sage/misc/fpickle.pyx", line 26, in sage.misc.fpickle.code_ctor (build/cythonized/sage/misc/fpickle.c:1536) return types.CodeType(*args) TypeError: code() takes at least 14 arguments (13 given) """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 709, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 1133, in compile_and_execute exec(compiled, globs) File "<doctest sage.manifolds.differentiable.affine_connection.AffineConnection.riemann[30]>", line 4, in <module> nab.add_coef(eV)[i,j,k] = nab.coef(eVW)[i,j,k,c_uvW].expr() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/manifolds/differentiable/affine_connection.py", line 663, in coef nab_evi = self(ev[i]) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/manifolds/differentiable/affine_connection.py", line 1386, in __call__ return self._derive_paral(tensor_r) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/manifolds/differentiable/affine_connection.py", line 1508, in _derive_paral for ii,val in make_CovDerivative(listParalInput): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/parallel/multiprocessing_sage.py", line 77, in parallel_iter for res in result: File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/multiprocessing/pool.py", line 868, in next raise value TypeError: code() takes at least 14 arguments (13 given)
 a deprecation warning:
File "src/sage/plot/plot3d/base.pyx", line 1560, in sage.plot.plot3d.base.Graphics3d.save_image Failed example: G.save_image(gif) Expected nothing Got: doctest:warning ... DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
 another one:
File "src/sage/doctest/forker.py", line 2351, in sage.doctest.forker.DocTestWorker.kill Failed example: time.sleep(0.2) # Worker doesn't die Expected nothing Got: doctest:warning ... DeprecationWarning: an integer is required (got type sage.rings.real_mpfr.RealLiteral). Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
I haven't gone through all of the failures yet, nor have I pored over the Python 3.8 changelog to try to figure any of these out. The first few types are easy to fix, I hope.
comment:102 Changed 3 months ago by
comment 77 above already mentions the hashing/ordering issues. I haven't looked at these patches
comment:103 Changed 3 months ago by
If doctests need patching because of sorting order changes, I hope we can do better than what I see in https://salsa.debian.org/scienceteam/sagemath//blob/master/debian/patches/u0versionpython3.8.patch
Many of such tests should in my opinion be fixed by using explicitly sorted
when comparing lists whose order has no mathematical meaning
comment:104 Changed 3 months ago by
In this one from the Debian patch, I'm wondering whether printing of Set
in doctests should be modified similar to what was done in #29042 for dict

sage/src/sage/schemes/elliptic_curves/ell_point.py
a b 822 822 sage: all(T.is_divisible_by(3) for T in tor) 823 823 True 824 824 sage: Set([T for T in tor if T.is_divisible_by(2)]) 825 {( 0 : 1 : 0), (1 : 0 : 1)}825 {(1 : 0 : 1), (0 : 1 : 0)} 826 826 sage: Set([2*T for T in tor]) 827 {( 0 : 1 : 0), (1 : 0 : 1)}827 {(1 : 0 : 1), (0 : 1 : 0)} 828 828
comment:105 Changed 3 months ago by
 Cc ghkliem added
comment:106 Changed 3 months ago by
For these ones from the Debian patch I have created #30162:
 a/sage/src/sage/symbolic/callable.py +++ b/sage/src/sage/symbolic/callable.py @@ 38,27 +38,27 @@ sage: f(1)=2 Traceback (most recent call last): ...  SyntaxError: can't assign to function call + SyntaxError: cannot assign to function call
comment:107 Changed 3 months ago by
 Cc tscrim added
For this one from the Debian patch, I also wonder whether some methods should be changed so that they return generators rather than lists  for better py3 style.
 a/sage/src/sage/categories/coxeter_groups.py +++ b/sage/src/sage/categories/coxeter_groups.py @@ 283,7 +283,7 @@ [[0, 1, 2, 1], [0, 2, 1, 2], [2, 0, 1, 2]] sage: W.braid_orbit([2,1,1,2,1])  [[2, 2, 1, 2, 2], [2, 1, 1, 2, 1], [1, 2, 1, 1, 2], [2, 1, 2, 1, 2]] + [[2, 1, 1, 2, 1], [1, 2, 1, 1, 2], [2, 2, 1, 2, 2], [2, 1, 2, 1, 2]] sage: W = ReflectionGroup(['A',3], index_set=["AA","BB",5]) # optional  gap3 sage: w = W.long_element() # optional  gap3
comment:108 followup: ↓ 109 Changed 3 months ago by
That is a big Sagewide issue/question: should things be iterators/generators or lists. Some places we do both (e.g., in graphs), in others we only have the lists, and even still some places only do iterators.
The simple fix here to make everything consistent is just to sort the output (as was done in the test before) as this should be considered more as a set than an enumerated list IIRC.
comment:109 in reply to: ↑ 108 Changed 3 months ago by
Replying to tscrim:
The simple fix here to make everything consistent is just to sort the output (as was done in the test before) as this should be considered more as a set than an enumerated list IIRC.
Sure, this is what we should do for this ticket.
comment:110 Changed 3 months ago by
 Dependencies changed from #28197, #29042, #29861 to #28197, #29042, #29861, #30162
comment:111 Changed 3 months ago by
comment:112 followup: ↓ 113 Changed 3 months ago by
I don't understand the point of checking specific hash values. For example, in src/sage/combinat/similarity_class_type.py
, the hashes depend on how Python determines hash(tuple([3,2,1]))
, for example. Since I think that is an implementation detail of Python, not Sage, doctesting for a specific value is the opposite of robust.
comment:113 in reply to: ↑ 112 Changed 3 months ago by
Replying to jhpalmieri:
I don't understand the point of checking specific hash values. For example, in
src/sage/combinat/similarity_class_type.py
, the hashes depend on how Python determineshash(tuple([3,2,1]))
, for example. Since I think that is an implementation detail of Python, not Sage, doctesting for a specific value is the opposite of robust.
My guess is that these tests are there to guard against thoughtless or accidental changes to hash functions.
I could imagine that we may want to change these tests to something like this:
sage: hash(x) in (5050909583595644741, 925386691174542829, 234567876545678) True
comment:114 Changed 3 months ago by
 Commit changed from 5a5fa82344129e34f03b62ec2c3bb0a72de0d808 to 3d6ae39a62049a0a02e71f56c9cdd970407147a4
Branch pushed to git repo; I updated commit sha1. New commits:
3d6ae39  trac 27754: fix some doctests for Python 3.8

comment:115 Changed 3 months ago by
Here are "fixes" for some doctests. Some are probably not ideal. This is also far from complete: I didn't have the energy to do more, and I don't know how to handle some of the failures. But maybe this is a start.
comment:116 Changed 3 months ago by
 Work issues set to merge #30162
comment:117 Changed 3 months ago by
For similarity_class_type.py
, back when this was written IIRC, there was less decided about how to test a __hash__
method and less concern about how changes in Python would affect this. So I think the changed tests here are good.
comment:118 Changed 3 months ago by
 Description modified (diff)
Python 3.8.5 was released as a security hotfix.
See #30184 for Python 3.9.x.
comment:119 followups: ↓ 120 ↓ 126 Changed 3 months ago by
I'm not going to be able to work on this for a few days. If someone else wants to work on doctests, please feel free. Do we have a Pillowupgrade ticket? I wonder if that would help with some of the tests. And #30176 (matplotlib upgrade) might help, too.
Should this be delayed to Sage 9.3, ideally early in the release cycle?
comment:120 in reply to: ↑ 119 Changed 3 months ago by
comment:121 Changed 3 months ago by
 Cc heluani added
comment:122 Changed 3 months ago by
 Commit changed from 3d6ae39a62049a0a02e71f56c9cdd970407147a4 to ccc461e59d246b87a99d8b3426ecd419bd8484ab
Branch pushed to git repo; I updated commit sha1. New commits:
8a43db6  build/pkgs/python3: Update to 3.8.5

60a8922  In doctests, accept both can't and cannot in error messages

71da37b  Fixup

2727aa8  src/sage/combinat/words/word.py: More wildcarding for respelled error messages

c43343c  Even more wildcarding for respelled error messages; remove some py2only tests

1a7bc97  Merge branch 'u/mkoeppe/make_doctests_accept_python_3_8_x_respellings_of_some_error_messages' of git://trac.sagemath.org/sage into t/27754/public/python3.8

ccc461e  Merge branch 'public/python3.8' of git://trac.sagemath.org/sage into t/27754/public/python3.8

comment:123 Changed 3 months ago by
 Summary changed from Upgrade: Python 3.8.4 to Upgrade: Python 3.8.5
 Work issues merge #30162 deleted
comment:124 Changed 3 months ago by
 Description modified (diff)
comment:125 Changed 3 months ago by
 Milestone changed from sage9.2 to sage9.3
comment:126 in reply to: ↑ 119 Changed 3 months ago by
Replying to jhpalmieri:
Should this be delayed to Sage 9.3, ideally early in the release cycle?
Yes, I think so. Let's do #30210 (Upgrade Python to 3.7.8) for 9.2
comment:127 Changed 3 months ago by
There are lots of failures caused by deprecation warnings on unpickling
DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
that come from pynac. These are fixed with https://github.com/pynac/pynac/pull/354, but pynac development seems stalled unfortunately
comment:128 followup: ↓ 130 Changed 3 months ago by
Could you please create a sage ticket for this pynac patch?
comment:129 Changed 3 months ago by
 Commit changed from ccc461e59d246b87a99d8b3426ecd419bd8484ab to b1fc3553e32333143adb763e74728b8d45d99f2e
Branch pushed to git repo; I updated commit sha1. New commits:
f243937  Merge remotetracking branch 'origin/develop' into t/27754/public/python3.8

73539f6  Make tests in categories/finite_enumerated_sets.py pass with Python 3.8

0c893db  Fix 'SyntaxWarning: "is" with a literal. Did you mean "=="?' with Python 3.8

b1fc355  Make tests in combinat/words/word_infinite_datatypes.py compatible with Python 3.8

comment:130 in reply to: ↑ 128 Changed 3 months ago by
 Dependencies changed from #28197, #29042, #29861, #30162 to #28197, #29042, #29861, #30162, #30225
comment:131 Changed 3 months ago by
 Commit changed from b1fc3553e32333143adb763e74728b8d45d99f2e to 6eb6a473d81c6bc3d04a3432f78ce3d8bfa192be
Branch pushed to git repo; I updated commit sha1. New commits:
6eb6a47  More sorted()

comment:132 Changed 3 months ago by
 Commit changed from 6eb6a473d81c6bc3d04a3432f78ce3d8bfa192be to 583042aaac9c36e11e76a4c9accf95188ec9d2d5
Branch pushed to git repo; I updated commit sha1. New commits:
583042a  Update compile flags for python 3.8

comment:133 Changed 3 months ago by
 Commit changed from 583042aaac9c36e11e76a4c9accf95188ec9d2d5 to d9ca5c5715da4b6c78adc66095eeaca7c4a26c96
comment:134 Changed 3 months ago by
comment:135 Changed 3 months ago by
 Commit changed from d9ca5c5715da4b6c78adc66095eeaca7c4a26c96 to 86d5e94dc2ce2f0b674b275e039cecaaa7d0e130
Branch pushed to git repo; I updated commit sha1. New commits:
86d5e94  Rewrite some hash tests to be independent of the actual hashes

comment:136 Changed 3 months ago by
 Commit changed from 86d5e94dc2ce2f0b674b275e039cecaaa7d0e130 to be98eb0a2721506f55a1a9dd30a647d006018e01
Branch pushed to git repo; I updated commit sha1. New commits:
be98eb0  Rewrite direct tests for hashes

comment:137 Changed 3 months ago by
 Commit changed from be98eb0a2721506f55a1a9dd30a647d006018e01 to 74f9352dccdd1d7e4f2c21da22b6625b88362884
comment:138 Changed 3 months ago by
 Commit changed from 74f9352dccdd1d7e4f2c21da22b6625b88362884 to a736d791ddeeea6dbc2e53e2c34a580dda32127b
comment:139 Changed 3 months ago by
 Commit changed from a736d791ddeeea6dbc2e53e2c34a580dda32127b to f384065cbf40774888040b1180ed0320add546ad
Branch pushed to git repo; I updated commit sha1. New commits:
f384065  Make computing graph faces deterministic

comment:140 Changed 3 months ago by
 Commit changed from f384065cbf40774888040b1180ed0320add546ad to 694c6b153eb1b2e69aa5345f5108ceeb6acfb1d9
comment:141 Changed 3 months ago by
 Commit changed from 694c6b153eb1b2e69aa5345f5108ceeb6acfb1d9 to 14c57e57275bdedce87b159d4cfe9658e94ec555
comment:142 Changed 3 months ago by
 Commit changed from 14c57e57275bdedce87b159d4cfe9658e94ec555 to cd346fa5bf72e14338a944626aa6fc3e8fe2833c
comment:143 Changed 3 months ago by
 Commit changed from cd346fa5bf72e14338a944626aa6fc3e8fe2833c to 093e9d61181d5ae221bc7d20e4b408b7efb251e8
comment:144 Changed 3 months ago by
 Commit changed from 093e9d61181d5ae221bc7d20e4b408b7efb251e8 to 73d6c7f47c6a203da2cca58f56e68df6fc3e6852
comment:145 Changed 3 months ago by
Just a handful of issues left, mostly pickling related
********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/matroids/linear_matroid.pyx", line 3674, in sage.matroids.linear_matroid.BinaryMatroid._projection_partition Failed example: N._projection_partition() Expected: 2 x 12 BinaryMatrix [110011001100] [001100110011] Got: 2 x 12 BinaryMatrix [001100110011] [110011001100] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/structure/sequence.py", line 824, in sage.structure.sequence.Sequence_generic.__copy__ Failed example: t.is_immutable == s.is_immutable Expected: True Got: False ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/structure/sequence.py", line 826, in sage.structure.sequence.Sequence_generic.__copy__ Failed example: t.is_mutable == s.is_mutable Expected: True Got: False ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/posets/hasse_diagram.py", line 2185, in sage.combinat.posets.hasse_diagram.HasseDiagram.antichains Failed example: TestSuite(A).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: An enumerated set with a forest structure != An enumerated set with a forest structure  The following tests failed: _test_pickling ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/posets/hasse_diagram.py", line 2190, in sage.combinat.posets.hasse_diagram.HasseDiagram.antichains Failed example: TestSuite(A).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: An enumerated set with a forest structure != An enumerated set with a forest structure  The following tests failed: _test_pickling ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/cluster_algebra_quiver/quiver_mutation_type.py", line 2171, in sage.combinat.cluster_algebra_quiver.quiver_mutation_type._construct_exceptional_mutation_classes Failed example: for mut_class in sorted(rank_3_exceptional.keys(), key=str): # long time print("{} {}".format(mut_class, rank_3_exceptional[mut_class])) Expected: ('G', 2, 1) [('BH?', (((1, 2), (1, 3)),)), ('BGO', (((2, 1), (3, 1)),)), ('BW?', (((0, 1), (3, 1)),)), ('BP?', (((0, 1), (1, 3)),)), ('BP_', (((0, 1), (1, 3)), ((2, 0), (3, 1)))), ('BP_', (((0, 1), (3, 1)), ((1, 2), (1, 3)), ((2, 0), (2, 2))))] ('G', 2, 1) [('BH?', (((1, 2), (3, 1)),)), ('BGO', (((2, 1), (1, 3)),)), ('BW?', (((0, 1), (1, 3)),)), ('BP?', (((0, 1), (3, 1)),)), ('BKO', (((1, 0), (3, 1)), ((2, 1), (1, 3)))), ('BP_', (((0, 1), (2, 2)), ((1, 2), (1, 3)), ((2, 0), (3, 1))))] Got: ('G', 2, 1) [('BH?', (((1, 2), (1, 3)),)), ('BGO', (((2, 1), (3, 1)),)), ('BW?', (((0, 1), (3, 1)),)), ('BP?', (((0, 1), (1, 3)),)), ('BP_', (((0, 1), (1, 3)), ((2, 0), (3, 1)))), ('BP_', (((0, 1), (1, 3)), ((1, 2), (2, 2)), ((2, 0), (3, 1))))] ('G', 2, 1) [('BH?', (((1, 2), (3, 1)),)), ('BGO', (((2, 1), (1, 3)),)), ('BW?', (((0, 1), (1, 3)),)), ('BP?', (((0, 1), (3, 1)),)), ('BKO', (((1, 0), (3, 1)), ((2, 1), (1, 3)))), ('BP_', (((0, 1), (3, 1)), ((1, 2), (2, 2)), ((2, 0), (1, 3)))), ('BP_', (((0, 1), (2, 2)), ((1, 2), (1, 3)), ((2, 0), (3, 1))))] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/cluster_algebra_quiver/mutation_type.py", line 1348, in sage.combinat.cluster_algebra_quiver.mutation_type._mutation_type_test Failed example: _mutation_type_test(3) # long time Expected: True ('A', (2, 1), 1) True ('A', 3) True ('B', 3) True ('BB', 2, 1) True ('BC', 2, 1) True ('C', 3) True ('CC', 2, 1) True ('G', 2, 1) True ('G', 2, 1) Got: True ('A', (2, 1), 1) True ('A', 3) True ('B', 3) True ('BB', 2, 1) True ('BC', 2, 1) True ('C', 3) True ('CC', 2, 1) False ('G', 2, 1) False ('G', 2, 1) ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/schemes/riemann_surfaces/riemann_surface.py", line 355, in sage.schemes.riemann_surfaces.riemann_surface.RiemannSurface Failed example: S.riemann_matrix() #abs tol 0.00000001 Expected: [0.500000000000000000000000... + 0.866025403784438646763723...*I] Got: [0.50000000000000000000000000000 + 0.86602540378443864676372317075*I] Tolerance exceeded in 1 of 2: 0.500000000000000000000000 vs 0.50000000000000000000000000000, tolerance 1e0 > 1e8 ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/homology/simplicial_complex_morphism.py", line 82, in sage.homology.simplicial_complex_morphism Failed example: z # this is the mapping path space Expected: Simplicial complex morphism: From: Simplicial complex with 6 vertices and 4 facets To: Minimal triangulation of the 2sphere Defn: ['L0R(0, 0)', 'L0R(0, 1)', 'L1R(1, 0)', 'L1R(1, 1)', 'L2R(2, 0)', 'L2R(2, 1)'] > [0, 0, 1, 1, 2, 2] Got: Simplicial complex morphism: From: Simplicial complex with 6 vertices and 5 facets To: Minimal triangulation of the 2sphere Defn: ['L0R(0, 0)', 'L0R(0, 1)', 'L1R(1, 0)', 'L1R(1, 1)', 'L2R(2, 0)', 'L2R(2, 1)'] > [0, 0, 1, 1, 2, 2] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/categories/homset.py", line 598, in sage.categories.homset.Homset.__init__ Failed example: TestSuite(H).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: Generic morphism: From: X To: Y != Generic morphism: From: X To: Y  The following tests failed: _test_pickling Failure in _test_elements The following tests failed: _test_elements ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/categories/homset.py", line 607, in sage.categories.homset.Homset.__init__ Failed example: TestSuite(H).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: Generic morphism: From: X To: Y != Generic morphism: From: X To: Y  The following tests failed: _test_pickling Failure in _test_elements The following tests failed: _test_elements ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/categories/homset.py", line 1253, in sage.categories.homset.HomsetWithBase.__init__ Failed example: TestSuite(H).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: Generic morphism: From: X To: Y != Generic morphism: From: X To: Y  The following tests failed: _test_pickling Failure in _test_elements The following tests failed: _test_elements
comment:146 Changed 3 months ago by
 Commit changed from 73d6c7f47c6a203da2cca58f56e68df6fc3e6852 to ce0f0e0404b23485c7c8b2e38e4c8a36eb1354e1
Branch pushed to git repo; I updated commit sha1. New commits:
687d070  Make vertices tuple of a simplicial complex deterministic

46114aa  Make test in matroids/linear_matroid.pyx pass regardless of the matrix row order

723b48f  Make _graph_without_edge_labels deterministic

ce0f0e0  Make riemann_matrix test more robust

comment:147 Changed 3 months ago by
The remaining issues are caused by this commit https://github.com/python/cpython/commit/ac20e0f98d6727ba97a9575bfa2a11b2f6247c35
In particular, if B=loads(dumps(A))
, then A.method
and B.method
have different hashes, so A.method==B.method
is always false.
comment:148 followup: ↓ 150 Changed 3 months ago by
Note that the issue of sorting vertices in simplicial complexes was discussed at length in #26966. I may propose a different solution to this problem, but give me a few days.
comment:149 Changed 3 months ago by
 Commit changed from ce0f0e0404b23485c7c8b2e38e4c8a36eb1354e1 to 5a98f9094bf29c5ddb7098ea21ca7076ab6d1a7d
comment:150 in reply to: ↑ 148 Changed 3 months ago by
Replying to jhpalmieri:
Note that the issue of sorting vertices in simplicial complexes was discussed at length in #26966. I may propose a different solution to this problem, but give me a few days.
Oh, I see now that you proposed the same fix there but reverted it.
comment:151 Changed 3 months ago by
With the current branch all tests are passing for me. There may be room for some improvements (besides the simplicial complex vertex ordering issue), but I hope there's still a chance that this can make it in 9.2. It would be quite unfortunate to not support python 3.8 one year after its release.
comment:152 Changed 3 months ago by
I wrote a long post about simplicial complexes with various solutions, but now I realize there is a simpler solution.
There are a few problems here: first, if you want consistency when dealing with products of simplicial complexes, some sorting is needed: if you triangulate a square, for example, you have two choices for how to add the diagonal, and different sortings will result in different choices. Second, Python 3.7 and 3.8 are handling some data differently from each other, and that's where the doctest failure in simplicial_complex_morphism.py
is coming from: different sortings yielding different triangulations of a product. Third, I think that this really means that the doctest in simplicial_complex_morphishm.py
is a bad one, because it is essentially relying on how intrinsically unsorted data is converted to an ordered object (a tuple). I tried putting in various ways of sorting, and I got at least three different results, all of which are mathematically valid, and I don't see any reason to force one sorting option over another.
As a result, we should make this change: get rid of the new sorting in simplicial_complex.py
and make the doctest more robust:

src/sage/homology/simplicial_complex.py
diff git a/src/sage/homology/simplicial_complex.py b/src/sage/homology/simplicial_complex.py index 8ee9a4f84c..1c9dd8da8e 100644
a b class SimplicialComplex(Parent, GenericCellComplex): 1045 1045 else: 1046 1046 vertex_set = range(n + 1) 1047 1047 1048 try: # use natural order if vertices are integers 1049 vertices = tuple(sorted(vertex_set)) 1050 except TypeError: # incomparable vertices 1051 vertices = tuple(sorted(vertex_set, key=str)) 1048 vertices = tuple(vertex_set) 1052 1049 1053 1050 gen_dict = {} 1054 1051 for v in vertices: 
src/sage/homology/simplicial_complex_morphism.py
diff git a/src/sage/homology/simplicial_complex_morphism.py b/src/sage/homology/simplicial_complex_morphism.py index d954dd90b4..8db933b9fe 100644
a b EXAMPLES:: 81 81 sage: z = y.fiber_product(x) 82 82 sage: z # this is the mapping path space 83 83 Simplicial complex morphism: 84 From: Simplicial complex with 6 vertices and 6facets84 From: Simplicial complex with 6 vertices and ... facets 85 85 To: Minimal triangulation of the 2sphere 86 86 Defn: ['L0R(0, 0)', 'L0R(0, 1)', 'L1R(1, 0)', 'L1R(1, 1)', 'L2R(2, 0)', 'L2R(2, 1)'] > [0, 0, 1, 1, 2, 2] 87 87 """
By the way, see ticket:26966#comment:25 for the an example of the evils of sorting using key=str
.
comment:153 Changed 3 months ago by
 Commit changed from 5a98f9094bf29c5ddb7098ea21ca7076ab6d1a7d to 196a30ad097999a707b71140abc6afd5265c652f
Branch pushed to git repo; I updated commit sha1. New commits:
196a30a  trac 27754: make doctest in simplicial_complex_morphism.py more robust

comment:154 Changed 3 months ago by
 Commit changed from 196a30ad097999a707b71140abc6afd5265c652f to 35cf0e197e4142b4849d09398a31063b5537eec9
Branch pushed to git repo; I updated commit sha1. New commits:
35cf0e1  Allow system python 3.8

comment:155 Changed 3 months ago by
 Commit changed from 35cf0e197e4142b4849d09398a31063b5537eec9 to f939f28b40da12dfc5b31f531dac380f5ef20343
comment:156 Changed 3 months ago by
 Status changed from new to needs_review
Setting to needs review for wider testing. Needs testing especially with system python<3.8, also with optional packages
comment:157 Changed 3 months ago by
 Commit changed from f939f28b40da12dfc5b31f531dac380f5ef20343 to c6002e8181f8b4bf2b9a4f2262a60aa354c16e4f
Branch pushed to git repo; I updated commit sha1. New commits:
c4769bf  Add pynac patch to fix deprecation warnings on unpickling with Python 3.8

ecc6f45  Add patch

c6002e8  Merge branch 'u/arojas/fix_deprecation_warnings_when_unpickling_pynac_objects_with_python_3_8' of git://trac.sagemath.org/sage into t/27754/public/python3.8

comment:158 followup: ↓ 161 Changed 3 months ago by
comment:159 Changed 3 months ago by
 Cc ghsheerluck added
comment:160 Changed 3 months ago by
 Commit changed from c6002e8181f8b4bf2b9a4f2262a60aa354c16e4f to 57c2d1c89515fe0716d04159bcdbe3fc398e1180
Branch pushed to git repo; I updated commit sha1. New commits:
57c2d1c  tox.ini, .github/workflows/tox.yml: Add gentoopython3.7

comment:161 in reply to: ↑ 158 Changed 3 months ago by
Replying to mkoeppe:
Tests run at https://github.com/mkoeppe/sage/actions/runs/193151775
What's the outcome of this? I see many failed jobs but I can't see any logs.
comment:162 Changed 3 months ago by
Looks like GH Action was having trouble. I have started another run at https://github.com/mkoeppe/sage/actions/runs/199465086
comment:163 Changed 3 months ago by
Clean on ubuntutrustystandard
(with python3 from SPKG, https://github.com/mkoeppe/sage/runs/962936559), except for
sage t randomseed=0 src/sage/doctest/test.py # Timed out
Clean on centos7standard
(with python3 from SPKG, https://github.com/mkoeppe/sage/runs/962936782), except for
sage t randomseed=0 src/sage/interfaces/sagespawn.pyx File "src/sage/interfaces/sagespawn.pyx", line 96, in sage.interfaces.sagespawn.SageSpawn.__repr__ Failed example: while s.isalive(): # Wait until the process finishes sleep(0.1) Expected nothing Got: doctest:warning DeprecationWarning: an integer is required (got type sage.rings.real_mpfr.RealLiteral). Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
Clean on gentoostandard
(using /usr/bin/python3.8
, https://github.com/mkoeppe/sage/runs/962936791) except for:
sage t randomseed=0 src/sage/schemes/elliptic_curves/ell_modular_symbols.py # Timed out
Clean on gentoostandardpython3.7
(using /usr/bin/python3.7
) except for:
sage t randomseed=0 src/sage/categories/finite_enumerated_sets.py File "src/sage/categories/finite_enumerated_sets.py", line 290, in sage.categories.finite_enumerated_sets.FiniteEnumeratedSets.ParentMethods._list_from_iterator Failed example: list(C) Expected: hello! hello! ... [1, 2, 3] Got: hello! hello! [1, 2, 3]
Lots of doctest failures because of LC_ALL: cannot change locale
messages (#30053) on linuxmint17minimal
(https://github.com/mkoeppe/sage/runs/962936711) and standard
Unfortunately most builds failed again because of unrelated technical problems on GH Actions
comment:164 Changed 3 months ago by
 Reviewers set to https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/runs/959545895
comment:165 Changed 3 months ago by
 Reviewers changed from https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/runs/959545895 to https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/actions/runs/199465096
comment:166 Changed 3 months ago by
 Commit changed from 57c2d1c89515fe0716d04159bcdbe3fc398e1180 to 389f1b2d7f5fe78ed232bbfb50d95ca5bee47d1e
comment:167 Changed 3 months ago by
 Commit changed from 389f1b2d7f5fe78ed232bbfb50d95ca5bee47d1e to fa2ad16159d452089a235796291a13f49d8d0477
Branch pushed to git repo; I updated commit sha1. New commits:
fa2ad16  Merge tag '9.2.beta8' into t/27754/public/python3.8

comment:168 Changed 3 months ago by
 Reviewers changed from https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/actions/runs/199465096 to github.com/mkoeppe/sage/actions/runs/206442118
comment:169 Changed 3 months ago by
debianjessieminimal
, debianstretchstandard
, centos7standard
(with python3 from SPKG) clean
ubuntubionicstandard
, debianbusterstandard
, linuxmint19.3standard
, fedora30standard
, fedora31standard
, gentoopython3.7standard
(with /usr/bin/python3.7
) clean
gentoostandard
(with /usr/bin/python3.8
) clean
ubuntueoanstandard
(with /usr/bin/python3.7
, https://github.com/mkoeppe/sage/runs/978617270) clean except for
sage t randomseed=0 src/sage/libs/eclib/interface.py # 2 doctests failed sage t randomseed=0 src/sage/schemes/cyclic_covers/charpoly_frobenius.py # 1 doctest failed
(seem unrelated to this ticket (mwrank; #28951?))
debianbullseyestandard
(with /usr/bin/python3.7
) clean except for doctest errors because of newer flint or arb from the system
linuxmint17standard
.... lots of doctest failures because of "setlocale" messages
fedora27standard
(with /usr/bin/python3.6
):
sage t randomseed=0 src/sage/lfunctions/sympow.py # 3 doctests failed sage t randomseed=0 src/sage/schemes/elliptic_curves/ell_rational_field.py # 2 doctests failed
(seems unrelated)
fedora29standard
(with /usr/bin/python3.7
): likewise
fedora28standard
(with /usr/bin/python3.6
) additionally:
sage t randomseed=0 src/sage/rings/polynomial/groebner_fan.py # 3 doctests failed
(seems unrelated)
condaforgestandard
(with /opt/conda/bin/python3.7
):
sage t randomseed=0 src/doc/en/constructions/algebraic_geometry.rst # 1 doctest failed sage t randomseed=0 src/doc/en/developer/coding_in_other.rst # 1 doctest failed sage t randomseed=0 src/sage/misc/cython.py # 3 doctests failed sage t randomseed=0 src/sage/modules/fg_pid/fgp_module.py # 3 doctests failed sage t randomseed=0 src/sage/modules/free_module_morphism.py # 3 doctests failed sage t randomseed=0 src/sage/rings/complex_arb.pyx # 6 doctests failed sage t randomseed=0 src/sage/parallel/map_reduce.py # Timed out sage t randomseed=0 src/sage/rings/real_arb.pyx # 2 doctests failed sage t randomseed=0 src/sage/schemes/hyperelliptic_curves/monsky_washnitzer.py # Killed due to abort
condaforgeminimal
(https://github.com/mkoeppe/sage/runs/978620286):
[python33.8.5] ImportError: The required _crypt module was not built as part of CPython [python33.8.5] crypt module failed to import [python33.8.5] ModuleNotFoundError: No module named 'zlib' [python33.8.5] zlib module failed to import [python33.8.5] ModuleNotFoundError: No module named '_sqlite3' [python33.8.5] sqlite3 module failed to import [python33.8.5] Error: One or more modules failed to import.
(but we had similar errors also before the update, on this platform)
ubuntufocalstandard
(https://github.com/mkoeppe/sage/runs/978617297)
hecking whether SageMath should install SPKG python3... checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no checking for python3 >= 3.6, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core... checking ... whether /usr/bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension checking ... whether /usr/bin/python3 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension checking ... whether /bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension checking ... whether /bin/python3 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension configure: no suitable system package found for SPKG python3
likewise on debiansidstandard
(https://github.com/mkoeppe/sage/runs/978620025)
comment:170 Changed 3 months ago by
homebrewmacosminimal
(using python3 SPKG): Clean except for various timeouts
sage t randomseed=0 src/sage/dynamics/arithmetic_dynamics/projective_ds.py # Timed out sage t randomseed=0 src/sage/interfaces/gap.py # 1 doctest failed sage t randomseed=0 src/sage/manifolds/differentiable/metric.py # Timed out sage t randomseed=0 src/sage/manifolds/differentiable/tensorfield.py # Timed out sage t randomseed=0 src/sage/plot/graphics.py # Timed out sage t randomseed=0 src/sage/plot/plot.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/base.pyx # Timed out sage t randomseed=0 src/sage/plot/plot3d/implicit_plot3d.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/parametric_plot3d.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/plot3d.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/shapes.pyx # Timed out sage t randomseed=0 src/sage/plot/plot3d/shapes2.py # Timed out sage t randomseed=0 src/sage/rings/function_field/function_field.py # Timed out sage t randomseed=0 src/sage/schemes/elliptic_curves/ell_number_field.py # Timed out sage t randomseed=0 src/sage/schemes/elliptic_curves/gp_simon.py
homebrewmacospython3_xcodestandard
(using /usr/bin/python3
)  build times out during docbuild  perhaps the same problem as reported on sagerelease
condaforgemacosminimal
(https://github.com/mkoeppe/sage/runs/978617180):
[python33.8.5] checking build system type... x86_64appledarwin19.6.0 [python33.8.5] checking host system type... x86_64appledarwin13.4.0 [python33.8.5] checking for python3.8... no [python33.8.5] checking for python3... python3 [python33.8.5] configure: error: Cross compiling required host=HOSTTUPLE and build=ARCH [python33.8.5] configure: WARNING: unrecognized options: disabletoolboxglue [python33.8.5] checking build system type... x86_64appledarwin19.6.0 [python33.8.5] checking host system type... x86_64appledarwin13.4.0 [python33.8.5] checking for python3.8... no [python33.8.5] checking for python3... python3 [python33.8.5] configure: error: Cross compiling required host=HOSTTUPLE and build=ARCH
???? But this platform also failed before the update
comment:171 Changed 3 months ago by
 Milestone changed from sage9.3 to sage9.2
Looks like we have a good chance of getting this ready for 9.2!
comment:173 Changed 3 months ago by
 Commit changed from fa2ad16159d452089a235796291a13f49d8d0477 to 89dc8c92ebd9f10c7d41e6b72e6d271e05fc1119
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
bccf5c1  Merge branch 't/30127/cygwin_minimal__pip_20_1_1_fails_to_install' into t/27754/public/python3.8

bc65004  tox.ini, tox.yml: Add ubuntugroovy, fedora33

8eb9cae  Merge branch 't/30339/tox_ini__update_for_latest_ubuntu__fedora_versions' into t/27754/public/python3.8

8d5f2ae  tox.ini, build/pkgs/python3/distros/: Switch to python 3.8

cf25b83  Move code for tagging/pushing docker images into tox.ini

51c6cde  fixup

231b812  .githubworkflows/tox*.yml: Update similarly

385571e  Merge branch 't/30144/tox_ini___tox_yml__move_code_for_tagging_pushing_docker_images_into_tox_ini' into t/27754/public/python3.8

701b4ce  build/make/Makefile.in: Add + before recursive make invocations

89dc8c9  Merge branch 't/30345/build_make_makefile_in__filter_out___j__from_sub_make_invocations_to_avoid_excessive_parallel_load' into t/27754/public/python3.8

comment:174 Changed 3 months ago by
 Dependencies changed from #28197, #29042, #29861, #30162, #30225 to #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345
comment:175 Changed 3 months ago by
 Reviewers changed from github.com/mkoeppe/sage/actions/runs/206442118 to github.com/mkoeppe/sage/actions/runs/206442118, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210
comment:176 Changed 3 months ago by
 Reviewers changed from github.com/mkoeppe/sage/actions/runs/206442118, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210 to github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210
comment:177 Changed 3 months ago by
 Priority changed from major to critical
comment:178 Changed 3 months ago by
ubuntufocalstandard
now accepts system python 3.8, clean doctests.
But:
debianbullseyestandard
(https://github.com/mkoeppe/sage/runs/982720084), now using /usr/bin/python3.8
(3.8.5), fails to build matplotlib
:
x86_64linuxgnugcc pthread shared Wl,O1 Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 Wl,rpathlink,/sage/local/lib L/sage/local/lib Wl,rpath,/sage/local/lib g fstackprotectorstrong Wformat Werror=formatsecurity Wdatetime D_FORTIFY_SOURCE=2 build/temp.linuxx86_643.8/src/qhull_wrap.o build/temp.linuxx86_643.8/extern/libqhull/geom.o build/temp.linuxx86_643.8/extern/libqhull/geom2.o build/temp.linuxx86_643.8/extern/libqhull/global.o build/temp.linuxx86_643.8/extern/libqhull/io.o build/temp.linuxx86_643.8/extern/libqhull/libqhull.o build/temp.linuxx86_643.8/extern/libqhull/mem.o build/temp.linuxx86_643.8/extern/libqhull/merge.o build/temp.linuxx86_643.8/extern/libqhull/poly.o build/temp.linuxx86_643.8/extern/libqhull/poly2.o build/temp.linuxx86_643.8/extern/libqhull/qset.o build/temp.linuxx86_643.8/extern/libqhull/random.o build/temp.linuxx86_643.8/extern/libqhull/rboxlib.o build/temp.linuxx86_643.8/extern/libqhull/stat.o build/temp.linuxx86_643.8/extern/libqhull/user.o build/temp.linuxx86_643.8/extern/libqhull/usermem.o build/temp.linuxx86_643.8/extern/libqhull/userprintf.o build/temp.linuxx86_643.8/extern/libqhull/userprintf_rbox.o lm o build/lib.linuxx86_643.8/matplotlib/_qhull.cpython38x86_64linuxgnu.so lto1: fatal error: bytecode stream in file ‘build/temp.linuxx86_643.8/src/qhull_wrap.o’ generated with GCC compiler older than 10.0 compilation terminated. ltowrapper: fatal error: x86_64linuxgnugcc returned 1 exit status compilation terminated. /usr/bin/ld: error: ltowrapper failed collect2: error: ld returned 1 exit status
Likewise on debiansidstandard
and ubuntugroovystandard
comment:179 followup: ↓ 182 Changed 3 months ago by
Getting rid of flto
would work around this problem. Not sure where it is coming from
comment:180 Changed 2 months ago by
(As expected, upgrading matplotlib to latest (3.3.1, #30358) does not solve this problem.)
comment:181 followup: ↓ 185 Changed 2 months ago by
I haven't experienced anything like that in Gentoo. But looking at build log for matplotlib 3.3.0 suggests reviving an idea: using an external qhull
install for matplotlib.
comment:182 in reply to: ↑ 179 Changed 2 months ago by
Replying to mkoeppe:
Getting rid of
flto
would work around this problem. Not sure where it is coming from
The flag is coming directly from matplotlib's setup.py
, I'll just disable it
comment:183 Changed 2 months ago by
 Dependencies changed from #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345 to #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365
comment:184 Changed 2 months ago by
 Commit changed from 89dc8c92ebd9f10c7d41e6b72e6d271e05fc1119 to 4daa290fef4356faaf56eefa4e58b948e2f8db84
comment:185 in reply to: ↑ 181 Changed 2 months ago by
Replying to fbissey:
I haven't experienced anything like that in Gentoo. But looking at build log for matplotlib 3.3.0 suggests reviving an idea: using an external
qhull
install for matplotlib.
I would support this in general, but not for 9.2; and as discussed in #30176, we would need to make cmake standard.
comment:186 Changed 2 months ago by
 Reviewers changed from github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210 to Matthias Koeppe, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210, ...
I think this ticket is now ready for review.
comment:187 Changed 2 months ago by
 Description modified (diff)
comment:188 Changed 2 months ago by
 Reviewers changed from Matthias Koeppe, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210, ... to Matthias Koeppe, François Bissey
 Status changed from needs_review to positive_review
Now is a good time to tests things on a bigger scale.
comment:189 Changed 2 months ago by
Thanks
comment:190 Changed 2 months ago by
 Commit changed from 4daa290fef4356faaf56eefa4e58b948e2f8db84 to 46c559c0b052e37d87c050fee879150318ba0463
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
46c559c  Merge tag '9.2.beta9' into t/27754/public/python3.8

comment:191 Changed 2 months ago by
 Status changed from needs_review to positive_review
comment:192 Changed 2 months ago by
 Status changed from positive_review to needs_work
STDOUT: CONFLICT (content): Merge conflict in src/sage/graphs/generic_graph.py
comment:193 Changed 2 months ago by
 Commit changed from 46c559c0b052e37d87c050fee879150318ba0463 to a625ed34796120b6d8bc7397c4a9ef4314649332
comment:194 Changed 2 months ago by
 Commit changed from a625ed34796120b6d8bc7397c4a9ef4314649332 to 53b569d68ddca39a0c5b33ab59ce77dd651da21a
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
3db87ad  build/pkgs/maxima: Update to 5.44.0

bcf3c98  Remove old patches

07aa2f9  build/pkgs/maxima/spkginstall.in: New workaround for makeinfo trouble

046634d  sage.interfaces.maxima_abstract.MaximaAbstract._command_runner: Skip all comment lines, rather than counting lines to skip

d1ad8d7  sage.interfaces.maxima_abstract.MaximaAbstract.completions: Remove 'impl' functions

2ea3e73  Merge branch 'u/mkoeppe/upgrade_maxima_to_5_44_0' of trac.sagemath.org:sage into HEAD

8f83f96  #30063 : Maxima gamma_greek is now called gamma_incomplete_lower

21e2fea  #30063 : mark Maxima's integration regression as known bug.

d6e0952  Trac #30063: Upgrade Maxima to 5.44.0

53b569d  Merge remotetracking branch 'vbraun/develop' into t/27754/public/python3.8

comment:195 Changed 2 months ago by
 Dependencies changed from #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365 to #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140
 Status changed from needs_work to positive_review
comment:196 Changed 2 months ago by
 Status changed from positive_review to needs_work
I'm getting various failures on 32bit apparently due to different iterator order:
********************************************************************** File "src/sage/combinat/rigged_configurations/tensor_product_kr_tableaux.py", line 333, in sage.combinat.rigged_configurations.tensor_product_kr_tableaux.TensorProductOfKirillovReshetikhinTableaux.__iter__ Failed example: next(g) Expected: [[2], [4]] (X) [[1]] Got: [[1], [2]] (X) [[1]] ********************************************************************** 1 item had failures: 1 of 5 in sage.combinat.rigged_configurations.tensor_product_kr_tableaux.TensorProductOfKirillovReshetikhinTableaux.__iter__ [95 tests, 1 failure, 21.93 s] ********************************************************************** File "src/sage/geometry/cone.py", line 4059, in sage.geometry.cone.?.semigroup_generators Failed example: halfplane.semigroup_generators() Expected: (N(0, 1), N(1, 0), N(1, 0)) Got: (N(1, 0), N(1, 0), N(0, 1)) ********************************************************************** 1 item had failures: 1 of 23 in sage.geometry.cone.?.semigroup_generators [1241 tests, 1 failure, 18.70 s] ********************************************************************** File "src/sage/geometry/fan_isomorphism.py", line 124, in sage.geometry.fan_isomorphism.fan_isomorphism_generator Failed example: tuple(fan_isomorphism_generator(fan1, fan2)) Expected: ( [18 1 5] [ 6 3 7] [ 4 0 1] [ 1 1 2] [ 5 0 1], [ 2 1 2] ) Got: ( [ 6 3 7] [18 1 5] [ 1 1 2] [ 4 0 1] [ 2 1 2], [ 5 0 1] ) ********************************************************************** File "src/sage/geometry/fan_isomorphism.py", line 130, in sage.geometry.fan_isomorphism.fan_isomorphism_generator Failed example: tuple(fan_isomorphism_generator(fan2, fan1)) Expected: ( [ 0 1 1] [ 0 1 1] [ 1 7 2] [ 2 2 5] [ 0 5 4], [ 1 0 3] ) Got: ( [ 0 1 1] [ 0 1 1] [ 2 2 5] [ 1 7 2] [ 1 0 3], [ 0 5 4] ) ********************************************************************** 1 item had failures: 2 of 21 in sage.geometry.fan_isomorphism.fan_isomorphism_generator [65 tests, 2 failures, 1.07 s] ********************************************************************** File "src/sage/geometry/polyhedron/library.py", line 2639, in sage.geometry.polyhedron.library.Polytopes.generalized_permutahedron Failed example: sorted(perm_a2_reg.vertices()) Expected: [A vertex at (1, 0), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1.000000000000000?, 0.?e18)] Got: [A vertex at (1.000000000000000?, 0.?e18), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1, 0)] ********************************************************************** 1 item had failures: 1 of 21 in sage.geometry.polyhedron.library.Polytopes.generalized_permutahedron [305 tests, 1 failure, 97.96 s] ********************************************************************** File "src/sage/geometry/triangulation/point_configuration.py", line 1401, in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits_support Failed example: list( p.circuits_support() ) Expected: [(0, 3, 4), (0, 1, 2), (1, 2, 3, 4)] Got: [(0, 1, 2), (0, 3, 4), (1, 2, 3, 4)] ********************************************************************** File "src/sage/geometry/triangulation/point_configuration.py", line 1471, in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits Failed example: p.circuits() Expected: (((0,), (1, 2), (3, 4)), ((0,), (3, 4), (1, 2)), ((1, 2), (0,), (3, 4))) Got: (((0,), (3, 4), (1, 2)), ((0,), (1, 2), (3, 4)), ((1, 2), (0,), (3, 4))) ********************************************************************** 2 items had failures: 1 of 6 in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits 1 of 3 in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits_support [214 tests, 2 failures, 19.74 s] ********************************************************************** File "src/sage/graphs/graph.py", line 7342, in sage.graphs.graph.Graph.cliques_containing_vertex Failed example: sorted(d[(0, 1)]) Expected: [[(0, 1), (0, 0)], [(0, 1), (0, 2)], [(0, 1), (1, 1)]] Got: [[(0, 0), (0, 1)], [(0, 2), (0, 1)], [(1, 1), (0, 1)]] ********************************************************************** 1 item had failures: 1 of 15 in sage.graphs.graph.Graph.cliques_containing_vertex [1190 tests, 1 failure, 20.84 s] ********************************************************************** File "src/sage/homology/homology_vector_space_with_basis.py", line 113, in sage.homology.homology_vector_space_with_basis.HomologyVectorSpaceWithBasis Failed example: x.cup_product(x) Expected: 0 Got: h^{2,0} ********************************************************************** File "src/sage/homology/homology_vector_space_with_basis.py", line 115, in sage.homology.homology_vector_space_with_basis.HomologyVectorSpaceWithBasis Failed example: x.cup_product(y) Expected: h^{2,0} Got: 0 ********************************************************************** 1 item had failures: 2 of 45 in sage.homology.homology_vector_space_with_basis.HomologyVectorSpaceWithBasis [194 tests, 2 failures, 16.93 s] R[write to console]: Warning messages: R[write to console]: 1: R[write to console]: In sage10 + sage6 : R[write to console]: R[write to console]: longer object length is not a multiple of shorter object length R[write to console]: 2: R[write to console]: In sqrt(sage10) : R[write to console]: NaNs produced R[write to console]: 3: R[write to console]: In sqrt(sage4) : R[write to console]: NaNs produced ********************************************************************** File "src/sage/matroids/graphic_matroid.py", line 1659, in sage.matroids.graphic_matroid.GraphicMatroid.graphic_coextensions Failed example: for N1 in I: N1.graph().edges(sort=True) Expected: [(0, 1, 0), (0, 3, 1), (0, 4, 'a'), (1, 2, 2), (2, 3, 3)] [(0, 1, 0), (0, 3, 1), (1, 4, 2), (2, 3, 3), (2, 4, 'a')] Got: [(0, 1, 0), (0, 3, 1), (0, 4, 'a'), (1, 2, 2), (2, 3, 3)] [(0, 3, 1), (0, 4, 0), (1, 2, 2), (1, 4, 'a'), (2, 3, 3)] ********************************************************************** 1 item had failures: 1 of 27 in sage.matroids.graphic_matroid.GraphicMatroid.graphic_coextensions [358 tests, 1 failure, 0.75 s] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1099, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.class_representatives Failed example: R Expected: [[V(2)*V(3)], [V(1)*V(2)]] Got: [[V(1)*V(2)], [V(2)*V(3)]] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1101, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.class_representatives Failed example: [v.continued_fraction()[1] for v in R] Expected: [(1, 2, 2), (3,)] Got: [(3,), (1, 2, 2)] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1318, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.reduced_elements Failed example: R Expected: [ [ 5*lam 3] [ 5*lam 7] [4*lam 3] [3*lam 1] [ 7 2*lam], [ 3 2*lam], [ 3 lam], [ 1 0] ] Got: [ [3*lam 1] [ 5*lam 3] [ 5*lam 7] [4*lam 3] [ 1 0], [ 7 2*lam], [ 3 2*lam], [ 3 lam] ] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1323, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.reduced_elements Failed example: [v.continued_fraction() for v in R] Expected: [((), (1, 2, 2)), ((), (2, 2, 1)), ((), (2, 1, 2)), ((), (3,))] Got: [((), (3,)), ((), (1, 2, 2)), ((), (2, 2, 1)), ((), (2, 1, 2))] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1350, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.simple_elements Failed example: G.simple_elements(D=14) Expected: [ [2*lam 1] [ lam 1] [2*lam 3] [ lam 3] [ 3 lam], [ 3 2*lam], [ 1 lam], [ 1 2*lam] ] Got: [ [2*lam 3] [ lam 3] [2*lam 1] [ lam 1] [ 1 lam], [ 1 2*lam], [ 3 lam], [ 3 2*lam] ] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1387, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.rational_period_functions Failed example: G.rational_period_functions(k=4, D=14) Expected: [z^4 + 1, 16*z^4  16, 16*z^4 + 16] Got: [z^4 + 1, 16*z^4 + 16, 16*z^4  16] ********************************************************************** 4 items had failures: 2 of 27 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.class_representatives 1 of 7 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.rational_period_functions 2 of 9 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.reduced_elements 1 of 5 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.simple_elements [228 tests, 6 failures, 4.90 s] ********************************************************************** File "src/sage/modular/modform_hecketriangle/readme.py", line 549, in sage.modular.modform_hecketriangle.readme Failed example: G.class_representatives(9*G.lam() + 5) Expected: [S*T^(2)*S*T^(1)*S, T*S*T^2] Got: [T*S*T^2, S*T^(2)*S*T^(1)*S] ********************************************************************** 1 item had failures: 1 of 383 in sage.modular.modform_hecketriangle.readme [382 tests, 1 failure, 19.56 s] ********************************************************************** File "src/sage/monoids/trace_monoid.py", line 959, in sage.monoids.trace_monoid.TraceMonoid._repr_ Failed example: M.<a,b,c,d> = TraceMonoid(I=I); M Expected: Trace monoid on 4 generators ([a], [b], [c], [d]) with independence relation {{a, d}, {b, c}} Got: Trace monoid on 4 generators ([a], [b], [c], [d]) with independence relation {{b, c}, {d, a}} ********************************************************************** File "src/sage/monoids/trace_monoid.py", line 976, in sage.monoids.trace_monoid.TraceMonoid._latex_ Failed example: M.<a,b,c,d> = TraceMonoid(I=I); latex(M) Expected: \langle a, b, c, d \mid ad=da,bc=cb \rangle Got: \langle a, b, c, d \mid bc=cb,da=ad \rangle ********************************************************************** 2 items had failures: 1 of 4 in sage.monoids.trace_monoid.TraceMonoid._latex_ 1 of 4 in sage.monoids.trace_monoid.TraceMonoid._repr_ [186 tests, 2 failures, 4.63 s] ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 4166, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis Failed example: f Expected: 1/2*y^2*z^7  1/4*y*z^8 + 2*x*z^5 + 95*z^6 + 1/2*y^5  1/4*y^4*z + x^2*y^2 + 3/2*x^2*y*z + 95*x*y*z^2 Got: 1/2*y^2*z^7  1/4*y*z^8 + 95*x*z^5 + 2*z^6 + 1/2*y^5  1/4*y^4*z + x^2*y^2 + 189/2*x^2*y*z + 2*x*y*z^2 ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 4168, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis Failed example: f.lift(I.gens()) Expected: [2*x + 95*z, 1/2*y^2  1/4*y*z, 0] Got: [95*x + 2*z, 1/2*y^2  1/4*y*z, 0] ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 4170, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis Failed example: l = f.lift(J.gens()); l Expected: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1/2*y^2 + 1/4*y*z, 1/2*y^2*z^2  1/4*y*z^3 + 2*x + 95*z] Got: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1/2*y^2 + 1/4*y*z, 1/2*y^2*z^2  1/4*y*z^3 + 95*x + 2*z] ********************************************************************** 1 item had failures: 3 of 111 in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis [917 tests, 3 failures, 10.04 s]  sage t long randomseed=0 src/sage/combinat/rigged_configurations/tensor_product_kr_tableaux.py # 1 doctest failed sage t long randomseed=0 src/sage/geometry/cone.py # 1 doctest failed sage t long randomseed=0 src/sage/geometry/fan_isomorphism.py # 2 doctests failed sage t long randomseed=0 src/sage/geometry/polyhedron/library.py # 1 doctest failed sage t long randomseed=0 src/sage/geometry/triangulation/point_configuration.py # 2 doctests failed sage t long randomseed=0 src/sage/graphs/graph.py # 1 doctest failed sage t long randomseed=0 src/sage/homology/homology_vector_space_with_basis.py # 2 doctests failed sage t long randomseed=0 src/sage/matroids/graphic_matroid.py # 1 doctest failed sage t long randomseed=0 src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py # 6 doctests failed sage t long randomseed=0 src/sage/modular/modform_hecketriangle/readme.py # 1 doctest failed sage t long randomseed=0 src/sage/monoids/trace_monoid.py # 2 doctests failed sage t long randomseed=0 src/sage/rings/polynomial/multi_polynomial_ideal.py # 3 doctests failed 
comment:197 Changed 2 months ago by
 Commit changed from 53b569d68ddca39a0c5b33ab59ce77dd651da21a to 42bfb1f7a85fdad131bd9b278105641952a54bfb
Branch pushed to git repo; I updated commit sha1. New commits:
42bfb1f  Merge tag '9.2.beta10' into t/27754/public/python3.8

comment:198 Changed 2 months ago by
 Commit changed from 42bfb1f7a85fdad131bd9b278105641952a54bfb to d167e08bb10a7c0b36be21edeffd75e8b222d89b
Branch pushed to git repo; I updated commit sha1. New commits:
d167e08  Make doctests more flexible for Python 3.8 on 32bit

comment:199 Changed 2 months ago by
 Status changed from needs_work to needs_review
comment:200 Changed 2 months ago by
This fixes all failures except for the following two:
sage t randomseed=0 src/sage/homology/homology_vector_space_with_basis.py # 2 doctests failed sage t randomseed=0 src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py # 1 doctest failed
Help with these would be welcome
comment:201 Changed 2 months ago by
There must be some way for me to run a 32bit machine via Docker so I can look into these, but I can't figure out how.
comment:202 followup: ↓ 203 Changed 2 months ago by
Yes, for example EXTRA_CONFIGURE_ARGS="withoutsystemsuitesparse" tox e dockerubuntueoani386standardpython3_spkg
comment:203 in reply to: ↑ 202 Changed 2 months ago by
Replying to mkoeppe:
Yes, for example
EXTRA_CONFIGURE_ARGS="withoutsystemsuitesparse" tox e dockerubuntueoani386standardpython3_spkg
Well, that didn't work for me (with EXTRA_CONFIGURE_ARGS="withsystemsuitesparse=no"
): the build failed for cypari
and giac
. But I used VirtualBox? to install a 32bit Debian machine, and I see the doctest failures. I actually haven't looked at the hecke_triangle_groups.py failure — not my area of expertise — but I understand what's happening with the homology failure. There are some unsorted pieces of data lying around, and when they get converted to lists or tuples, how they get converted affects the answer. Both answers (32bit and 64bit) are mathematically correct, it's just that you get a different choice of bases in the two cases.
I propose changing to a case where the sorting won't affect the answer.
comment:204 Changed 2 months ago by
 Commit changed from d167e08bb10a7c0b36be21edeffd75e8b222d89b to d85c4ba66d5bf4b03e4f480da0ca8d49ade49fb2
Branch pushed to git repo; I updated commit sha1. New commits:
d85c4ba  trac 27754: homology doctest fix

comment:205 followup: ↓ 206 Changed 2 months ago by
For the hecke_triangle_groups.py problem, this seems to fix it, but I don't know the mathematics of what's going on, nor have I looked at the code:

src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py
diff git a/src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py b/src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py index 47ba82eb8e..cb336f64c4 100644
a b class HeckeTriangleGroup(FinitelyGeneratedMatrixGroup_generic, 1096 1096 [[U], [U^(1)]] 1097 1097 1098 1098 sage: R = G.class_representatives(14) 1099 sage: R1099 sage: sorted(R) 1100 1100 [[V(2)*V(3)], [V(1)*V(2)]] 1101 1101 sage: sorted(v.continued_fraction()[1] for v in R) 1102 1102 [(1, 2, 2), (3,)]
comment:206 in reply to: ↑ 205 ; followup: ↓ 207 Changed 2 months ago by
Replying to jhpalmieri:
For the hecke_triangle_groups.py problem, this seems to fix it, but I don't know the mathematics of what's going on, nor have I looked at the code:
docs say that it's a list of representatives, with no meaning attached to the order, so your fix is OK.
comment:207 in reply to: ↑ 206 Changed 2 months ago by
Replying to dimpase:
Replying to jhpalmieri:
For the hecke_triangle_groups.py problem, this seems to fix it, but I don't know the mathematics of what's going on, nor have I looked at the code:
docs say that it's a list of representatives, with no meaning attached to the order, so your fix is OK.
I'll push a branch after I do a bit more testing, unless someone beats me to it. (One question is how the elements in the list R are sorted. I can't tell after looking at the code for a few minutes. But this does seem to work.)
comment:208 Changed 2 months ago by
 Commit changed from d85c4ba66d5bf4b03e4f480da0ca8d49ade49fb2 to b9d451c7740613f38194cba3c4bd6f428cdc19e5
Branch pushed to git repo; I updated commit sha1. New commits:
b9d451c  trac 27754: 32bit Python 3.8 doctest fix in hecke_triangle_groups.py

comment:209 Changed 2 months ago by
Okay, this works for me on OS X with the system Python and Sage's Python 3.8, and the same on a virtual Debian 32bit machine.
comment:211 Changed 2 months ago by
 Status changed from positive_review to needs_work
Merge conflict
comment:212 Changed 2 months ago by
 Commit changed from b9d451c7740613f38194cba3c4bd6f428cdc19e5 to a9b69650667022f4a46b59ac90994254389931c1
Branch pushed to git repo; I updated commit sha1. New commits:
488bba4  on macOS set OMP_NUM_THREADS to 1 for docbuild and tests

4b11a70  use the correct import location, not six.*

a9b6965  Merge commit '4b11a706e1267b385099d8691b4468efd974f94d' of git://trac.sagemath.org/sage into t/27754/public/python3.8

comment:213 Changed 2 months ago by
 Status changed from needs_work to positive_review
comment:214 Changed 2 months ago by
 Dependencies changed from #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140 to #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140, #30351
comment:215 Changed 2 months ago by
 Commit changed from a9b69650667022f4a46b59ac90994254389931c1 to f1cda120e6c715ec6bdd6d8bb776b3b0aa921794
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
f1cda12  Fixup merge

comment:216 Changed 2 months ago by
 Status changed from needs_review to positive_review
comment:217 Changed 8 weeks ago by
 Priority changed from critical to blocker
comment:218 Changed 8 weeks ago by
FWIW, worked for MeToo(TM) :
Rebuilding 2.9.beta11
+#27754 on Debian sesting running on core i7 + 16 GB RAM gives results identical to those obtained on the same machine without #27754 (see sagerelease). And $sSAGE_LOCAL\bin\python*
are symlinks to \usr\bin\
...
comment:219 Changed 8 weeks ago by
it even works on Rasberry Pi 4 (armv7l).
comment:220 Changed 7 weeks ago by
 Branch changed from public/python3.8 to f1cda120e6c715ec6bdd6d8bb776b3b0aa921794
 Resolution set to fixed
 Status changed from positive_review to closed
comment:221 Changed 7 weeks ago by
 Commit f1cda120e6c715ec6bdd6d8bb776b3b0aa921794 deleted
Wonderful to have this in Sage 9.2.
Next episode:
 #30184: Support Python 3.9
(Python 3.9.rc2 is expected on 20200914 and Python 3.9 final on 20201005.)
If people want to start testing, Python 3.8.0a3 is out: