#27754 closed enhancement (fixed)
Upgrade: Python 3.8.5
Reported by:  Samuel Lelièvre  Owned by:  

Priority:  blocker  Milestone:  sage9.2 
Component:  packages: standard  Keywords:  days101 
Cc:  Frédéric Chapoton, Erik Bray, François Bissey, Jeroen Demeyer, Matthias Köppe, Samuel Lelièvre, Timo Kaufmann, Isuru Fernando, ghkliem, Travis Scrimshaw, Reimundo Heluani, Andrew  Merged in:  
Authors:  Jeroen Demeyer, John Palmieri, Matthias Koeppe, Antonio Rojas  Reviewers:  Matthias Koeppe, François Bissey 
Report Upstream:  Reported upstream. Developers acknowledge bug.  Work issues:  
Branch:  f1cda12 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140, #30351  Stopgaps: 
Description (last modified by )
This is to upgrade to Python 3.8.x.
Tarball: see checksums.ini
Previous and next Python upgrades:
See also:
 #30384 Adopt the “time windowbased” policy for support of Python versions from NEP 29
Attachments (1)
Change History (222)
comment:1 Changed 4 years ago by
comment:2 Changed 4 years ago by
I think it's a bit early for testing. I'd rather wait for the first beta, which is supposed to be released later this month.
comment:6 Changed 3 years ago by
Dependencies:  → #28012 

Keywords:  days101 added 
comment:8 Changed 3 years ago by
Dependencies:  #28012 → #27886 

comment:9 Changed 3 years ago by
Authors:  → Jeroen Demeyer 

Description:  modified (diff) 
Milestone:  sagewishlist → sage8.9 
Type:  task → enhancement 
comment:10 Changed 3 years ago by
Branch:  → u/jdemeyer/upgrade__python_3_8_x 

comment:11 Changed 3 years ago by
Commit:  → 46837b74871a41b8f79af5c9d477428b183dc2d7 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
46837b7  Python 3.8.0

comment:12 Changed 3 years ago by
Dependencies:  #27886 

Description:  modified (diff) 
Actually, thanks to two patches which are likely going to be accepted upstream, it no longer depends on the Cython upgrade.
comment:13 Changed 3 years ago by
Report Upstream:  N/A → Reported upstream. Developers acknowledge bug. 

comment:14 Changed 3 years ago by
There is a mysterious build failure of matplotlib:
Processing /home/jdemeyer/sagepython3/local/var/tmp/sage/build/matplotlib2.2.3.p0/src Created temporary directory: /tmp/pipreqbuildvq4okaxf Added file:///home/jdemeyer/sagepython3/local/var/tmp/sage/build/matplotlib2.2.3.p0/src to build tracker '/tmp/pipreqtrackerxcv165b_' Running setup.py (path:/tmp/pipreqbuildvq4okaxf/setup.py) egg_info for package from file:///home/jdemeyer/sagepython3/local/var/tmp/sage/build/matplotlib2.2.3.p0/src Running command python setup.py egg_info Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pipreqbuildvq4okaxf/setup.py", line 172, in <module> result = package.check() File "/tmp/pipreqbuildvq4okaxf/setupext.py", line 963, in check import numpy File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/__init__.py", line 142, in <module> from . import core File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/core/__init__.py", line 16, in <module> from . import multiarray File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/core/multiarray.py", line 12, in <module> from . import overrides File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/core/overrides.py", line 8, in <module> from numpy.compat._inspect import getargspec File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/compat/__init__.py", line 14, in <module> from . import py3k File "/home/jdemeyer/sagepython3/local/lib/python3.8/sitepackages/numpy/compat/py3k.py", line 15, in <module> from pathlib import Path, PurePath File "/home/jdemeyer/sagepython3/local/lib/python3.8/pathlib.py", line 391, in <module> class _NormalAccessor(_Accessor): File "/home/jdemeyer/sagepython3/local/lib/python3.8/pathlib.py", line 415, in _NormalAccessor link_to = os.link AttributeError: module 'os' has no attribute 'link'
But when running Python normally, os.link
does exist:
jdemeyer@sage4:~/sagepython3$ ./sage python3 Python 3.8.0b1 (default, Jun 19 2019, 16:54:26) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.link <builtin function link>
I have no clue how this could happen.
comment:15 followups: 29 64 Changed 3 years ago by
I tried this with OS X and ran into a problem with Pillow:
File "setup.py", line 316, in build_extensions _add_directory(library_dirs, sysroot+"/usr/lib") TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")
Python 3.7.3 does this:
Python 3.7.3 (default, Jun 17 2019, 15:38:22) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' >>>
The new Python does this:
Python 3.8.0b1 (default, Jun 19 2019, 11:09:54) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') >>>
comment:16 followup: 17 Changed 3 years ago by
I see the same problem with matplotlib. For laughs, I put a print(dir(os))
command right before the line link_to = os.link
. The output is identical to what I get when I do import os; dir(os)
from sage python3
, except it is missing link
.
If I run make k
, I also see problems building the Sage library, by the way: lots of errors of the form
RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable.
comment:17 followup: 18 Changed 3 years ago by
Replying to jhpalmieri:
I see the same problem with matplotlib. For laughs, I put a
print(dir(os))
command right before the linelink_to = os.link
. The output is identical to what I get when I doimport os; dir(os)
fromsage python3
, except it is missinglink
.
That makes no sense. Somebody is fooling with us...
Could you try also print(os)
? Maybe there is a different os
module shadowing the standard one or something...
Just in case, I checked that the problem unfortunately persists after make distclean
.
And a Google search for this error doesn't find anything.
comment:18 Changed 3 years ago by
I'm guessing the problem is coming from matplotlib
's setup.py:
import os try: del os.link except AttributeError: pass
comment:19 followup: 20 Changed 3 years ago by
My question is why it worked with Python 3.7.3 but not 3.8.0.
comment:20 Changed 3 years ago by
Replying to jhpalmieri:
My question is why it worked with Python 3.7.3 but not 3.8.0.
A different implementation of pathlib
: https://bugs.python.org/issue26978
comment:21 Changed 3 years ago by
Description:  modified (diff) 

I reported this to matplotlib: https://github.com/matplotlib/matplotlib/issues/14580
comment:22 Changed 3 years ago by
Dependencies:  → #28023 

Description:  modified (diff) 
comment:23 Changed 3 years ago by
Commit:  46837b74871a41b8f79af5c9d477428b183dc2d7 → cc78db22810ca605ed28235b45eeafe72418b269 

comment:24 Changed 3 years ago by
With matplotlib fixes, I'm getting these errors at runtime:
Exception ignored in: <sage.misc.weak_dict.WeakValueDictEraser object at 0x7f1bf60b5460> Traceback (most recent call last): File "sage/misc/weak_dict.pyx", line 219, in sage.misc.weak_dict.WeakValueDictEraser.__call__ (build/cythonized/sage/misc/weak_dict.c:1968) File "sage/cpython/dict_del_by_value.pyx", line 375, in sage.cpython.dict_del_by_value.del_dictitem_by_exact_value (build/cythonized/sage/cpython/dict_del_by_value.c:2311) File "sage/cpython/dict_del_by_value.pyx", line 294, in sage.cpython.dict_del_by_value.ensure_allows_deletions (build/cythonized/sage/cpython/dict_del_by_value.c:2025) AssertionError:
comment:25 Changed 3 years ago by
Dependencies:  #28023 → #28023, #28025 

comment:28 Changed 3 years ago by
comment:29 followup: 33 Changed 3 years ago by
Replying to jhpalmieri:
I tried this with OS X and ran into a problem with Pillow: Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")
This is actually added in a Sage patch. It's failing because of the removal of macos_no_include.patch
, which was added in #27631. I'll have to see if that patch is necessary anymore. If not, we need to unpatch Pillow.
comment:30 Changed 3 years ago by
I still gets lots of errors as in comment:16. The build process for sagelib
seems to go into an infinite loop. I just hit ctrlC, and there are about 2400 instances of
************************************************************************ Error building the Sage library ************************************************************************ Please email sagedevel (http://groups.google.com/group/sagedevel) explaining the problem and including the relevant part of the log file /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/logs/pkgs/sagelib8.8.rc2.log Describe your computer, operating system, etc.
in the log file, all connected with the same error message.
comment:31 followup: 32 Changed 3 years ago by
On which system are you seeing that? The build of the Sage library worked for me on Linux.
comment:32 Changed 3 years ago by
Replying to jdemeyer:
On which system are you seeing that? The build of the Sage library worked for me on Linux.
OS X, both 10.13 and 10.14.
comment:33 Changed 3 years ago by
Replying to jhpalmieri:
Replying to jhpalmieri:
I tried this with OS X and ran into a problem with Pillow: Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")This is actually added in a Sage patch. It's failing because of the removal of
macos_no_include.patch
, which was added in #27631. I'll have to see if that patch is necessary anymore. If not, we need to unpatch Pillow.
On an OS X 10.14 machine, one affected by #27631 (no /usr/include
), pip
builds just fine, so I think Python and pip are finding zlib, but Pillow does not.
If I keep the Pillow patch, it fails to build at all, with the error mentioned above. If I delete the Pillow patch and build, I see in its log file "The headers and library files could not be found for zlib".
Changed 3 years ago by
Attachment:  sagelib8.8.rc2.log added 

comment:34 Changed 3 years ago by
Here is the start of the log file. I've deleted more than 2000 repeated tracebacks.
comment:35 Changed 3 years ago by
I think the problem is this (from the 3.8 changelog):
on macOS, the spawn start method is now used by default in multiprocessing
The documentation for multiprocessing
then points to a bug report which discusses our old friend OBJC_DISABLE_INITIALIZE_FORK_SAFETY
(#25921). So if we get multiprocessing to work properly on OS X, maybe we can remove the line
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
from sageenv.
In the meantime, I don't know how to get it working properly; using fork
instead of spawn
in src/setup.py
is the only way I've found to proceed so far. I don't understand the distinctions in the multiprocessing documentation between fork
and spawn
to know what should be set differently to use spawn
correctly.
comment:36 Changed 3 years ago by
Using fork
instead of spawn
lets me build the Sage library:

src/setup.py
diff git a/src/setup.py b/src/setup.py index 42ba3a63da..5e2d65f9cc 100755
a b def execute_list_of_commands_in_parallel(command_list, nthreads): 493 493 progress = progress_fmt.format(i+1, N) 494 494 command_list[i] = command_list[i] + (progress,) 495 495 496 from multiprocessing import Pool 496 import multiprocessing as mp 497 mp.set_start_method('fork') 497 498 # map_async handles KeyboardInterrupt correctly if an argument is 498 499 # given to get(). Plain map() and apply_async() do not work 499 500 # correctly, see Trac #16113. 500 pool = Pool(nthreads)501 pool = mp.Pool(nthreads) 501 502 result = pool.map_async(apply_func_progress, command_list, 1).get(99999) 502 503 pool.close() 503 504 pool.join()
I made a similar change to allow docbuilding to work. Now I get failures with docbuilding. First, lots of warnings:
..../local/lib/python3.8/sitepackages/matplotlib/backends/backend_agg.py:531: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
Then an error:
Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__main__.py", line 2, in <module> main() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 1731, in main builder() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 352, in _wrapper getattr(get_builder(document), 'inventory')(*args, **kwds) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 548, in _wrapper build_many(build_ref_doc, L) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/sitepackages/sage_setup/docbuild/__init__.py", line 289, in build_many ret = x.get(99999) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage8.8.rc2/local/lib/python3.8/multiprocessing/pool.py", line 768, in get raise self._value OSError: WARNING: error while formatting arguments for sage.calculus.riemann.Riemann_Map.get_szego: bad operand type for unary : 'NoneType'
comment:37 Changed 3 years ago by
And of course I can't run any doctests because they use multiprocessing
, and I haven't figured out how to tell doctesting (forker.py in particular) to use fork
instead of spawn
. Sage starts and some basic things work. The Jupyter notebook is giving me problems.
comment:38 Changed 3 years ago by
I also didn't manage to build the docs because the same error. I know it's a problem with sage_getargspec
but I haven't checked further.
comment:39 Changed 3 years ago by
I don't have a good solution for the problem with Pillow (comment:33). I have been deleting the patch and force installing zlib
, which works for now.
comment:40 Changed 3 years ago by
For the record: for the spawn
vs. fork
issue with OS X, I have had to make changes in three files: src/setup.py
, src/sage_setup/docbuild/__init__.py
, and Cython/Build/Dependencies.py
. The changes are all pretty much the same; for example in docbuild/__init__.py
:

src/sage_setup/docbuild/__init__.py
diff git a/src/sage_setup/docbuild/__init__.py b/src/sage_setup/docbuild/__init__.py index 0f2700168a..6d1ff4e377 100644
a b if not (CYGWIN_VERSION and CYGWIN_VERSION[0] < 3): 279 279 # serialize this by taking the pool (and thus the call to fork()) out 280 280 # completely: The call to Sphinx leaks memory, so we need to build each 281 281 # document in its own process to control the RAM usage. 282 from multiprocessing import Pool 283 pool = Pool(NUM_THREADS, maxtasksperchild=1) 282 import multiprocessing 283 ctx = multiprocessing.get_context('fork') 284 pool = ctx.Pool(NUM_THREADS, maxtasksperchild=1) 284 285 # map_async handles KeyboardInterrupt correctly. Plain map and 285 286 # apply_async does not, so don't use it. 286 287 x = pool.map_async(target, args, 1)
comment:41 followup: 42 Changed 3 years ago by
I am also seeing the error "ImportError?: cannot import name 'clock' from 'time' (unknown location)". I don't think this is coming from the Sage library, but rather from other packages, and in particular brial and sympy may need to be changed.
$ grep R "from time import clock" local/lib/python3.8/ local/lib/python3.8//sitepackages/sympy/plotting/pygletplot/plot_window.py:from time import clock local/lib/python3.8//sitepackages/brial/interpolate.py:from time import clock local/lib/python3.8//sitepackages/future/backports/test/pystone.py:from time import clock local/lib/python3.8//sitepackages/zmq/backend/cffi/_poll.py: from time import clock as monotonic local/lib/python3.8//sitepackages/sage/coding/information_set_decoder.py: from time import clock as process_time
Changing the import to something like
try: from time import perf_counter as clock # or process_time instead of perf_counter? except ImportError: from time import clock
might work.
comment:42 Changed 3 years ago by
Replying to jhpalmieri:
I am also seeing the error "ImportError?: cannot import name 'clock' from 'time' (unknown location)". I don't think this is coming from the Sage library, but rather from other packages, and in particular brial and sympy may need to be changed.
$ grep R "from time import clock" local/lib/python3.8/ local/lib/python3.8//sitepackages/sympy/plotting/pygletplot/plot_window.py:from time import clock local/lib/python3.8//sitepackages/brial/interpolate.py:from time import clock local/lib/python3.8//sitepackages/future/backports/test/pystone.py:from time import clock local/lib/python3.8//sitepackages/zmq/backend/cffi/_poll.py: from time import clock as monotonic local/lib/python3.8//sitepackages/sage/coding/information_set_decoder.py: from time import clock as process_timeChanging the import to something like
try: from time import perf_counter as clock # or process_time instead of perf_counter? except ImportError: from time import clockmight work.
 The sympy issue has been fixed 16 days ago.
 I have just made a pull request for brial
 The zmq issue is already fixed in https://github.com/zeromq/pyzmq/blob/master/zmq/backend/cffi/_poll.py
 The last line (our own information_set_decoder.py) is already correctly using a try/except for this import.
comment:43 Changed 3 years ago by
Dependencies:  #28023, #28025 → #28023, #28025, #28116 

Description:  modified (diff) 
comment:44 Changed 3 years ago by
Dependencies:  #28023, #28025, #28116 → #28025 

Description:  modified (diff) 
comment:45 Changed 3 years ago by
Python 3.8 is getting closer, it would be great if somebody could review the dependency #28025.
comment:46 Changed 3 years ago by
Dependencies:  #28025 → #28025, #28428 

comment:47 Changed 3 years ago by
Description:  modified (diff) 

Python 3.8.0 is out. Added tarball link to ticket description.
comment:48 Changed 3 years ago by
Branch:  u/jdemeyer/upgrade__python_3_8_x → public/python3.8 

Commit:  cc78db22810ca605ed28235b45eeafe72418b269 → 193e9f49517a95e483af785d9879948468e9ed98 
Dependencies:  #28025, #28428 
Milestone:  sage8.9 → sage9.0 
New commits:
193e9f4  Python 3.8.0

comment:50 Changed 3 years ago by
It looks to me as though the two new patches were actually merged into the release, and so they are no longer needed. I'm not positive, though.
comment:51 Changed 3 years ago by
On OS X, the Pillow problem is still there. I tried upgrading to the most recent version of Pillow, but it did not help. I'm going to force installation of zlib
and proceed from there.
comment:52 Changed 3 years ago by
There is still a problem with a from time import clock
statement. I don't know which one is causing the problem, but I'm guessing either brial or sympy. Neither package has fixed it in a release, as far as I can tell, but I think it's been fixed upstream in both.
comment:53 Changed 3 years ago by
I am also, not surprisingly, having the same multiprocessing
problems on OS X as before. I can get the Sage library and the documentation to build, but I don't know how to get doctesting to work.
comment:54 Changed 3 years ago by
FWIW after updating to python 3.8 on Arch I'm getting test failures on 223 modules, so things don't look good. Most of them seem to stem from the same 34 issues though.
comment:55 followups: 72 77 Changed 3 years ago by
After taking a closer look, besides some string changes, most of the failures fall into one of the following categories:
(1) Hash changes  all hashes have changed in 3.8. This requires updating many tests, making them conditional on the python version. Not sure if it makes sense to fork so many tests at this point rather than just drop python2 support or at least not run those tests.
(2) Sorting changes  many lists are now output in a different order. Same comment as in (1) about the solution.
(3) "DeprecationWarning?: In future, it will be an error for 'np.bool_' scalars to be interpreted as an index" from numpy. These are fixed with https://github.com/matplotlib/matplotlib/commit/4651a125
(4) "DeprecationWarning?: PY_SSIZE_T_CLEAN will be required for '#' formats"  this happens mostly when pickling/unpickling functions. No idea where the actual code that throws the warning is.
(5) "DeprecationWarning?: an integer is required (got type sage.rings.real_mpfr.RealLiteral?). Implicit conversion to integers using int is deprecated, and may be removed in a future version of Python." when calling time.sleep with a Sage (sage.rings.real_mpfr.RealLiteral?) float. Coercing to float first fixes the issue.
(6) "SyntaxWarning?: "is" with a literal. Did you mean "=="?" when comparing strings with 'is'
(7) Use of the removed cgi.escape in sage/interacts/debugger.py. Should be changed to html.escape.
(8) The signature of code() (used in sage/misc/fpickle.pyx) has changed and now requires a 'posonlyargcount' argument.
(9) Mismatch after pickling/unpickling for some objects:
sage: A = Poset()._hasse_diagram.antichains() sage: A == loads(dumps(A)) False
For packagers that don't care about python2 compatibility (or 32 bit hashes) a WIP patch that fixes or works around most of these issues is at https://aur.archlinux.org/cgit/aur.git/tree/sagemathpython3.8.patch?h=sagemathgit. A backport of https://github.com/BRiAl/BRiAl/commit/74d86170 is also needed.
comment:56 Changed 3 years ago by
Cc:  Timo Kaufmann added 

comment:58 followup: 63 Changed 3 years ago by
It would be great if someone with access to OS X could look at this. I don't know how to solve all of the problems mentioned above. In particular:
 multiprocessing on OS X has a different default behavior than on linux (comment:35). As a result, the Sage library doesn't build and the docs don't build without some modification, and I don't know how to get doctesting to work.
 Pillow does not find the system
zlib
. Is there a better solution than forceinstalling Sage'szlib
?
comment:60 followup: 61 Changed 3 years ago by
Reports of issues with SageMath and Python 3.8:
 Sage error from module
typing
and python 3.8 https://ask.sagemath.org/question/49326
 Sage crash report: Manjaro, Python 3.8.1 https://trac.sagemath.org/ticket/28946
comment:61 Changed 3 years ago by
Replying to slelievre:
Reports of issues with SageMath and Python 3.8:
 Sage error from module
typing
and python 3.8 https://ask.sagemath.org/question/49326
 Sage crash report: Manjaro, Python 3.8.1 https://trac.sagemath.org/ticket/28946
None of these is a Sage issue: the first one is caused by the user mixing up distro and pip packages, the second one is a ntl soversion mismatch in Manjaro. The errors that still need to be figured out (besides the macOS problems) are (4), (5) and (9) in comment:55 above.
comment:62 Changed 3 years ago by
Milestone:  sage9.0 → sage9.2 

Sage 9.0 is out. I think this should wait until after 9.1 to give Sage with Python 3.7 a little more time to breathe, but of course this is debatable.
comment:63 Changed 3 years ago by
Replying to jhpalmieri:
 Pillow does not find the system
zlib
. Is there a better solution than forceinstalling Sage'szlib
?
I have created #29019 for the problems with Pillow because they also affect #27824.
comment:64 Changed 3 years ago by
Replying to jhpalmieri:
I tried this with OS X and ran into a problem with Pillow:
File "setup.py", line 316, in build_extensions _add_directory(library_dirs, sysroot+"/usr/lib") TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'Here is the problem code from Pillow's setup.py:
sysroot = sysconfig.get_config_var('Py_MACOS_SYSROOT') _add_directory(library_dirs, sysroot+"/usr/lib")Python 3.7.3 does this:
Python 3.7.3 (default, Jun 17 2019, 15:38:22) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' >>>The new Python does this:
Python 3.8.0b1 (default, Jun 19 2019, 11:09:54) [Clang 10.0.1 (clang1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_config_var('Py_MACOS_SYSROOT') >>>
As far as I can see, this config variable was invented in #27631 and is not available on any other Python than our patched Python.
comment:65 Changed 3 years ago by
indeed, Py_MACOS_SYSROOT
comes from #27631.
Note that #27631 followed a cpython patch we proposed on
https://bugs.python.org/issue36231
which was rejected in favour of
https://github.com/python/cpython/commit/c421c66a58a6caae30f0679d7e61411418e67cec
and so in 3.7.x with x>3 and in 3.y with y>7 the problem apparently is fixed, but I don't quite know how.
I don't have any access to a modern MacOS box with Xcode without sane /usr/incude
etc, so it's hard to experiment for me (I guess the patch for pillow in #27631 needs to be replaced with whatever hack they came up with for MacOS.
comment:66 followup: 67 Changed 3 years ago by
See comment:33 — I could not build Pillow (after removing Sage's patch) with Python 3.8. So I don't know if I believe that the problem has been fixed.
comment:67 followup: 69 Changed 3 years ago by
Replying to jhpalmieri:
See comment:33 — I could not build Pillow (after removing Sage's patch) with Python 3.8. So I don't know if I believe that the problem has been fixed.
Please try again with the latest commit in #29019.
comment:68 Changed 3 years ago by
Commit:  193e9f49517a95e483af785d9879948468e9ed98 → 9cf6f0ebf83c57384161fdc1594bc8156e1c49c3 

comment:69 Changed 3 years ago by
Replying to mkoeppe:
Replying to jhpalmieri:
See comment:33 — I could not build Pillow (after removing Sage's patch) with Python 3.8. So I don't know if I believe that the problem has been fixed.
Please try again with the latest commit in #29019.
That works for me. With this branch plus #29019, the build succeeds until I get to the Sage library, at which point the changes to multiprocessing start causing problems (as in comment:16, comment:35, comment:36).
comment:70 Changed 3 years ago by
See https://github.com/cython/cython/issues/3262 and https://github.com/condaforge/cythonfeedstock/pull/59 (?) for multiprocessing issues with Cython + Python 3.8 + OS X. One proposed solution there is to disable parallel processing on OS X (or more generally if multiprocessing.get_start_method() == 'spawn'
). Do we want to do that, or are we willing to use a patch like the proposal in comment:40, applied to Cython/Build/Dependencies.py
?
comment:71 Changed 3 years ago by
And by the way, even with the patches as in comment:40, I still can't get the documentation to build. It ends with
[dochtml] ForkingPickler(file, protocol).dump(obj) [dochtml] AttributeError: Can't pickle local object 'build_many.<locals>.run_worker'
comment:72 followup: 73 Changed 3 years ago by
Replying to arojas: [...]
For packagers that don't care about python2 compatibility (or 32 bit hashes) a WIP patch that fixes or works around most of these issues is at https://aur.archlinux.org/cgit/aur.git/tree/sagemathpython3.8.patch?h=sagemathgit. A backport of https://github.com/BRiAl/BRiAl/commit/74d86170 is also needed.
@arojas, I'm trying to update sage on nix to 9.0 with python 3.8. I'm using your 3.8 patch in the 9.0 version [1]. I'm still seeing some test failures caused by the python update though:
sage t long src/sage/categories/homset.py # 3 doctests failed sage t long src/sage/combinat/cluster_algebra_quiver/quiver_mutation_type.py # 1 doctest failed sage t long src/sage/combinat/finite_state_machine.py # 1 doctest failed sage t long src/sage/combinat/posets/hasse_diagram.py # 2 doctests failed sage t long src/sage/groups/cubic_braid.py # 1 doctest failed sage t long src/sage/homology/simplicial_complex_morphism.py # 1 doctest failed sage t long src/sage/structure/sequence.py # 2 doctests failed
Here[2] is the full log. You said that your patch fixes "most" failures, so is that to be expected? Do you have the same failures on arch?
[1] https://aur.archlinux.org/cgit/aur.git/plain/sagemathpython3.8.patch?h=sagemathgit&id=138b3db82d8d13f9c66bf8d7515aa37671710e0a [2] https://gist.github.com/timokau/13c97bf8f45162f3d4155999fc5bec13
comment:73 Changed 3 years ago by
Replying to ghtimokau:
Here[2] is the full log. You said that your patch fixes "most" failures, so is that to be expected? Do you have the same failures on arch?
Yes, I have the same failures (the one in cubic_braid no longer happens in the latest beta). They are mostly related to pickling, which I don't understand.
comment:74 Changed 3 years ago by
Thanks!
I had a look at the python 3.8 changelog, the most obvious change seemed to be the change of the default pickle protocol (3 > 4). But I have patched python to revert that change and that did not fix the issue.
I'll look into gitbisect'ing the cpython update, though that is of course not exactly trivial. Maybe it will help.
comment:75 Changed 3 years ago by
there is a change of API in the ast, so this breaks old versions of ipython with python 3.8. In particular, it breaks ipython 5.8 so you cannot run sage under jupyter in python 3.8 so far.
comment:76 Changed 3 years ago by
Dependencies:  → #28197 

I've added as a dependency: #28197  upgrade to ipython 7
comment:77 Changed 3 years ago by
Replying to arojas:
For packagers that don't care about python2 compatibility (or 32 bit hashes) a WIP patch that fixes or works around most of these issues is at https://aur.archlinux.org/cgit/aur.git/tree/sagemathpython3.8.patch?h=sagemathgit.
Thanks for the patch. A version of this patch including the 32 bit hashes can now be found at https://salsa.debian.org/scienceteam/sagemath//blob/master/debian/patches/u0versionpython3.8.patch.
Unfortunately many of the ordering issues addressed in this patch also depend on 32/64 bit now. See 32bit build log at https://buildd.debian.org/status/fetch.php?pkg=sagemath&arch=i386&ver=9.03&stamp=1586083972&raw=0.
comment:78 Changed 3 years ago by
Dependencies:  #28197 → #28197, #29042 

comment:79 Changed 2 years ago by
Cc:  Isuru Fernando added 

comment:81 Changed 2 years ago by
I am seeing negative progress: I can't even get the Sage library to build on OS X anymore: I just see errors as in comment:16. I assume the problem is parallel building with spawn
instead of fork
, but I can't figure out where to change things to get it to work.
For the record, I tried this with Python 3.8.4, not 3.8.0. I'll try with 3.8.0, but I don't expect any difference. I tried this but it didn't help:

src/sage_setup/run_parallel.py
diff git a/src/sage_setup/run_parallel.py b/src/sage_setup/run_parallel.py index 7a9b787b4e..a4cf73801d 100644
a b def execute_list_of_commands_in_parallel(command_list, nthreads): 73 73 progress = progress_fmt.format(i+1, N) 74 74 command_list[i] = command_list[i] + (progress,) 75 75 76 from multiprocessing import Pool 76 import multiprocessing 77 ctx = multiprocessing.get_context('fork') 77 78 # map_async handles KeyboardInterrupt correctly if an argument is 78 79 # given to get(). Plain map() and apply_async() do not work 79 80 # correctly, see Trac #16113. 80 pool = Pool(nthreads)81 pool = ctx.Pool(nthreads, maxtasksperchild=1) 81 82 result = pool.map_async(apply_func_progress, command_list, 1).get(99999) 82 83 pool.close() 83 84 pool.join()
comment:82 Changed 2 years ago by
Commit:  9cf6f0ebf83c57384161fdc1594bc8156e1c49c3 → 9688a9c6064603725c6aca4278f7098b758743c9 

comment:83 Changed 2 years ago by
Description:  modified (diff) 

Summary:  Upgrade: Python 3.8.x → Upgrade: Python 3.8.4 
comment:86 Changed 2 years ago by
Commit:  9688a9c6064603725c6aca4278f7098b758743c9 → c44c448acfb4152e981d91ec02b4eb6ae34c57a7 

comment:87 Changed 2 years ago by
Not sure if the Cython update (from #29861) was necessary. But 0.29.21 does have a change specifically for 3.8.
comment:89 Changed 2 years ago by
Right, thanks, I forgot to deal with Cython. I can now build sagelib. Also the docs, with the following changes:

src/sage_setup/docbuild/utils.py
diff git a/src/sage_setup/docbuild/utils.py b/src/sage_setup/docbuild/utils.py index 272f7e2d0f..2691339ea6 100644
a b def build_many(target, args, processes=None): 112 112 ... 113 113 WorkerDiedException: worker for 4 died with nonzero exit code 9 114 114 """ 115 from multiprocessing import Process, Queue, cpu_count 115 import multiprocessing 116 ctx = multiprocessing.get_context('fork') 117 Process = ctx.Process 118 Queue = ctx.Queue 116 119 from six.moves.queue import Empty 117 120 118 121 if processes is None: 119 processes = cpu_count()122 processes = multiprocessing.cpu_count() 120 123 121 124 workers = [None] * processes 122 125 tasks = enumerate(args)
Doctesting doesn't work, though; first, it failed completely, and then I tried to patch a few files in src/sage/doctest
, but I was not successful. (I got some tests to run, but there were doctest failures, and then the process just quit.)
comment:90 Changed 2 years ago by
According to https://docs.python.org/3/library/multiprocessing.html#multiprocessing.get_context,
get_context
is available for python >= 3.4. We support python >= 3.6, so I would guess it is safe to make this change to utils.py
. Could you push it to the branch please?
comment:91 Changed 2 years ago by
Commit:  c44c448acfb4152e981d91ec02b4eb6ae34c57a7 → 6a8c543c70ee90b5ad332ace5aebf5e498fb1444 

Branch pushed to git repo; I updated commit sha1. New commits:
6a8c543  trac 27754: docbuilding on OS X

comment:92 Changed 2 years ago by
This builds sagelib
plus the documentation for me on, either using the system Python or Python 3.8.4.
comment:93 Changed 2 years ago by
Dependencies:  #28197, #29042 → #28197, #29042, #29861 

comment:94 Changed 2 years ago by
Thanks. Do we know if the problem with doctesting is specific to macOS?
comment:95 Changed 2 years ago by
With this branch, everything builds on an Ubuntu virtual machine. I didn't run doctests.
I also didn't know about the
multiprocessing.set_start_method('fork', force=True)
command. With this change, doctests on OS X run, although there are a lot of failures:

src/sage/doctest/external.py
diff git a/src/sage/doctest/external.py b/src/sage/doctest/external.py index b0db07b47a..af49ca29e5 100644
a b AUTHORS: 25 25 # http://www.gnu.org/licenses/ 26 26 #***************************************************************************** 27 27 28 from multiprocessing import Array 28 import multiprocessing 29 import os 30 if os.uname().sysname == 'Darwin': 31 multiprocessing.set_start_method('fork', force=True) 32 Array = multiprocessing.Array 29 33 30 34 import urllib.error 31 35 from urllib.request import Request, urlopen 
src/sage/doctest/forker.py
diff git a/src/sage/doctest/forker.py b/src/sage/doctest/forker.py index 75559667d1..8f5f2bff2b 100644
a b from sage.repl.user_globals import set_globals 63 63 from sage.cpython.atexit import restore_atexit 64 64 from sage.cpython.string import bytes_to_str, str_to_bytes 65 65 66 if os.uname().sysname == 'Darwin': 67 multiprocessing.set_start_method('fork', force=True) 66 68 67 69 # All doctests run as if the following future imports are present 68 70 import __future__
comment:97 Changed 2 years ago by
Without the changes from comment 95, I get the following error:
$ ./sage t src/sage/geometry/polyhedron/base.py too many failed tests, not using stored timings Running doctests with ID 202007151942599c236f91. Git branch: bigupdate Using optional=build,cryptominisat,dochtml,sage Doctesting 1 file.  Doctests interrupted: 0/1 files tested  Total time for all tests: 0.0 seconds cpu time: 0.0 seconds cumulative wall time: 0.0 seconds Traceback (most recent call last): File "/Users/mkoeppe/s/sage/sagerebasing/src/bin/sageruntests", line 177, in <module> err = DC.run() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 1207, in run self.run_doctests() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 908, in run_doctests self.dispatcher.dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2039, in dispatch self.parallel_dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 1934, in parallel_dispatch w.start() # This might take some time File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2206, in start super(DocTestWorker, self).start() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'DocTestDispatcher.parallel_dispatch.<locals>.sel_exit'
Is this the same error that you were getting?
comment:98 Changed 2 years ago by
Commit:  6a8c543c70ee90b5ad332ace5aebf5e498fb1444 → 5a5fa82344129e34f03b62ec2c3bb0a72de0d808 

Branch pushed to git repo; I updated commit sha1. New commits:
5a5fa82  trac 27754: on OS X, use 'fork' with multiprocessing when doctesting

comment:99 Changed 2 years ago by
Yes, that's the error I was getting. Now I'm getting the same doctest failures on OS X as on the Ubuntu virtual machine (approximately: I think I get 113 files with failures on OS X, 114 with failures on Ubuntu).
comment:100 Changed 2 years ago by
I tried make ptest
, and it ended with:
sage t src/sage/calculus/riemann.pyx # 18 doctests failed sage t src/sage/functions/log.py # 2 doctests failed sage t src/sage/graphs/graph_plot.py # 4 doctests failed Doctests interrupted: 344/4127 files tested File "/Users/mkoeppe/s/sage/sagerebasing/src/bin/sageruntests", line 177, in <module> err = DC.run() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 1207, in run self.run_doctests() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/control.py", line 908, in run_doctests self.dispatcher.dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2044, in dispatch self.parallel_dispatch() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 1939, in parallel_dispatch w.start() # This might take some time File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 2211, in start super(DocTestWorker, self).start() File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.8/multiprocessing/popen_fork.py", line 69, in _launch child_r, parent_w = os.pipe() OSError: [Errno 24] Too many open files make: *** [ptest] Error 1
comment:101 Changed 2 years ago by
make ptestlong
completed for me (with make j6
). I see several types of failures:
 sorting issues:
File "src/sage/dynamics/arithmetic_dynamics/projective_ds.py", line 3148, in sage.dynamics.arithmetic_dynamics.projective_ds.?.automorphism_group Failed example: D6.automorphism_group() Expected: [ [1 0] [0 w] [0 1] [w 0] [w  1 0] [ 0 w  1] [0 1], [1 0], [1 0], [0 1], [ 0 1], [ 1 0] ] Got: [ [1 0] [0 1] [ 0 w  1] [w 0] [w  1 0] [0 w] [0 1], [1 0], [ 1 0], [0 1], [ 0 1], [1 0] ]
 hashes have changed:
File "src/sage/combinat/similarity_class_type.py", line 411, in sage.combinat.similarity_class_type.?.__hash__ Failed example: hash(PT1) Expected: 5050909583595644741 Got: 925386691174542829
 pickling?
File "src/sage/manifolds/differentiable/affine_connection.py", line 1785, in sage.manifolds.differentiable.affine_connection.AffineConnection.riemann Failed example: for i in M.irange(): for j in M.irange(): for k in M.irange(): nab.add_coef(eV)[i,j,k] = nab.coef(eVW)[i,j,k,c_uvW].expr() Exception raised: multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "sage/misc/fpickle.pyx", line 99, in sage.misc.fpickle.call_pickled_function (build/cythonized/sage/misc/fpickle.c:2279) f = eval("unpickle_function(fp)", sage.all.__dict__, {'fp':fp}) File "<string>", line 1, in <module> File "sage/misc/fpickle.pyx", line 90, in sage.misc.fpickle.unpickle_function (build/cythonized/sage/misc/fpickle.c:1975) recovered = pickle.loads(pickled) File "sage/misc/fpickle.pyx", line 26, in sage.misc.fpickle.code_ctor (build/cythonized/sage/misc/fpickle.c:1536) return types.CodeType(*args) TypeError: code() takes at least 14 arguments (13 given) """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 709, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/doctest/forker.py", line 1133, in compile_and_execute exec(compiled, globs) File "<doctest sage.manifolds.differentiable.affine_connection.AffineConnection.riemann[30]>", line 4, in <module> nab.add_coef(eV)[i,j,k] = nab.coef(eVW)[i,j,k,c_uvW].expr() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/manifolds/differentiable/affine_connection.py", line 663, in coef nab_evi = self(ev[i]) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/manifolds/differentiable/affine_connection.py", line 1386, in __call__ return self._derive_paral(tensor_r) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/manifolds/differentiable/affine_connection.py", line 1508, in _derive_paral for ii,val in make_CovDerivative(listParalInput): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/sitepackages/sage/parallel/multiprocessing_sage.py", line 77, in parallel_iter for res in result: File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage9.2.beta5/local/lib/python3.8/multiprocessing/pool.py", line 868, in next raise value TypeError: code() takes at least 14 arguments (13 given)
 a deprecation warning:
File "src/sage/plot/plot3d/base.pyx", line 1560, in sage.plot.plot3d.base.Graphics3d.save_image Failed example: G.save_image(gif) Expected nothing Got: doctest:warning ... DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
 another one:
File "src/sage/doctest/forker.py", line 2351, in sage.doctest.forker.DocTestWorker.kill Failed example: time.sleep(0.2) # Worker doesn't die Expected nothing Got: doctest:warning ... DeprecationWarning: an integer is required (got type sage.rings.real_mpfr.RealLiteral). Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
I haven't gone through all of the failures yet, nor have I pored over the Python 3.8 changelog to try to figure any of these out. The first few types are easy to fix, I hope.
comment:102 Changed 2 years ago by
comment 77 above already mentions the hashing/ordering issues. I haven't looked at these patches
comment:103 Changed 2 years ago by
If doctests need patching because of sorting order changes, I hope we can do better than what I see in https://salsa.debian.org/scienceteam/sagemath//blob/master/debian/patches/u0versionpython3.8.patch
Many of such tests should in my opinion be fixed by using explicitly sorted
when comparing lists whose order has no mathematical meaning
comment:104 Changed 2 years ago by
In this one from the Debian patch, I'm wondering whether printing of Set
in doctests should be modified similar to what was done in #29042 for dict

sage/src/sage/schemes/elliptic_curves/ell_point.py
a b 822 822 sage: all(T.is_divisible_by(3) for T in tor) 823 823 True 824 824 sage: Set([T for T in tor if T.is_divisible_by(2)]) 825 {( 0 : 1 : 0), (1 : 0 : 1)}825 {(1 : 0 : 1), (0 : 1 : 0)} 826 826 sage: Set([2*T for T in tor]) 827 {( 0 : 1 : 0), (1 : 0 : 1)}827 {(1 : 0 : 1), (0 : 1 : 0)} 828 828
comment:105 Changed 2 years ago by
Cc:  ghkliem added 

comment:106 Changed 2 years ago by
For these ones from the Debian patch I have created #30162:
 a/sage/src/sage/symbolic/callable.py +++ b/sage/src/sage/symbolic/callable.py @@ 38,27 +38,27 @@ sage: f(1)=2 Traceback (most recent call last): ...  SyntaxError: can't assign to function call + SyntaxError: cannot assign to function call
comment:107 Changed 2 years ago by
Cc:  Travis Scrimshaw added 

For this one from the Debian patch, I also wonder whether some methods should be changed so that they return generators rather than lists  for better py3 style.
 a/sage/src/sage/categories/coxeter_groups.py +++ b/sage/src/sage/categories/coxeter_groups.py @@ 283,7 +283,7 @@ [[0, 1, 2, 1], [0, 2, 1, 2], [2, 0, 1, 2]] sage: W.braid_orbit([2,1,1,2,1])  [[2, 2, 1, 2, 2], [2, 1, 1, 2, 1], [1, 2, 1, 1, 2], [2, 1, 2, 1, 2]] + [[2, 1, 1, 2, 1], [1, 2, 1, 1, 2], [2, 2, 1, 2, 2], [2, 1, 2, 1, 2]] sage: W = ReflectionGroup(['A',3], index_set=["AA","BB",5]) # optional  gap3 sage: w = W.long_element() # optional  gap3
comment:108 followup: 109 Changed 2 years ago by
That is a big Sagewide issue/question: should things be iterators/generators or lists. Some places we do both (e.g., in graphs), in others we only have the lists, and even still some places only do iterators.
The simple fix here to make everything consistent is just to sort the output (as was done in the test before) as this should be considered more as a set than an enumerated list IIRC.
comment:109 Changed 2 years ago by
Replying to tscrim:
The simple fix here to make everything consistent is just to sort the output (as was done in the test before) as this should be considered more as a set than an enumerated list IIRC.
Sure, this is what we should do for this ticket.
comment:110 Changed 2 years ago by
Dependencies:  #28197, #29042, #29861 → #28197, #29042, #29861, #30162 

comment:111 Changed 2 years ago by
Authors:  Jeroen Demeyer → Jeroen Demeyer, John Palmieri, Matthias Koeppe 

comment:112 followup: 113 Changed 2 years ago by
I don't understand the point of checking specific hash values. For example, in src/sage/combinat/similarity_class_type.py
, the hashes depend on how Python determines hash(tuple([3,2,1]))
, for example. Since I think that is an implementation detail of Python, not Sage, doctesting for a specific value is the opposite of robust.
comment:113 Changed 2 years ago by
Replying to jhpalmieri:
I don't understand the point of checking specific hash values. For example, in
src/sage/combinat/similarity_class_type.py
, the hashes depend on how Python determineshash(tuple([3,2,1]))
, for example. Since I think that is an implementation detail of Python, not Sage, doctesting for a specific value is the opposite of robust.
My guess is that these tests are there to guard against thoughtless or accidental changes to hash functions.
I could imagine that we may want to change these tests to something like this:
sage: hash(x) in (5050909583595644741, 925386691174542829, 234567876545678) True
comment:114 Changed 2 years ago by
Commit:  5a5fa82344129e34f03b62ec2c3bb0a72de0d808 → 3d6ae39a62049a0a02e71f56c9cdd970407147a4 

Branch pushed to git repo; I updated commit sha1. New commits:
3d6ae39  trac 27754: fix some doctests for Python 3.8

comment:115 Changed 2 years ago by
Here are "fixes" for some doctests. Some are probably not ideal. This is also far from complete: I didn't have the energy to do more, and I don't know how to handle some of the failures. But maybe this is a start.
comment:116 Changed 2 years ago by
Work issues:  → merge #30162 

comment:117 Changed 2 years ago by
For similarity_class_type.py
, back when this was written IIRC, there was less decided about how to test a __hash__
method and less concern about how changes in Python would affect this. So I think the changed tests here are good.
comment:118 Changed 2 years ago by
Description:  modified (diff) 

Python 3.8.5 was released as a security hotfix.
See #30184 for Python 3.9.x.
comment:119 followups: 120 126 Changed 2 years ago by
I'm not going to be able to work on this for a few days. If someone else wants to work on doctests, please feel free. Do we have a Pillowupgrade ticket? I wonder if that would help with some of the tests. And #30176 (matplotlib upgrade) might help, too.
Should this be delayed to Sage 9.3, ideally early in the release cycle?
comment:120 Changed 2 years ago by
comment:121 Changed 2 years ago by
Cc:  Reimundo Heluani added 

comment:122 Changed 2 years ago by
Commit:  3d6ae39a62049a0a02e71f56c9cdd970407147a4 → ccc461e59d246b87a99d8b3426ecd419bd8484ab 

Branch pushed to git repo; I updated commit sha1. New commits:
8a43db6  build/pkgs/python3: Update to 3.8.5

60a8922  In doctests, accept both can't and cannot in error messages

71da37b  Fixup

2727aa8  src/sage/combinat/words/word.py: More wildcarding for respelled error messages

c43343c  Even more wildcarding for respelled error messages; remove some py2only tests

1a7bc97  Merge branch 'u/mkoeppe/make_doctests_accept_python_3_8_x_respellings_of_some_error_messages' of git://trac.sagemath.org/sage into t/27754/public/python3.8

ccc461e  Merge branch 'public/python3.8' of git://trac.sagemath.org/sage into t/27754/public/python3.8

comment:123 Changed 2 years ago by
Summary:  Upgrade: Python 3.8.4 → Upgrade: Python 3.8.5 

Work issues:  merge #30162 
comment:124 Changed 2 years ago by
Description:  modified (diff) 

comment:125 Changed 2 years ago by
Milestone:  sage9.2 → sage9.3 

comment:126 Changed 2 years ago by
Replying to jhpalmieri:
Should this be delayed to Sage 9.3, ideally early in the release cycle?
Yes, I think so. Let's do #30210 (Upgrade Python to 3.7.8) for 9.2
comment:127 Changed 2 years ago by
There are lots of failures caused by deprecation warnings on unpickling
DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
that come from pynac. These are fixed with https://github.com/pynac/pynac/pull/354, but pynac development seems stalled unfortunately
comment:128 followup: 130 Changed 2 years ago by
Could you please create a sage ticket for this pynac patch?
comment:129 Changed 2 years ago by
Commit:  ccc461e59d246b87a99d8b3426ecd419bd8484ab → b1fc3553e32333143adb763e74728b8d45d99f2e 

Branch pushed to git repo; I updated commit sha1. New commits:
f243937  Merge remotetracking branch 'origin/develop' into t/27754/public/python3.8

73539f6  Make tests in categories/finite_enumerated_sets.py pass with Python 3.8

0c893db  Fix 'SyntaxWarning: "is" with a literal. Did you mean "=="?' with Python 3.8

b1fc355  Make tests in combinat/words/word_infinite_datatypes.py compatible with Python 3.8

comment:130 Changed 2 years ago by
Dependencies:  #28197, #29042, #29861, #30162 → #28197, #29042, #29861, #30162, #30225 

comment:131 Changed 2 years ago by
Commit:  b1fc3553e32333143adb763e74728b8d45d99f2e → 6eb6a473d81c6bc3d04a3432f78ce3d8bfa192be 

Branch pushed to git repo; I updated commit sha1. New commits:
6eb6a47  More sorted()

comment:132 Changed 2 years ago by
Commit:  6eb6a473d81c6bc3d04a3432f78ce3d8bfa192be → 583042aaac9c36e11e76a4c9accf95188ec9d2d5 

Branch pushed to git repo; I updated commit sha1. New commits:
583042a  Update compile flags for python 3.8

comment:133 Changed 2 years ago by
Commit:  583042aaac9c36e11e76a4c9accf95188ec9d2d5 → d9ca5c5715da4b6c78adc66095eeaca7c4a26c96 

comment:134 Changed 2 years ago by
Authors:  Jeroen Demeyer, John Palmieri, Matthias Koeppe → Jeroen Demeyer, John Palmieri, Matthias Koeppe, Antonio Rojas 

comment:135 Changed 2 years ago by
Commit:  d9ca5c5715da4b6c78adc66095eeaca7c4a26c96 → 86d5e94dc2ce2f0b674b275e039cecaaa7d0e130 

Branch pushed to git repo; I updated commit sha1. New commits:
86d5e94  Rewrite some hash tests to be independent of the actual hashes

comment:136 Changed 2 years ago by
Commit:  86d5e94dc2ce2f0b674b275e039cecaaa7d0e130 → be98eb0a2721506f55a1a9dd30a647d006018e01 

Branch pushed to git repo; I updated commit sha1. New commits:
be98eb0  Rewrite direct tests for hashes

comment:137 Changed 2 years ago by
Commit:  be98eb0a2721506f55a1a9dd30a647d006018e01 → 74f9352dccdd1d7e4f2c21da22b6625b88362884 

comment:138 Changed 2 years ago by
Commit:  74f9352dccdd1d7e4f2c21da22b6625b88362884 → a736d791ddeeea6dbc2e53e2c34a580dda32127b 

comment:139 Changed 2 years ago by
Commit:  a736d791ddeeea6dbc2e53e2c34a580dda32127b → f384065cbf40774888040b1180ed0320add546ad 

Branch pushed to git repo; I updated commit sha1. New commits:
f384065  Make computing graph faces deterministic

comment:140 Changed 2 years ago by
Commit:  f384065cbf40774888040b1180ed0320add546ad → 694c6b153eb1b2e69aa5345f5108ceeb6acfb1d9 

comment:141 Changed 2 years ago by
Commit:  694c6b153eb1b2e69aa5345f5108ceeb6acfb1d9 → 14c57e57275bdedce87b159d4cfe9658e94ec555 

comment:142 Changed 2 years ago by
Commit:  14c57e57275bdedce87b159d4cfe9658e94ec555 → cd346fa5bf72e14338a944626aa6fc3e8fe2833c 

comment:143 Changed 2 years ago by
Commit:  cd346fa5bf72e14338a944626aa6fc3e8fe2833c → 093e9d61181d5ae221bc7d20e4b408b7efb251e8 

comment:144 Changed 2 years ago by
Commit:  093e9d61181d5ae221bc7d20e4b408b7efb251e8 → 73d6c7f47c6a203da2cca58f56e68df6fc3e6852 

comment:145 Changed 2 years ago by
Just a handful of issues left, mostly pickling related
********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/matroids/linear_matroid.pyx", line 3674, in sage.matroids.linear_matroid.BinaryMatroid._projection_partition Failed example: N._projection_partition() Expected: 2 x 12 BinaryMatrix [110011001100] [001100110011] Got: 2 x 12 BinaryMatrix [001100110011] [110011001100] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/structure/sequence.py", line 824, in sage.structure.sequence.Sequence_generic.__copy__ Failed example: t.is_immutable == s.is_immutable Expected: True Got: False ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/structure/sequence.py", line 826, in sage.structure.sequence.Sequence_generic.__copy__ Failed example: t.is_mutable == s.is_mutable Expected: True Got: False ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/posets/hasse_diagram.py", line 2185, in sage.combinat.posets.hasse_diagram.HasseDiagram.antichains Failed example: TestSuite(A).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: An enumerated set with a forest structure != An enumerated set with a forest structure  The following tests failed: _test_pickling ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/posets/hasse_diagram.py", line 2190, in sage.combinat.posets.hasse_diagram.HasseDiagram.antichains Failed example: TestSuite(A).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: An enumerated set with a forest structure != An enumerated set with a forest structure  The following tests failed: _test_pickling ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/cluster_algebra_quiver/quiver_mutation_type.py", line 2171, in sage.combinat.cluster_algebra_quiver.quiver_mutation_type._construct_exceptional_mutation_classes Failed example: for mut_class in sorted(rank_3_exceptional.keys(), key=str): # long time print("{} {}".format(mut_class, rank_3_exceptional[mut_class])) Expected: ('G', 2, 1) [('BH?', (((1, 2), (1, 3)),)), ('BGO', (((2, 1), (3, 1)),)), ('BW?', (((0, 1), (3, 1)),)), ('BP?', (((0, 1), (1, 3)),)), ('BP_', (((0, 1), (1, 3)), ((2, 0), (3, 1)))), ('BP_', (((0, 1), (3, 1)), ((1, 2), (1, 3)), ((2, 0), (2, 2))))] ('G', 2, 1) [('BH?', (((1, 2), (3, 1)),)), ('BGO', (((2, 1), (1, 3)),)), ('BW?', (((0, 1), (1, 3)),)), ('BP?', (((0, 1), (3, 1)),)), ('BKO', (((1, 0), (3, 1)), ((2, 1), (1, 3)))), ('BP_', (((0, 1), (2, 2)), ((1, 2), (1, 3)), ((2, 0), (3, 1))))] Got: ('G', 2, 1) [('BH?', (((1, 2), (1, 3)),)), ('BGO', (((2, 1), (3, 1)),)), ('BW?', (((0, 1), (3, 1)),)), ('BP?', (((0, 1), (1, 3)),)), ('BP_', (((0, 1), (1, 3)), ((2, 0), (3, 1)))), ('BP_', (((0, 1), (1, 3)), ((1, 2), (2, 2)), ((2, 0), (3, 1))))] ('G', 2, 1) [('BH?', (((1, 2), (3, 1)),)), ('BGO', (((2, 1), (1, 3)),)), ('BW?', (((0, 1), (1, 3)),)), ('BP?', (((0, 1), (3, 1)),)), ('BKO', (((1, 0), (3, 1)), ((2, 1), (1, 3)))), ('BP_', (((0, 1), (3, 1)), ((1, 2), (2, 2)), ((2, 0), (1, 3)))), ('BP_', (((0, 1), (2, 2)), ((1, 2), (1, 3)), ((2, 0), (3, 1))))] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/combinat/cluster_algebra_quiver/mutation_type.py", line 1348, in sage.combinat.cluster_algebra_quiver.mutation_type._mutation_type_test Failed example: _mutation_type_test(3) # long time Expected: True ('A', (2, 1), 1) True ('A', 3) True ('B', 3) True ('BB', 2, 1) True ('BC', 2, 1) True ('C', 3) True ('CC', 2, 1) True ('G', 2, 1) True ('G', 2, 1) Got: True ('A', (2, 1), 1) True ('A', 3) True ('B', 3) True ('BB', 2, 1) True ('BC', 2, 1) True ('C', 3) True ('CC', 2, 1) False ('G', 2, 1) False ('G', 2, 1) ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/schemes/riemann_surfaces/riemann_surface.py", line 355, in sage.schemes.riemann_surfaces.riemann_surface.RiemannSurface Failed example: S.riemann_matrix() #abs tol 0.00000001 Expected: [0.500000000000000000000000... + 0.866025403784438646763723...*I] Got: [0.50000000000000000000000000000 + 0.86602540378443864676372317075*I] Tolerance exceeded in 1 of 2: 0.500000000000000000000000 vs 0.50000000000000000000000000000, tolerance 1e0 > 1e8 ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/homology/simplicial_complex_morphism.py", line 82, in sage.homology.simplicial_complex_morphism Failed example: z # this is the mapping path space Expected: Simplicial complex morphism: From: Simplicial complex with 6 vertices and 4 facets To: Minimal triangulation of the 2sphere Defn: ['L0R(0, 0)', 'L0R(0, 1)', 'L1R(1, 0)', 'L1R(1, 1)', 'L2R(2, 0)', 'L2R(2, 1)'] > [0, 0, 1, 1, 2, 2] Got: Simplicial complex morphism: From: Simplicial complex with 6 vertices and 5 facets To: Minimal triangulation of the 2sphere Defn: ['L0R(0, 0)', 'L0R(0, 1)', 'L1R(1, 0)', 'L1R(1, 1)', 'L2R(2, 0)', 'L2R(2, 1)'] > [0, 0, 1, 1, 2, 2] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/categories/homset.py", line 598, in sage.categories.homset.Homset.__init__ Failed example: TestSuite(H).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: Generic morphism: From: X To: Y != Generic morphism: From: X To: Y  The following tests failed: _test_pickling Failure in _test_elements The following tests failed: _test_elements ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/categories/homset.py", line 607, in sage.categories.homset.Homset.__init__ Failed example: TestSuite(H).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: Generic morphism: From: X To: Y != Generic morphism: From: X To: Y  The following tests failed: _test_pickling Failure in _test_elements The following tests failed: _test_elements ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/categories/homset.py", line 1253, in sage.categories.homset.HomsetWithBase.__init__ Failed example: TestSuite(H).run() Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/usr/lib/python3.8/sitepackages/sage/misc/sage_unittest.py", line 296, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 639, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:4953) tester.assertEqual(loads(dumps(self)), self) File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: Generic morphism: From: X To: Y != Generic morphism: From: X To: Y  The following tests failed: _test_pickling Failure in _test_elements The following tests failed: _test_elements
comment:146 Changed 2 years ago by
Commit:  73d6c7f47c6a203da2cca58f56e68df6fc3e6852 → ce0f0e0404b23485c7c8b2e38e4c8a36eb1354e1 

Branch pushed to git repo; I updated commit sha1. New commits:
687d070  Make vertices tuple of a simplicial complex deterministic

46114aa  Make test in matroids/linear_matroid.pyx pass regardless of the matrix row order

723b48f  Make _graph_without_edge_labels deterministic

ce0f0e0  Make riemann_matrix test more robust

comment:147 Changed 2 years ago by
The remaining issues are caused by this commit https://github.com/python/cpython/commit/ac20e0f98d6727ba97a9575bfa2a11b2f6247c35
In particular, if B=loads(dumps(A))
, then A.method
and B.method
have different hashes, so A.method==B.method
is always false.
comment:148 followup: 150 Changed 2 years ago by
Note that the issue of sorting vertices in simplicial complexes was discussed at length in #26966. I may propose a different solution to this problem, but give me a few days.
comment:149 Changed 2 years ago by
Commit:  ce0f0e0404b23485c7c8b2e38e4c8a36eb1354e1 → 5a98f9094bf29c5ddb7098ea21ca7076ab6d1a7d 

comment:150 Changed 2 years ago by
Replying to jhpalmieri:
Note that the issue of sorting vertices in simplicial complexes was discussed at length in #26966. I may propose a different solution to this problem, but give me a few days.
Oh, I see now that you proposed the same fix there but reverted it.
comment:151 Changed 2 years ago by
With the current branch all tests are passing for me. There may be room for some improvements (besides the simplicial complex vertex ordering issue), but I hope there's still a chance that this can make it in 9.2. It would be quite unfortunate to not support python 3.8 one year after its release.
comment:152 Changed 2 years ago by
I wrote a long post about simplicial complexes with various solutions, but now I realize there is a simpler solution.
There are a few problems here: first, if you want consistency when dealing with products of simplicial complexes, some sorting is needed: if you triangulate a square, for example, you have two choices for how to add the diagonal, and different sortings will result in different choices. Second, Python 3.7 and 3.8 are handling some data differently from each other, and that's where the doctest failure in simplicial_complex_morphism.py
is coming from: different sortings yielding different triangulations of a product. Third, I think that this really means that the doctest in simplicial_complex_morphishm.py
is a bad one, because it is essentially relying on how intrinsically unsorted data is converted to an ordered object (a tuple). I tried putting in various ways of sorting, and I got at least three different results, all of which are mathematically valid, and I don't see any reason to force one sorting option over another.
As a result, we should make this change: get rid of the new sorting in simplicial_complex.py
and make the doctest more robust:

src/sage/homology/simplicial_complex.py
diff git a/src/sage/homology/simplicial_complex.py b/src/sage/homology/simplicial_complex.py index 8ee9a4f84c..1c9dd8da8e 100644
a b class SimplicialComplex(Parent, GenericCellComplex): 1045 1045 else: 1046 1046 vertex_set = range(n + 1) 1047 1047 1048 try: # use natural order if vertices are integers 1049 vertices = tuple(sorted(vertex_set)) 1050 except TypeError: # incomparable vertices 1051 vertices = tuple(sorted(vertex_set, key=str)) 1048 vertices = tuple(vertex_set) 1052 1049 1053 1050 gen_dict = {} 1054 1051 for v in vertices: 
src/sage/homology/simplicial_complex_morphism.py
diff git a/src/sage/homology/simplicial_complex_morphism.py b/src/sage/homology/simplicial_complex_morphism.py index d954dd90b4..8db933b9fe 100644
a b EXAMPLES:: 81 81 sage: z = y.fiber_product(x) 82 82 sage: z # this is the mapping path space 83 83 Simplicial complex morphism: 84 From: Simplicial complex with 6 vertices and 6facets84 From: Simplicial complex with 6 vertices and ... facets 85 85 To: Minimal triangulation of the 2sphere 86 86 Defn: ['L0R(0, 0)', 'L0R(0, 1)', 'L1R(1, 0)', 'L1R(1, 1)', 'L2R(2, 0)', 'L2R(2, 1)'] > [0, 0, 1, 1, 2, 2] 87 87 """
By the way, see ticket:26966#comment:25 for the an example of the evils of sorting using key=str
.
comment:153 Changed 2 years ago by
Commit:  5a98f9094bf29c5ddb7098ea21ca7076ab6d1a7d → 196a30ad097999a707b71140abc6afd5265c652f 

Branch pushed to git repo; I updated commit sha1. New commits:
196a30a  trac 27754: make doctest in simplicial_complex_morphism.py more robust

comment:154 Changed 2 years ago by
Commit:  196a30ad097999a707b71140abc6afd5265c652f → 35cf0e197e4142b4849d09398a31063b5537eec9 

Branch pushed to git repo; I updated commit sha1. New commits:
35cf0e1  Allow system python 3.8

comment:155 Changed 2 years ago by
Commit:  35cf0e197e4142b4849d09398a31063b5537eec9 → f939f28b40da12dfc5b31f531dac380f5ef20343 

comment:156 Changed 2 years ago by
Status:  new → needs_review 

Setting to needs review for wider testing. Needs testing especially with system python<3.8, also with optional packages
comment:157 Changed 2 years ago by
Commit:  f939f28b40da12dfc5b31f531dac380f5ef20343 → c6002e8181f8b4bf2b9a4f2262a60aa354c16e4f 

Branch pushed to git repo; I updated commit sha1. New commits:
c4769bf  Add pynac patch to fix deprecation warnings on unpickling with Python 3.8

ecc6f45  Add patch

c6002e8  Merge branch 'u/arojas/fix_deprecation_warnings_when_unpickling_pynac_objects_with_python_3_8' of git://trac.sagemath.org/sage into t/27754/public/python3.8

comment:158 followup: 161 Changed 2 years ago by
comment:159 Changed 2 years ago by
Cc:  Andrew added 

comment:160 Changed 2 years ago by
Commit:  c6002e8181f8b4bf2b9a4f2262a60aa354c16e4f → 57c2d1c89515fe0716d04159bcdbe3fc398e1180 

Branch pushed to git repo; I updated commit sha1. New commits:
57c2d1c  tox.ini, .github/workflows/tox.yml: Add gentoopython3.7

comment:161 Changed 2 years ago by
Replying to mkoeppe:
Tests run at https://github.com/mkoeppe/sage/actions/runs/193151775
What's the outcome of this? I see many failed jobs but I can't see any logs.
comment:162 Changed 2 years ago by
Looks like GH Action was having trouble. I have started another run at https://github.com/mkoeppe/sage/actions/runs/199465086
comment:163 Changed 2 years ago by
Clean on ubuntutrustystandard
(with python3 from SPKG, https://github.com/mkoeppe/sage/runs/962936559), except for
sage t randomseed=0 src/sage/doctest/test.py # Timed out
Clean on centos7standard
(with python3 from SPKG, https://github.com/mkoeppe/sage/runs/962936782), except for
sage t randomseed=0 src/sage/interfaces/sagespawn.pyx File "src/sage/interfaces/sagespawn.pyx", line 96, in sage.interfaces.sagespawn.SageSpawn.__repr__ Failed example: while s.isalive(): # Wait until the process finishes sleep(0.1) Expected nothing Got: doctest:warning DeprecationWarning: an integer is required (got type sage.rings.real_mpfr.RealLiteral). Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
Clean on gentoostandard
(using /usr/bin/python3.8
, https://github.com/mkoeppe/sage/runs/962936791) except for:
sage t randomseed=0 src/sage/schemes/elliptic_curves/ell_modular_symbols.py # Timed out
Clean on gentoostandardpython3.7
(using /usr/bin/python3.7
) except for:
sage t randomseed=0 src/sage/categories/finite_enumerated_sets.py File "src/sage/categories/finite_enumerated_sets.py", line 290, in sage.categories.finite_enumerated_sets.FiniteEnumeratedSets.ParentMethods._list_from_iterator Failed example: list(C) Expected: hello! hello! ... [1, 2, 3] Got: hello! hello! [1, 2, 3]
Lots of doctest failures because of LC_ALL: cannot change locale
messages (#30053) on linuxmint17minimal
(https://github.com/mkoeppe/sage/runs/962936711) and standard
Unfortunately most builds failed again because of unrelated technical problems on GH Actions
comment:164 Changed 2 years ago by
Reviewers:  → https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/runs/959545895 

comment:165 Changed 2 years ago by
Reviewers:  https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/runs/959545895 → https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/actions/runs/199465096 

comment:166 Changed 2 years ago by
Commit:  57c2d1c89515fe0716d04159bcdbe3fc398e1180 → 389f1b2d7f5fe78ed232bbfb50d95ca5bee47d1e 

comment:167 Changed 2 years ago by
Commit:  389f1b2d7f5fe78ed232bbfb50d95ca5bee47d1e → fa2ad16159d452089a235796291a13f49d8d0477 

Branch pushed to git repo; I updated commit sha1. New commits:
fa2ad16  Merge tag '9.2.beta8' into t/27754/public/python3.8

comment:168 Changed 2 years ago by
Reviewers:  https://github.com/mkoeppe/sage/actions/runs/199465086, https://github.com/mkoeppe/sage/actions/runs/199465096 → github.com/mkoeppe/sage/actions/runs/206442118 

comment:169 Changed 2 years ago by
debianjessieminimal
, debianstretchstandard
, centos7standard
(with python3 from SPKG) clean
ubuntubionicstandard
, debianbusterstandard
, linuxmint19.3standard
, fedora30standard
, fedora31standard
, gentoopython3.7standard
(with /usr/bin/python3.7
) clean
gentoostandard
(with /usr/bin/python3.8
) clean
ubuntueoanstandard
(with /usr/bin/python3.7
, https://github.com/mkoeppe/sage/runs/978617270) clean except for
sage t randomseed=0 src/sage/libs/eclib/interface.py # 2 doctests failed sage t randomseed=0 src/sage/schemes/cyclic_covers/charpoly_frobenius.py # 1 doctest failed
(seem unrelated to this ticket (mwrank; #28951?))
debianbullseyestandard
(with /usr/bin/python3.7
) clean except for doctest errors because of newer flint or arb from the system
linuxmint17standard
.... lots of doctest failures because of "setlocale" messages
fedora27standard
(with /usr/bin/python3.6
):
sage t randomseed=0 src/sage/lfunctions/sympow.py # 3 doctests failed sage t randomseed=0 src/sage/schemes/elliptic_curves/ell_rational_field.py # 2 doctests failed
(seems unrelated)
fedora29standard
(with /usr/bin/python3.7
): likewise
fedora28standard
(with /usr/bin/python3.6
) additionally:
sage t randomseed=0 src/sage/rings/polynomial/groebner_fan.py # 3 doctests failed
(seems unrelated)
condaforgestandard
(with /opt/conda/bin/python3.7
):
sage t randomseed=0 src/doc/en/constructions/algebraic_geometry.rst # 1 doctest failed sage t randomseed=0 src/doc/en/developer/coding_in_other.rst # 1 doctest failed sage t randomseed=0 src/sage/misc/cython.py # 3 doctests failed sage t randomseed=0 src/sage/modules/fg_pid/fgp_module.py # 3 doctests failed sage t randomseed=0 src/sage/modules/free_module_morphism.py # 3 doctests failed sage t randomseed=0 src/sage/rings/complex_arb.pyx # 6 doctests failed sage t randomseed=0 src/sage/parallel/map_reduce.py # Timed out sage t randomseed=0 src/sage/rings/real_arb.pyx # 2 doctests failed sage t randomseed=0 src/sage/schemes/hyperelliptic_curves/monsky_washnitzer.py # Killed due to abort
condaforgeminimal
(https://github.com/mkoeppe/sage/runs/978620286):
[python33.8.5] ImportError: The required _crypt module was not built as part of CPython [python33.8.5] crypt module failed to import [python33.8.5] ModuleNotFoundError: No module named 'zlib' [python33.8.5] zlib module failed to import [python33.8.5] ModuleNotFoundError: No module named '_sqlite3' [python33.8.5] sqlite3 module failed to import [python33.8.5] Error: One or more modules failed to import.
(but we had similar errors also before the update, on this platform)
ubuntufocalstandard
(https://github.com/mkoeppe/sage/runs/978617297)
hecking whether SageMath should install SPKG python3... checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no checking for python3 >= 3.6, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core... checking ... whether /usr/bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension checking ... whether /usr/bin/python3 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension checking ... whether /bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension checking ... whether /bin/python3 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension configure: no suitable system package found for SPKG python3
likewise on debiansidstandard
(https://github.com/mkoeppe/sage/runs/978620025)
comment:170 Changed 2 years ago by
homebrewmacosminimal
(using python3 SPKG): Clean except for various timeouts
sage t randomseed=0 src/sage/dynamics/arithmetic_dynamics/projective_ds.py # Timed out sage t randomseed=0 src/sage/interfaces/gap.py # 1 doctest failed sage t randomseed=0 src/sage/manifolds/differentiable/metric.py # Timed out sage t randomseed=0 src/sage/manifolds/differentiable/tensorfield.py # Timed out sage t randomseed=0 src/sage/plot/graphics.py # Timed out sage t randomseed=0 src/sage/plot/plot.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/base.pyx # Timed out sage t randomseed=0 src/sage/plot/plot3d/implicit_plot3d.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/parametric_plot3d.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/plot3d.py # Timed out sage t randomseed=0 src/sage/plot/plot3d/shapes.pyx # Timed out sage t randomseed=0 src/sage/plot/plot3d/shapes2.py # Timed out sage t randomseed=0 src/sage/rings/function_field/function_field.py # Timed out sage t randomseed=0 src/sage/schemes/elliptic_curves/ell_number_field.py # Timed out sage t randomseed=0 src/sage/schemes/elliptic_curves/gp_simon.py
homebrewmacospython3_xcodestandard
(using /usr/bin/python3
)  build times out during docbuild  perhaps the same problem as reported on sagerelease
condaforgemacosminimal
(https://github.com/mkoeppe/sage/runs/978617180):
[python33.8.5] checking build system type... x86_64appledarwin19.6.0 [python33.8.5] checking host system type... x86_64appledarwin13.4.0 [python33.8.5] checking for python3.8... no [python33.8.5] checking for python3... python3 [python33.8.5] configure: error: Cross compiling required host=HOSTTUPLE and build=ARCH [python33.8.5] configure: WARNING: unrecognized options: disabletoolboxglue [python33.8.5] checking build system type... x86_64appledarwin19.6.0 [python33.8.5] checking host system type... x86_64appledarwin13.4.0 [python33.8.5] checking for python3.8... no [python33.8.5] checking for python3... python3 [python33.8.5] configure: error: Cross compiling required host=HOSTTUPLE and build=ARCH
???? But this platform also failed before the update
comment:171 Changed 2 years ago by
Milestone:  sage9.3 → sage9.2 

Looks like we have a good chance of getting this ready for 9.2!
comment:173 Changed 2 years ago by
Commit:  fa2ad16159d452089a235796291a13f49d8d0477 → 89dc8c92ebd9f10c7d41e6b72e6d271e05fc1119 

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
bccf5c1  Merge branch 't/30127/cygwin_minimal__pip_20_1_1_fails_to_install' into t/27754/public/python3.8

bc65004  tox.ini, tox.yml: Add ubuntugroovy, fedora33

8eb9cae  Merge branch 't/30339/tox_ini__update_for_latest_ubuntu__fedora_versions' into t/27754/public/python3.8

8d5f2ae  tox.ini, build/pkgs/python3/distros/: Switch to python 3.8

cf25b83  Move code for tagging/pushing docker images into tox.ini

51c6cde  fixup

231b812  .githubworkflows/tox*.yml: Update similarly

385571e  Merge branch 't/30144/tox_ini___tox_yml__move_code_for_tagging_pushing_docker_images_into_tox_ini' into t/27754/public/python3.8

701b4ce  build/make/Makefile.in: Add + before recursive make invocations

89dc8c9  Merge branch 't/30345/build_make_makefile_in__filter_out___j__from_sub_make_invocations_to_avoid_excessive_parallel_load' into t/27754/public/python3.8

comment:174 Changed 2 years ago by
Dependencies:  #28197, #29042, #29861, #30162, #30225 → #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345 

comment:175 Changed 2 years ago by
Reviewers:  github.com/mkoeppe/sage/actions/runs/206442118 → github.com/mkoeppe/sage/actions/runs/206442118, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210 

comment:176 Changed 2 years ago by
Reviewers:  github.com/mkoeppe/sage/actions/runs/206442118, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210 → github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210 

comment:177 Changed 2 years ago by
Priority:  major → critical 

comment:178 Changed 2 years ago by
ubuntufocalstandard
now accepts system python 3.8, clean doctests.
But:
debianbullseyestandard
(https://github.com/mkoeppe/sage/runs/982720084), now using /usr/bin/python3.8
(3.8.5), fails to build matplotlib
:
x86_64linuxgnugcc pthread shared Wl,O1 Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 Wl,rpathlink,/sage/local/lib L/sage/local/lib Wl,rpath,/sage/local/lib g fstackprotectorstrong Wformat Werror=formatsecurity Wdatetime D_FORTIFY_SOURCE=2 build/temp.linuxx86_643.8/src/qhull_wrap.o build/temp.linuxx86_643.8/extern/libqhull/geom.o build/temp.linuxx86_643.8/extern/libqhull/geom2.o build/temp.linuxx86_643.8/extern/libqhull/global.o build/temp.linuxx86_643.8/extern/libqhull/io.o build/temp.linuxx86_643.8/extern/libqhull/libqhull.o build/temp.linuxx86_643.8/extern/libqhull/mem.o build/temp.linuxx86_643.8/extern/libqhull/merge.o build/temp.linuxx86_643.8/extern/libqhull/poly.o build/temp.linuxx86_643.8/extern/libqhull/poly2.o build/temp.linuxx86_643.8/extern/libqhull/qset.o build/temp.linuxx86_643.8/extern/libqhull/random.o build/temp.linuxx86_643.8/extern/libqhull/rboxlib.o build/temp.linuxx86_643.8/extern/libqhull/stat.o build/temp.linuxx86_643.8/extern/libqhull/user.o build/temp.linuxx86_643.8/extern/libqhull/usermem.o build/temp.linuxx86_643.8/extern/libqhull/userprintf.o build/temp.linuxx86_643.8/extern/libqhull/userprintf_rbox.o lm o build/lib.linuxx86_643.8/matplotlib/_qhull.cpython38x86_64linuxgnu.so lto1: fatal error: bytecode stream in file ‘build/temp.linuxx86_643.8/src/qhull_wrap.o’ generated with GCC compiler older than 10.0 compilation terminated. ltowrapper: fatal error: x86_64linuxgnugcc returned 1 exit status compilation terminated. /usr/bin/ld: error: ltowrapper failed collect2: error: ld returned 1 exit status
Likewise on debiansidstandard
and ubuntugroovystandard
comment:179 followup: 182 Changed 2 years ago by
Getting rid of flto
would work around this problem. Not sure where it is coming from
comment:180 Changed 2 years ago by
(As expected, upgrading matplotlib to latest (3.3.1, #30358) does not solve this problem.)
comment:181 followup: 185 Changed 2 years ago by
I haven't experienced anything like that in Gentoo. But looking at build log for matplotlib 3.3.0 suggests reviving an idea: using an external qhull
install for matplotlib.
comment:182 Changed 2 years ago by
Replying to mkoeppe:
Getting rid of
flto
would work around this problem. Not sure where it is coming from
The flag is coming directly from matplotlib's setup.py
, I'll just disable it
comment:183 Changed 2 years ago by
Dependencies:  #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345 → #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365 

comment:184 Changed 2 years ago by
Commit:  89dc8c92ebd9f10c7d41e6b72e6d271e05fc1119 → 4daa290fef4356faaf56eefa4e58b948e2f8db84 

comment:185 Changed 2 years ago by
Replying to fbissey:
I haven't experienced anything like that in Gentoo. But looking at build log for matplotlib 3.3.0 suggests reviving an idea: using an external
qhull
install for matplotlib.
I would support this in general, but not for 9.2; and as discussed in #30176, we would need to make cmake standard.
comment:186 Changed 2 years ago by
Reviewers:  github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210 → Matthias Koeppe, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210, ... 

I think this ticket is now ready for review.
comment:187 Changed 2 years ago by
Description:  modified (diff) 

comment:188 Changed 2 years ago by
Reviewers:  Matthias Koeppe, github.com/mkoeppe/sage/actions/runs/207801205, github.com/mkoeppe/sage/actions/runs/207801208, github.com/mkoeppe/sage/actions/runs/207801210, ... → Matthias Koeppe, François Bissey 

Status:  needs_review → positive_review 
Now is a good time to tests things on a bigger scale.
comment:190 Changed 2 years ago by
Commit:  4daa290fef4356faaf56eefa4e58b948e2f8db84 → 46c559c0b052e37d87c050fee879150318ba0463 

Status:  positive_review → needs_review 
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
46c559c  Merge tag '9.2.beta9' into t/27754/public/python3.8

comment:191 Changed 2 years ago by
Status:  needs_review → positive_review 

comment:192 Changed 2 years ago by
Status:  positive_review → needs_work 

STDOUT: CONFLICT (content): Merge conflict in src/sage/graphs/generic_graph.py
comment:193 Changed 2 years ago by
Commit:  46c559c0b052e37d87c050fee879150318ba0463 → a625ed34796120b6d8bc7397c4a9ef4314649332 

comment:194 Changed 2 years ago by
Commit:  a625ed34796120b6d8bc7397c4a9ef4314649332 → 53b569d68ddca39a0c5b33ab59ce77dd651da21a 

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
3db87ad  build/pkgs/maxima: Update to 5.44.0

bcf3c98  Remove old patches

07aa2f9  build/pkgs/maxima/spkginstall.in: New workaround for makeinfo trouble

046634d  sage.interfaces.maxima_abstract.MaximaAbstract._command_runner: Skip all comment lines, rather than counting lines to skip

d1ad8d7  sage.interfaces.maxima_abstract.MaximaAbstract.completions: Remove 'impl' functions

2ea3e73  Merge branch 'u/mkoeppe/upgrade_maxima_to_5_44_0' of trac.sagemath.org:sage into HEAD

8f83f96  #30063 : Maxima gamma_greek is now called gamma_incomplete_lower

21e2fea  #30063 : mark Maxima's integration regression as known bug.

d6e0952  Trac #30063: Upgrade Maxima to 5.44.0

53b569d  Merge remotetracking branch 'vbraun/develop' into t/27754/public/python3.8

comment:195 Changed 2 years ago by
Dependencies:  #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365 → #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140 

Status:  needs_work → positive_review 
comment:196 Changed 2 years ago by
Status:  positive_review → needs_work 

I'm getting various failures on 32bit apparently due to different iterator order:
********************************************************************** File "src/sage/combinat/rigged_configurations/tensor_product_kr_tableaux.py", line 333, in sage.combinat.rigged_configurations.tensor_product_kr_tableaux.TensorProductOfKirillovReshetikhinTableaux.__iter__ Failed example: next(g) Expected: [[2], [4]] (X) [[1]] Got: [[1], [2]] (X) [[1]] ********************************************************************** 1 item had failures: 1 of 5 in sage.combinat.rigged_configurations.tensor_product_kr_tableaux.TensorProductOfKirillovReshetikhinTableaux.__iter__ [95 tests, 1 failure, 21.93 s] ********************************************************************** File "src/sage/geometry/cone.py", line 4059, in sage.geometry.cone.?.semigroup_generators Failed example: halfplane.semigroup_generators() Expected: (N(0, 1), N(1, 0), N(1, 0)) Got: (N(1, 0), N(1, 0), N(0, 1)) ********************************************************************** 1 item had failures: 1 of 23 in sage.geometry.cone.?.semigroup_generators [1241 tests, 1 failure, 18.70 s] ********************************************************************** File "src/sage/geometry/fan_isomorphism.py", line 124, in sage.geometry.fan_isomorphism.fan_isomorphism_generator Failed example: tuple(fan_isomorphism_generator(fan1, fan2)) Expected: ( [18 1 5] [ 6 3 7] [ 4 0 1] [ 1 1 2] [ 5 0 1], [ 2 1 2] ) Got: ( [ 6 3 7] [18 1 5] [ 1 1 2] [ 4 0 1] [ 2 1 2], [ 5 0 1] ) ********************************************************************** File "src/sage/geometry/fan_isomorphism.py", line 130, in sage.geometry.fan_isomorphism.fan_isomorphism_generator Failed example: tuple(fan_isomorphism_generator(fan2, fan1)) Expected: ( [ 0 1 1] [ 0 1 1] [ 1 7 2] [ 2 2 5] [ 0 5 4], [ 1 0 3] ) Got: ( [ 0 1 1] [ 0 1 1] [ 2 2 5] [ 1 7 2] [ 1 0 3], [ 0 5 4] ) ********************************************************************** 1 item had failures: 2 of 21 in sage.geometry.fan_isomorphism.fan_isomorphism_generator [65 tests, 2 failures, 1.07 s] ********************************************************************** File "src/sage/geometry/polyhedron/library.py", line 2639, in sage.geometry.polyhedron.library.Polytopes.generalized_permutahedron Failed example: sorted(perm_a2_reg.vertices()) Expected: [A vertex at (1, 0), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1.000000000000000?, 0.?e18)] Got: [A vertex at (1.000000000000000?, 0.?e18), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1/2, 0.866025403784439?), A vertex at (1, 0)] ********************************************************************** 1 item had failures: 1 of 21 in sage.geometry.polyhedron.library.Polytopes.generalized_permutahedron [305 tests, 1 failure, 97.96 s] ********************************************************************** File "src/sage/geometry/triangulation/point_configuration.py", line 1401, in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits_support Failed example: list( p.circuits_support() ) Expected: [(0, 3, 4), (0, 1, 2), (1, 2, 3, 4)] Got: [(0, 1, 2), (0, 3, 4), (1, 2, 3, 4)] ********************************************************************** File "src/sage/geometry/triangulation/point_configuration.py", line 1471, in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits Failed example: p.circuits() Expected: (((0,), (1, 2), (3, 4)), ((0,), (3, 4), (1, 2)), ((1, 2), (0,), (3, 4))) Got: (((0,), (3, 4), (1, 2)), ((0,), (1, 2), (3, 4)), ((1, 2), (0,), (3, 4))) ********************************************************************** 2 items had failures: 1 of 6 in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits 1 of 3 in sage.geometry.triangulation.point_configuration.PointConfiguration.circuits_support [214 tests, 2 failures, 19.74 s] ********************************************************************** File "src/sage/graphs/graph.py", line 7342, in sage.graphs.graph.Graph.cliques_containing_vertex Failed example: sorted(d[(0, 1)]) Expected: [[(0, 1), (0, 0)], [(0, 1), (0, 2)], [(0, 1), (1, 1)]] Got: [[(0, 0), (0, 1)], [(0, 2), (0, 1)], [(1, 1), (0, 1)]] ********************************************************************** 1 item had failures: 1 of 15 in sage.graphs.graph.Graph.cliques_containing_vertex [1190 tests, 1 failure, 20.84 s] ********************************************************************** File "src/sage/homology/homology_vector_space_with_basis.py", line 113, in sage.homology.homology_vector_space_with_basis.HomologyVectorSpaceWithBasis Failed example: x.cup_product(x) Expected: 0 Got: h^{2,0} ********************************************************************** File "src/sage/homology/homology_vector_space_with_basis.py", line 115, in sage.homology.homology_vector_space_with_basis.HomologyVectorSpaceWithBasis Failed example: x.cup_product(y) Expected: h^{2,0} Got: 0 ********************************************************************** 1 item had failures: 2 of 45 in sage.homology.homology_vector_space_with_basis.HomologyVectorSpaceWithBasis [194 tests, 2 failures, 16.93 s] R[write to console]: Warning messages: R[write to console]: 1: R[write to console]: In sage10 + sage6 : R[write to console]: R[write to console]: longer object length is not a multiple of shorter object length R[write to console]: 2: R[write to console]: In sqrt(sage10) : R[write to console]: NaNs produced R[write to console]: 3: R[write to console]: In sqrt(sage4) : R[write to console]: NaNs produced ********************************************************************** File "src/sage/matroids/graphic_matroid.py", line 1659, in sage.matroids.graphic_matroid.GraphicMatroid.graphic_coextensions Failed example: for N1 in I: N1.graph().edges(sort=True) Expected: [(0, 1, 0), (0, 3, 1), (0, 4, 'a'), (1, 2, 2), (2, 3, 3)] [(0, 1, 0), (0, 3, 1), (1, 4, 2), (2, 3, 3), (2, 4, 'a')] Got: [(0, 1, 0), (0, 3, 1), (0, 4, 'a'), (1, 2, 2), (2, 3, 3)] [(0, 3, 1), (0, 4, 0), (1, 2, 2), (1, 4, 'a'), (2, 3, 3)] ********************************************************************** 1 item had failures: 1 of 27 in sage.matroids.graphic_matroid.GraphicMatroid.graphic_coextensions [358 tests, 1 failure, 0.75 s] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1099, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.class_representatives Failed example: R Expected: [[V(2)*V(3)], [V(1)*V(2)]] Got: [[V(1)*V(2)], [V(2)*V(3)]] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1101, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.class_representatives Failed example: [v.continued_fraction()[1] for v in R] Expected: [(1, 2, 2), (3,)] Got: [(3,), (1, 2, 2)] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1318, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.reduced_elements Failed example: R Expected: [ [ 5*lam 3] [ 5*lam 7] [4*lam 3] [3*lam 1] [ 7 2*lam], [ 3 2*lam], [ 3 lam], [ 1 0] ] Got: [ [3*lam 1] [ 5*lam 3] [ 5*lam 7] [4*lam 3] [ 1 0], [ 7 2*lam], [ 3 2*lam], [ 3 lam] ] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1323, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.reduced_elements Failed example: [v.continued_fraction() for v in R] Expected: [((), (1, 2, 2)), ((), (2, 2, 1)), ((), (2, 1, 2)), ((), (3,))] Got: [((), (3,)), ((), (1, 2, 2)), ((), (2, 2, 1)), ((), (2, 1, 2))] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1350, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.simple_elements Failed example: G.simple_elements(D=14) Expected: [ [2*lam 1] [ lam 1] [2*lam 3] [ lam 3] [ 3 lam], [ 3 2*lam], [ 1 lam], [ 1 2*lam] ] Got: [ [2*lam 3] [ lam 3] [2*lam 1] [ lam 1] [ 1 lam], [ 1 2*lam], [ 3 lam], [ 3 2*lam] ] ********************************************************************** File "src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py", line 1387, in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.rational_period_functions Failed example: G.rational_period_functions(k=4, D=14) Expected: [z^4 + 1, 16*z^4  16, 16*z^4 + 16] Got: [z^4 + 1, 16*z^4 + 16, 16*z^4  16] ********************************************************************** 4 items had failures: 2 of 27 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.class_representatives 1 of 7 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.rational_period_functions 2 of 9 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.reduced_elements 1 of 5 in sage.modular.modform_hecketriangle.hecke_triangle_groups.?.simple_elements [228 tests, 6 failures, 4.90 s] ********************************************************************** File "src/sage/modular/modform_hecketriangle/readme.py", line 549, in sage.modular.modform_hecketriangle.readme Failed example: G.class_representatives(9*G.lam() + 5) Expected: [S*T^(2)*S*T^(1)*S, T*S*T^2] Got: [T*S*T^2, S*T^(2)*S*T^(1)*S] ********************************************************************** 1 item had failures: 1 of 383 in sage.modular.modform_hecketriangle.readme [382 tests, 1 failure, 19.56 s] ********************************************************************** File "src/sage/monoids/trace_monoid.py", line 959, in sage.monoids.trace_monoid.TraceMonoid._repr_ Failed example: M.<a,b,c,d> = TraceMonoid(I=I); M Expected: Trace monoid on 4 generators ([a], [b], [c], [d]) with independence relation {{a, d}, {b, c}} Got: Trace monoid on 4 generators ([a], [b], [c], [d]) with independence relation {{b, c}, {d, a}} ********************************************************************** File "src/sage/monoids/trace_monoid.py", line 976, in sage.monoids.trace_monoid.TraceMonoid._latex_ Failed example: M.<a,b,c,d> = TraceMonoid(I=I); latex(M) Expected: \langle a, b, c, d \mid ad=da,bc=cb \rangle Got: \langle a, b, c, d \mid bc=cb,da=ad \rangle ********************************************************************** 2 items had failures: 1 of 4 in sage.monoids.trace_monoid.TraceMonoid._latex_ 1 of 4 in sage.monoids.trace_monoid.TraceMonoid._repr_ [186 tests, 2 failures, 4.63 s] ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 4166, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis Failed example: f Expected: 1/2*y^2*z^7  1/4*y*z^8 + 2*x*z^5 + 95*z^6 + 1/2*y^5  1/4*y^4*z + x^2*y^2 + 3/2*x^2*y*z + 95*x*y*z^2 Got: 1/2*y^2*z^7  1/4*y*z^8 + 95*x*z^5 + 2*z^6 + 1/2*y^5  1/4*y^4*z + x^2*y^2 + 189/2*x^2*y*z + 2*x*y*z^2 ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 4168, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis Failed example: f.lift(I.gens()) Expected: [2*x + 95*z, 1/2*y^2  1/4*y*z, 0] Got: [95*x + 2*z, 1/2*y^2  1/4*y*z, 0] ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 4170, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis Failed example: l = f.lift(J.gens()); l Expected: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1/2*y^2 + 1/4*y*z, 1/2*y^2*z^2  1/4*y*z^3 + 2*x + 95*z] Got: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1/2*y^2 + 1/4*y*z, 1/2*y^2*z^2  1/4*y*z^3 + 95*x + 2*z] ********************************************************************** 1 item had failures: 3 of 111 in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_basis [917 tests, 3 failures, 10.04 s]  sage t long randomseed=0 src/sage/combinat/rigged_configurations/tensor_product_kr_tableaux.py # 1 doctest failed sage t long randomseed=0 src/sage/geometry/cone.py # 1 doctest failed sage t long randomseed=0 src/sage/geometry/fan_isomorphism.py # 2 doctests failed sage t long randomseed=0 src/sage/geometry/polyhedron/library.py # 1 doctest failed sage t long randomseed=0 src/sage/geometry/triangulation/point_configuration.py # 2 doctests failed sage t long randomseed=0 src/sage/graphs/graph.py # 1 doctest failed sage t long randomseed=0 src/sage/homology/homology_vector_space_with_basis.py # 2 doctests failed sage t long randomseed=0 src/sage/matroids/graphic_matroid.py # 1 doctest failed sage t long randomseed=0 src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py # 6 doctests failed sage t long randomseed=0 src/sage/modular/modform_hecketriangle/readme.py # 1 doctest failed sage t long randomseed=0 src/sage/monoids/trace_monoid.py # 2 doctests failed sage t long randomseed=0 src/sage/rings/polynomial/multi_polynomial_ideal.py # 3 doctests failed 
comment:197 Changed 2 years ago by
Commit:  53b569d68ddca39a0c5b33ab59ce77dd651da21a → 42bfb1f7a85fdad131bd9b278105641952a54bfb 

Branch pushed to git repo; I updated commit sha1. New commits:
42bfb1f  Merge tag '9.2.beta10' into t/27754/public/python3.8

comment:198 Changed 2 years ago by
Commit:  42bfb1f7a85fdad131bd9b278105641952a54bfb → d167e08bb10a7c0b36be21edeffd75e8b222d89b 

Branch pushed to git repo; I updated commit sha1. New commits:
d167e08  Make doctests more flexible for Python 3.8 on 32bit

comment:199 Changed 2 years ago by
Status:  needs_work → needs_review 

comment:200 Changed 2 years ago by
This fixes all failures except for the following two:
sage t randomseed=0 src/sage/homology/homology_vector_space_with_basis.py # 2 doctests failed sage t randomseed=0 src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py # 1 doctest failed
Help with these would be welcome
comment:201 Changed 2 years ago by
There must be some way for me to run a 32bit machine via Docker so I can look into these, but I can't figure out how.
comment:202 followup: 203 Changed 2 years ago by
Yes, for example EXTRA_CONFIGURE_ARGS="withoutsystemsuitesparse" tox e dockerubuntueoani386standardpython3_spkg
comment:203 Changed 2 years ago by
Replying to mkoeppe:
Yes, for example
EXTRA_CONFIGURE_ARGS="withoutsystemsuitesparse" tox e dockerubuntueoani386standardpython3_spkg
Well, that didn't work for me (with EXTRA_CONFIGURE_ARGS="withsystemsuitesparse=no"
): the build failed for cypari
and giac
. But I used VirtualBox? to install a 32bit Debian machine, and I see the doctest failures. I actually haven't looked at the hecke_triangle_groups.py failure — not my area of expertise — but I understand what's happening with the homology failure. There are some unsorted pieces of data lying around, and when they get converted to lists or tuples, how they get converted affects the answer. Both answers (32bit and 64bit) are mathematically correct, it's just that you get a different choice of bases in the two cases.
I propose changing to a case where the sorting won't affect the answer.
comment:204 Changed 2 years ago by
Commit:  d167e08bb10a7c0b36be21edeffd75e8b222d89b → d85c4ba66d5bf4b03e4f480da0ca8d49ade49fb2 

Branch pushed to git repo; I updated commit sha1. New commits:
d85c4ba  trac 27754: homology doctest fix

comment:205 followup: 206 Changed 2 years ago by
For the hecke_triangle_groups.py problem, this seems to fix it, but I don't know the mathematics of what's going on, nor have I looked at the code:

src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py
diff git a/src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py b/src/sage/modular/modform_hecketriangle/hecke_triangle_groups.py index 47ba82eb8e..cb336f64c4 100644
a b class HeckeTriangleGroup(FinitelyGeneratedMatrixGroup_generic, 1096 1096 [[U], [U^(1)]] 1097 1097 1098 1098 sage: R = G.class_representatives(14) 1099 sage: R1099 sage: sorted(R) 1100 1100 [[V(2)*V(3)], [V(1)*V(2)]] 1101 1101 sage: sorted(v.continued_fraction()[1] for v in R) 1102 1102 [(1, 2, 2), (3,)]
comment:206 followup: 207 Changed 2 years ago by
Replying to jhpalmieri:
For the hecke_triangle_groups.py problem, this seems to fix it, but I don't know the mathematics of what's going on, nor have I looked at the code:
docs say that it's a list of representatives, with no meaning attached to the order, so your fix is OK.
comment:207 Changed 2 years ago by
Replying to dimpase:
Replying to jhpalmieri:
For the hecke_triangle_groups.py problem, this seems to fix it, but I don't know the mathematics of what's going on, nor have I looked at the code:
docs say that it's a list of representatives, with no meaning attached to the order, so your fix is OK.
I'll push a branch after I do a bit more testing, unless someone beats me to it. (One question is how the elements in the list R are sorted. I can't tell after looking at the code for a few minutes. But this does seem to work.)
comment:208 Changed 2 years ago by
Commit:  d85c4ba66d5bf4b03e4f480da0ca8d49ade49fb2 → b9d451c7740613f38194cba3c4bd6f428cdc19e5 

Branch pushed to git repo; I updated commit sha1. New commits:
b9d451c  trac 27754: 32bit Python 3.8 doctest fix in hecke_triangle_groups.py

comment:209 Changed 2 years ago by
Okay, this works for me on OS X with the system Python and Sage's Python 3.8, and the same on a virtual Debian 32bit machine.
comment:212 Changed 2 years ago by
Commit:  b9d451c7740613f38194cba3c4bd6f428cdc19e5 → a9b69650667022f4a46b59ac90994254389931c1 

Branch pushed to git repo; I updated commit sha1. New commits:
488bba4  on macOS set OMP_NUM_THREADS to 1 for docbuild and tests

4b11a70  use the correct import location, not six.*

a9b6965  Merge commit '4b11a706e1267b385099d8691b4468efd974f94d' of git://trac.sagemath.org/sage into t/27754/public/python3.8

comment:213 Changed 2 years ago by
Status:  needs_work → positive_review 

comment:214 Changed 2 years ago by
Dependencies:  #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140 → #28197, #29042, #29861, #30162, #30225, #30127, #30144, #30345, #30365, #30140, #30351 

comment:215 Changed 2 years ago by
Commit:  a9b69650667022f4a46b59ac90994254389931c1 → f1cda120e6c715ec6bdd6d8bb776b3b0aa921794 

Status:  positive_review → needs_review 
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
f1cda12  Fixup merge

comment:216 Changed 2 years ago by
Status:  needs_review → positive_review 

comment:217 Changed 2 years ago by
Priority:  critical → blocker 

comment:218 Changed 2 years ago by
FWIW, worked for MeToo(TM) :
Rebuilding 2.9.beta11
+#27754 on Debian sesting running on core i7 + 16 GB RAM gives results identical to those obtained on the same machine without #27754 (see sagerelease). And $sSAGE_LOCAL\bin\python*
are symlinks to \usr\bin\
...
comment:220 Changed 2 years ago by
Branch:  public/python3.8 → f1cda120e6c715ec6bdd6d8bb776b3b0aa921794 

Resolution:  → fixed 
Status:  positive_review → closed 
comment:221 Changed 2 years ago by
Commit:  f1cda120e6c715ec6bdd6d8bb776b3b0aa921794 

Wonderful to have this in Sage 9.2.
Next episode:
 #30184: Support Python 3.9
(Python 3.9.rc2 is expected on 20200914 and Python 3.9 final on 20201005.)
If people want to start testing, Python 3.8.0a3 is out: