Opened 7 years ago
Closed 16 months ago
#15995 closed enhancement (fixed)
Python 3 preparation: Adapt to the changed integer division semantics (/ vs. //)
Reported by: | wluebbe | Owned by: | embray |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | python3 | Keywords: | division |
Cc: | embray, jdemeyer, fbissey, tscrim, jhpalmieri, vklein | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
See original ticket below. We can still use this as a meta-ticket for tracking all division-related changes needed for Python 3. Existing tickets include:
Progress Report
Open
No results
In Progress
No results
Done
- #15995
- Python 3 preparation: Adapt to the changed integer division semantics (/ vs. //)
- #20533
- Change again integer divisions from / to // in combinat
- #21127
- py3: turn some divisions in compatible format
- #22767
- py3: some care for division
- #23729
- py3: some care for division in graphs + various improvements
- #23731
- py3: future division in matrix2
- #24208
- py3: add some future division imports
- #24805
- py3: miscellaneous division-related fixes, particularly for sage_setup.autogen
- #25180
- py3: fixes to sage.dynamics
- #26135
- py3: fix bad divisions in givaro finite fields
- #29139
- *.pxi files might not be Python3-ready
Modules needing work
Based on some grepping results, the following modules appear to contain a class that implements __div__
but not __truediv__
(the latter should be preferred in all cases):
Ticket | Module |
---|---|
#????? | src/sage/algebras/lie_algebras/lie_algebra_element.pyx |
#????? | src/sage/categories/examples/lie_algebras.py |
#24805 | (this is just from a Python 2 only test case from the relevant ticket) |
#????? | src/sage/groups/additive_abelian/qmodnz_element.py |
#????? | src/sage/matrix/matrix_gfpn_dense.pyx |
#????? | src/sage/rings/integer.pyx |
#????? | src/sage/rings/rational.pyx |
#????? | src/sage/structure/category_object.pyx |
The new semantics can be enabled in Python 2 with from __future__ import division
.
This will require checking where to replace /
by //
Changes according to libfuturize/fixes/division.py
:
Add from __future__ import division
This ticket is tracked as a dependency of meta-ticket ticket:15980.
To test what remains to be done, one can use:
export PYTHONIOENCODING="utf-8" source ./local/bin/sage-env python -Qnew ./local/bin/sage-runtests --all
Attachments (2)
Change History (26)
comment:1 follow-up: ↓ 10 Changed 6 years ago by
comment:2 Changed 6 years ago by
I finally manage to run the doctests with the Python command line option -3
. This is how I did it:
...:~/sage-6.2.beta8$ . src/bin/sage-env ...:~/sage-6.2.beta8$ python -3 ./local/bin/sage-runtests src/sage 1>logs/sage-warnings3-stdout.txt 2>logs/sage-warnings3-stderr.txt
A summary of stderr
output is
nbr_lines= 133 83 Overriding __eq__ blocks inheritance of __hash__ in 3.x 27 classic int division 18 type inequality comparisons not supported in 3.x 2 the cmp argument is not supported in 3.x 1 execfile() not supported in 3.x; use exec() 1 file.softspace not supported in 3.x 1 tmpfile has been removed in 3.x; use the tempfile module
and of stdout
nbr_lines= 39630 19386 type inequality comparisons not supported in 3.x 8779 Overriding __eq__ blocks inheritance of __hash__ in 3.x 4039 CObject type is not supported in 3.x. Please use capsule objects instead. 2139 reduce() not supported in 3.x; use functools.reduce() 2017 classic int division 2000 the cmp argument is not supported in 3.x 789 comparing unequal types not supported in 3.x 262 __getitem__ not supported for exception classes in 3.x; use args attribute 88 sys.exc_clear() not supported in 3.x; use except clauses 57 in 3.x, __getslice__ has been removed; use __getitem__ 14 file.softspace not supported in 3.x 12 the commands module has been removed in Python 3.0; use the subprocess module instead 7 execfile() not supported in 3.x; use exec() 7 tmpfile has been removed in 3.x; use the tempfile module 5 dict inequality comparisons not supported in 3.x 3 2 Use the Permutations object instead. 2 This class is replaced by Matrix_modn_dense_float/Matrix_modn_dense_double. 2 to_permutation() is deprecated. Use instead reading_word_permutation() 2 this class is deprecated. Use RibbonShapedTableau instead 2 BaseException.message has been deprecated as of Python 2.6 2 The 'new' module has been removed in Python 3.0; use the 'types' module instead. 2 this is done automatically by the doctest framework 2 derangements() is deprecated. Use Derangements instead. 1 The output of Transducer.cartesian_product will change. Please use Transducer.intersection for the original output. 1 Substitution using function-call syntax and unnamed arguments is deprecated and will be removed from a future release of Sage; you can use named arguments instead, like EXPR(x=..., y=...) 1 This method is deprecated. Use triangulate() instead. 1 In 3.x, reload() is renamed to imp.reload() 1 dict.has_key() not supported in 3.x; use the in operator 1 The output of 1 Substitution using 1 This method is deprecated. Use triangulate().simplicial_complex() instead. 1 Variable of integration should be specified explicitly. 1 OrderedAlphabet is deprecated; use Alphabet instead.
Unfortunately typical stdout
printout looks like
********************************************************************** File "src/sage/modular/modsym/space.py", line 929, in sage.modular.modsym.space.ModularSymbolsSpace._q_expansion_module Failed example: ModularSymbols(11, 2, base_ring=GF(4,'a')).cuspidal_submodule()._q_expansion_module(prec=4, algorithm="hecke") Expected: Vector space of degree 4 and dimension 1 over Finite Field in a of size 2^2 Basis matrix: [0 1 0 1] Got: doctest:127: DeprecationWarning: type inequality comparisons not supported in 3.x doctest:511: DeprecationWarning: classic int division doctest:250: DeprecationWarning: type inequality comparisons not supported in 3.x doctest:585: DeprecationWarning: classic int division doctest:286: DeprecationWarning: classic int division doctest:396: DeprecationWarning: classic int division doctest:6464: DeprecationWarning: classic int division Vector space of degree 4 and dimension 1 over Finite Field in a of size 2^2 Basis matrix: [0 1 0 1] **********************************************************************
and I don't know how to correlate line likes doctest:511
with the original line.
The line in stderr are more useful, but are few and far between
~/sage-6.2.beta8/local/lib/python2.7/site-packages/sage/groups/perm_gps/cubegroup.py:368: DeprecationWarning: classic int division 2: [1/2,3/2,5/2], # ub
comment:3 Changed 6 years ago by
- Milestone changed from sage-6.2 to sage-6.3
comment:4 Changed 6 years ago by
- Dependencies set to #16371
Hi Wilfried,
in #16371 I changed some /
to //
, maybe you can review it.
BTW: how you got the summary of stdout
and stderr
?
comment:5 Changed 6 years ago by
For the summary see the small attached Py script ;-)
I will look at your ticket.
comment:6 Changed 6 years ago by
- Milestone changed from sage-6.3 to sage-6.4
comment:7 Changed 5 years ago by
- Dependencies changed from #16371 to #16371, #18578
comment:8 Changed 5 years ago by
One very small effort in the spirit of this ticket: #19535
comment:9 Changed 5 years ago by
Also #19536.
comment:10 in reply to: ↑ 1 Changed 5 years ago by
Replying to wluebbe:
But currently I am stuck:
- I don't know how to pass the option
-Qwarnall
to the Python interpreter when running the doctests
Could you try the following:
From a terminal, run sage -sh
From this shell, run:
export PYTHONIOENCODING="utf-8" python -Qwarnall $(which sage-runtests) <options>
Where <options>
is something like --all --long
(depending on the options you are using for doctesting) ?
comment:11 Changed 4 years ago by
- Component changed from distribution to python3
comment:12 Changed 4 years ago by
- Description modified (diff)
- Milestone changed from sage-6.4 to sage-7.3
comment:13 Changed 4 years ago by
- Milestone changed from sage-7.3 to sage-7.4
one step done in #21127
comment:14 Changed 4 years ago by
- Milestone changed from sage-7.4 to sage-7.6
comment:15 Changed 3 years ago by
another step in #22767
comment:16 Changed 3 years ago by
- Milestone changed from sage-7.6 to sage-8.0
comment:17 Changed 3 years ago by
- Milestone changed from sage-8.0 to sage-8.1
another step in #23731
comment:18 Changed 3 years ago by
another step in #24548
comment:19 Changed 3 years ago by
- Description modified (diff)
- Keywords division added; python3 removed
- Owner changed from (none) to embray
comment:20 Changed 3 years ago by
- Description modified (diff)
Some more status tracking for division-related work.
comment:21 Changed 3 years ago by
- Description modified (diff)
comment:22 Changed 16 months ago by
- Cc embray jdemeyer fbissey tscrim jhpalmieri vklein added
- Dependencies #16371, #18578 deleted
- Milestone changed from sage-8.1 to sage-duplicate/invalid/wontfix
- Status changed from new to needs_review
I think that we are now ready to close this one. Agreed ?
comment:23 Changed 16 months ago by
- Status changed from needs_review to positive_review
comment:24 Changed 16 months ago by
- Resolution set to fixed
- Status changed from positive_review to closed
The first attempt went like this:
futurize -w -f division src/
.This added
from __future__ import division
to 411 py modules.Running the doctests
./sage -t -p --all
gave billions of lines of doctest failures ... - I had to cancel the run.But this result should not have come as a surprise because I had not fixed anything yet ...
The task is to identify those
/
and/=
that operate solely on ints and longs and then change them to//
and//=
.The Python source distribution has a script
Tools/scripts/finddiv.py
which has identified similar 416 py modules as the above programs (it performs clearly better as a simple grep since it tokenizes the code). It also outputs those 2625 lines that contain the / and /= operators - but for all types of operands (the analysis is purely syntactical).There is another script
Tools/scripts/fixdiv.py
that uses the output ofpython -Qwarnall yourscripts 2>the-warnings
to create a simple (commented) patch file for those division operators that only deal with ints and longs. This is possible since it uses the type information obtained by running the Python programs. This should be much more useful!But currently I am stuck:
-Qwarnall
to the Python interpreter when running the doctestsIt may be possible to use the
warnings
module to write the warnings to a file instead if stderr (by replacingwarnings.showwarning()
with a custom function).