Opened 3 years ago
Closed 2 years ago
#28190 closed enhancement (invalid)
Allow different versions of python modules depending on the python version used
Reported by: | arojas | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | packages: standard | Keywords: | |
Cc: | vdelecroix, gh-timokau, fbissey, embray, jdemeyer, chapoton, vbraun | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | public/ticket/28190 (Commits, GitHub, GitLab) | Commit: | ecb7133e0357c8693c63efad85998df2500b713d |
Dependencies: | Stopgaps: |
Description
See https://groups.google.com/forum/#!topic/sage-packaging/i9VosTO9-cM for details.
Until python2 is dropped, many packages which have dropped python 2 support upstream (ipython, sphinx, scipy, networkx, matplotlib...) are stuck to old, unmaintained versions in sage. This will pose a particular problem for distros once Sage defaults to python 3, since those distros will probably be carrying newer versions of these dependencies and sage will have to be heavily patched to work with them.
The aim of this ticket is to allow different versions for python2/python3 packages, so the python3 versions can be kept up to date in sage itself and the relevant patches can be upstreamed.
Change History (20)
comment:1 Changed 3 years ago by
- Cc vdelecroix added
comment:2 Changed 3 years ago by
- Cc gh-timokau added
comment:3 Changed 3 years ago by
- Cc fbissey added
comment:4 Changed 3 years ago by
comment:5 follow-up: ↓ 6 Changed 3 years ago by
Perhaps it would be simpler to keep the python2 and python3 versions as separate pkgs? (build/pkgs/sphinx vs build/pkgs/python2-sphinx)
comment:6 in reply to: ↑ 5 Changed 3 years ago by
Replying to arojas:
Perhaps it would be simpler to keep the python2 and python3 versions as separate pkgs? (build/pkgs/sphinx vs build/pkgs/python2-sphinx)
+1
It will also be simpler to remove the python2-*
repos when the life of Python2 terminates.
comment:7 Changed 3 years ago by
That follows some model of binary distros, and I am OK with that. Implementation will have some kinks as it relies on the relationship between the folder name, the name of the package for dependencies and the name of the source tarball. I would say the latest is the hardest bit to implement - dependencies not so much, it is just tedious work.
comment:8 Changed 3 years ago by
- Cc embray jdemeyer added
Who is going to take care of that ?
- I would suggest not to change the name of the packages, not to duplicate.
- Rather have (when required) inside build/pkg folders some new files
checksums-legacy.ini
andpackage-version-legacy.txt
- Change the script in the appropriate place (where ?) so that it chooses the version to install according to the python version.
Erik, Jeroen : your opinions on the problem, please ?
comment:9 Changed 3 years ago by
- Branch set to public/ticket/28190
- Commit set to ecb7133e0357c8693c63efad85998df2500b713d
as a kind of provocation, here is a sketch of tentative..
New commits:
ecb7133 | first tentative of allowing a legacy version of packages
|
comment:10 in reply to: ↑ description ; follow-ups: ↓ 11 ↓ 15 Changed 3 years ago by
Let's start by analyzing the problem and possible solutions before discussing implementation details.
For a given package, let V2 be the latest version supporting Python 2 and let V3 be the latest version supporting Python 3. There are several possibilities (assuming obviously that V3 >= V2):
(A) V2 == V3: the package continues to support Python 2.
(B) V2 < V3 but the versions are compatible: the functionality that Sage uses works the same way with both versions.
(C) V2 and V3 are incompatible.
In case (A), nothing need to be done, so we don't need to discuss this further. In case (B), it doesn't really matter what we do: even if a distro uses a different version from Sage, there is no problem. So we really need to worry only about case (C). So first of all a serious question: are there packages where (C) currently applies? If not, we're solving a non-problem.
For (C), I see three possible solutions:
(i) continue using V2 in Sage and leave it up to the distros to deal with that.
(ii) use V2 when compiling with Python 2 and V3 when compiling with Python 3.
(iii) use V3 and drop support for Python 2 in Sage completely.
This ticket proposes (ii) but I'm not convinced that this is what we should do. It means that code in the Sage library needs to deal with supporting two incompatible versions of a package. That will complicate the code and increase chances of breakage.
If we can wait, maybe (iii) is actually a viable option: Python 2 is officially unsupported half a year from now. If Sage fully works under Python 3 by then, maybe we can drop Python 2 support at that time.
comment:11 in reply to: ↑ 10 ; follow-up: ↓ 12 Changed 3 years ago by
Replying to jdemeyer:
So first of all a serious question: are there packages where (C) currently applies? If not, we're solving a non-problem.
Yes (see the linked sage-packaging post)
If we can wait, maybe (iii) is actually a viable option: Python 2 is officially unsupported half a year from now. If Sage fully works under Python 3 by then, maybe we can drop Python 2 support at that time.
That would be a solution, but would you really consider going from defaulting to python2 to completely dropping it without any deprecation period?
comment:12 in reply to: ↑ 11 ; follow-up: ↓ 14 Changed 3 years ago by
Replying to arojas:
would you really consider going from defaulting to python2 to completely dropping it without any deprecation period?
What would that deprecation mean in practice?
From the user's point of view, Sage uses either Python 2 or Python 3 (whatever we default to at installation time), so there is going to be a sudden break anyway sooner or later. Once you're defaulting to Python 3, I don't know if we should still put effort in supporting Python 2.
comment:13 Changed 3 years ago by
As mentioned by Samuel on the mailing list, there is a distinct situation where we might want to upgrade packages to Python3 before the transition py2 -> py3: the jupyter notebook. It is not exactly covered by the ticket description.
More concretely, we could
- python3 install the modules:
jupyter_core
,jupyter_client
,traitlets
,decorator
,enum34
,pyzmq
,dateutil
- remove python2 installation of
jupyter_core
andjupyter_client
(and dead dependencies) - run the jupyter notebook under python3
Question: would that really be useful?
comment:14 in reply to: ↑ 12 Changed 3 years ago by
Replying to jdemeyer:
Replying to arojas:
would you really consider going from defaulting to python2 to completely dropping it without any deprecation period?
What would that deprecation mean in practice?
From the user's point of view, Sage uses either Python 2 or Python 3 (whatever we default to at installation time), so there is going to be a sudden break anyway sooner or later. Once you're defaulting to Python 3, I don't know if we should still put effort in supporting Python 2.
+1 on that point. If users don't want to migrate right away, they can continue using an older sage version. That may have security implications of course, but since some dependencies won't receive any updates anyway those implications are there either way.
There is little point to the "python2 deadline" if half of the projects decide "just a little longer".
comment:15 in reply to: ↑ 10 Changed 3 years ago by
Replying to jdemeyer:
This ticket proposes (ii) but I'm not convinced that this is what we should do. It means that code in the Sage library needs to deal with supporting two incompatible versions of a package. That will complicate the code and increase chances of breakage.
I don't think that's such a problem. Yes, it's annoying. But with care, it can be done fairly cleanly, and explicitly supporting multiple versions of a depdency can be more stable.
E.g. in Astropy we support something like 4 different Numpy versions, all of which occasionally need different version-specific worksarounds or features. This is usually done explicitly in the code with pre-defined constants like if NUMPY_LT_1_11: ...
.
comment:16 Changed 2 years ago by
- Milestone changed from sage-8.9 to sage-9.1
Ticket retargeted after milestone closed
comment:17 Changed 2 years ago by
Now that 9.0 is out, what is the plan for this? Is python2 going to be dropped for 9.1? My current ipython 7 patch is almost 5000 LOC and getting harder to maintain, it would be nice to upstream it. Also, updating to python 3.8 would be more complicated if python2 support is to be kept, see https://trac.sagemath.org/ticket/27754#comment:55
comment:18 Changed 2 years ago by
I don't think there should be a rush to upgrade to Python 3.8. With the first Python 3-based release just out, let's give it a little breathing room for things to shake out, as I'm sure it will still be a rocky transition for some users at first (hopefully not too much so though).
comment:19 Changed 2 years ago by
- Cc chapoton vbraun added
- Milestone changed from sage-9.1 to sage-duplicate/invalid/wontfix
- Status changed from new to needs_review
comment:20 Changed 2 years ago by
- Resolution set to invalid
- Status changed from needs_review to closed
I agree, let us move forward.
Do you have ideas about how to implement this? Maybe a directory
build/pkgs/python2-packages/
and if building with Python 2, first look there for the directory, and if that fails, look inbuild/pkgs
? Or instead, withinbuild/pkgs/sphinx/
(for example), have apython2
directory with py2 build information. In any case, I think that if there is a default setting, it should be Python 3.