Opened 14 months ago

Closed 5 months ago

#30551 closed task (fixed)

Drop Python 3.6 support

Reported by: slelievre Owned by:
Priority: major Milestone: sage-9.4
Component: python3 Keywords:
Cc: chapoton, mkoeppe, mjo, dimpase, vbraun Merged in:
Authors: Tobias Diez, Matthias Koeppe Reviewers: Matthias Koeppe, Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: b06731c (Commits, GitHub, GitLab) Commit: b06731c29501c81a35bba514ce11a957174732f0
Dependencies: #30913 Stopgaps:

Status badges

Description (last modified by dimpase)

This can be a dependency for other tickets.

For instance tickets updating dependencies to versions that dropped support for Python 3.6.

Related:

  • #30384: Adopt "time window-based" Python versions support policy
  • #29756: Meta-ticket: Review of Python 3 features that sagelib should use systematically
  • #30053 - can be dropped if 3.6 is gone

Change History (53)

comment:1 follow-up: Changed 14 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-pending

I don't think there is anything that says that we "should" drop Python 3.6 support.

comment:2 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:3 in reply to: ↑ 1 Changed 14 months ago by slelievre

  • Cc chapoton added
  • Description modified (diff)

Replying to mkoeppe:

I don't think there is anything that says that we "should" drop Python 3.6 support.

Oh good, then let's support it as long as we can.

By the way #15530 and #15980 which started support for Python 3.6 are almost ready to be closed.

comment:4 Changed 13 months ago by dimpase

#30053 says that we must drop Python 3.6 - unless you want to have a go at it.

it seems that some of our code related to docbuilding is too new for 3.6, and perhaps our Sphinx is too new too.

Besides, the EOL for 3.6 is late 2021, which is soon.

comment:5 Changed 13 months ago by dimpase

  • Description modified (diff)

comment:6 follow-up: Changed 11 months ago by gh-tobiasdiez

What do you think about adopting the Python version strategy of numpy, according to which "a project with a major or minor version release in November 2020 should support Python 3.7 and newer"?

comment:7 in reply to: ↑ 6 Changed 11 months ago by mkoeppe

Replying to gh-tobiasdiez:

What do you think about adopting the Python version strategy of numpy, according to which "a project with a major or minor version release in November 2020 should support Python 3.7 and newer"?

That's #30384

comment:8 Changed 11 months ago by gh-tobiasdiez

  • Milestone changed from sage-pending to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

According to NEP 29, "a project with a major or minor version release in November 2020 should support Python 3.7 and newer." and thus this ticket can be closed in my opinion.

(Documentation for following NEP 29 is added in #30384.)

comment:9 Changed 11 months ago by mkoeppe

  • Milestone changed from sage-duplicate/invalid/wontfix to sage-pending

??!

This ticket is for dropping support for Python 3.6 -- this involves changing what system python versions we accept (build/pkgs/python3/spkg-configure.m4) and removing python3.6 workarounds from the code.

NEP 29 would already allow us to drop support now -- but I don't think we want to do it now (because of Sage's traditional role of providing a distribution that people can install onto ancient machines) -- hence the milestone is set to "pending".

comment:10 Changed 11 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:11 Changed 11 months ago by gh-tobiasdiez

  • Branch set to public/python3/drop36
  • Cc mjo dimpase added
  • Commit set to c5436df859dece3ca97fb892ed95d990ff8503ec
  • Status changed from needs_work to needs_review

Sorry, it wasn't clear to me from the ticket description that this is more than a placeholder to mark dependencies on Python 3.6.

NEP 29 is actually quite explicit about that one should drop support after the 42 months, e.g "On Jun 23, 2020 drop support for Python 3.6 (initially released on Dec 23, 2016)". Of course, it could be sage's policy to support also older python versions, but that's then no longer NEP 29 (in it's strict form). However, one should also keep in mind that supporting old Python versions results in technical debt that might be bigger than the gains from supporting old systems.

Also correct me if I'm wrong, but since sage is also installing its own python version, sage can still be used on older systems, right? It is just that the system python is not used.

Anyway, I've tried to locate all workarounds that are in place for Python 3.6 support, and removed them (I've probably missed a few, but maybe its good anyway to pay of the technical debt in follow up tickets). Moreover, I've upgraded the requirements to 3.7 (again, I might have missed something).


New commits:

7d97a56Partially revert "Allow Sage to work with a system Python 3.6."
2ed040bRevert #30576
01000e8Partially revert #30008
c5436dfRequire at least Python 3.7

comment:12 Changed 10 months ago by slelievre

Other projects dropping support for Python 3.6:

Last edited 10 months ago by slelievre (previous) (diff)

comment:13 follow-up: Changed 10 months ago by mkoeppe

In this change:

 - **ar** and **ranlib**: can be obtained as part of GNU binutils.
 - **tar**: GNU tar version 1.17 or later, or BSD tar.
-- **python**: Python 3, 3.6 or later, or Python 2.7 (deprecated).
+- **python**: Python 3.7 or later (Python 2.7 or Python <= 3.6 support is deprecated).

Both the original text and the new one are not correct. This part refers to the sage-bootstrap-python which has much wider range of supported python versions, including Python >= 2.6 and Python >= 3.4 - according to build/tox.ini.

comment:14 Changed 10 months ago by git

  • Commit changed from c5436df859dece3ca97fb892ed95d990ff8503ec to 5c9983da275bd46317ad39d0694afecbff53af6e

Branch pushed to git repo; I updated commit sha1. New commits:

5c9983dCorrect versions for installation from source

comment:15 in reply to: ↑ 13 Changed 10 months ago by gh-tobiasdiez

Replying to mkoeppe:

In this change:

 - **ar** and **ranlib**: can be obtained as part of GNU binutils.
 - **tar**: GNU tar version 1.17 or later, or BSD tar.
-- **python**: Python 3, 3.6 or later, or Python 2.7 (deprecated).
+- **python**: Python 3.7 or later (Python 2.7 or Python <= 3.6 support is deprecated).

Both the original text and the new one are not correct. This part refers to the sage-bootstrap-python which has much wider range of supported python versions, including Python >= 2.6 and Python >= 3.4 - according to build/tox.ini.

I've changed the versions now.

comment:16 Changed 10 months ago by mkoeppe

  • Authors set to Tobias Diez
--- a/src/doc/en/installation/source.rst
+++ b/src/doc/en/installation/source.rst
@@ -88,7 +88,7 @@ computer:
 - **perl**: version 5.8.0 or later.
 - **ar** and **ranlib**: can be obtained as part of GNU binutils.
 - **tar**: GNU tar version 1.17 or later, or BSD tar.
-- **python**: Python 3, 3.6 or later, or Python 2.7 (deprecated).
+- **python**: Python 3.7 or later (Python >= 2.6 and <= 3.6 support is deprecated).
 
 Other versions of these may work, but they are untested.

I don't think we have deprecated anything regarding sage-bootstrap-python versions. (see #20023 - which proposes to drop support for 2.6)

comment:17 Changed 10 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:18 Changed 10 months ago by mkoeppe

  • Milestone changed from sage-pending to sage-9.4

comment:19 Changed 10 months ago by mkoeppe

I'm splitting this documentation update out to #31192 to get it into Sage 9.3

comment:20 Changed 10 months ago by mkoeppe

  • Dependencies set to #31192

comment:21 Changed 10 months ago by git

  • Commit changed from 5c9983da275bd46317ad39d0694afecbff53af6e to dd9b8e5ee88fa2476455584f9244257d95ccdc6f

Branch pushed to git repo; I updated commit sha1. New commits:

4265656src/doc/en/installation/source.rst: Fix documentation of python prerequisite (sage-bootstrap-python)
dd9b8e5Merge branch 't/31192/installation_manual__fix_documentation_of_python_prerequisite__sage_bootstrap_python_' into t/30551/public/python3/drop36

comment:22 Changed 9 months ago by gh-tobiasdiez

  • Status changed from needs_work to needs_review

comment:23 Changed 9 months ago by gh-tobiasdiez

  • Milestone changed from sage-9.4 to sage-9.3

comment:24 Changed 9 months ago by dimpase

what's going on in src/sage/combinat/subset.py change in this branch?

comment:25 follow-up: Changed 9 months ago by gh-tobiasdiez

That's a revert of commit "Allow Sage to work with a system Python 3.6." abb5607710a2a5cc5415423489fa452965cbef68 which is introduced a workaround that is no longer required for Python 3.7.

comment:26 Changed 9 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

comment:27 in reply to: ↑ 25 Changed 9 months ago by dimpase

Replying to gh-tobiasdiez:

That's a revert of commit "Allow Sage to work with a system Python 3.6." abb5607710a2a5cc5415423489fa452965cbef68 which is introduced a workaround that is no longer required for Python 3.7.

I mean, what's

    def __len__....
...
-        return int(self.cardinality())
+        return Integer(1) << self._s.cardinality()
+
+    __len__ = cardinality

in

commit 7d97a56c5aaab99af75de58b71b144a28d4f9dd3
Author: Tobias Diez <code@tobiasdiez.com>
Date:   Mon Dec 7 23:52:28 2020 +0100

    Partially revert "Allow Sage to work with a system Python 3.6."
    

why do we need an implementation for __len__ ? (before abb5607710a it was just __len__ = cardinality)

comment:28 Changed 9 months ago by gh-tobiasdiez

Sorry, I don't understand your question. In pseudo-language:

Before abb5607710a: __len__ = cardinality

After abb5607710a: __len__ = int(cardinality)

After 7d97a56c5a (this ticket): __len__ = cardinality

So we are back to the original behavior before abb5607710a "Allow Sage to work with a system Python 3.6."

Maybe this representation of the diff is clearer: https://github.com/sagemath/sagetrac-mirror/commit/7d97a56c5aaab99af75de58b71b144a28d4f9dd3

Last edited 9 months ago by gh-tobiasdiez (previous) (diff)

comment:29 Changed 9 months ago by dimpase

Oh, OK, I misread the diff, sorry.

comment:30 Changed 9 months ago by mkoeppe

  • Dependencies changed from #31192 to #31192, #31318

comment:31 Changed 8 months ago by mkoeppe

  • Work issues set to also revert #31191

comment:32 Changed 6 months ago by mkoeppe

  • Dependencies changed from #31192, #31318 to #30913

comment:33 Changed 6 months ago by git

  • Commit changed from dd9b8e5ee88fa2476455584f9244257d95ccdc6f to b06731c29501c81a35bba514ce11a957174732f0

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

f691585Merge tag '9.3.beta9' into t/30913/sagelib__add_setup_cfg__install_requires_
66c2987build/pkgs/{sage_sws2rst,sage_docbuild}/install-requires.txt: New
7f8ec6cbuild/pkgs/sagelib/src/tox.ini: Add factor sitepackages
0283da5build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system
f720722build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation
c451b31src/setup.cfg.m4 (install_requires): Add sage_conf
6700223Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_
04da2c6build/pkgs/ipywidgets: Patch out declaring install-requires of nbformat and jupyterlab-widgets
815c944Merge #30913
b06731cRemove Python 3.6 support from metadata and documentation

comment:34 Changed 6 months ago by mkoeppe

  • Authors changed from Tobias Diez to Tobias Diez, Matthias Koeppe
  • Reviewers set to Matthias Koeppe, ...
  • Work issues also revert #31191 deleted

Rebased on 9.3.rc4 and #30913 (which added relevant metadata).

Let's merge this at the beginning of the 9.4 cycle.

comment:35 Changed 6 months ago by dimpase

  • Reviewers changed from Matthias Koeppe, ... to Matthias Koeppe, Dima Pasechnik
  • Status changed from needs_review to positive_review

lgtm

comment:36 Changed 5 months ago by vdelecroix

Did you take into account that Ubuntu 18.04 (LTS supported until 2028) still uses Python 3.6 (see https://packages.ubuntu.com/bionic/python3)? I am not sure what they intend to do with Python though.

comment:37 Changed 5 months ago by mkoeppe

Yes, these systems are out there. From Sage 9.4 on we will no longer be able to use the system python, but this will not stop Sage from building and running on this system.

comment:38 Changed 5 months ago by slelievre

There are two relevant Python minimum versions (see comment:13):

  • the minimum Python version Sage can use as its underlying Python
  • the minimum Python version Sage can use to bootstrap, building is own Python if necessary

On Ubuntu 18.04, building from source will still work, but if the system Python is 3.6, Sage will build a more recent Python to base itself on.

comment:39 Changed 5 months ago by dimpase

no need to build it on Debian/Ubuntu, even the old LTS provide modern pythons. E.g.

sudo apt-get install python3.8 python3.8-dev python3.8-distutils python3.8-venv

and if you don't have root you can use pyenv to get modern python.

We should be thinking about stopping with shipping CPython sources, we don't patch them, and there are enough ways to get it in binary.

Last edited 5 months ago by dimpase (previous) (diff)

comment:40 Changed 5 months ago by dimpase

a slight inconvenience is that you'd need to specify the Python to use to ./configure

comment:41 follow-up: Changed 5 months ago by mkoeppe

  • Status changed from positive_review to needs_work

Commit 721ede18042cfee9ca044823599346e1ee7420f1 is not good.

comment:42 Changed 5 months ago by mkoeppe

  • Cc vbraun added

comment:43 in reply to: ↑ 41 ; follow-up: Changed 5 months ago by dimpase

Replying to mkoeppe:

Commit 721ede18042cfee9ca044823599346e1ee7420f1 is not good.

this is

  • build/bin/sage-bootstrap-python

    Author: Tobias Diez <code@tobiasdiez.com>
    Date:   Tue Dec 8 00:05:41 2020 +0100
    
        Partially revert #30008
    
    diff --git a/build/bin/sage-bootstrap-python b/build/bin/sage-bootstrap-python
    index faee444e8d..c3afc1261a 100755
    a b fi 
    3030# is accessible by this python; this is to guard on Cygwin against Pythons
    3131# installed somewhere else in Windows.
    3232
    33 # Trac #30008: Make it work even if the environment tries to sabotage UTF-8
    34 # operation in Python 3.0.x-3.6.x by setting LC_ALL=C or similar.
    35 
    36 if [ "$LC_ALL" = "C" -o "$LANG" = "C" -o "$LC_CTYPE" = "C" ]; then
    37     LC_ALL=$(locale -a | grep -E -i '^(c|en_us)[-.]utf-?8$' | head -n 1)
    38     LANG=$LC_ALL
    39     export LC_ALL
    40     export LANG
    41 fi
    4233
    4334PYTHONS="python python3 python3.8 python3.7 python2.7 python3.6 python2"
    4435for PY in $PYTHONS; do

what's the problem here?

comment:44 in reply to: ↑ 43 Changed 5 months ago by mkoeppe

Replying to dimpase:

what's the problem here?

It is a change to sage-bootstrap-python; but we are NOT changing support for bootstrapping with old Python versions.

comment:45 Changed 5 months ago by git

  • Commit changed from b06731c29501c81a35bba514ce11a957174732f0 to bc27c450b4dbbf0d833e35586e93098cd6201d46

Branch pushed to git repo; I updated commit sha1. New commits:

bc27c45Revert "Partially revert #30008"

comment:46 Changed 5 months ago by mkoeppe

  • Status changed from needs_work to needs_review

comment:47 Changed 5 months ago by mkoeppe

@vbraun - if it's too late for this change, I can also put this reverting commit on a new ticket.

comment:48 Changed 5 months ago by dimpase

  • Status changed from needs_review to positive_review

ok.

comment:49 Changed 5 months ago by mkoeppe

Looks like the earlier version, b06731c29501c81a35bba514ce11a957174732f0, was merged in 9.4.beta0.


New commits:

bc27c45Revert "Partially revert #30008"

comment:50 Changed 5 months ago by git

  • Commit changed from bc27c450b4dbbf0d833e35586e93098cd6201d46 to b06731c29501c81a35bba514ce11a957174732f0
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. This was a forced push. New commits:

comment:51 Changed 5 months ago by mkoeppe

I've put the missing commit on #31860.

comment:52 Changed 5 months ago by mkoeppe

  • Status changed from needs_review to positive_review

comment:53 Changed 5 months ago by vbraun

  • Branch changed from public/python3/drop36 to b06731c29501c81a35bba514ce11a957174732f0
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.