Opened 6 months ago

Closed 5 months ago

#27886 closed enhancement (fixed)

Misc pip upgrades again

Reported by: chapoton Owned by:
Priority: major Milestone: sage-8.9
Component: packages: standard Keywords: upgrade, cython
Cc: slelievre, vbraun, embray, jdemeyer, fbissey Merged in:
Authors: Frédéric Chapoton Reviewers: John Palmieri
Report Upstream: N/A Work issues:
Branch: 34aba48 (Commits) Commit: 34aba4806942cd2478530faeaac4c36a18085f95
Dependencies: Stopgaps:

Change History (28)

comment:1 Changed 6 months ago by jhpalmieri

Some of these are only for Python 3: ipython, ipykernel, matplotlib, Sphinx, rpy2 (?), possibly others. In some cases (matplotlib, rpy2) we may be able to upgrade one or two minor versions and still maintain compatibility with Python 2, but we can't upgrade to the latest version yet.

comment:2 Changed 6 months ago by slelievre

To take into account py2 / py3, upgrade this:

package upstream             version    latest   wheel  py3     spkg name in Sage
official name                in Sage    py2-ok   sdist  only    if different
-------------------------------------------------------------------------------
Babel                        2.6.0      2.7.0    wheel          babel
backports.ssl-match-hostname 3.5.0.1    3.7.0.1  sdist          backports_ssl_match_hostname
bleach                       3.0.2      3.1.0    wheel
certifi                      2018.11.29 2019.3.9 wheel
configparser                 3.5.0      3.7.4    wheel
cvxopt                       1.1.8      1.2.3    wheel
cypari2                      2.1.0      2.1.1    sdist
Cython                       0.29.5     0.29.8   wheel          cython
decorator                    4.3.0      4.4.0    wheel
defusedxml                   0.5.0      0.6.0    wheel
entrypoints                  0.2.3      0.3      wheel
Flask                        0.10.1     1.0.3    wheel          flask
Flask-AutoIndex              0.6        0.6.2    sdist          flask_autoindex
Flask-Babel                  0.9        0.12.2   sdist          flask_babel
ipykernel                    4.8.2      4.10.0   wheel  5+
Jinja2                       2.10       2.10.1   wheel          jinja2
jsonschema                   2.6.0      3.0.1    wheel
kiwisolver                   1.0.1      1.1.0    wheel
MarkupSafe                   1.1.0      1.1.1    wheel          markupsafe
nbconvert                    5.4.0      5.5.0    wheel
networkx                     2.2        2.3      sdist
notebook                     5.7.6      5.7.8    wheel
numpy                        1.16.1     1.16.3   wheel
packaging                    18.0       19.0     wheel
path.py                      7.1        11.5.2   wheel  12+     pathpy
pexpect                      4.6.0      4.7.0    wheel
Pillow                       5.3.0      6.0.0    wheel          pillow
pip                          18.1       19.1.1   wheel
pkgconfig                    1.4.0      1.5.1    wheel
prometheus-client            0.5.0      0.6.0    sdist          prometheus_client
prompt-toolkit               1.0.15     2.0.9    wheel          prompt_toolkit
psutil                       5.2.0      5.6.2    sdist
ptyprocess                   0.5.1      0.6.0    wheel
Pygments                     2.3.1      2.4.2    wheel          pygments
pyparsing                    2.3.0      2.4.0    wheel
python-dateutil              2.5.3      2.8.0    wheel          dateutil
pytz                         2018.7     2019.1   wheel
pyzmq                        17.1.2     18.0.1   wheel
requests                     2.13.0     2.22.0   wheel
rpy2                         2.8.2      2.8.6    sdist  2.9+
scandir                      1.9.0      1.10.0   sdist
scipy                        1.2.0      1.2.1    wheel  1.3+
setuptools                   40.6.3     41.0.1   wheel
setuptools-scm               3.1.0      3.3.3    wheel          setuptools_scm
sphinxcontrib-websupport     1.1.0      1.1.2    wheel          sphinxcontrib_websupport
terminado                    0.8.1      0.8.2    wheel
tornado                      4.5.2      5.1.1    sdist  6+
Twisted                      16.3.0     19.2.0   sdist          twisted
Werkzeug                     0.14.1     0.15.4   wheel          werkzeug

Cannot upgrade this until we drop Python 2:

ipython                      5.8.0      5.8.0    wheel  6+
matplotlib                   2.2.3      2.2.3    wheel  3+
Sphinx                       1.8.5      1.8.5    wheel  2+     sphinx

Additional possible PyPI upgrades:

readline 6.3.008.p0 --> gnureadline 6.3.8
pyx 0.12.1.p0 --> 0.14.1
scons 1.2.0 --> 3.0.5

comment:3 Changed 6 months ago by slelievre

  • Cc slelievre added
  • Component changed from PLEASE CHANGE to packages: standard

comment:4 follow-up: Changed 6 months ago by jhpalmieri

It looks to me as though matplotlib 2.2.4 would be okay for Python 2, but nothing beyond that.

comment:5 in reply to: ↑ 4 Changed 6 months ago by fbissey

Replying to jhpalmieri:

It looks to me as though matplotlib 2.2.4 would be okay for Python 2, but nothing beyond that.

Using it in sage-on-gentoo.

comment:6 Changed 6 months ago by chapoton

  • Cc vbraun added

as in #26969, we may need to exclude "ptyprocess" and "tornado"

Volker, do you have an easy way to do this kind of batch upgrade, as you did it before ?

comment:7 Changed 5 months ago by chapoton

  • Branch set to public/27886
  • Commit set to 1f1ec27c3a35d4c62e825a47567d67cb0e291431

New commits:

1f1ec27some packages upgrade

comment:8 Changed 5 months ago by git

  • Commit changed from 1f1ec27c3a35d4c62e825a47567d67cb0e291431 to 7d825c350afc01515a67b9827c628567c5c057f8

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7d825c3some packages upgrade

comment:9 Changed 5 months ago by chapoton

  • Authors set to Frédéric Chapoton
  • Cc embray jdemeyer fbissey added
  • Status changed from new to needs_review

ok, maybe this is a good first step. Includes an update of cython.

comment:10 Changed 5 months ago by chapoton

  • Status changed from needs_review to needs_work

cypari does not build. Should I add back the cython trashcan patch ?

comment:11 Changed 5 months ago by git

  • Commit changed from 7d825c350afc01515a67b9827c628567c5c057f8 to e226ab8649d4bd3686acb8ed5919e3eaf9ab50ff

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

e226ab8some packages upgrade, including cython

comment:12 Changed 5 months ago by chapoton

  • Status changed from needs_work to needs_review

ok, I have added back the cython patch. Back to needs review.

comment:13 Changed 5 months ago by slelievre

  • Description modified (diff)
  • Keywords upgrade cython added

Updating the ticket description to reflect the actual changes.

comment:14 Changed 5 months ago by jhpalmieri

  • Description modified (diff)
  • Reviewers set to John Palmieri

Works for me on OS X, both Python 2 and 3. Others should try on other platforms.

Unfortunately it doesn't fix the Python 3 doctest failure in cpython.

comment:15 Changed 5 months ago by chapoton

Which failure remains in the cpython folder under python3 ? Both of these :

┣ debug.pyx  # 1
┗ dict_del_by_value.pyx  # 2

or just one ? The upgrade is supposed only to fix the second file.

Version 0, edited 5 months ago by chapoton (next)

comment:16 Changed 5 months ago by jhpalmieri

I still get both failures. I'll try make distclean and then rebuild to see if that helps.

comment:17 Changed 5 months ago by jhpalmieri

After make distclean, same result: both files have failures.

comment:18 Changed 5 months ago by chapoton

And Jeroen, who knows about cython and prevented me to fix that in a naive way, seems not to be very active here at the moment.

comment:19 Changed 5 months ago by jhpalmieri

I see:

test_del_dictitem_by_exact_value(D, ZZ, 2)

fails, but

test_del_dictitem_by_exact_value(D, ZZ, int(2))

works. What was your naive fix? I would be tempted to use

test_del_dictitem_by_exact_value(D, ZZ, hash(2))

This is more robust, since it works in this kind of case:

sage: D = {1230981308409180310928308123: ZZ}
sage: test_del_dictitem_by_exact_value(D, ZZ, hash(1230981308409180310928308123))

while

sage: D = {1230981308409180310928308123: ZZ}
sage: test_del_dictitem_by_exact_value(D, ZZ, 1230981308409180310928308123)

fails with both Python 2 and 3. The current doctest relies on the fact that the hash of the Sage integer 2 is equal to 2, and this doesn't generalize to integers that are too large.

comment:20 Changed 5 months ago by chapoton

ok, I will do that in another ticket, thanks for the suggestion.

see #27972

Last edited 5 months ago by chapoton (previous) (diff)

comment:21 Changed 5 months ago by jdemeyer

The test_del_dictitem_by_exact_value failure is fixed upstream at https://github.com/cython/cython/commit/28251032f86c266065e4976080230481b1a1bb29

I suggest to just apply that patch instead of "fixing" the doctest.

comment:22 Changed 5 months ago by chapoton

But this is supposed to be in cython 0.29.3 ??

6 months ago, in #26855 you said : Fixed upstream, but not in a stable release

comment:23 Changed 5 months ago by jdemeyer

It's not fixed in a Cython bugfix release it seems, but only in the next major release.

comment:24 Changed 5 months ago by jdemeyer

...or maybe it was meant to be fixed in 0.29.x but forgotten.

comment:25 Changed 5 months ago by git

  • Commit changed from e226ab8649d4bd3686acb8ed5919e3eaf9ab50ff to 34aba4806942cd2478530faeaac4c36a18085f95

Branch pushed to git repo; I updated commit sha1. New commits:

34aba48Add Cython patch to fix Py_hash_t conversion

comment:26 Changed 5 months ago by jhpalmieri

This change does fix the py3 doctest failures in cpython/dict_del_by_value.pyx. I was hoping it would fix some of the doctests in integer.pyx, but it doesn't seem to have done that.

comment:27 Changed 5 months ago by jhpalmieri

  • Milestone changed from sage-8.8 to sage-8.9
  • Status changed from needs_review to positive_review

Let's merge this into an early 8.9 beta.

comment:28 Changed 5 months ago by vbraun

  • Branch changed from public/27886 to 34aba4806942cd2478530faeaac4c36a18085f95
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.