Opened 2 years ago

Closed 16 months ago

Last modified 16 months ago

#30416 closed enhancement (fixed)

Make tox a standard package

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-9.4
Component: packages: standard Keywords:
Cc: John Palmieri, Tobias Diez, Frédéric Chapoton, Samuel Lelièvre, Michael Orlitzky Merged in:
Authors: Matthias Koeppe Reviewers: John Palmieri
Report Upstream: N/A Work issues:
Branch: 65e6330 (Commits, GitHub, GitLab) Commit:
Dependencies: #31699 Stopgaps:

Status badges

Attachments (1)

py-1.10.0.log (26.3 KB) - added by John Palmieri 17 months ago.

Download all attachments as: .zip

Change History (37)

comment:1 Changed 2 years ago by Samuel Lelièvre

Cc: Frédéric Chapoton Samuel Lelièvre added; chapooton removed

comment:2 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:3 Changed 2 years ago by Matthias Köppe

Branch: u/mkoeppe/make_tox_a_standard_package

comment:4 Changed 2 years ago by Matthias Köppe

Authors: Matthias Koeppe
Commit: 6913cd8cf3cb18a6873696bcaf7678e95f60f37a
Status: newneeds_review

New commits:

f8a7ee6build/pkgs/tox: Make it a standard package
471ab84build/pkgs/tox/SPKG.rst: New
6913cd8build/pkgs/tox/dependencies: New

comment:5 Changed 2 years ago by John Palmieri

Should there also be an spkg-configure.m4 file? (As a policy issue, should all new standard packages be encouraged, required if at all possible, to have such files, to try to maintain the momentum toward working well with distros?)

comment:6 Changed 2 years ago by Matthias Köppe

Status: needs_reviewneeds_work

That's a good idea, and I would support adding this policy too.

comment:7 Changed 2 years ago by git

Commit: 6913cd8cf3cb18a6873696bcaf7678e95f60f37a3b510ddb24aac1161817570f09b336f34c8185af

Branch pushed to git repo; I updated commit sha1. New commits:

3b510ddbuild/pkgs/tox/spkg-configure.m4: New

comment:8 Changed 2 years ago by Matthias Köppe

Status: needs_workneeds_review

comment:9 in reply to:  6 Changed 2 years ago by John Palmieri

Replying to mkoeppe:

That's a good idea, and I would support adding this policy too.

See #30543 for documentation for spkg-configure.m4 and the statement of this policy.

comment:10 Changed 2 years ago by John Palmieri

We're missing some dependencies:

install_requires = 
	filelock>=3.0.0
	packaging>=14
	pluggy>=0.12.0
	py>=1.4.17
	six>=1.14.0 # required when virtualenv>=20
	toml>=0.9.4
        virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
	colorama>=0.4.1 ;platform_system=="Windows"
	importlib-metadata>=0.12,<2;python_version<"3.8"

When I run ./sage --tox --version, I get

ModuleNotFoundError: No module named 'pluggy'

comment:11 Changed 2 years ago by Matthias Köppe

Status: needs_reviewneeds_work

Indeed, thanks for catching this.

comment:12 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:13 Changed 2 years ago by Matthias Köppe

Dependencies: #30544
Description: modified (diff)
Milestone: sage-9.2sage-9.3

comment:14 Changed 2 years ago by git

Commit: 3b510ddb24aac1161817570f09b336f34c8185aff8a7ee6c4e498c455b96701c159f55edeb8348bb

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

comment:15 Changed 2 years ago by Michael Orlitzky

Cc: Michael Orlitzky added

comment:16 Changed 22 months ago by git

Commit: f8a7ee6c4e498c455b96701c159f55edeb8348bb9e7cc2aed7f075cba0ff6b0b60ae382f3af58638

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7db399cbuild/pkgs/tox: Make it a standard package
2b3b26dbuild/pkgs: Add tox dependencies: filelock py toml virtualenv pluggy
c50e9f2build/pkgs/packaging: Update to 20.8
9e7cc2abuild/pkgs/tox: Add to dependencies

comment:17 Changed 22 months ago by Matthias Köppe

Added these dependencies using the new tool described at https://wiki.sagemath.org/ReleaseTours/sage-9.3#For_developers:_Setting_up_Python_packages_from_PyPI_as_Sage_packages

Untested so far - possibly needs more dependencies

And probably the dependencies that are only needed by tox should be marked as not required if system tox is found.

comment:18 Changed 21 months ago by Matthias Köppe

virtualenv has the following additional dependencies:

        appdirs>=1.4.3,<2
        distlib>=0.3.1,<1
        filelock>=3.0.0,<4
        six>=1.9.0,<2   # keep it >=1.9.0 as it may cause problems on LTS platforms
        importlib-metadata>=0.12;python_version<"3.8"
        importlib-resources>=1.0;python_version<"3.7"

comment:19 Changed 21 months ago by git

Commit: 9e7cc2aed7f075cba0ff6b0b60ae382f3af58638b51cf6b6e467f1cc8f761ca6bb0698c6a49e534c

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

6f3b864build/pkgs/[s-t]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'
23558f8build/pkgs/[t-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all standard packages
17789dfbuild/pkgs/[z-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all optional packages
944408dbuild/pkgs/[a-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all experimental packages
feafa60src/doc/en/installation/standard_packages.rst: Remove
58c63e1src/doc/bootstrap: Add brief explanation of the standard/optional/experimental package types
54d4fb1extra line after version
565dd46Merge branch 't/29655/improve_build_pkgs___spkg_rst' into t/30416/make_tox_a_standard_package
7f4c0b2build/pkgs/{distlib,appdirs,importlib_resources}: Add dependencies of virtualenv
b51cf6bbuild/pkgs/virtualenv/dependencies: Add deps

comment:20 Changed 21 months ago by Matthias Köppe

Dependencies: #30544#29655

comment:21 Changed 19 months ago by Matthias Köppe

Milestone: sage-9.3sage-9.4

comment:22 Changed 17 months ago by Matthias Köppe

Dependencies: #29655#31699

comment:23 Changed 17 months ago by git

Commit: b51cf6b6e467f1cc8f761ca6bb0698c6a49e534ccc2f4ae779d1dc5b03ee94f0f5e1e17c422363ad

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

012774fbuild/pkgs/packaging: Update to 20.9
fff89bcbuild/pkgs/importlib_metadata: Update to 4.0.1
e14a5edbuild/pkgs/pip: Update to 21.1.1
3386fa1build/pkgs/toml: New
71aeec7build/pkgs/typing_extensions: New
ecdf3e1build/pkgs/importlib_metadata/dependencies: Update
2e74014Merge #31699
e694b15build/pkgs/tox: Make it a standard package
9c7a1c2build/pkgs: Add tox dependencies: filelock py virtualenv pluggy
cc2f4aebuild/pkgs/{distlib,appdirs,importlib_resources}: Add dependencies of virtualenv

comment:24 Changed 17 months ago by git

Commit: cc2f4ae779d1dc5b03ee94f0f5e1e17c422363adb9c4ec8aadeb3c2977e9899cbd65bae7d9bc2761

Branch pushed to git repo; I updated commit sha1. New commits:

5488ecdbuild/pkgs: Update tox and dependencies to latest
b9c4ec8build/pkgs/packaging/checksums.ini: Add upstream_url

comment:25 Changed 17 months ago by Matthias Köppe

Status: needs_workneeds_review

comment:26 Changed 17 months ago by John Palmieri

py doesn't build for me; even if it did, it looks like it's trying to download things at the start of its build process. Log attached.

Changed 17 months ago by John Palmieri

Attachment: py-1.10.0.log added

comment:27 Changed 17 months ago by git

Commit: b9c4ec8aadeb3c2977e9899cbd65bae7d9bc2761ce2ec11d5f9cdb37e4b8d746836dbb5461e64ed7

Branch pushed to git repo; I updated commit sha1. New commits:

ef12a3ebuild/pkgs/pkgconfig/checksums.ini: Add upstream_url
7a6d0b6build/pkgs/packaging/checksums.ini: Add upstream_url
6abfe84Merge #31699
ce2ec11build/pkgs/py/dependencies: Add setuptools_scm (a setup-requires)

comment:28 Changed 17 months ago by John Palmieri

It's a little unfortunate that the top-level configure recognizes that

configure:38013: checking for tox >= 2.5.0
configure:38096: result: /usr/local/bin/tox
configure:38107: will use system package and not install SPKG tox

but our build system isn't smart enough to skip its dependencies.

comment:29 Changed 17 months ago by Matthias Köppe

Great point! I'll see what I can do about this

comment:30 Changed 17 months ago by git

Commit: ce2ec11d5f9cdb37e4b8d746836dbb5461e64ed765e6330c118a89e21b8ba13bcfc9a407550bad63

Branch pushed to git repo; I updated commit sha1. New commits:

65e6330build/pkgs{appdirs,distlib,filelock,virtualenv}/spkg-configure.m4: Mark as not required if system tox is used

comment:31 Changed 17 months ago by Matthias Köppe

Here's something that will avoid installing 4 packages in this case:

  • appdirs, distlib, filelock, virtualenv

I haven't added the same logic to the other packages:

  • pluggy, py, toml -- because they will also be needed as dependencies of pytest (see #31110)
  • importlib_resources -- because it is useful as a backport package (see #31306)

comment:32 Changed 16 months ago by John Palmieri

Reviewers: John Palmieri
Status: needs_reviewpositive_review

Okay, thanks, looks good.

comment:33 Changed 16 months ago by Matthias Köppe

Thanks!

comment:34 Changed 16 months ago by Volker Braun

Branch: u/mkoeppe/make_tox_a_standard_package65e6330c118a89e21b8ba13bcfc9a407550bad63
Resolution: fixed
Status: positive_reviewclosed

comment:35 in reply to:  26 ; Changed 16 months ago by gh-kliem

Commit: 65e6330c118a89e21b8ba13bcfc9a407550bad63

Replying to jhpalmieri:

py doesn't build for me; even if it did, it looks like it's trying to download things at the start of its build process. Log attached.

What was the resolution to this?

I'm getting build failure of sage because of this now. It's not terrible and I guess py isn't needed by anything. But I'm also wondering, why it is a dependency of tox and pytest in the first place. Neither of them lists py as a dependency. Where do we use it?

comment:36 in reply to:  35 Changed 16 months ago by Matthias Köppe

Replying to gh-kliem:

Replying to jhpalmieri:

py doesn't build for me; even if it did, it looks like it's trying to download things at the start of its build process. Log attached.

What was the resolution to this?

The missing dependency on setuptools_scm was added

I'm getting build failure of sage because of this now. It's not terrible and I guess py isn't needed by anything. But I'm also wondering, why it is a dependency of tox and pytest in the first place. Neither of them lists py as a dependency. Where do we use it?

It's an install_requires of tox: https://github.com/tox-dev/tox/blob/master/setup.cfg

Note: See TracTickets for help on using tickets.