Opened 18 months ago
Last modified 2 weeks ago
#28936 new task
Meta-ticket: Adopt mainstream Python testing/linting infrastructure: tox, pytest, ..., describe in Developer's Guide
Reported by: | mkoeppe | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.4 |
Component: | doctest framework | Keywords: | |
Cc: | jdemeyer, dimpase, embray, vdelecroix, chapoton, tscrim, jmantysalo, gh-tobiasdiez, gh-kliem, nthiery | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
This metaticket proposes to use mainstream Python testing and linting infrastructure for Sage -- by adding configuration files and documentation.
tox:
- #28987 Add
src/tox.ini
: As an alternative interface to running the sage doctester. - #30467
src/tox.ini
: Check patchbot plugin patterns - #30410 (Command "sage -tox")
- #30416 (Make tox a standard package)
- #30453 Document "sage -tox"
- #30411 (src/tox.ini: Add environment pyright)
- #30448 (src/tox.ini: Add validation of rst files and docstrings via a quick docbuild)
pytest:
- #31003 Add minimal pytest configuration
- #31103
sage.numerical.backends
: Replace use ofTestSuite
bypytest
- #31110 Make
pytest
a standard package - #31123 Improve pytest integration
- #30738 (Use pytest instead of custom test discovery using
TestSuite
) - #31104
sage.geometry.polyhedron
: Replace use ofTestSuite
bypytest
Other tickets:
- #30452 ("sage -startuptime" should accept file names too)
- #30415 (Remove pyflakes?)
Documentation tickets:
- #29784 Update section "Sage Development Process" in Developer's Guide
- #29520: Mention .lgtm.yml in the Developer's Guide
Infrastructure tickets:
- #30363 Replace patchbot by github actions (part of https://trac.sagemath.org/ticket/)
- Use pre-commit https://pre-commit.com/
- Use pipenv in place of venv https://pipenv.pypa.io/en/latest/#
See also:
- Meta-ticket #29060: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages
- Generalize
sage.numerical.backends.logging_backend.LoggingBackend
to help generate_test
methods
- #28998: Add script-type check_dependencies package that installs tox, mock, pytest etc.
- Extensive description of different linters: https://github.com/vintasoftware/python-linters-and-code-analysis
Change History (32)
comment:1 Changed 18 months ago by
- Cc vdelecroix added
comment:2 Changed 18 months ago by
- Milestone changed from sage-9.0 to sage-9.1
- Summary changed from Create tox plugin for running the sage doctester to Create tox / pytest plugins for running the sage doctester
comment:3 in reply to: ↑ description ; follow-up: ↓ 5 Changed 17 months ago by
comment:5 in reply to: ↑ 3 Changed 17 months ago by
Replying to embray:
Replying to mkoeppe: The main advantage to tox is the ability to run a package's tests across multiple runtime environments (esp. different Python versions). This isn't really possible currently with Sage; it would actually be very difficult to use tox in any sensible manner with Sage.
FWIW, I managed to use tox
on top of the sage installation, see https://github.com/mkoeppe/sage-numerical-backends-gurobi/blob/master/tox.ini: The trick is to run tox
with sitepackages = true
and an install_command
that installs into the user scheme. With this trick, after all, no new plugin for tox
is needed to handle the sage environment.
comment:6 Changed 17 months ago by
- Cc chapoton added
- Description modified (diff)
- Summary changed from Create tox / pytest plugins for running the sage doctester to Meta-ticket: Adopt mainstream Python testing/linting infrastructure
I have broadened the ticket description.
comment:7 Changed 17 months ago by
- Description modified (diff)
comment:8 Changed 14 months ago by
- Milestone changed from sage-9.1 to sage-9.2
pushing these forward to 9.2
comment:9 Changed 14 months ago by
- Cc tscrim jmantysalo added
- Description modified (diff)
- Summary changed from Meta-ticket: Adopt mainstream Python testing/linting infrastructure to Meta-ticket: Adopt mainstream Python testing/linting infrastructure, describe in Developer's Guide
comment:10 Changed 14 months ago by
- Description modified (diff)
comment:11 Changed 12 months ago by
- Cc gh-tobiasdiez added
- Description modified (diff)
comment:12 Changed 12 months ago by
- Cc gh-kliem added
- Description modified (diff)
comment:13 Changed 10 months ago by
- Description modified (diff)
comment:14 Changed 10 months ago by
- Description modified (diff)
comment:15 Changed 10 months ago by
- Description modified (diff)
comment:16 Changed 10 months ago by
- Description modified (diff)
comment:17 Changed 10 months ago by
- Description modified (diff)
comment:18 Changed 10 months ago by
- Description modified (diff)
comment:19 Changed 10 months ago by
- Description modified (diff)
comment:20 Changed 9 months ago by
- Description modified (diff)
comment:21 Changed 9 months ago by
- Description modified (diff)
comment:22 Changed 9 months ago by
- Description modified (diff)
comment:23 Changed 8 months ago by
- Description modified (diff)
comment:24 Changed 8 months ago by
- Milestone changed from sage-9.2 to sage-9.3
comment:25 Changed 6 months ago by
- Cc nthiery added
comment:26 Changed 6 months ago by
- Description modified (diff)
comment:27 Changed 6 months ago by
- Description modified (diff)
comment:28 Changed 6 months ago by
- Description modified (diff)
- Summary changed from Meta-ticket: Adopt mainstream Python testing/linting infrastructure, describe in Developer's Guide to Meta-ticket: Adopt mainstream Python testing/linting infrastructure: tox, pytest, ..., describe in Developer's Guide
comment:29 Changed 5 weeks ago by
- Milestone changed from sage-9.3 to sage-9.4
Moving to 9.4, as 9.3 has been released.
comment:30 Changed 2 weeks ago by
- Description modified (diff)
comment:31 Changed 2 weeks ago by
What do you think about using pytest-cov for generating the test coverage report? The results can then be uploded to codecov as part of the build process (github action). This gives a nice overview of the actual test coverage (i.e. code covered by tests) and could replace the sage-coverage scripts in the future.
comment:32 Changed 2 weeks ago by
+1 on investigating this. Myself I have no experience with the available tools (including the sage-coverage
scripts even though I have fixed them up at least once). As usual, it's best to build a prototype and then demonstrate to the sage-devel community that it is useful.
Replying to mkoeppe:
I don't know what you mean by "preferred interface" here. There isn't really any such thing as a preferred interface. It's just being proposed here as a recommendation over
setup.py test
. Sage doesn't usesetup.py test
anyways so there's nothing that needs to be "replaced" with a different interface if/whensetup.py
goes away.The main advantage to tox is the ability to run a package's tests across multiple runtime environments (esp. different Python versions). This isn't really possible currently with Sage; it would actually be very difficult to use tox in any sensible manner with Sage.
If someone wants to do this anyways I don't see the harm really, but I'm not sure what problem it would be solving.