Opened 3 years ago
Last modified 3 months ago
#28936 new task
Meta-ticket: Adopt mainstream Python testing/linting infrastructure: tox, pytest, ..., describe in Developer's Guide
Reported by: | mkoeppe | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.7 |
Component: | doctest framework | Keywords: | |
Cc: | jdemeyer, dimpase, embray, vdelecroix, chapoton, tscrim, jmantysalo, gh-tobiasdiez, gh-kliem, nthiery, tornaria | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
This metaticket proposes to use mainstream Python testing and linting infrastructure for Sage -- by adding configuration files and documentation.
tox:
- #28987 Add
src/tox.ini
: As an alternative interface to running the sage doctester. - #30467
src/tox.ini
: Check patchbot plugin patterns - #30410 (Command "sage -tox")
- #30416 (Make tox a standard package)
- #30453 Document "sage -tox"
- #30411 (src/tox.ini: Add environment pyright)
- #30448 (src/tox.ini: Add validation of rst files and docstrings via a quick docbuild)
pytest:
- #31003 Add minimal pytest configuration
- #31103
sage.numerical.backends
: Replace use ofTestSuite
bypytest
- #30362 Add symplectic structures
- #32975 Improve doctest interaction with pytest
- #33549 Rename
test_...
functions to avoid the naming scheme reserved by pytest - #31924
sage -t
: Do not run pytest on individual Python files unless they match the pytest file pattern - #33572
sage -pytest
- #33560 pytest: ignore cython files
- #33561 Rename
test_...
classes to avoid the naming scheme reserved by pytest - #33550 Preparse sage files before passing them to pytest
- #31123 Document pytest
- #33521 doctesting with pytest fails on system install of sage
- #33531 pytest in sage is broken, please make it really optional
- #31110 Make
pytest
a standard package - #33546 Replace custom Sage doctest discovery by pytest (using pytest_collect_file)
- #30738 Replace custom Sage unit test discovery by pytest
- #31104
sage.geometry.polyhedron
: Replace use ofTestSuite
bypytest
Other tickets:
- #30452 ("sage -startuptime" should accept file names too)
- #33232 (
sage -t
: Write report in JUnitXML format)
- #30415 (Remove pyflakes?)
Documentation tickets:
- #29784 Update section "Sage Development Process" in Developer's Guide
- #29520: Mention .lgtm.yml in the Developer's Guide
Infrastructure tickets:
- #30363 Replace patchbot by github actions (part of https://trac.sagemath.org/ticket/)
- Use pre-commit https://pre-commit.com/
- Use pipenv in place of venv https://pipenv.pypa.io/en/latest/#
See also:
- Meta-ticket #29060: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages
- Generalize
sage.numerical.backends.logging_backend.LoggingBackend
to help generate_test
methods
- #28998: Add script-type check_dependencies package that installs tox, mock, pytest etc.
- Extensive description of different linters: https://github.com/vintasoftware/python-linters-and-code-analysis
Change History (44)
comment:1 Changed 3 years ago by
- Cc vdelecroix added
comment:2 Changed 3 years ago by
- Milestone changed from sage-9.0 to sage-9.1
- Summary changed from Create tox plugin for running the sage doctester to Create tox / pytest plugins for running the sage doctester
comment:3 in reply to: ↑ description ; follow-up: ↓ 5 Changed 3 years ago by
comment:5 in reply to: ↑ 3 Changed 3 years ago by
Replying to embray:
Replying to mkoeppe: The main advantage to tox is the ability to run a package's tests across multiple runtime environments (esp. different Python versions). This isn't really possible currently with Sage; it would actually be very difficult to use tox in any sensible manner with Sage.
FWIW, I managed to use tox
on top of the sage installation, see https://github.com/mkoeppe/sage-numerical-backends-gurobi/blob/master/tox.ini: The trick is to run tox
with sitepackages = true
and an install_command
that installs into the user scheme. With this trick, after all, no new plugin for tox
is needed to handle the sage environment.
comment:6 Changed 3 years ago by
- Cc chapoton added
- Description modified (diff)
- Summary changed from Create tox / pytest plugins for running the sage doctester to Meta-ticket: Adopt mainstream Python testing/linting infrastructure
I have broadened the ticket description.
comment:7 Changed 3 years ago by
- Description modified (diff)
comment:8 Changed 2 years ago by
- Milestone changed from sage-9.1 to sage-9.2
pushing these forward to 9.2
comment:9 Changed 2 years ago by
- Cc tscrim jmantysalo added
- Description modified (diff)
- Summary changed from Meta-ticket: Adopt mainstream Python testing/linting infrastructure to Meta-ticket: Adopt mainstream Python testing/linting infrastructure, describe in Developer's Guide
comment:10 Changed 2 years ago by
- Description modified (diff)
comment:11 Changed 2 years ago by
- Cc gh-tobiasdiez added
- Description modified (diff)
comment:12 Changed 2 years ago by
- Cc gh-kliem added
- Description modified (diff)
comment:13 Changed 2 years ago by
- Description modified (diff)
comment:14 Changed 2 years ago by
- Description modified (diff)
comment:15 Changed 2 years ago by
- Description modified (diff)
comment:16 Changed 2 years ago by
- Description modified (diff)
comment:17 Changed 2 years ago by
- Description modified (diff)
comment:18 Changed 2 years ago by
- Description modified (diff)
comment:19 Changed 2 years ago by
- Description modified (diff)
comment:20 Changed 2 years ago by
- Description modified (diff)
comment:21 Changed 23 months ago by
- Description modified (diff)
comment:22 Changed 23 months ago by
- Description modified (diff)
comment:23 Changed 22 months ago by
- Description modified (diff)
comment:24 Changed 22 months ago by
- Milestone changed from sage-9.2 to sage-9.3
comment:25 Changed 20 months ago by
- Cc nthiery added
comment:26 Changed 20 months ago by
- Description modified (diff)
comment:27 Changed 20 months ago by
- Description modified (diff)
comment:28 Changed 20 months ago by
- Description modified (diff)
- Summary changed from Meta-ticket: Adopt mainstream Python testing/linting infrastructure, describe in Developer's Guide to Meta-ticket: Adopt mainstream Python testing/linting infrastructure: tox, pytest, ..., describe in Developer's Guide
comment:29 Changed 15 months ago by
- Milestone changed from sage-9.3 to sage-9.4
Moving to 9.4, as 9.3 has been released.
comment:30 Changed 15 months ago by
- Description modified (diff)
comment:31 Changed 15 months ago by
What do you think about using pytest-cov for generating the test coverage report? The results can then be uploded to codecov as part of the build process (github action). This gives a nice overview of the actual test coverage (i.e. code covered by tests) and could replace the sage-coverage scripts in the future.
comment:32 Changed 15 months ago by
+1 on investigating this. Myself I have no experience with the available tools (including the sage-coverage
scripts even though I have fixed them up at least once). As usual, it's best to build a prototype and then demonstrate to the sage-devel community that it is useful.
comment:33 Changed 13 months ago by
- Milestone changed from sage-9.4 to sage-9.5
comment:34 Changed 8 months ago by
- Milestone changed from sage-9.5 to sage-9.6
comment:35 Changed 5 months ago by
- Cc tornaria added
- Description modified (diff)
comment:36 Changed 5 months ago by
- Description modified (diff)
comment:37 Changed 5 months ago by
- Description modified (diff)
comment:38 Changed 5 months ago by
- Description modified (diff)
comment:39 Changed 5 months ago by
- Description modified (diff)
comment:40 Changed 5 months ago by
- Description modified (diff)
comment:41 Changed 5 months ago by
- Description modified (diff)
comment:42 Changed 4 months ago by
- Description modified (diff)
comment:43 Changed 4 months ago by
- Description modified (diff)
comment:44 Changed 3 months ago by
- Milestone changed from sage-9.6 to sage-9.7
Replying to mkoeppe:
I don't know what you mean by "preferred interface" here. There isn't really any such thing as a preferred interface. It's just being proposed here as a recommendation over
setup.py test
. Sage doesn't usesetup.py test
anyways so there's nothing that needs to be "replaced" with a different interface if/whensetup.py
goes away.The main advantage to tox is the ability to run a package's tests across multiple runtime environments (esp. different Python versions). This isn't really possible currently with Sage; it would actually be very difficult to use tox in any sensible manner with Sage.
If someone wants to do this anyways I don't see the harm really, but I'm not sure what problem it would be solving.