Opened 22 months ago

Last modified 3 months ago

#28936 new task

Meta-ticket: Adopt mainstream Python testing/linting infrastructure: tox, pytest, ..., describe in Developer's Guide

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.5
Component: doctest framework Keywords:
Cc: jdemeyer, dimpase, embray, vdelecroix, chapoton, tscrim, jmantysalo, gh-tobiasdiez, gh-kliem, nthiery Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

This metaticket proposes to use mainstream Python testing and linting infrastructure for Sage -- by adding configuration files and documentation.

tox:

  • #28987 Add src/tox.ini: As an alternative interface to running the sage doctester.
  • #30467 src/tox.ini: Check patchbot plugin patterns
  • #30410 (Command "sage -tox")
  • #30416 (Make tox a standard package)
  • #30453 Document "sage -tox"
  • #30411 (src/tox.ini: Add environment pyright)
  • #30448 (src/tox.ini: Add validation of rst files and docstrings via a quick docbuild)

pytest:

  • #31003 Add minimal pytest configuration
  • #31103 sage.numerical.backends: Replace use of TestSuite by pytest
  • #31110 Make pytest a standard package
  • #31123 Improve pytest integration
  • #30738 (Use pytest instead of custom test discovery using TestSuite)
  • #31104 sage.geometry.polyhedron: Replace use of TestSuite by pytest

Other tickets:

  • #30452 ("sage -startuptime" should accept file names too)
  • #30361 (Add pyright config and linting documentation; dup: #17992)

Documentation tickets:

  • #29784 Update section "Sage Development Process" in Developer's Guide
  • #29520: Mention .lgtm.yml in the Developer's Guide

Infrastructure tickets:

See also:

  • Meta-ticket #29060: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages
  • Generalize sage.numerical.backends.logging_backend.LoggingBackend to help generate _test methods
  • #28998: Add script-type check_dependencies package that installs tox, mock, pytest etc.

Change History (33)

comment:1 Changed 22 months ago by mkoeppe

  • Cc vdelecroix added

comment:2 Changed 22 months ago by mkoeppe

  • Milestone changed from sage-9.0 to sage-9.1
  • Summary changed from Create tox plugin for running the sage doctester to Create tox / pytest plugins for running the sage doctester

comment:3 in reply to: ↑ description ; follow-up: Changed 22 months ago by embray

Replying to mkoeppe:

tox seems to be the preferred interface for running tests - see for example https://github.com/pypa/setuptools/issues/931 (deprecation of setup.py test).

I don't know what you mean by "preferred interface" here. There isn't really any such thing as a preferred interface. It's just being proposed here as a recommendation over setup.py test. Sage doesn't use setup.py test anyways so there's nothing that needs to be "replaced" with a different interface if/when setup.py goes away.

The main advantage to tox is the ability to run a package's tests across multiple runtime environments (esp. different Python versions). This isn't really possible currently with Sage; it would actually be very difficult to use tox in any sensible manner with Sage.

If someone wants to do this anyways I don't see the harm really, but I'm not sure what problem it would be solving.

comment:4 Changed 22 months ago by mkoeppe

  • Type changed from enhancement to task

A first step: #28987

comment:5 in reply to: ↑ 3 Changed 22 months ago by mkoeppe

Replying to embray:

Replying to mkoeppe: The main advantage to tox is the ability to run a package's tests across multiple runtime environments (esp. different Python versions). This isn't really possible currently with Sage; it would actually be very difficult to use tox in any sensible manner with Sage.

FWIW, I managed to use tox on top of the sage installation, see https://github.com/mkoeppe/sage-numerical-backends-gurobi/blob/master/tox.ini: The trick is to run tox with sitepackages = true and an install_command that installs into the user scheme. With this trick, after all, no new plugin for tox is needed to handle the sage environment.

comment:6 Changed 22 months ago by mkoeppe

  • Cc chapoton added
  • Description modified (diff)
  • Summary changed from Create tox / pytest plugins for running the sage doctester to Meta-ticket: Adopt mainstream Python testing/linting infrastructure

I have broadened the ticket description.

comment:7 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:8 Changed 19 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

pushing these forward to 9.2

comment:9 Changed 19 months ago by mkoeppe

  • Cc tscrim jmantysalo added
  • Description modified (diff)
  • Summary changed from Meta-ticket: Adopt mainstream Python testing/linting infrastructure to Meta-ticket: Adopt mainstream Python testing/linting infrastructure, describe in Developer's Guide

comment:10 Changed 18 months ago by mkoeppe

  • Description modified (diff)

comment:11 Changed 17 months ago by mkoeppe

  • Cc gh-tobiasdiez added
  • Description modified (diff)

comment:12 Changed 17 months ago by mkoeppe

  • Cc gh-kliem added
  • Description modified (diff)

comment:13 Changed 15 months ago by gh-tobiasdiez

  • Description modified (diff)

comment:14 Changed 14 months ago by gh-tobiasdiez

  • Description modified (diff)

comment:15 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:16 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:17 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:18 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:19 Changed 14 months ago by gh-tobiasdiez

  • Description modified (diff)

comment:20 Changed 14 months ago by gh-tobiasdiez

  • Description modified (diff)

comment:21 Changed 13 months ago by mkoeppe

  • Description modified (diff)

comment:22 Changed 13 months ago by gh-tobiasdiez

  • Description modified (diff)

comment:23 Changed 13 months ago by gh-tobiasdiez

  • Description modified (diff)

comment:24 Changed 12 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:25 Changed 10 months ago by mkoeppe

  • Cc nthiery added

comment:26 Changed 10 months ago by mkoeppe

  • Description modified (diff)

comment:27 Changed 10 months ago by mkoeppe

  • Description modified (diff)

comment:28 Changed 10 months ago by mkoeppe

  • Description modified (diff)
  • Summary changed from Meta-ticket: Adopt mainstream Python testing/linting infrastructure, describe in Developer's Guide to Meta-ticket: Adopt mainstream Python testing/linting infrastructure: tox, pytest, ..., describe in Developer's Guide

comment:29 Changed 6 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Moving to 9.4, as 9.3 has been released.

comment:30 Changed 5 months ago by mkoeppe

  • Description modified (diff)

comment:31 Changed 5 months ago by gh-tobiasdiez

What do you think about using pytest-cov for generating the test coverage report? The results can then be uploded to codecov as part of the build process (github action). This gives a nice overview of the actual test coverage (i.e. code covered by tests) and could replace the sage-coverage scripts in the future.

comment:32 Changed 5 months ago by mkoeppe

+1 on investigating this. Myself I have no experience with the available tools (including the sage-coverage scripts even though I have fixed them up at least once). As usual, it's best to build a prototype and then demonstrate to the sage-devel community that it is useful.

comment:33 Changed 3 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5
Note: See TracTickets for help on using tickets.