Testsuite for src/setup.py

[mkoeppe]

We should have a testsuite that tests the correct operation and interaction of various commands that a distutils-based setup.py should support.

Does something like this exist in the Python world already?

[vbraun]

Not that I know of; Usually you realize pretty quickly that you can't install your project.

Any non-trivial functionality of setup.py should be implemented in the sage_setup library which can be unit tested in the usual way.

[mkoeppe]

Replying to vbraun:

Not that I know of; Usually you realize pretty quickly that you can't install your project.

Any non-trivial functionality of setup.py should be implemented in the sage_setup library which can be unit tested in the usual way.

Yes, but I am thinking of higher-level tests, for example to ensure that making an sdist and then unpacking-building-cleaning succeeds and gives a tree that is the same as in the sdist archive. This is currently not tested at all and broken (#21516).

[embray]

That's not a bad idea. astropy-helper has a number of functional tests to this effect: ​https://github.com/astropy/astropy-helpers/tree/master/astropy_helpers/tests

They're specifically geared toward testing the code in astropy-helpers itself, but this tests a lot of the other expected outputs of setup.py in the process.

[mkoeppe]

Ticket #21516 (Fix sagelib sdist) now has a test (sdistcheck) but it currently fails.

[jdemeyer]

Just a pointer: I implemented a pretty extensive installation test suite for cysignals. It tests installation with --prefix, --root and --user. It tests setup.py sdist and tests distclean.

See ​https://github.com/sagemath/cysignals/blob/master/Makefile#L71 and following.