Meta-ticket: Clean up src/setup.py to bring it to standard setuptools behavior

[mkoeppe]

Sub-tasks:

• #21480: Make sagelib setup.py self-contained and independent of SAGE_ROOT
• #21613: Make setup.py not depend on make
• #21604: Cleaning up stale installed files in setup()
  • See also #21654: Disentangle cleaning of stale installed files in build directory and in install directory
• #21600: Use custom build_ext to compile Cython code

In particular,

• src/setup.py --help should return immediately.
• src/setup.py clean should not build anything but just clean (and everything that is now done by make clean in the src directory should be done by it; see #29411)

Also the following should be done:

• #29702: Move all code from src/setup.py, src/fpickle_setup.py to sage_setup
• #21516: Fix sagelib sdist (src/setup.py sdist)
• #21535: Make src/setup.py respect --build-base and --inplace, independent of SAGE_CYTHONIZED
• #21573: Make sure src/setup.py respects --install-base and --root
• #21678 Testsuite for src/setup.py
• #30580 sage_setup: Remove import-time dependency (setup_requires) on pkgconfig, numpy

Also see:

• #20108: Use package_data instead of data_files in setup.py
• #21682: Add a separate "cythonize" command to setup.py
• #12659: build the sage library in place
• #13190: make sage use setuptools instead of distutils
• #29701: Meta-ticket: Replace use of OptionalExtension? by namespace packages

[embray]

and everything that is now done by make clean should be done by it

By which you mean the makefile in src/ right?

[mkoeppe]

Yes -- I've clarified it in the description.

[embray]

Definitely keep in touch with me as a point of contact on any work you do on this--I know distutils + setuptools from top to bottom.

[mkoeppe]

Let me just say that I consider #21613 not an important step towards this task ("... standard distutils behavior"). The present task ticket (#21508) is concerned with getting the behavior of setup.py standard (how does it respond to standard commands and options), not necessarily its implementation. The fact that setup.py with #21480 invokes make is an implementation detail, which is not very relevant for the goal of #21507.

[embray]

Except it is pretty non-standard for it to have non-trivial side-effects (e.g. running a subprocess that generates files) unconditionally. Behavior-wise that should only happen when it's actually important for that to happen, such as during the build command.

[mkoeppe]

Replying to embray:

Except it is pretty non-standard for it to have non-trivial side-effects (e.g. running a subprocess that generates files) unconditionally. Behavior-wise that should only happen when it's actually important for that to happen, such as during the build command.

I agree that it should happen during the build command.

[embray]

I will work on #21613, and you'll see it's not that big a deal.

[jdemeyer]

Replying to embray:

I will work on #21613, and you'll see it's not that big a deal.

Beware of conflicts with other tickets, in particular #21600 and its dependencies.

[embray]

Yep. I don't think there will be significant conflicts, but I can make those tickets dependencies of #21613 and ensure any work I do there integrates well with the others before suggesting it to be merged.

[mkoeppe]

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.