pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf

[mkoeppe]

Similar to #29411 + #29950, which made sagelib a pip-installable distribution (which is then installed by the sage-the-distribution script package sagelib), we change the build/installation scheme of the sage documentation.

We create pip-installable packages sagemath-doc-html and sagemath-doc-pdf. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.

We create a package sagemath-doc-inventory, which is in charge of building and installing the inventory files. It is a build dependency of sagemath-doc-html and sagemath-doc-pdf.

All of these packages depend on sagemath-doc-src. It has a build-system dependency on sagelib. An sdist of it consists of SAGE_SRC/doc. A wheel of sagemath-doc-src (and thus an installation) contains a copy of the sources (including the auto-generated rst files) in share/doc/sage/src/ and the doctrees in share/doc/sage/doctrees. The incremental build features of the docbuild will only be active when the package sagemath-doc-src is installed directly using setup.py install, setup.py develop or pip install --editable. In these cases, the doctrees (including environment pickes) are kept in the source directory. pip wheel will always make a from-scratch build of the documentation, eliminating incremental build errors.

The script sage-grepdoc will be provided by sagemath-doc-html.

Follow-up tickets:

• When installed using pip, the packages provide nbextensions that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see #30306 (3)).
  • ​https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Distributing%20Jupyter%20Extensions%20as%20Python%20Packages.html
• Or alternatively as a JupyterLab extension
  • ​https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html

See also:

• #30010: Split sage_setup.docbuild out to a separate package
• #29097: build/make/Makefile.in: Rename make targets SPKG-clean to SPKG-uninstall
• #29231: Add intersphinx mapping for SciPy -- with discussion about docbuild and inventory files

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

Input and help from docbuild experts is very welcome.

[mkoeppe]

Some questions:

• Who generates files such as src/doc/en/reference/monoids/sage/monoids/free_monoid.rst, and should they be part of what sagemath-doc-src installs (and thus ships with the wheel)?

• Is there a way to only generate the doctrees but not build HTML or PDF?

• Do the HTML and PDF builds need to be able to run the Sage library, or can they build these formats if given only the doctrees?

[fbissey]

Replying to mkoeppe:

Some questions:

• Is there a way to only generate the doctrees but not build HTML or PDF?

I do believe there is a sphinx option for doing just that.

• Do the HTML and PDF builds need to be able to run the Sage library, or can they build these formats if given only the doctrees?

As far as I know they need to be able to run the sage library. In particular all graphs/plots are generated during the documentation build.

[fbissey]

Replying to fbissey:

Replying to mkoeppe:

Some questions:

• Is there a way to only generate the doctrees but not build HTML or PDF?

I do believe there is a sphinx option for doing just that.

I cannot find such an option anymore. If it existed. There is an option to say where the doctrees should be stored and found, but nothing on producing it in isolation.

[jhpalmieri]

Well, ./sage --docbuild all inventory builds the inventory files and the doctree files, but not html or pdf.

[jhpalmieri]

Replying to mkoeppe:

Some questions:

• Who generates files such as src/doc/en/reference/monoids/sage/monoids/free_monoid.rst, and should they be part of what sagemath-doc-src installs (and thus ships with the wheel)?

The autodoc procedure does this, with sage/monoids/free_monoid.py as input. (Edit: or maybe just src/doc/en/reference/monoids/index.rst as input?) I'm not sure precisely how this is done, but the file sage_docbuild/ext/sage_autodoc.py is certainly relevant. (This is Sage's fork of the Sphinx autodoc module.)

[jhpalmieri]

No, I'm wrong about that. It's done in sage_docbuild/__init__.py, in the write_auto_rest_file method.

[mkoeppe]

According to ​https://trac.sagemath.org/ticket/33064#comment:19, there is no need to install doctrees in SAGE_LOCAL, not even for introspection (sage.misc.sphinxify)