Restructure docbuild as script packages sagemath_doc_html, sagemath_doc_pdf

[mkoeppe]

The specific code in Makefile.in for the documentation and its dependencies is moved to the new script packages.

In preparation for #32759:

• We introduce another installation tree (in addition to SAGE_LOCAL and SAGE_VENV), called SAGE_DOCS. When the installation tree variable is empty, we suppress building the package.
• We introduce a mechanism to select an installation tree, the file trees.txt. Using this mechanism, we associate the new script packages with the installation tree SAGE_DOCS.

Next step:

• #32759 ./configure --disable-doc
• #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf
• rename src/doc/bootstrap to build/pkgs/sagemath_doc_html/bootstrap.

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

New commits:

​42b7714

WIP sagemath_doc_html sagemath_doc_pdf

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​c262e44	build/pkgs/sage_docbuild/src: Update metadata
​52778c3	build/pkgs/sage_docbuild/spkg-install: Use sdh_setup_bdist_wheel
​b6abe5e	src/sage/docs/conf.py: Use absolute module names for sage_docbuild extensions
​b42920b	trac 30010: (typo) change sage_doctest -> sage_docbuild
​0a472f2	build/pkgs/sage_docbuild/src/setup.py: Restrict find_namespace_packages to sage_docbuild
​1dfeb84	build/make/Makefile.in (sage_docbuild-clean): New, run it from build-clean
​881c4af	src/sage_setup/__init__.py: Restore
​4eca937	build/pkgs/sage_docbuild/src/setup.{cfg,py}: Use an explicit list of packages, add install_requires
​aaab1d3	build/pkgs/sage_docbuild/dependencies: Add dependencies on .py files
​f68feab	build/pkgs/sagemath_doc_{html,pdf}: New, delegate 'make doc-{html,pdf}' to here

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0e05512

build/pkgs/sagemath_doc_html/dependencies: Add sage_docbuild

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d39fd70	build/pkgs/sagemath_doc_html: Add SPKG.rst, type
​5c19f46	build/pkgs/sagemath_doc_html: Add package-version.txt

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​6dd12bd	build/pkgs/sagemath_doc_{html,pdf}: New, delegate 'make doc-{html,pdf}' to here
​ec43b4c	build/pkgs/sagemath_doc_html/dependencies: Add sage_docbuild
​862c6ac	build/pkgs/sagemath_doc_html: Add SPKG.rst, type
​3cd70ff	build/pkgs/sagemath_doc_html: Add package-version.txt

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​d62fc48

WIP sagemath_doc_html sagemath_doc_pdf

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​0968c1c

WIP sagemath_doc_html sagemath_doc_pdf

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6ab935d	m4/sage_spkg_collect.m4: Determine install tree for python3 via new trees.txt mechanism
​37718fb	build/make/Makefile.in: Install doc via trees.txt = SAGE_DOC_PREFIX

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8a6db35

build/make/Makefile.in: WIP Disablable trees

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​8e26780	build/pkgs/sagemath_doc_{html,pdf}: New; delegate to them from targets doc-html etc.
​eeb8777	m4/sage_spkg_collect.m4: Determine install tree for python3 via new trees.txt mechanism

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c8e19a0

fixup

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5a4424b

build/make/Makefile.in (all-sage): Do not include packages installed in SAGE_DOCS

[jhpalmieri]

I feel like trees.txt should be tree.txt, unless there is some way to specify more than one tree there. Obviously not a big deal. I would like this file to be documented, though, and maybe a comment in m4/sage_spkg_collect.m4 is the right place.

[mkoeppe]

Yes, actually several trees are allowed to be specified there. We don't need this quite yet, but a separate environment for the Jupyter notebook could make sense; and then some packages such as jupyter_core would have to be installed in both SAGE_VENV and the new notebook environment.

[mkoeppe]

I'll add a bit to the developer's guide

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cb78741

src/doc/en/developer/packaging.rst: Exlpain trees.txt

[jhpalmieri]

It's not clear to me what ./configure --enable-sagemath_doc_pdf is supposed to do just by reading ./configure --help. If I run make, will it also run make doc-pdf, or does it just do some prep work for the PDF build? I see this:

  --enable-sagemath_doc_pdf={no|if_installed (default)|yes}
                          enable build and use of the optional package sagemath_doc_pdf
                          * package info: ./sage -info sagemath_doc_pdf

and then unfortunately ./sage -info sagemath_doc_pdf is not very useful.

[mkoeppe]

I'll add SPKG.rst files to make the info more useful

[jhpalmieri]

I think that sage_docbuild should be a dependency of each of the new packages: I just got this error (sagemath_doc_html-none.log):

/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/bin/python3: No module named sage_docbuild
make doc-inventory--
cd /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5 && ./sage --docbuild --no-pdf-links .o inventory 
/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/bin/python3: No module named sage_docbuild
make[6]: *** [doc-inventory--.o] Error 1
make[5]: *** [doc-inventory-reference] Error 2

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7ad6cd9

build/pkgs/sagemath_doc_*/SPKG.rst: New

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​eb65e03

build/pkgs/sagemath_doc_*/dependencies: Add sage_docbuild

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7989d87

src/doc/Makefile: Handle errors from './sage --docbuild --all-documents'

[jhpalmieri]

I'm getting an error with the pdf build, one that I don't get with the develop branch: en/reference/plotting is failing, and the file local/share/doc/sage/latex/en/reference/plotting/plotting.log ends with

Package pdftex.def Info: plot-23.pdf  used on input line 864.
(pdftex.def)             Requested size: 462.52684pt x 346.89516pt.
<plot-24.pdf, id=486, 462.52798pt x 346.89601pt>
File: plot-24.pdf Graphic file (type pdf)
<use plot-24.pdf>
Package pdftex.def Info: plot-24.pdf  used on input line 884.
(pdftex.def)             Requested size: 462.52684pt x 346.89516pt.

[23 <./plot-22.pdf>] [24 <./plot-23.pdf>] [25 <./plot-24.pdf
!pdfTeX error: pdflatex (file ./plot-24.pdf): PDF inclusion: type <cmd> cannot 
be copied
 ==> Fatal error occurred, no output PDF file produced!

Any ideas?

[mkoeppe]

No idea, I haven't seen this failure

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d64fb76	build/make/Makefile.in: Undo reintroduction of SAGE_SKIP_PLOT_DIRECTIVE, SAGE_DOC_MATHJAX settings
​6afc0d5	build/pkgs/sagemath_doc_html/spkg-install: Move handling of SAGE_DOC_JSMATH environment variable here from src/bin/sage-env

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b2cf40c

build/make/Makefile.in: Undo reintroduction of SAGE_SKIP_PLOT_DIRECTIVE, SAGE_DOC_MATHJAX settings (fixup)

[jhpalmieri]

I'm also getting an error with the html build:

[developer] The HTML pages are in local/share/doc/sage/html/en/developer.
Error building the documentation.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__main__.py", line 2, in <module>
    main()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 1814, in main
    builder()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 133, in f
    runsphinx()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/sphinxbuild.py", line 323, in runsphinx
    sys.stderr.raise_errors()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/sphinxbuild.py", line 258, in raise_errors
    raise OSError(self._error)
OSError: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/src/doc/en/developer/packaging.rst:721: WARNING: Duplicate explicit target name: "section-dependencies".

This is after make distclean && ./configure && make.

[mkoeppe]

Thanks, I see this one now too, after make distclean

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​37a03bd

src/doc/en/developer/packaging.rst: Fix up section label

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fc89a66

build/pkgs/sagemath_doc_pdf/spkg-install: Set SAGE_DOC_MATHJAX as a workaround

[mkoeppe]

PDF builds for me without error

[jhpalmieri]

I think that sagemath_doc_html should be a dependency for sagemath_doc_pdf. Otherwise both may try to build at the same time, and they may both try to build the inventory files at the same time, which could easily lead to race conditions.

[mkoeppe]

Yes, I agree

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fd298c4

build/pkgs/sagemath_doc_pdf/dependencies: Depend on sagemath_doc_html

[jhpalmieri]

Everything now builds for me.

[mkoeppe]

Thanks for testing!

[jhpalmieri]

I think this is ready to go. Merge now or wait until 9.6? I can imagine it breaking something for unforeseen reasons.

[mkoeppe]

I think we can merge it now

[mkoeppe]

Thank you!