#29651 closed defect (fixed)

sage -docbuild: error building docs of single file

Reported by: gh-DaveWitteMorris Owned by:
Priority: major Milestone: sage-9.2
Component: build Keywords:
Cc: Merged in:
Authors: John Palmieri Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: b0310af (Commits, GitHub, GitLab) Commit: b0310af578810d6b99e525c68bbc01fc523f672e
Dependencies: Stopgaps:

Status badges

Description

I tried a few different ways to build the docs of a single file, and got a few different errors. (But one of the methods works, so there is a workaround.) This is with sage 9.1.rc2 on Mac OS 10.15.3.

(1) My first attempt failed because 'DOT_SAGE' is not defined.

% sage --docbuild file=src/sage/plot/text.py pdf
Error building the documentation.
Traceback (most recent call last):
...
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__init__.py", line 1165, in __init__
    base_dir = os.path.join(DOT_SAGE, 'docbuild', module_name)
NameError: name 'DOT_SAGE' is not defined

(2) To avoid the need for DOT_SAGE, I gave an explicit build path. This worked better (it made pdf files of all of the plots), but failed with an AssertionError (and subsequent OSError): assert app.outdir.startswith(SAGE_DOC):

% sage --docbuild -o/Users/dmorris/tmp file=src/sage/plot/text.py pdf
...
[latex    ] Exception occurred:
[latex    ]   File "/Users/dmorris/Documents/misc/Programs/sage3/src/sage_setup/docbuild/ext/multidocs.py", line 220, in citation_dir
[latex    ]     assert app.outdir.startswith(SAGE_DOC)
...
OSError: Exception occurred:

(3) To eliminate the AssertionError, I added a print statement to tell me the value of SAGE_DOC, and I used it for the build path. This worked! (The release number and author show up as "unknown" in the pdf file, but I am guessing that this is the expected behavior.)

% sage --docbuild -o/Users/dmorris/Documents/misc/Programs/sage3/local/share/doc/sage file=src/sage/plot/text.py pdf
[latex    ] building [latex]: all documents
...
Output written on text.pdf (23 pages, 527224 bytes).
Transcript written on text.log.
=== TeX engine is 'pdfTeX'
Latexmk: Index file 'text.idx' was written
Latexmk: Log file says output to 'text.pdf'
Latexmk: All targets () are up-to-date
Build finished.  The built documents can be found in /Users/dmorris/Documents/misc/Programs/sage3/local/share/doc/sage/text/output/pdf

(4) To save myself some typing, I tried again with a local path for the build directory, but got

RuntimeError: failed to run $MAKE all-pdf in local/share/doc/sage/text/output/latex

because

mv: rename text.pdf to local/share/doc/sage/text/output/pdf: No such file or directory`.

The directory actually did get created (but is empty). However, it seems that the local path fails because the $MAKE command is not running in my working directory (where I ran the sage --docbuild command). I don't know whether this is a bug, or the expected behavior.

% sage --docbuild -o local/share/doc/sage file=src/sage/plot/text.py pdf
[latex    ] building [latex]: all documents
[latex    ] updating environment: 1 added, 0 changed, 0 removed
[latex    ] Merging environment/index files...
[latex    ] ... done (0 todos, 1 index, 0 citations, 0 modules)
[latex    ] WARNING: autodoc: failed to import module 'text'; the following exception was raised:
[latex    ] No module named 'text'
[latex    ] processing text.tex...index
[latex    ] resolving references...
[latex    ] writing... done
[latex    ] copying TeX support files...
[latex    ] done
[latex    ] The LaTeX files are in local/share/doc/sage/text/output/latex.
[latex    ] Run 'make' in that directory to run these through (pdf)latex
[latex    ] (use `make latexpdf' here to do that automatically).
LaTeX file written to local/share/doc/sage/text/output/latex
latexmk -pdf -dvi- -ps-  'text.tex'
Latexmk: This is Latexmk, John Collins, 26 Dec. 2019, version: 4.67.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex   -recorder  "text.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
...
Transcript written in text.ilg.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'text.aux'
      'text.ind'
      'text.out'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex   -recorder  "text.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
...
Output written on text.pdf (4 pages, 7317 bytes).
Transcript written on text.log.
=== TeX engine is 'pdfTeX'
Latexmk: Index file 'text.idx' was written
Latexmk: Log file says output to 'text.pdf'
Latexmk: All targets () are up-to-date
mv: rename text.pdf to local/share/doc/sage/text/output/pdf: No such file or directory
Error building the documentation.
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__main__.py", line 2, in <module>
    main()
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__init__.py", line 1720, in main
    builder()
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__init__.py", line 252, in pdf
    raise RuntimeError(error_message%(command, tex_dir))
RuntimeError: failed to run $MAKE all-pdf in local/share/doc/sage/text/output/latex

Change History (19)

comment:1 Changed 21 months ago by jhpalmieri

This fixes the problem for me. The only part I'm not happy about is silencing the warning in sphinxbuild.py, because that message could indicate a problem in some circumstances:

  • src/sage_setup/docbuild/__init__.py

    diff --git a/src/sage_setup/docbuild/__init__.py b/src/sage_setup/docbuild/__init__.py
    index 128117b960..6c4761859b 100644
    a b import sphinx.ext.intersphinx 
    5959import sage.all
    6060from sage.misc.cachefunc import cached_method
    6161from sage.misc.misc import sage_makedirs
    62 from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SRC
     62from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SRC, DOT_SAGE
    6363
    6464from .build_options import (LANGUAGES, SPHINXOPTS, PAPER, OMIT,
    6565     PAPEROPTS, ALLSPHINXOPTS, NUM_THREADS, WEBSITESPHINXOPTS,
    html_title = project 
    11851185html_short_title = project
    11861186htmlhelp_basename = name
    11871187
     1188extensions.remove('multidocs')
     1189extensions.remove('inventory_builder')
     1190
    11881191latex_domain_indices = False
    11891192latex_documents = [
    11901193  ('index', name + '.tex', u'Documentation for {}',
  • src/sage_setup/docbuild/sphinxbuild.py

    diff --git a/src/sage_setup/docbuild/sphinxbuild.py b/src/sage_setup/docbuild/sphinxbuild.py
    index fe7eba43b2..31fdaf4bfb 100644
    a b class SageSphinxLogger(object): 
    110110            re.compile('WARNING: Any IDs not assiend for figure node'),
    111111            re.compile('WARNING: .* is not referenced'),
    112112            re.compile('WARNING: Build finished'),
     113            re.compile('WARNING: unknown config value \'multidoc_first_pass\''),
    113114            )
    114115        self._useless_chatter += self._ignored_warnings
    115116

comment:2 Changed 21 months ago by gh-DaveWitteMorris

Thanks! This does make (1) and (2) work for me (and (3) was already working). Do you want to make the patches into a branch (or should I)? I don't understand the docbuild system, so I don't have a suggestion for an alternative to silencing the warning (and I can't review this fix), but maybe I will before 9.2 comes out.

Method (4) still doesn't work, but I don't think it really needs to be fixed. However, unless there is already some reason that a user shouldn't expect relative paths to work for the build directory, then I would suggest mentioning the need for absolute paths in the documentation (and in sage --docbuild -h). That's a patch that I could probably do.

comment:3 Changed 21 months ago by jhpalmieri

Let me think more about the change to sphinxbuild.py. There should be another way to do this.

comment:4 Changed 17 months ago by mkoeppe

  • Priority changed from minor to major

comment:5 Changed 17 months ago by jhpalmieri

This branch partially works for me:

  • it works with ./sage --docbuild file=src/sage/plot/text.py html, same with pdf instead of html.
  • it does not work with ./sage --docbuild file=src/sage/algebras/steenrod/steenrod_algebra.py html, because that file has citations to references in src/doc/en/reference/references/index.rst.
  • it does not work with a file outside of the Sage library unless that file is in Python's path (for example in the current directory): "WARNING: autodoc: failed to import module 'foo'" . The documentation is ambiguous; should it work for such files?

comment:6 Changed 17 months ago by jhpalmieri

  • Branch set to u/jhpalmieri/single-file-docbuild

comment:7 Changed 17 months ago by jhpalmieri

  • Commit set to 4e167e9965f92b73597adc90968ec0162ab08c13
  • Status changed from new to needs_review

I'm marking this as "needs review", but it could perhaps use improvements, as indicated in comment:5.


New commits:

4e167e9trac 29651: docbuild a single file

comment:8 Changed 17 months ago by mkoeppe

$ ./sage -docbuild file=src/sage/geometry/polyhedron/base.py html 
[html     ] building [html]: targets for 1 source files that are out of date
[html     ] updating environment: [new config] 1 added, 0 changed, 0 removed
[html     ] WARNING: autodoc: failed to import module 'base'; the following exception was raised:
[html     ] attempted relative import with no known parent package
[html     ] writing additional pages...  searchdone
[html     ] copying static files... ... done
[html     ] dumping search index in English (code: en)... done
[html     ] The HTML pages are in ../../../../.sage/docbuild/base/output/html.
Error building the documentation.

comment:9 follow-up: Changed 17 months ago by mkoeppe

Note "attempted relative import with no known parent package"

comment:10 Changed 17 months ago by mkoeppe

I can confirm that ./sage --docbuild file=src/sage/plot/text.py html works

comment:11 in reply to: ↑ 9 Changed 17 months ago by jhpalmieri

Replying to mkoeppe:

Note "attempted relative import with no known parent package"

I don't know what that means.

comment:12 Changed 17 months ago by git

  • Commit changed from 4e167e9965f92b73597adc90968ec0162ab08c13 to b0310af578810d6b99e525c68bbc01fc523f672e

Branch pushed to git repo; I updated commit sha1. New commits:

b0310aftrac 29651: fix module names with underscores

comment:13 follow-up: Changed 17 months ago by jhpalmieri

Maybe the module name needs to be set more intelligently if the path starts with SAGE_SRC, or something like that.

Last edited 17 months ago by jhpalmieri (previous) (diff)

comment:14 Changed 17 months ago by mkoeppe

Looks like the approach of SingleFileBuilder, creating a symlink to the source file, is not compatible with from .module import ...

comment:15 in reply to: ↑ 13 Changed 17 months ago by mkoeppe

Replying to jhpalmieri:

Maybe the module name needs to be set more intelligently if the path starts with SAGE_SRC, or something like that.

I just wrote something like this for #30452

comment:16 Changed 17 months ago by mkoeppe

  • Summary changed from error building docs of single file on Mac OS to sage -docbuild: error building docs of single file

comment:17 follow-up: Changed 17 months ago by mkoeppe

  • Authors set to John Palmieri
  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

In any case, this is already an improvement. We can improve it further on a follow-up ticket.

comment:18 in reply to: ↑ 17 Changed 17 months ago by mkoeppe

Replying to mkoeppe:

We can improve it further on a follow-up ticket.

I have created #30475 for this.

comment:19 Changed 17 months ago by vbraun

  • Branch changed from u/jhpalmieri/single-file-docbuild to b0310af578810d6b99e525c68bbc01fc523f672e
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.