sagenb docs fail to build

[Konrad127123]

Building sage in a clean environment with make distclean && make, fails on building the documentation for sagenb:

Running Sphinx v1.7.6
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 19 source files that are out of date
updating environment: 19 added, 0 changed, 0 removed
reading sources... [  5%] index
reading sources... [ 10%] misc/introspect
reading sources... [ 15%] misc/misc

Exception occurred:
  File "sage/misc/lazy_import.pyx", line 218, in sage.misc.lazy_import.LazyImport._
get_object (build/cythonized/sage/misc/lazy_import.c:2412)
    raise RuntimeError(f"resolving lazy import {self._name} during startup")
RuntimeError: resolving lazy import dumps during startup
The full traceback has been saved in /tmp/sphinx-err-k0V6v7.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make[4]: *** [Makefile:20: html] Error 2
make[4]: Leaving directory '/home/sage/sage-8.4/local/var/tmp/sage/build/sagenb-1.1.1/src/doc'

make distclean && make sagenb && make succeeds.

Installing all the dependencies of sagelib before sagenb with make distclean && make arb boost_cropped brial bzip2 cddlib cliquer cypari cysignals cython decorator docutils ecl eclib ecm enum34 fflas_ffpack flint fplll freetype gap gc gf2x giac givaro glpk gsl iconv iml ipython_genutils jinja2 jupyter_core lcalc libatomic_ops libbraiding libgap libgd libhomfly libpng linbox lrcalc m4ri m4rie markupsafe mpc mpfi mpfr mpir ncurses ntl numpy openblas pari pari_galdata pari_seadata_small pip pkgconf pkgconfig planarity ppl pycygwin pynac python2 python3 ratpoints readline rw setuptools singular six sqlite symmetrica traitlets xz yasm zlib zn_poly && make sagenb succeeds.

make distclean && make sagelib && make sagenb fails.

The problem seems to occur if sagenb is built after sagelib, but before $(SAGERUNTIME).

[Konrad127123]

New commits:

​a964c94

sagenb should depend on $(SAGERUNTIME)

[Konrad127123]

Also possibly worth noting: building all the packages that sagenb and $(SAGERUNTIME) depend on without actually building $(SAGERUNTIME) is not sufficient, so it looks like $(SAGERUNTIME) really is the minimal dependency that needs to be added.

i.e. doing make distclean && make alabaster appnope arb babel backports_abc backports_shutil_get_terminal_size backports_ssl_match_hostname boost_cropped brial bzip2 cddlib certifi cliquer cypari cysignals cython decorator docutils ecl eclib ecm enum34 fflas_ffpack flask flask_autoindex flask_babel flask_oldsessions flask_openid flask_silk flint fplll freetype future gap gc gf2x giac givaro glpk gsl iconv imagesize iml ipython ipython_genutils itsdangerous jinja2 jupyter_core lcalc libatomic_ops libbraiding libgap libgd libhomfly libpng linbox lrcalc m4ri m4rie markupsafe mathjax mpc mpfi mpfr mpir ncurses ntl numpy openblas packaging pari pari_galdata pari_seadata_small pathlib2 pathpy pexpect pickleshare pip pkgconf pkgconfig planarity ppl prompt_toolkit psutil ptyprocess pycygwin pygments pynac pyparsing python2 python3 python_openid pytz pyzmq ratpoints readline requests rw sagelib scandir setuptools setuptools_scm simplegeneric singledispatch singular six snowballstemmer speaklater sphinx sphinxcontrib_websupport sqlite symmetrica tornado traitlets twisted typing wcwidth werkzeug xz yasm zeromq zlib zn_poly zope_interface && make sagenb still fails, although it fails slightly later:

[sagenb-1.1.1] Running Sphinx v1.7.6
[sagenb-1.1.1] making output directory...
[sagenb-1.1.1] loading pickled environment... not yet created
[sagenb-1.1.1] building [mo]: targets for 0 po files that are out of date
[sagenb-1.1.1] building [html]: targets for 19 source files that are out of date
[sagenb-1.1.1] updating environment: 19 added, 0 changed, 0 removed
[sagenb-1.1.1] reading sources... [  5%] index
[sagenb-1.1.1] reading sources... [ 10%] misc/introspect
[sagenb-1.1.1] reading sources... [ 15%] misc/misc
[sagenb-1.1.1] reading sources... [ 21%] misc/sageinspect
[sagenb-1.1.1] reading sources... [ 26%] misc/sphinxify
[sagenb-1.1.1] reading sources... [ 31%] misc/support
[sagenb-1.1.1] reading sources... [ 36%] notebook/cell
[sagenb-1.1.1] reading sources... [ 42%] notebook/challenge
[sagenb-1.1.1] reading sources... [ 47%] notebook/config
[sagenb-1.1.1] reading sources... [ 52%] notebook/css
[sagenb-1.1.1] reading sources... [ 57%] notebook/docHTMLProcessor
[sagenb-1.1.1] reading sources... [ 63%] notebook/interact
[sagenb-1.1.1] reading sources... [ 68%] notebook/js
[sagenb-1.1.1] reading sources... [ 73%] notebook/notebook
[sagenb-1.1.1] WARNING: autodoc: failed to import module u'sagenb.misc.introspect'; the following exception was raised:
[sagenb-1.1.1] Traceback (most recent call last):
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sphinx/ext/autodoc/importer.py", line 152, in import_module
[sagenb-1.1.1]     __import__(modname)
[sagenb-1.1.1]   File "../sagenb/__init__.py", line 3, in <module>
[sagenb-1.1.1]     from . import storage
[sagenb-1.1.1]   File "../sagenb/storage/__init__.py", line 3, in <module>
[sagenb-1.1.1]     from .filesystem_storage import FilesystemDatastore
[sagenb-1.1.1]   File "../sagenb/storage/filesystem_storage.py", line 52, in <module>
[sagenb-1.1.1]     from sagenb.misc.misc import set_restrictive_permissions, encoded_str
[sagenb-1.1.1]   File "../sagenb/misc/misc.py", line 216, in <module>
[sagenb-1.1.1]     import sage.all
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/all.py", line 83, in <module>
[sagenb-1.1.1]     from sage.misc.all       import *         # takes a while
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/misc/all.py", line 84, in <module>
[sagenb-1.1.1]     from .functional import (additive_order,
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/misc/functional.py", line 27, in <module>
[sagenb-1.1.1]     from sage.rings.complex_double import CDF
[sagenb-1.1.1]   File "sage/rings/complex_double.pyx", line 101, in init sage.rings.complex_double (build/cythonized/sage/rings/complex_double.c:24114)
[sagenb-1.1.1]     cdef CC = ComplexField()
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/complex_field.py", line 114, in ComplexField
[sagenb-1.1.1]     C = ComplexField_class(prec)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/complex_field.py", line 208, in __init__
[sagenb-1.1.1]     self._populate_coercion_lists_(coerce_list=[RRtoCC(self._real_field(), self)])
[sagenb-1.1.1]   File "sage/rings/complex_number.pyx", line 2580, in sage.rings.complex_number.RRtoCC.__init__ (build/cythonized/sage/rings/complex_number.c:21896)
[sagenb-1.1.1]     Map.__init__(self, RR, CC)
[sagenb-1.1.1]   File "sage/categories/map.pyx", line 125, in sage.categories.map.Map.__init__ (build/cythonized/sage/categories/map.c:3588)
[sagenb-1.1.1]     parent = homset.Hom(parent, codomain)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/categories/homset.py", line 397, in Hom
[sagenb-1.1.1]     H = Hom(X, Y, category, check=False)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/categories/homset.py", line 424, in Hom
[sagenb-1.1.1]     H = X._Hom_(Y, category)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/categories/rings.py", line 361, in _Hom_
[sagenb-1.1.1]     from sage.rings.homset import RingHomset
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/homset.py", line 19, in <module>
[sagenb-1.1.1]     from . import quotient_ring
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/quotient_ring.py", line 116, in <module>
[sagenb-1.1.1]     import sage.rings.polynomial.multi_polynomial_ideal
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/polynomial/multi_polynomial_ideal.py", line 239, in <module>
[sagenb-1.1.1]     from sage.interfaces.all import (singular as singular_default,
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/all.py", line 24, in <module>
[sagenb-1.1.1]     from .maxima import maxima, Maxima
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/maxima.py", line 1238, in <module>
[sagenb-1.1.1]     script_subdirectory=None)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/maxima.py", line 529, in __init__
[sagenb-1.1.1]     raise RuntimeError('You must get the file local/bin/sage-maxima.lisp')
[sagenb-1.1.1] RuntimeError: You must get the file local/bin/sage-maxima.lisp
[sagenb-1.1.1]
[sagenb-1.1.1] WARNING: autodoc: failed to import module u'sagenb.misc.misc'; the following exception was raised:
[sagenb-1.1.1] Traceback (most recent call last):
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sphinx/ext/autodoc/importer.py", line 152, in import_module
[sagenb-1.1.1]     __import__(modname)
[sagenb-1.1.1]   File "../sagenb/__init__.py", line 3, in <module>
[sagenb-1.1.1]     from . import storage
[sagenb-1.1.1]   File "../sagenb/storage/__init__.py", line 3, in <module>
[sagenb-1.1.1]     from .filesystem_storage import FilesystemDatastore
[sagenb-1.1.1]   File "../sagenb/storage/filesystem_storage.py", line 52, in <module>
[sagenb-1.1.1]     from sagenb.misc.misc import set_restrictive_permissions, encoded_str
[sagenb-1.1.1]   File "../sagenb/misc/misc.py", line 352, in <module>
[sagenb-1.1.1]     from sage.plot.colors import Color
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/plot/__init__.py", line 2, in <module>
[sagenb-1.1.1]     from . import all
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/plot/all.py", line 3, in <module>
[sagenb-1.1.1]     from .plot import plot, graphics_array, list_plot, parametric_plot, polar_plot
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/plot/plot.py", line 577, in <module>
[sagenb-1.1.1]     from sage.ext.fast_eval import fast_float, is_fast_float
[sagenb-1.1.1]   File "sage/ext/fast_eval.pyx", line 93, in init sage.ext.fast_eval (build/cythonized/sage/ext/fast_eval.c:14529)
[sagenb-1.1.1]     from sage.ext.fast_callable import fast_callable, Wrapper
[sagenb-1.1.1]   File "sage/ext/fast_callable.pyx", line 308, in init sage.ext.fast_callable (build/cythonized/sage/ext/fast_callable.c:20784)
[sagenb-1.1.1]     from sage.rings.all import RDF, CDF
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/all.py", line 31, in <module>
[sagenb-1.1.1]     from .quotient_ring import QuotientRing
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/quotient_ring.py", line 116, in <module>
[sagenb-1.1.1]     import sage.rings.polynomial.multi_polynomial_ideal
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/polynomial/multi_polynomial_ideal.py", line 239, in <module>
[sagenb-1.1.1]     from sage.interfaces.all import (singular as singular_default,
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/all.py", line 24, in <module>
[sagenb-1.1.1]     from .maxima import maxima, Maxima
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/maxima.py", line 1238, in <module>
[sagenb-1.1.1]     script_subdirectory=None)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/maxima.py", line 529, in __init__
[sagenb-1.1.1]     raise RuntimeError('You must get the file local/bin/sage-maxima.lisp')
[sagenb-1.1.1] RuntimeError: You must get the file local/bin/sage-maxima.lisp
[sagenb-1.1.1]
[sagenb-1.1.1] WARNING: autodoc: failed to import module u'sagenb.misc.support'; the following exception was raised:
[sagenb-1.1.1] Traceback (most recent call last):
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sphinx/ext/autodoc/importer.py", line 152, in import_module
[sagenb-1.1.1]     __import__(modname)
[sagenb-1.1.1]   File "../sagenb/misc/support.py", line 525, in <module>
[sagenb-1.1.1]     from sage.symbolic.all import Expression, SR
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/symbolic/all.py", line 3, in <module>
[sagenb-1.1.1]     from sage.libs.pynac.pynac import I
[sagenb-1.1.1]   File "sage/symbolic/expression.pxd", line 4, in init sage.libs.pynac.pynac (build/cythonized/sage/libs/pynac/pynac.cpp:31945)
[sagenb-1.1.1]     cdef class Expression(CommutativeRingElement):
[sagenb-1.1.1]   File "sage/symbolic/expression.pyx", line 161, in init sage.symbolic.expression (build/cythonized/sage/symbolic/expression.cpp:80707)
[sagenb-1.1.1]     from . import ring
[sagenb-1.1.1]   File "sage/symbolic/ring.pyx", line 32, in init sage.symbolic.ring (build/cythonized/sage/symbolic/ring.cpp:16908)
[sagenb-1.1.1]     from sage.rings.all import RR, CC, ZZ
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/all.py", line 31, in <module>
[sagenb-1.1.1]     from .quotient_ring import QuotientRing
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/quotient_ring.py", line 116, in <module>
[sagenb-1.1.1]     import sage.rings.polynomial.multi_polynomial_ideal
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/rings/polynomial/multi_polynomial_ideal.py", line 239, in <module>
[sagenb-1.1.1]     from sage.interfaces.all import (singular as singular_default,
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/all.py", line 24, in <module>
[sagenb-1.1.1]     from .maxima import maxima, Maxima
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/maxima.py", line 1238, in <module>
[sagenb-1.1.1]     script_subdirectory=None)
[sagenb-1.1.1]   File "/home/sage/sage-8.4/local/lib/python2.7/site-packages/sage/interfaces/maxima.py", line 529, in __init__
[sagenb-1.1.1]     raise RuntimeError('You must get the file local/bin/sage-maxima.lisp')
[sagenb-1.1.1] RuntimeError: You must get the file local/bin/sage-maxima.lisp
[sagenb-1.1.1]
[sagenb-1.1.1]
[sagenb-1.1.1] Exception occurred:
[sagenb-1.1.1]   File "sage/misc/lazy_import.pyx", line 218, in sage.misc.lazy_import.LazyImport._get_object (build/cythonized/sage/misc/lazy_import.c:2412)
[sagenb-1.1.1]     raise RuntimeError(f"resolving lazy import {self._name} during startup")
[sagenb-1.1.1] RuntimeError: resolving lazy import load during startup
[sagenb-1.1.1] The full traceback has been saved in /tmp/sphinx-err-JkWjJW.log, if you want to report the issue to the developers.
[sagenb-1.1.1] Please also report this if it was a user error, so that a better error message can be provided next time.
[sagenb-1.1.1] A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
[sagenb-1.1.1] make[2]: *** [Makefile:20: html] Error 2
[sagenb-1.1.1] make[2]: Leaving directory '/home/sage/sage-8.4/local/var/tmp/sage/build/sagenb-1.1.1/src/doc'

[dimpase]

Thanks, this appears to work for me.

[embray]

+1 Haven't tested this but this looks like the right fix to me.

[tmonteil]

Apparently, this is not the right fix. I fall into this issue while trying to build SDL for 8.6 (for which the patch is applied).

Note that i manually disabled the build of openblas to fix some other issue, but i guess this should not be related.

Let me attach two logs in attachment.

[dimpase]

This error means that your sagelib is broken. (I have seen such errors, and they always boiled down to this). Can you start Sage?

[dimpase]

It might be a bug in Sage's handling of dependencies, but I am not sure. Indeed, $(SAGERUNTIME) is a dependency of sagenb,

[dimpase]

Note that $(SAGERUNTIME) is not a dependency of sphinx, which sounds fishy. Can you add it to its dependencies and try again?

[zimmerma]

log of the sagenb buid with Sage 8.6

[zimmerma]

I did hit this error while building sage-8.6 from source on my laptop (Intel(R) Core(TM) i5-6300U under Ubuntu 16.04. I did attach the log file sagenb-1.1.1.log.pz.

Paul Zimmermann

[Konrad127123]

I'm unable to reproduce this bug using make -j1 or make -j1 sagenb. Does it still occur with 8.7.beta3?

If so, please could you open a new ticket to avoid confusion with the original bug (the original bug was fixed, so this looks like a separate issue, albeit one which causes the same error). Could you also include a list of the files in logs/pkgs when the build fails?

[dimpase]

Replying to zimmerma:

I did hit this error while building sage-8.6 from source on my laptop (Intel(R) Core(TM) i5-6300U under Ubuntu 16.04. I did attach the log file sagenb-1.1.1.log.pz.

this looks as if the problem was earlier with building sagelib itself; as sagelib is needed to build sagenb docs, it should work. Does starting Sage work?

[zimmerma]

I'm not sure one can say this is a separate issue, since it corresponds fully to the description: in a clean environment (after extracting sage-8.6.tar.gz), running make produces exactly the same result as in the description (see the log file I attached):

Running Sphinx v1.7.6
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 19 source files that are out of date
updating environment: 19 added, 0 changed, 0 removed
reading sources... [  5%] index
reading sources... [ 10%] misc/introspect
reading sources... [ 15%] misc/misc

Exception occurred:
  File "sage/misc/lazy_import.pyx", line 218, in sage.misc.lazy_import.LazyImpo\
rt._get_object (build/cythonized/sage/misc/lazy_import.c:2489)
    raise RuntimeError(f"resolving lazy import {self._name} during startup")
RuntimeError: resolving lazy import dumps during startup
The full traceback has been saved in /tmp/sphinx-err-luxsJe.log, if you want to\
 report the issue to the developers.
Please also report this if it was a user error, so that a better error message \
can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphi\
nx/issues>. Thanks!

This was with make without any argument, thus make -j1.

[dimpase]

How about the mentioned /tmp/sphinx-err-luxsJe.log ?

[dimpase]

In order to isolate this better, you can try removing the lines

cd doc
$MAKE html || sdh_die "Error building the documentation"
docdir="$SAGE_LOCAL/share/doc/sagenb"
rm -rf "$docdir"
mv build/html "$docdir"

from build/pkgs/sagenb/spkg-install and see whether building Sage completes and you can start it, etc.

[zimmerma]

Replying to dimpase:

How about the mentioned /tmp/sphinx-err-luxsJe.log ?

it is now attached.

[dimpase]

Replying to zimmerma:

Replying to dimpase:

How about the mentioned /tmp/sphinx-err-luxsJe.log ?

it is now attached.

Thanks - no clues there, unfortunately. Could you try the experiment proposed in comment 18 ?

[jdemeyer]

Please create a new ticket instead of discussing things on an old ticket which is most likely unrelated.

[jdemeyer]

I think the underlying problem is all these except ImportError in sagenb/misc/misc.py. They hide real errors and just break stuff. I'll try to fix that upstream.

It won't solve your problem but at least it you'll get a proper error message.

[jdemeyer]

Please continue the discussion at #27257.