Make sure sage starts before building packages that depend on sagelib

[embray]

A problem that is occurring frequently in reports of build problems is failures during building sagenb, like

[sagenb-1.1.2] reading sources... [ 15%] misc/misc
[sagenb-1.1.2]
[sagenb-1.1.2] Exception occurred:
[sagenb-1.1.2]   File "sage/misc/lazy_import.pyx", line 218, in sage.misc.lazy_import.LazyImport._get_object (build/cythonized/sage/misc/lazy_import.c:2502)
[sagenb-1.1.2]     raise RuntimeError(f"resolving lazy import {self._name} during startup")
[sagenb-1.1.2] RuntimeError: resolving lazy import dumps during startup
[sagenb-1.1.2] The full traceback has been saved in /var/folders/qf/wr6_n5s56m780kv130nznmlc0000gn/T/sphinx-err-MzyE1A.log, if you want to report the issue to the developers.
[sagenb-1.1.2] Please also report this if it was a user error, so that a better error message can be provided next time.

However, the problem has nothing to do with sagenb--rather it's because sagenb's own build process uses sagelib, and if there was a problem building sagelib properly (such that the build succeeded without failure, but still has runtime problems) it means the build will fail for any packages that depend on sagelib.

We should somehow update the Makefile so that after building sagelib it is tested immediately, and the build is failed before building any further packages, if sage fails to start (and also it would be helpful to display any actual errors to the install log / stdout).

(Currently sagenb is the only standard package that depends on sagelib being built, and it is being slowly phased out. However, there are optional packages that depend on sage, and may be more standard packages in the future.)

[dimpase]

Do you know how reliably break Sage so that ./sage -f sagenb will break as above?

[embray]

You could do almost anything really--e.g. remove a module that's imported at startup. The last two times I saw this come up it's because the sqlite3 module for Python didn't build (something which should also be checked; that's #27705).

[dimpase]

Thanks,

mv local/lib/python2.7/lib-dynload/_sqlite3.so local/lib/python2.7/lib-dynload/_sqlite3.so.bak

does the "job" :-)

[dimpase]

The following makes ./sage -f sagenb fail with a meaningful error if Sage is broken

build/pkgs/sagenb/dependencies

@@ -1 @@
-$(PYTHON) | $(SAGERUNTIME) pip babel flask flask_autoindex flask_babel flask_oldsessions flask_openid mathjax twisted sphinx
+$(PYTHON) $(STARTED) | $(SAGERUNTIME) pip babel flask flask_autoindex flask_babel flask_oldsessions flask_openid mathjax twisted sphinx
 
 ----------
 All lines of this file are ignored except the first.

(and works if Sage is OK) This obviously may be simplified to

build/pkgs/sagenb/dependencies

@@ -1 @@
-$(PYTHON) | $(SAGERUNTIME) pip babel flask flask_autoindex flask_babel flask_oldsessions flask_openid mathjax twisted sphinx
+$(STARTED) | pip babel flask flask_autoindex flask_babel flask_oldsessions flask_openid mathjax twisted sphinx
 
 ----------
 All lines of this file are ignored except the first.

and possibly even more. What's a bit worrying me here is why this is not a circular dependency.

[dimpase]

I have added the change in comment:4 to the branch in #28710.

[mkoeppe]

The package used as an example in the description has been removed. Should this same problem surface for any other packages, its dependencies should be changed as discussed here. Let's close this.

[dimpase]

ok