Upgrade NumPy to 1.19.1, scipy to 1.5.2, networkx to 2.4, add pybind11 package

[dimpase]

Follow-up from the update of numpy to 1.16.6 (#29429):

We update

• ​https://pypi.org/project/numpy/ to 1.19.1 (latest as of 2020-07-21)
• ​https://pypi.org/project/scipy/ to 1.5.2 (latest as of 2020-07-31)
• networkx to the latest release, 2.4 (latest as of 2020-07-04)

All of the above versions support Python >= 3.6.

One deprecation warning is removed here, another suppressed (from #29425), the rest it trivial.

Tarballs: see checksums.ini [upstream_url]

[dimpase]

Last 10 new commits:

​fa20111	build/pkgs/pybind11: New, dependency of scipy
​a5c17a8	build/pkgs/cython: Update to 0.29.16
​c7cc999	build/pkgs/cython/checksums.ini: Use predictable URL for upstream_url pattern
​d54b715	Update Cython to 0.29.17
​5958d38	build/pkgs/numpy: Update numpy to 1.18.2, back out numpy-1.10.2-no-hardcode-blas.patch (does not apply)
​0075beb	Update numpy to 1.18.4
​204213d	Update scipy to 1.4.1
​14bcb1f	build/pkgs/pybind11: New, dependency of scipy
​1d5ea9e	Merge branch 'u/mkoeppe/local_conda_forge_macos_standard__build_error_with_numpy' of trac.sagemath.org:sage into networkx24
​2310c79	update networkx to version 2.4

[fbissey]

You may get more stuff if you are using scipy 1.4 with that. I patched all the deprecation I could in sage-on-gentoo ​https://github.com/cschwan/sage-on-gentoo/blob/master/dev-python/networkx/files/networkx-2.4-scipy-1.4.patch

[dimpase]

for scipy 1.4 (cf #29425 - which is a dependency of this ticket!) I chose to suppress these warnings in a Pythonic way, rather than doing patching already done by upstream (just not yet released)

[tscrim]

Minor (perhaps trivial) point, but pulling it from that link gave a zip file named networkx-networkx-2.4.zip, which caused it to fail without manually downloading and naming the tarball correctly.

With this ticket, I get two deprecation warnings in generic_graph.py on these two tests:

File "src/sage/graphs/generic_graph.py", line 9786, in sage.graphs.generic_graph.GenericGraph.?
Failed example:
    G.pagerank(algorithm="Scipy") # abs tol 1e-9

File "src/sage/graphs/generic_graph.py", line 9793, in sage.graphs.generic_graph.GenericGraph.?
Failed example:
    G.pagerank(algorithm="Scipy", by_weight=True) # abs tol 1e-9

[dimpase]

Could you post deprecation warnings you get? Cause I thought these were fixed on #29425

[tscrim]

Both of them are like this:

    doctest:warning
      File "/home/uqtscrim/sage-build/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1226, in run
        self.run_doctests()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 927, in run_doctests
        self.dispatcher.dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2011, in dispatch
        self.parallel_dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1906, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in start
        super(DocTestWorker, self).start()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2150, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2478, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 878, in run
        return self._run(test, compileflags, out)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 680, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1104, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generic_graph.GenericGraph.?[8]>", line 1, in <module>
        G.pagerank(algorithm="Scipy", by_weight=True) # abs tol 1e-9
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 9877, in pagerank
        dangling=dangling)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/networkx/algorithms/link_analysis/pagerank_alg.py", line 432, in pagerank_scipy
        S = scipy.array(M.sum(axis=1)).flatten()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 19, in call
        stacklevel=stacklevel)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: scipy.array is deprecated and will be removed in SciPy 2.0.0, use numpy.array instead
    doctest:warning
      File "/home/uqtscrim/sage-build/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1226, in run
        self.run_doctests()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 927, in run_doctests
        self.dispatcher.dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2011, in dispatch
        self.parallel_dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1906, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in start
        super(DocTestWorker, self).start()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2150, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2478, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 878, in run
        return self._run(test, compileflags, out)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 680, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1104, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generic_graph.GenericGraph.?[8]>", line 1, in <module>
        G.pagerank(algorithm="Scipy", by_weight=True) # abs tol 1e-9
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 9877, in pagerank
        dangling=dangling)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/networkx/algorithms/link_analysis/pagerank_alg.py", line 439, in pagerank_scipy
        x = scipy.repeat(1.0 / N, N)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 19, in call
        stacklevel=stacklevel)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: scipy.repeat is deprecated and will be removed in SciPy 2.0.0, use numpy.repeat instead
    doctest:warning
      File "/home/uqtscrim/sage-build/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1226, in run
        self.run_doctests()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 927, in run_doctests
        self.dispatcher.dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2011, in dispatch
        self.parallel_dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1906, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in start
        super(DocTestWorker, self).start()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2150, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2478, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 878, in run
        return self._run(test, compileflags, out)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 680, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1104, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generic_graph.GenericGraph.?[8]>", line 1, in <module>
        G.pagerank(algorithm="Scipy", by_weight=True) # abs tol 1e-9
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 9877, in pagerank
        dangling=dangling)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/networkx/algorithms/link_analysis/pagerank_alg.py", line 446, in pagerank_scipy
        p = scipy.repeat(1.0 / N, N)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 19, in call
        stacklevel=stacklevel)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: scipy.repeat is deprecated and will be removed in SciPy 2.0.0, use numpy.repeat instead
    doctest:warning
      File "/home/uqtscrim/sage-build/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1226, in run
        self.run_doctests()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 927, in run_doctests
        self.dispatcher.dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2011, in dispatch
        self.parallel_dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1906, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in start
        super(DocTestWorker, self).start()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2150, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2478, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 878, in run
        return self._run(test, compileflags, out)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 680, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1104, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generic_graph.GenericGraph.?[8]>", line 1, in <module>
        G.pagerank(algorithm="Scipy", by_weight=True) # abs tol 1e-9
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 9877, in pagerank
        dangling=dangling)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/networkx/algorithms/link_analysis/pagerank_alg.py", line 459, in pagerank_scipy
        is_dangling = scipy.where(S == 0)[0]
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 19, in call
        stacklevel=stacklevel)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: scipy.where is deprecated and will be removed in SciPy 2.0.0, use numpy.where instead
    doctest:warning
      File "/home/uqtscrim/sage-build/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1226, in run
        self.run_doctests()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/control.py", line 927, in run_doctests
        self.dispatcher.dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2011, in dispatch
        self.parallel_dispatch()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1906, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in start
        super(DocTestWorker, self).start()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2150, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2478, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 878, in run
        return self._run(test, compileflags, out)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 680, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1104, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generic_graph.GenericGraph.?[8]>", line 1, in <module>
        G.pagerank(algorithm="Scipy", by_weight=True) # abs tol 1e-9
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 9877, in pagerank
        dangling=dangling)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/networkx/algorithms/link_analysis/pagerank_alg.py", line 467, in pagerank_scipy
        err = scipy.absolute(x - xlast).sum()
      File "/home/uqtscrim/sage-build/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 19, in call
        stacklevel=stacklevel)
      File "/home/uqtscrim/sage-build/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: scipy.absolute is deprecated and will be removed in SciPy 2.0.0, use numpy.absolute instead

[tscrim]

When I run the file as a standalone, I get

File "src/sage/graphs/generic_graph.py", line 4762, in sage.graphs.generic_graph.GenericGraph.minimum_cycle_basis
Failed example:
    sorted(g.minimum_cycle_basis(by_weight=True, algorithm='NetworkX'))
Expected:
    [[1, 2, 3], [1, 2, 3, 4], [5, 6, 7]]
Got:
    doctest:warning
      File "/home/uqtscrim/sage/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1226, in run
        self.run_doctests()
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 927, in run_doctests
        self.dispatcher.dispatch()
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2011, in dispatch
        self.parallel_dispatch()
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1906, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in start
        super(DocTestWorker, self).start()
      File "/home/uqtscrim/sage/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/uqtscrim/sage/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/uqtscrim/sage/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/uqtscrim/sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/uqtscrim/sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/uqtscrim/sage/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2150, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2478, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 878, in run
        return self._run(test, compileflags, out)
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 680, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1104, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generic_graph.GenericGraph.minimum_cycle_basis[3]>", line 1, in <module>
        sorted(g.minimum_cycle_basis(by_weight=True, algorithm='NetworkX'))
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 4805, in minimum_cycle_basis
        return networkx.minimum_cycle_basis(G, weight='weight')
      File "</home/uqtscrim/sage/local/lib/python3.7/site-packages/decorator.py:decorator-gen-354>", line 2, in minimum_cycle_basis
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/networkx/utils/decorators.py", line 82, in _not_implemented_for
        return not_implement_for_func(*args, **kwargs)
      File "</home/uqtscrim/sage/local/lib/python3.7/site-packages/decorator.py:decorator-gen-353>", line 2, in minimum_cycle_basis
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/networkx/utils/decorators.py", line 82, in _not_implemented_for
        return not_implement_for_func(*args, **kwargs)
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/networkx/algorithms/cycles.py", line 525, in minimum_cycle_basis
        nx.connected_component_subgraphs(G)), [])
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/networkx/algorithms/cycles.py", line 524, in <genexpr>
        return sum((_min_cycle_basis(c, weight) for c in
      File "/home/uqtscrim/sage/local/lib/python3.7/site-packages/networkx/algorithms/components/connected.py", line 86, in connected_component_subgraphs
        _warnings.warn(msg, DeprecationWarning)
      File "/home/uqtscrim/sage/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: connected_component_subgraphs is deprecated and will be removedin 2.2. Use (G.subgraph(c).copy() for c in connected_components(G))
    [[1, 2, 3], [1, 2, 3, 4], [5, 6, 7]]

[fbissey]

You shouldn't have this warning with networkx 2.4. It implies you are still using 2.2.

[dimpase]

I didn't fully merge #29425 into this branch. I can do it (must I?), or leave it to the release manager.

as to the url - I guess it's something to do with GitHub naming of archives.

[tscrim]

@fbissey I am using 2.4 as far as Sage knows:

./sage --installed
...
networkx................................2.4 (2.4)

@dimpase I didn't realize the entire branch of #29425 had not been merged in. It looked like it had from a quick glance. Once I did that then indeed I do not get the deprecation warnings.

[tscrim]

Since the doctests do pass for me with #29425 (and you don't need to merge it in any further I think), positive review.

[dimpase]

Replying to tscrim:

Minor (perhaps trivial) point, but pulling it from that link gave a zip file named networkx-networkx-2.4.zip, which caused it to fail without manually downloading and naming the tarball correctly.

I don't quite understand what "pulling it from that link" means. The idea of upstream_url in checksum.ini is to allow automatic download of the package archive from that URL. (which can either be enabled with -o, as follows, ./sage -i -o networkx of if one runs ./configure with --enable-download-from-upstream-url option).

And this automation works for me on Linux. The ULR is copied verbatim from ​https://github.com/networkx/networkx/releases/tag/networkx-2.4

[tscrim]

Replying to dimpase:

Replying to tscrim:

Minor (perhaps trivial) point, but pulling it from that link gave a zip file named networkx-networkx-2.4.zip, which caused it to fail without manually downloading and naming the tarball correctly.

I don't quite understand what "pulling it from that link" means. The idea of upstream_url in checksum.ini is to allow automatic download of the package archive from that URL. (which can either be enabled with -o, as follows, ./sage -i -o networkx of if one runs ./configure with --enable-download-from-upstream-url option).

And this automation works for me on Linux. The ULR is copied verbatim from ​https://github.com/networkx/networkx/releases/tag/networkx-2.4

Okay, I tried with just make instead of directly with sage -i (which worked by itself when I did this on another spkg previously), so maybe that is why it didn't work. I was just thinking it came from the zip file having an unexpected name.

[mkoeppe]

From #29425: running the testsuite of networkx requires pytest.

[mkoeppe]

Replying to mkoeppe:

From #29425: running the testsuite of networkx requires pytest.

I'll add pytest in #29813.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9a3617d	build/pkgs/cython: Update to 0.29.16
​4a606b6	build/pkgs/cython/checksums.ini: Use predictable URL for upstream_url pattern
​e9a7cb0	Update Cython to 0.29.17
​15911cb	build/pkgs/numpy: Update numpy to 1.18.2, back out numpy-1.10.2-no-hardcode-blas.patch (does not apply)
​96e7a41	Update numpy to 1.18.4
​78b4907	Update scipy to 1.4.1
​54595b0	build/pkgs/pybind11: New, dependency of scipy
​c005562	suppress scipy deprecation warnings
​56a6fe1	build/pkgs/numpy: Update to 1.18.5
​e592bdc	Merge branch 't/29425/trac/local_conda_forge_macos_standard__build_error_with_numpy' into t/29766/packages/networkx24

[mkoeppe]

Merged in updated #29425 and #29813.

[mkoeppe]

Tests at ​https://github.com/mkoeppe/sage/pull/38/checks

[mkoeppe]

#29813 needs to be fixed.

[mkoeppe]

Merged ticket #29425 into this ticket because #29245 could not be separately tested because of networkx 2.2 testsuite errors triggered by updates.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bedc7ae	build/make/Makefile.in: Allow pip packages as dependencies
​11bf67f	Merge branch 't/29813/add_pytest_as_a_type_optional__source_pip_package' into t/29766/packages/networkx24

[mkoeppe]

Tests at ​https://github.com/mkoeppe/sage/actions/runs/128112212:

On ubuntu-eoan-standard (​https://github.com/mkoeppe/sage/runs/748390625), LOTS of failures in the networkx testsuite:

Ran 3618 tests in 386.630s

FAILED (errors=808)

Is this expected?

[mkoeppe]

Failure on cygwin-standard (​https://github.com/mkoeppe/sage/runs/748608397):

  g++: scipy/optimize/rectangular_lsap/rectangular_lsap.cpp
  scipy/optimize/rectangular_lsap/rectangular_lsap.cpp:117:43: error: ‘int64_t’ has not been declared
    117 | solve(int nr, int nc, double* input_cost, int64_t* output_col4row)
        |                                           ^~~~~~~
  scipy/optimize/rectangular_lsap/rectangular_lsap.cpp:186:41: error: ‘int64_t’ has not been declared
    186 |                                         int64_t* col4row)
        |                                         ^~~~~~~

[mkoeppe]

The int64_t issue is fixed in ​https://github.com/scipy/scipy/pull/11320 for the upcoming 1.5.0, marked "backport-candidate", but so far no 1.4.x with the fix has been released...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​db0fa41

build/pkgs/scipy: Add patch for scipy pr 11320

[tscrim]

Everything builds and works for me on Ubuntu.

[dimpase]

lgtm

[mkoeppe]

Comment 29 is unresolved

[dimpase]

a package fails its own testsuite. it's not the 1st one (happens on Gentoo as well)

[mkoeppe]

OK, good to know. No objection then

[vbraun]

Lots of failures of the form

**********************************************************************
File "src/sage/modular/modform_hecketriangle/abstract_space.py", line 2235, in sage.modular.modform_hecketriangle.abstract_space.FormsSpace_abstract.ZZ
Failed example:
    qexp_int = WF.rationalize_series(qexp)
Expected nothing
Got:
    doctest:warning
      File "/home/release/Sage/src/bin/sage-runtests", line 178, in <module>
        err = DC.run()
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1231, in run
        self.run_doctests()
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 931, in run_doctests
        self.dispatcher.dispatch()
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2045, in dispatch
        self.parallel_dispatch()
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1940, in parallel_dispatch
        w.start()  # This might take some time
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2212, in start
        super(DocTestWorker, self).start()
      File "/home/release/Sage/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/home/release/Sage/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/home/release/Sage/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/home/release/Sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/home/release/Sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/home/release/Sage/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2184, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2513, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2562, in _run
        result = runner.run(test)
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 912, in run
        return self._run(test, compileflags, out)
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 707, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1138, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.modular.modform_hecketriangle.abstract_space.FormsSpace_abstract.ZZ[4]>", line 1, in <module>
        qexp_int = WF.rationalize_series(qexp)
      File "/home/release/Sage/local/lib/python3.7/site-packages/sage/modular/modform_hecketriangle/abstract_space.py", line 2309, in rationalize_series
        warn("Using an experimental rationalization of coefficients, please check the result for correctness!")
      File "/home/release/Sage/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    UserWarning: Using an experimental rationalization of coefficients, please check the result for correctness!
**********************************************************************
2 items had failures:
   1 of  29 in sage.modular.modform_hecketriangle.abstract_space.FormsSpace_abstract.ZZ
   1 of  41 in sage.modular.modform_hecketriangle.abstract_space.FormsSpace_abstract.construct_quasi_form
    [566 tests, 2 failures, 7.37 s]
----------------------------------------------------------------------
sage -t --long --warn-long 37.2 src/sage/modular/modform_hecketriangle/abstract_space.py  # 2 doctests failed
----------------------------------------------------------------------

[fbissey]

This looks strange. I have done a test run of sage 9.2.beta2 with numpy-1.19/scipy-1.5/networkx-2.4 in sage-on-gentoo and I have seen no such things [in fact the number of doctest failures in sage-on-gentoo is at its lowest in years - 3 doctests across 2 files].

I would think another ticket would be the cause of this, although I am at a loss as to which one when inspecting your last commits. Running my own tests on your branch ASAP.

[arojas]

This is caused by the

warnings.filterwarnings('ignore', '.*Deprec.*scipy.*instead',)

line. This is run on every examples block, and resets the registry of already displayed warnings - so warnings which are not affected by a filter, which by default are only displayed the first time they are hit, are shown again.

[dimpase]

oh dear. ok, we can suppress these, too

[slelievre]

From the SciPy 1.5.1 Release Notes:

SciPy 1.5.1 is a bug-fix release with no new features compared to 1.5.0. In particular, an issue where DLL loading can fail for SciPy wheels on Windows with Python 3.6 has been fixed.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2efe841	Merge tag '9.2.beta3' into t/29766/packages/networkx24
​634ce73	build/pkgs: Update numpy to 1.19, scipy to 1.5.1
​a2e1978	build/pkgs/scipy/patches/pr-11320.patch: Remove

[mkoeppe]

Tests run at ​https://github.com/mkoeppe/sage/actions/runs/157933479

[mkoeppe]

The warnings.filterwarnings issue needs to be addressed

[dimpase]

Replying to slelievre:

From the SciPy 1.5.1 Release Notes:

SciPy 1.5.1 is a bug-fix release with no new features compared to 1.5.0. In particular, an issue where DLL loading can fail for SciPy wheels on Windows with Python 3.6 has been fixed.

no worries, the ticket description was not updated, that's all.

[dimpase]

wrong tarball hash for numpy. See ​https://github.com/numpy/numpy/issues/16754

• the usual GH crappy tarballs stuff I guess.

[dimpase]

Currently it is

build/pkgs/numpy/checksums.ini

@@ -1 +1 @@
 tarball=numpy-VERSION.tar.gz
-sha1=0b1792c8a828d6a9fb148fb33dfa11549391d858
-md5=3f5ce88a859302f0a1aceb5f75b563fc
-cksum=1603544858
+sha1=a6c099c0929e0b437a72fd60aad76d23061087c6
+md5=e3baa59da67348c2d6596803514129db
+cksum=1607167529
 upstream_url=https://github.com/numpy/numpy/releases/download/vVERSION/numpy-VERSION.tar.gz

but of course it's not really stable...

[dimpase]

And now md5 as advertised on Numpy pages:

build/pkgs/numpy/checksums.ini

@@ -1 +1 @@
 tarball=numpy-VERSION.tar.gz
-sha1=0b1792c8a828d6a9fb148fb33dfa11549391d858
-md5=3f5ce88a859302f0a1aceb5f75b563fc
-cksum=1603544858
+sha1=ba35592527bd01f53244ae1bb55c896101bbf0f3
+md5=d59aadf47354bd10c7b9996032ba4da0
+cksum=3500020755

Sucks.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0879c3b

build/pkgs/{numpy,scipy}/checksums.ini: Set upstream_url to pypi.io

[mkoeppe]

Hopefully the sources packages on PyPI are more stable.

[dimpase]

Replying to mkoeppe:

The warnings.filterwarnings issue needs to be addressed

It appears I figured out this issue; doing

src/sage/all.py

@@ -87 +87 @@
 # Ignore all deprecations from IPython etc.
 warnings.filterwarnings('ignore', category=DeprecationWarning,
     module='.*(IPython|ipykernel|jupyter_client|jupyter_core|nbformat|notebook|ipywidgets|storemagic|jedi)')
+warnings.filterwarnings('ignore', category=DeprecationWarning,
+    module='.*(scipy|networkx)')
 # Ignore collections.abc warnings, there are a lot of them but they are
 # harmless.
 warnings.filterwarnings('ignore', category=DeprecationWarning,

src/sage/doctest/forker.py

@@ -881 +881 @@
         #             and will be removed in SciPy 2.0.0, use numpy.array instead
         # This affects networkx 2.2 up and including 2.4
         # We filter them out here:
-        warnings.filterwarnings('ignore', '.*Deprec.*scipy.*instead',) # cf trac #29425
+ #       warnings.filterwarnings('ignore', '.*Deprec.*scipy.*instead',) # cf trac #29425
 
         warnings.showwarning = showwarning_with_traceback
         self.running_doctest_digest = hashlib.md5()

appears to fix it - still testing, but expect a success. Basically, we already did suppression of deprecation warnings from other packages right, now the same has to be done for scipy and networkx

[arojas]

Basically, we already did suppression of deprecation warnings from other packages right, now the same has to be done for scipy and networkx

Doesn't that also hide deprecation warnings coming from sage code, and thus would prevent detecting such deprecated code use with future scipy/networkx updates? That sort of defeats the point of the previous

    deprecationWarning = ('ignore', None, DeprecationWarning, None, 0)
    if deprecationWarning in warnings.filters: warnings.filters.remove(deprecationWarning)

[dimpase]

Replying to arojas:

Basically, we already did suppression of deprecation warnings from other packages right, now the same has to be done for scipy and networkx

Doesn't that also hide deprecation warnings coming from sage code, and thus would prevent detecting such deprecated code use with future scipy/networkx updates? That sort of defeats the point of the previous

    deprecationWarning = ('ignore', None, DeprecationWarning, None, 0)
    if deprecationWarning in warnings.filters: warnings.filters.remove(deprecationWarning)

module='.*(scipy|networkx)' suppresses deprecation warnings emitted by code in scipy or in networkx. It does not do anything to these warnings from Sage code itself. Indeed, the tests in src/sage/tests/deprecation_test.py still pass.

[dimpase]

needless to say, each upgrade to scipy|networkx will need to re-evaluate these.

We should add something to the Developer manual on this.

[dimpase]

correct suppression of warnings

---

New commits:

​8a70ebe

right place for warnings.filterwarnings() for networkx & scipy

[mkoeppe]

Last 10 new commits:

​b7bf43b	build/bin/write-dockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884
​365ce61	Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD
​fb61a31	Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+ci-fixes
​bfb2fb4	tox.ini: Add environment local-homebrew-usrlocal
​aded967	Merge commit '8195cb821e01e0cb627240d7135d5634c265cab0' of git://trac.sagemath.org/sage into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​07d657d	Merge tag '9.2.beta4' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​bd15fb9	build/bin/write-dockerfile.sh, tox.ini: Do not run testsuites that need pip (ssl)
​25393b0	Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sage-spkg
​5aca238	Merge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​0356be9	Merge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29766/packages/networkx24

[mkoeppe]

Elevating this upgrade ticket to "blocker" because homebrew builds are currently broken because they now ship gfortran 10.

[mkoeppe]

(see for example ​https://github.com/sagemath/sage/runs/862451287)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6954deb

build/pkgs/networkx/dependencies: Conditionalize on SAGE_CHECK_networkx

[mkoeppe]

Tests as part of Meta-ticket #29456 (Support gcc/gfortran 10) at ​https://github.com/mkoeppe/sage/actions/runs/167913617

On debian-jessie-standard (​https://github.com/mkoeppe/sage/runs/866989168) and debian-jessie-minimal:

gcc: build/src.linux-x86_64-3.7/numpy/core/src/umath/loops.c
In file included from numpy/core/src/umath/loops.c.src:50:0:
numpy/core/src/umath/simd.inc.src: In function ‘avx512_conjugate_ps’:
numpy/core/src/umath/simd.inc.src:1843:5: warning: implicit declaration of function ‘_mm512_castps_si512’ [-Wimplicit-function-declaration]
     __m512i cast_x = _mm512_cast@vsub@_si512(x);
     ^
numpy/core/src/umath/simd.inc.src:1843:22: error: incompatible types when initializing type ‘__vector(8) long long int’ using type ‘int’
     __m512i cast_x = _mm512_cast@vsub@_si512(x);
                      ^
numpy/core/src/umath/simd.inc.src:1846:5: warning: implicit declaration of function ‘_mm512_castsi512_ps’ [-Wimplicit-function-declaration]
     return _mm512_castsi512_@vsub@(res);
     ^
numpy/core/src/umath/simd.inc.src:1846:5: error: incompatible types when returning type ‘int’ but ‘__vector(16) float’ was expected

[mkoeppe]

debian-jessie uses gcc version 4.9.2 (Debian 4.9.2-10+deb8u2), and a fix for something that looks related is in gcc 4.9.3 - ​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61878

[mkoeppe]

This was added to numpy in ​https://github.com/numpy/numpy/commit/5562a8c93fe18f0a51d9051f0c25c7cf525312fe (new in 1.19.0) (​https://github.com/numpy/numpy/pull/15408)

[mkoeppe]

Let's go back to 1.18.5 (released Jun 3, 2020).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d39817f

build/pkgs/numpy: Downgrade to 1.18.5 to fix build error on debian-jessie

[mkoeppe]

New tests at ​https://github.com/mkoeppe/sage/actions/runs/168163111

[mkoeppe]

Actually ​https://github.com/mkoeppe/sage/actions/runs/168165380

[mkoeppe]

In these tests, fedora-32-standard uses gcc/gfortran 10 from the system, fedora-32-minimal uses gcc 10 from the system and gfortran from spkg, and the various homebrew-macos tests use gcc = apple clang, gfortran 10 from homebrew.

They all succeed. Still waiting for other tests to finish.

[mkoeppe]

On local-conda-forge-macos-maximal (​https://github.com/mkoeppe/sage/runs/867731009), numpy 1.18.5 builds but building sagelib fails as follows:

  [sagelib-9.2.beta5] error installing, exit status 1. End of log file:
  [sagelib-9.2.beta5]       from . import core
  [sagelib-9.2.beta5]     File "/Users/runner/work/sage/sage/.tox/local-conda-forge-macos-maximal/local/lib/python3.7/site-packages/numpy/core/__init__.py", line 50, in <module>
  [sagelib-9.2.beta5]       raise ImportError(msg)
  [sagelib-9.2.beta5]   ImportError: 
  [sagelib-9.2.beta5]   
  [sagelib-9.2.beta5]   IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
  [sagelib-9.2.beta5]   
  [sagelib-9.2.beta5]   Importing the numpy C-extensions failed. This error can happen for
  [sagelib-9.2.beta5]   many reasons, often due to issues with your setup or how NumPy was
  [sagelib-9.2.beta5]   installed.
...
  [sagelib-9.2.beta5]   Original error was: dlopen(/Users/runner/work/sage/sage/.tox/local-conda-forge-macos-maximal/local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-darwin.so, 2): Symbol not found: _aheapsort_bool
  [sagelib-9.2.beta5]     Referenced from: /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-maximal/local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-darwin.so
  [sagelib-9.2.beta5]     Expected in: flat namespace
  [sagelib-9.2.beta5]    in /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-maximal/local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-darwin.so

Also building matplotlib fails with a similar error.

[mkoeppe]

numpy 1.19 had worked OK on local-conda-forge-macos when that was tested (​https://github.com/mkoeppe/sage/runs/858960241).

[mkoeppe]

Trying next with ​https://github.com/numpy/numpy/pull/16871

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fcddf69	Revert "build/pkgs/numpy: Downgrade to 1.18.5 to fix build error on debian-jessie"
​3558737	build/pkgs/numpy/patches/16871.patch: New

[mkoeppe]

Running at ​https://github.com/mkoeppe/sage/actions/runs/169304521

[mkoeppe]

Unfortunately the patch does not apply by itself

[mkoeppe]

Running tests with numpy master + pr-16871 at ​https://github.com/mkoeppe/sage/actions/runs/169620460

[mkoeppe]

​https://github.com/numpy/numpy/pull/16871 has been merged in numpy master. Hoping for a backport

[dimpase]

there are more deprecation warnings to suppress, e.g.

2020-07-15T10:18:00.4452989Z File "src/sage/calculus/interpolators.pyx", line 48, in sage.calculus.interpolators.polygon_spline
2020-07-15T10:18:00.4453182Z Failed example:
2020-07-15T10:18:00.4453370Z     ps = polygon_spline(pts)
2020-07-15T10:18:00.4453562Z Expected nothing
2020-07-15T10:18:00.4453746Z Got:
2020-07-15T10:18:00.4453928Z     doctest:warning
2020-07-15T10:18:00.4454284Z       File "/sage/src/bin/sage-runtests", line 177, in <module>
2020-07-15T10:18:00.4454494Z         err = DC.run()
2020-07-15T10:18:00.4454877Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1207, in run
2020-07-15T10:18:00.4455104Z         self.run_doctests()
2020-07-15T10:18:00.4455498Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 908, in run_doctests
2020-07-15T10:18:00.4455721Z         self.dispatcher.dispatch()
2020-07-15T10:18:00.4456125Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2039, in dispatch
2020-07-15T10:18:00.4456344Z         self.parallel_dispatch()
2020-07-15T10:18:00.4456736Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1934, in parallel_dispatch
2020-07-15T10:18:00.4456963Z         w.start()  # This might take some time
2020-07-15T10:18:00.4457348Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2206, in start
2020-07-15T10:18:00.4457571Z         super(DocTestWorker, self).start()
2020-07-15T10:18:00.4457778Z       File "/sage/local/lib/python3.7/multiprocessing/process.py", line 112, in start
2020-07-15T10:18:00.4457985Z         self._popen = self._Popen(self)
2020-07-15T10:18:00.4458194Z       File "/sage/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
2020-07-15T10:18:00.4458409Z         return _default_context.get_context().Process._Popen(process_obj)
2020-07-15T10:18:00.4458621Z       File "/sage/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
2020-07-15T10:18:00.4458822Z         return Popen(process_obj)
2020-07-15T10:18:00.4459027Z       File "/sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
2020-07-15T10:18:00.4459245Z         self._launch(process_obj)
2020-07-15T10:18:00.4459447Z       File "/sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
2020-07-15T10:18:00.4459645Z         code = process_obj._bootstrap()
2020-07-15T10:18:00.4459852Z       File "/sage/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
2020-07-15T10:18:00.4460066Z         self.run()
2020-07-15T10:18:00.4460446Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2178, in run
2020-07-15T10:18:00.4461480Z         task(self.options, self.outtmpfile, msgpipe, self.result_queue)
2020-07-15T10:18:00.4462380Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2507, in __call__
2020-07-15T10:18:00.4462669Z         doctests, extras = self._run(runner, options, results)
2020-07-15T10:18:00.4463154Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2553, in _run
2020-07-15T10:18:00.4463441Z         result = runner.run(test)
2020-07-15T10:18:00.4463902Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 906, in run
2020-07-15T10:18:00.4464158Z         return self._run(test, compileflags, out)
2020-07-15T10:18:00.4464793Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 707, in _run
2020-07-15T10:18:00.4465103Z         self.compile_and_execute(example, compiler, test.globs)
2020-07-15T10:18:00.4465621Z       File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1132, in compile_and_execute
2020-07-15T10:18:00.4465963Z         exec(compiled, globs)
2020-07-15T10:18:00.4466212Z       File "<doctest sage.calculus.interpolators.polygon_spline[1]>", line 1, in <module>
2020-07-15T10:18:00.4466479Z         ps = polygon_spline(pts)
2020-07-15T10:18:00.4466937Z       File "/sage/local/lib/python3.7/site-packages/numpy/__init__.py", line 237, in __getattr__
2020-07-15T10:18:00.4467232Z         warnings.warn(msg, DeprecationWarning, stacklevel=2)
2020-07-15T10:18:00.4467504Z       File "/sage/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
2020-07-15T10:18:00.4467750Z         msg.file, msg.line)
2020-07-15T10:18:00.4468009Z     :
2020-07-15T10:18:01.5210352Z     DeprecationWarning: `np.complex` is a deprecated alias for the builtin `complex`. Use `complex` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.complex_` here.

[mkoeppe]

these deprecation warnings are coming from numpy master, they will not hit us now, only later

[mkoeppe]

numpy 1.19.1 is out but unfortunately does not have a backport of ​​https://github.com/numpy/numpy/pull/16871

[mkoeppe]

However, thanks to other backported fixes, the pull request is now easier to apply.

[mkoeppe]

numpy 1.19.1 needs newest Cython (#29861)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1bf6249	Update numpy to 1.19.1, add patches
​c24d109	build/pkgs/cython: Updatt to 0.29.21
​061e975	Merge branch 'u/mkoeppe/upgrade_cython_to_0_29_20' of git://trac.sagemath.org/sage into t/29766/packages/networkx24

[mkoeppe]

Tests run at ​https://github.com/mkoeppe/sage/actions/runs/177662666

[mkoeppe]

I see a number of doctest failures related to printing of deprecation warnings. Did something on this ticket cause this? For example (on ubuntu-trusty-standard, ​https://github.com/mkoeppe/sage/runs/896153611):

sage -t src/sage/algebras/lie_algebras/free_lie_algebra.py
**********************************************************************
File "src/sage/algebras/lie_algebras/free_lie_algebra.py", line 194, in sage.algebras.lie_algebras.free_lie_algebra.FreeLieBasis_abstract.basis
Failed example:
    L.Hall().basis()
Expected:
    Disjoint union of Lazy family (graded basis(i))_{i in Positive integers}
Got:
    doctest:warning
      File "/sage/src/bin/sage-runtests", line 177, in <module>
        err = DC.run()
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1207, in run
        self.run_doctests()
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 908, in run_doctests
        self.dispatcher.dispatch()
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2045, in dispatch
        self.parallel_dispatch()
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1940, in parallel_dispatch
        w.start()  # This might take some time
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2212, in start
        super(DocTestWorker, self).start()
      File "/sage/local/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/sage/local/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/sage/local/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/sage/local/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/sage/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2184, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2513, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2559, in _run
        result = runner.run(test)
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 912, in run
        return self._run(test, compileflags, out)
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 707, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1138,       compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.algebras.lie_algebras.free_lie_algebra.FreeLieBasis_abstract.basis[1]>", line 1, in <module>
        L.Hall().basis()
      File "/sage/local/lib/python3.7/site-packages/sage/structure/unique_representation.py", line 1008, in __classcall__
        instance = typecall(cls, *args, **options)
      File "/sage/local/lib/python3.7/site-packages/sage/algebras/lie_algebras/free_lie_algebra.py", line 445, in __init__
        experimental_warning(16823, "The Hall basis has not been fully proven correct,"
      File "/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 181, in experimental_warning
        warning(trac_number, message, FutureWarning, stacklevel)
      File "/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 146, in warning
        warn(message, warning_class, stacklevel)
      File "/sage/local/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    FutureWarning: The Hall basis has not been fully proven correct, but currently no bugs are known
    See http://trac.sagemath.org/16823 for details.
    Disjoint union of Lazy family (graded basis(i))_{i in Positive integers}
**********************************************************************

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d1a2cbf	Merge branch 't/30088/fix__make_sdist___add_test_run_to_github_actions' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​b62f775	Merge tag '9.2.beta6' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​fc7592c	Merge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29766/packages/networkx24

[mkoeppe]

Merged 9.2.beta6

[dimpase]

Replying to mkoeppe:

I see a number of doctest failures related to printing of deprecation warnings. Did something on this ticket cause this?

​https://git.sagemath.org/sage.git/diff/src/sage/doctest/forker.py?id=fc7592cdafb6ed3b1d947faeeb67841c3491e843&id2=ddaba54f6d69276ed1b9c60193ace839542eb5a9

this - which is actually a wrong place to deal with this stuff. This branch seems to be obsolete at this place - there is a better place to handle these warnings, and I think I've put it on this ticket some time back - but then it did not stick for some reason.

[dimpase]

namely, you need ​https://git.sagemath.org/sage.git/commit?id=8a70ebedcf063d0a77564c01537a9600dd2accda

(cf. comment:64)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7fe5985

right place for warnings.filterwarnings() for networkx & scipy

[mkoeppe]

Tests run at ​https://github.com/mkoeppe/sage/actions/runs/183520034

[vdelecroix]

scipy 1.5.2 is now available.

[vdelecroix]

I report that compilation succeeded with gcc 10.1.0 and this ticket merged.

[mkoeppe]

I think this is good to go. Reviewers?

[vdelecroix]

Don't you want to make the upgrade scipy 1.5.1 -> 1.5.2?

[dimpase]

self-tests of networkx need pytest, but logs show that pytest might bot be installed. I don't understand the meaning of this poorly half-implemented LISPy thing:

$(PYTHON) decorator | $(PYTHON_TOOLCHAIN) scipy $(and $(filter-out no,$(SAGE_CHECK_networkx)), nose pytest)

[mkoeppe]

which log? link please

[dimpase]

Replying to mkoeppe:

which log? link please

​https://github.com/mkoeppe/sage/runs/912558796?check_suite_focus=true

[dimpase]

anyhow, how about upgrading to scipy 1.5.2 along the way?

[mkoeppe]

Replying to dimpase:

I don't understand that meaning of this poorly half-implemented LISPy thing:

$(PYTHON) decorator | $(PYTHON_TOOLCHAIN) scipy $(and $(filter-out no,$(SAGE_CHECK_networkx)), nose pytest)

It keeps "nose pytest" only when running the networkx test suite is requested.

Because these are testsuite-only dependencies.

[dimpase]

Replying to mkoeppe:

Replying to dimpase:

I don't understand that meaning of this poorly half-implemented LISPy thing:

$(PYTHON) decorator | $(PYTHON_TOOLCHAIN) scipy $(and $(filter-out no,$(SAGE_CHECK_networkx)), nose pytest)

It keeps "nose pytest" only when running the networkx test suite is requested.

Because these are testsuite-only dependencies.

but somehow it does not work in the GH Actions realm (or maybe at all), no?

[mkoeppe]

Yes, there is a problem - thanks for catching it.

I am running GH Actions build with SAGE_CHECK=warn, and the code added in #30118 does not handle the exclusions given in SAGE_CHECK_PACKAGES in that case.

Let's change that.

[mkoeppe]

Replying to dimpase:

anyhow, how about upgrading to scipy 1.5.2 along the way?

Sure, let's do it.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5fa0bba	build/pkgs/scipy: Update to 1.5.2
​3890e1a	build/make/Makefile.in: Improve handling of SAGE_CHECK_PACKAGES

[mkoeppe]

Tests run at ​https://github.com/mkoeppe/sage/actions/runs/190765555

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5d4b013

build/make/Makefile.in: Handle the case in which SAGE_CHECK_PACKAGES is set using a command argument

[mkoeppe]

Replying to mkoeppe:

I am running GH Actions build with SAGE_CHECK=warn, and the code added in #30118 does not handle the exclusions given in SAGE_CHECK_PACKAGES in that case.

That was not quite the right explanation, but now I have fixed it. Subtleties of Makefile variables and who overrides whom.

[mkoeppe]

Let's please get this in?

[dimpase]

lgtm

[mkoeppe]

Thank you!

[jhpalmieri]

Why is there no SPKG.rst file for pybind11? Are these suddenly optional?

[mkoeppe]

Replying to jhpalmieri:

Why is there no SPKG.rst file for pybind11? Are these suddenly optional?

A simple mistake. There's actually nothing in our scripts that checks the presence of these files (see e.g. #29334). Let's fix it in #30357.

[slelievre]

Somehow this broke building documentation on macOS. See #30351.

[jhpalmieri]

I have no idea why this would broke docbuilding, but that's what it looks like. Does anyone have ideas how to debug it? Please discuss at #30351.