Build documentation of cvxopt

[SimonKing]

When building Sage with SAGE_SPKG_INSTALL_DOCS=yes, cvxopt fails as follows:

[cvxopt-1.1.8.p2]     running install_scripts
[cvxopt-1.1.8.p2]     writing list of installed files to '/tmp/pip-QeTOxN-record/install-record.txt'
[cvxopt-1.1.8.p2]     Running setup.py install for cvxopt: finished with status 'done'
[cvxopt-1.1.8.p2]   Removing source in /tmp/pip-efK92a-build
[cvxopt-1.1.8.p2] Successfully installed cvxopt-1.1.8
[cvxopt-1.1.8.p2] Cleaning up...
[cvxopt-1.1.8.p2] cp: cannot stat 'html/*': No such file or directory
[cvxopt-1.1.8.p2] 
[cvxopt-1.1.8.p2] real  1m5.661s
[cvxopt-1.1.8.p2] user  0m56.644s
[cvxopt-1.1.8.p2] sys   0m3.588s
[cvxopt-1.1.8.p2] ************************************************************************
[cvxopt-1.1.8.p2] Error installing package cvxopt-1.1.8.p2
[cvxopt-1.1.8.p2] ************************************************************************

I guess one either has let the install script create a missing directory, or needs to configure with --docdir=....

[SimonKing]

From spkg-install:

if [ "x$SAGE_SPKG_INSTALL_DOCS" = xyes ] ; then
   cd doc
# This part would be used to build the documentation with sphinx.
# cvxopt would then have to depend on sphinx.
# in 1.1.5 the documentation is shipped already built and up to date.
#   ${MAKE} -B html
#   if [ $? -ne 0 ]; then
#      echo "Error building the documentation"
#      exit 1
#   fi
# checking to see if there is previously installed documentation.
   if [ -d $SAGE_LOCAL/share/doc/cvxopt/html ] ; then
      rm -rf $SAGE_LOCAL/share/doc/cvxopt/html
   fi
   mkdir -p $SAGE_LOCAL/share/doc/cvxopt/html
   cp -r html/* $SAGE_LOCAL/share/doc/cvxopt/html/
fi

The script mentionas 1.1.5. However, we now have 1.1.8.p2. And in the upstream tarball, I find

cvxopt24447/cvxopt-1.1.8$ ls doc/
Makefile  source

So, the docs are in fact *not* shipped by upstream. And building the documentation still requires Sphinx.

But isn't the Sage documentation built using something related to Sphinx? I guess one should (on a different ticket) enhance Sage's machinery of building its docs, so that it can be used to build documentation of a third-party Python/Cython? package. Admittedly I'm being selfish here: My group cohomology spkg would be the same case.

[SimonKing]

I tried to build the documentation. But alas, it won't work. Apparently there is something missing in Sage's sphinx:

[cvxopt-1.1.8.p2] pkg_resources.DistributionNotFound: The 'sphinxcontrib-websupport' distribution was not found and is required by Sphinx

So, is it acceptable to add 'sphinxcontrib-websupport' into Sage's sphinx package?

[SimonKing]

Replying to SimonKing:

So, is it acceptable to add 'sphinxcontrib-websupport' into Sage's sphinx package?

Is sphinx used at all in Sage? I just found that Sage's sphinx wouldn't even do sphinx --help without websupport.

Therefore, I propose that the spkg-install script for sphinx is extended by

pip install sphinxcontrib-websupport

Then, as I just tested, the documentation for cvxopt builds fine.

[SimonKing]

WTF? I tried to add pip install sphinxcontrib-websupport, but it failed with

[sphinx-1.6.3.p1] Collecting sphinxcontrib-websupport
[sphinx-1.6.3.p1]   Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f15799b6650>: Failed to establish a new connection: [Errno 101] Network is unreachable',))': /simple/sphinxcontrib-websupport/
[sphinx-1.6.3.p1]   Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f15799b6590>: Failed to establish a new connection: [Errno 101] Network is unreachable',))': /simple/sphinxcontrib-websupport/
[sphinx-1.6.3.p1]   Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f15799b6e90>: Failed to establish a new connection: [Errno 101] Network is unreachable',))': /simple/sphinxcontrib-websupport/
[sphinx-1.6.3.p1]   Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f15799b6990>: Failed to establish a new connection: [Errno 101] Network is unreachable',))': /simple/sphinxcontrib-websupport/
[sphinx-1.6.3.p1]   Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f15799b66d0>: Failed to establish a new connection: [Errno 101] Network is unreachable',))': /simple/sphinxcontrib-websupport/
[sphinx-1.6.3.p1]   Could not find a version that satisfies the requirement sphinxcontrib-websupport (from versions: )
[sphinx-1.6.3.p1] No matching distribution found for sphinxcontrib-websupport
[sphinx-1.6.3.p1] Error building Sphinx: 'Error installing websupport'

However, if I do pip install sphinxcontrib-websupport in a Sage shell, it just works.

Why is that? How can one use pip in spkg-install? Apparently it isn't sdh_pip_install, as that is only for installation of a local package.

[SimonKing]

Perhaps

python -m pip install sphinxcontrib-websupport

?

At least it seems to work!

[SimonKing]

New commits:

​515ea59	Install sphinx with websupport
​ac5b90b	Allow to build cvxopt's documentation

[fbissey]

For info, sphinxcontrib-websupport used to be shipped as part of sphinx prior to version 1.6. Which is why it suddenly broke. Jeroen and I didn't think sage needed it when we upgraded it. Although I noticed it and got all the right bits into Gentoo. I should raise the bar of the packaging in sage I guess.

More concretely, I think sphinxcontrib-websupport should be a separate standard spkg, that gives interesting dependency problems but it is the right thing to do. And that means inclusion of the tarball for it to become a local install. Stuff in standard package need to be available locally - so even if you put it in cvxopt's spkg-install it should be in upstream. Then because it is an extra on top we don't have a checksum for it. Which is why it needs to be a separate spkg - so not only is it in upstream but we checksum it.

[Konrad127123]

Duplicate of #14645.