Add support for staged installations in packages that use sdh_pip_install

[embray]

This implements #22509 for any Python packages that are installed with sdh_pip_install. Passing --root= to pip is essentially the same as using DESTDIR with make install.

This required a few additional changes to a small handful of packages for them to work with staged installation.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​a591ffb	Added the ability for packages to have a post-install script called spkg-postinst.
​599668c	The postinst script should probably be run with SAGE_SUDO if it is set
​ac34ecb	Add a paragraph to the docs about spkg-postinst
​6519c3f	Use $SAGE_SUDO, in case it's set, when moving files from the $SAGE_DESTDIR to $SAGE_LOCAL
​454b75b	Don't use SAGE_SUDO in these helpers if SAGE_DESTDIR is set
​fcd6e5d	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​fd1719e

Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation

[vdelecroix]

merge failure on 8.3.b0

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​d03844b

Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation

[embray]

Rebased since its dependency #24645 has been satisfied.

[jdemeyer]

What's up with build/pkgs/widgetsnbextension/spkg-postinst? That should just be removed.

[jdemeyer]

For sagenb, why not keep

if [ ! -d mathjax ]; then
    echo >&2 "Error: Cannot symlink mathjax into the SageNB data directory."
    exit 1
fi

It seems like a useful sanity check.

[embray]

Replying to jdemeyer:

What's up with build/pkgs/widgetsnbextension/spkg-postinst?

Let me check whether or not that was intended to be in this branch.

That should just be removed.

I don't see how you can assert that without knowing what it is.

[embray]

Replying to jdemeyer:

For sagenb, why not keep

You can see just above that it checks that the directory being symlinked exists before making the symlink.

[jdemeyer]

Replying to embray:

You can see just above that it checks that the directory being symlinked exists before making the symlink.

It's not at all obvious to me that the check before is equivalent to the ln -s succeeding. The check after is simpler and therefore safer.

[jdemeyer]

Replying to embray:

I don't see how you can assert that without knowing what it is.

I know what it is because I wrote the original. The code which is in that spkg-postinst script used to be in spkg-install but it's no longer needed because of Jupyter improvements. At some point, you must have had a merge/rebase conflict with spkg-install and this left-over spkg-postinst script is fallout from that.

[embray]

I see, you even removed the jupyter nbextension call a while ago (but probably after I first wrote this). I believe you then, that it's no longer needed.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​6bbff40	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation
​00bd3f3	try installing the symlink anyways and don't poke around SAGE_LOCAL

[embray]

Ok, I dropped the spkg-postinst for widgetsnbextension and updated the sagenb install. I think this is better now.

[jdemeyer]

For sagenb, you still removed the check for a working symlink. Why?

[embray]

The symlink wouldn't resolve because this is being installed in DESTDIR.

[jdemeyer]

The changes to Sphinx conflict with #24935. It is possible to just revert those?

[embray]

Unfortunately they're necessary for the package to install correctly. You could merge this into #24935.

[saraedum]

there are merge conflicts

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​5cb0a24	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation
​208d754	try installing the symlink anyways and don't poke around SAGE_LOCAL

[embray]

I'm surprised this hasn't been merged yet. It really ought to be.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​49aa08f	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation
​449a6a2	try installing the symlink anyways and don't poke around SAGE_LOCAL

[embray]

Here we go again...

[vbraun]

See patchbot

[embray]

Apparently I had asked Jeroen to make this a prerequisite to #24935 (after all, this ticket was opened first and had no reason to wait so long to be merged). But he forgot about that or ignored it (and I likewise forgot about it).

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b1412f5	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation
​4d20506	try installing the symlink anyways and don't poke around SAGE_LOCAL

[embray]

The spkg-postinst for Sphinx was simply no longer necessary so I removed it.

[embray]

I believe this issue can reasonably be addressed for Sage 8.4.

[vbraun]

You obviously didn't try to build Sage with the ticket

No record that 'sagenb' was ever installed; skipping uninstall
./spkg-install: line 33: syntax error near unexpected token `fi'
./spkg-install: line 33: `fi'

[embray]

Replying to vbraun:

You obviously didn't try to build Sage with the ticket

That's a bit flippant. Of course I built Sage with this ticket. It's been opened for 6 months and has had to be rebased many times. Perhaps there was a small merge error along the way. But that's not as if it was never tested...

[embray]

Also, the ticket doesn't bump the package versions of the affected packages, so you would only see the issue if rebuilding from scratch. The "obviously" is pretty unnecessary.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​7643e66	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation
​32f5bbe	try installing the symlink anyways and don't poke around SAGE_LOCAL
​ff1ccd3	bump package version on sagenb since its spkg-install changes are slightly non-trivial

[embray]

New commits:

​7643e66	Add support for SAGE_DESTDIR to sdh_pip_install, thus making most Python packages work with staged installation
​32f5bbe	try installing the symlink anyways and don't poke around SAGE_LOCAL
​ff1ccd3	bump package version on sagenb since its spkg-install changes are slightly non-trivial