Sage: Ticket #22509: Install packages in temporary root before copying to $SAGE_LOCAL
https://trac.sagemath.org/ticket/22509
<p>
Currently we build packages, in most cases, by setting <code>--prefix=$SAGE_LOCAL</code> (in <code>./configure</code>, or the equivalent in non-autoconf packages), and then installing directly to that prefix.
</p>
<p>
This is fine to keep as is. But with automake packages we can also set <code>DESTDIR</code> (<a class="ext-link" href="https://www.gnu.org/software/automake/manual/html_node/DESTDIR.html#DESTDIR"><span class="icon"></span>https://www.gnu.org/software/automake/manual/html_node/DESTDIR.html#DESTDIR</a>) to an alternative (typically temporary) directory into which to install. This has a couple advantages:
</p>
<ul><li>Although it's not as usual for <code>make install</code> to fail after a successful <code>make</code>, it <em>can</em> happen. This prevents messy partial installs in the case of a failed <code>make install</code> run.
</li></ul><ul><li>It gives us the opportunity to make an exact list of the files that were installed. This is a prerequisite to improving package uninstallation/reinstallation in sage-dist (see <a class="closed ticket" href="https://trac.sagemath.org/ticket/22510" title="enhancement: Enhanced package uninstallation for Sage spkgs (closed: worksforme)">#22510</a>).
</li></ul><p>
This is also standard operating procedure in most other packaging systems, so it would be good for Sage to adopt too. The general approach is the same as that used for converting an arbitrary Makefile to support this convention--everywhere a file is installed, prefix the installation path with <code>$(DESTDIR)</code> (or in this case a variable we'll call <code>$SAGE_DESTDIR</code>).
</p>
<p>
For packages that already support the <code>DESTDIR</code> convention, then, we pass <code>DESTDIR=$SAGE_DESTDIR</code> to <code>make install</code>. It is also already possible for Python packages (e.g. <code>pip install --root</code>).
</p>
<p>
Since the introduction of the build helper scripts in <a class="closed ticket" href="https://trac.sagemath.org/ticket/23160" title="enhancement: Add a library of common helper functions for use in spkg-install (closed: fixed)">#23160</a>, it makes the most sense to make these changes directly in the helper scripts, and then update more packages to use those helper scripts where possible.
</p>
<p>
Dependent tickets:
</p>
<ul><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/24106" title="enhancement: Install packages in temporary root before copying to $SAGE_LOCAL ... (closed: fixed)">#24106</a> - implement the basic functionality of supporting staged installs through <code>sage-spkg</code>
</li><li><a class="needs_work ticket" href="https://trac.sagemath.org/ticket/24024" title="task: Meta-ticket: Update other packages to using the sage-dist-helpers ... (needs_work)">#24024</a> - update more <code>spkg-build/install</code> scripts to use the <code>sdh_</code> helper functions, since that is the fastest way to add <code>DESTDIR</code> support for many packages.
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/24646" title="enhancement: Add support for staged installations in packages that use sdh_pip_install (closed: fixed)">#24646</a> - add support for <code>--root=${SAGE_DESTDIR}</code> to Python spkgs
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/24645" title="enhancement: Support for spkg-postinst scripts (closed: fixed)">#24645</a> - add support for post-install scripts; this can be useful for cases where, for example, local files are generated after the package is installed, or to handle cases where one file is shared by multiple packages and needs to be updated when a package that uses it is installed (see e.g. <a class="ext-link" href="https://trac.sagemath.org/ticket/22510#comment:5"><span class="icon"></span>https://trac.sagemath.org/ticket/22510#comment:5</a>)
</li><li>#????? - convert all spkgs to support staged-install; for many packages this can be covered by using the helper functions, but some packages require additional steps; in particular, packages that have files that are manually copied into <code>$SAGE_LOCAL</code>, rather than through a standard installer
</li></ul>en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/22509
Trac 1.1.6mkoeppeFri, 03 Mar 2017 16:33:58 GMTcc set
https://trac.sagemath.org/ticket/22509#comment:1
https://trac.sagemath.org/ticket/22509#comment:1
<ul>
<li><strong>cc</strong>
<em>mkoeppe</em> added
</li>
</ul>
TicketembrayTue, 18 Apr 2017 11:42:12 GMTdescription, milestone changed
https://trac.sagemath.org/ticket/22509#comment:2
https://trac.sagemath.org/ticket/22509#comment:2
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=2">diff</a>)
</li>
<li><strong>milestone</strong>
changed from <em>sage-7.6</em> to <em>sage-wishlist</em>
</li>
</ul>
TicketembrayWed, 31 May 2017 12:13:28 GMTstatus changed; commit, dependencies, branch, author set
https://trac.sagemath.org/ticket/22509#comment:3
https://trac.sagemath.org/ticket/22509#comment:3
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
<li><strong>commit</strong>
set to <em>a25a42a02a89a20058d41d95c56efc1cd0875f26</em>
</li>
<li><strong>dependencies</strong>
set to <em>#23096</em>
</li>
<li><strong>branch</strong>
set to <em>u/embray/build/destdir</em>
</li>
<li><strong>author</strong>
set to <em>Erik Bray</em>
</li>
</ul>
<p>
Started work on this, and converted I think all standard packages to work with a staged installation (install first to a temporary directory, then copy from the temp directory into <code>$SAGE_LOCAL</code>).
</p>
<p>
As I wrote in the description, most packages are easy--either they use automake, or at least a Makefile that's compatible with the <code>DESTDIR</code> convention, or they are Python packages in which case the <code>--root</code> option does the same thing. A handful of packages needed hand-tweaking.
</p>
<p>
This is currently implemented in such a way that if a package <em>doesn't</em> yet support staged install, it will still install just fine. The only major downside is that the installed files are not recorded yet, so they wouldn't be able to take advantage of improved uninstall.
</p>
<p>
That said, I'm going to continue converting the remaining packages. My only question is whether to continue work in this ticket, or separate the general changes (i.e. to <code>sage-spkg</code>) from the updates to the packages themselves.
</p>
<hr />
<p>
Last 10 new commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=3fa088a137af80e1e5a6caf0e7ecdde76ac48cd3"><span class="icon"></span>3fa088a</a></td><td><code>Update pip install to use --root</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=30aad2f0fc94fb4e3a09dfa5e14c28ab95e165c8"><span class="icon"></span>30aad2f</a></td><td><code>Update freetype and pkgconf to use DESTDIR</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=b1aabaa32decdde4e8573d06b034e1df1dabfaca"><span class="icon"></span>b1aabaa</a></td><td><code>Updated various packages with non-standard installers or that are otherwise</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=223608f53595217d2a66b5ddf06a65020a961670"><span class="icon"></span>223608f</a></td><td><code>Update cephes to work with staged install.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=27878c313369cb6b3116eb7b7f0df0be09982ea7"><span class="icon"></span>27878c3</a></td><td><code>Update openblas to use staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=ecd214c42ca6863a249318ae6de6cce1340a9e9d"><span class="icon"></span>ecd214c</a></td><td><code>Update lcalc to support staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=3d64db8dbc2f89d40a65f949d15e87e47a4647fc"><span class="icon"></span>3d64db8</a></td><td><code>Update zn_poly, which doesn't really have a proper installer, to use staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=609393ad73188016b15137552dc6e481886686ff"><span class="icon"></span>609393a</a></td><td><code>Roll back accidental change from dc46ea2dfa97be79d7ef8ddbe9130d4c717acc3d that broke build of Python 2 on Cygwin.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=03424e19de9385c6a30cae01fedfac065cf3a53c"><span class="icon"></span>03424e1</a></td><td><code>Merge remote-tracking branch 'upstream/u/embray/cygwin/ticket-23059' into u/embray/build/destdir</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=a25a42a02a89a20058d41d95c56efc1cd0875f26"><span class="icon"></span>a25a42a</a></td><td><code>Updated the python2/3 pacakges to use staged install.</code>
</td></tr></table>
TicketembrayWed, 31 May 2017 12:14:31 GMTdependencies changed
https://trac.sagemath.org/ticket/22509#comment:4
https://trac.sagemath.org/ticket/22509#comment:4
<ul>
<li><strong>dependencies</strong>
changed from <em>#23096</em> to <em>#23059 #23096</em>
</li>
</ul>
TicketembrayWed, 31 May 2017 12:19:19 GMT
https://trac.sagemath.org/ticket/22509#comment:5
https://trac.sagemath.org/ticket/22509#comment:5
<p>
By way of explanation, the idea here is that there is a variable exported by <code>sage-spkg</code> called <code>$SAGE_INST_TEMP</code> (the name is open to bikeshedding), so this is expected to be passed to each <code>spkg-install</code> to use an alternative install root. In principle this could be defined in each <code>spkg-install</code> individually, but it's less trouble to standardize in one place, and in any case <code>sage-spkg</code> still needs to know this path to copy files from it to <code>$SAGE_LOCAL</code>.
</p>
<p>
Of course, if we take this approach, the developer docs will also need to be updated with some guidance on how to implement staged installation for new packages.
</p>
TicketgitWed, 31 May 2017 12:25:33 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:6
https://trac.sagemath.org/ticket/22509#comment:6
<ul>
<li><strong>commit</strong>
changed from <em>a25a42a02a89a20058d41d95c56efc1cd0875f26</em> to <em>8a621af7a5f3e10d1337c07b42d04edee6a8f1b8</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=3a32187468218cd79e6a5c0defb1f365a488c147"><span class="icon"></span>3a32187</a></td><td><code>Roll back accidental change from dc46ea2dfa97be79d7ef8ddbe9130d4c717acc3d that broke build of Python 2 on Cygwin.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=8a621af7a5f3e10d1337c07b42d04edee6a8f1b8"><span class="icon"></span>8a621af</a></td><td><code>Updated the python2/3 pacakges to use staged install.</code>
</td></tr></table>
TicketgitWed, 31 May 2017 13:13:19 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:7
https://trac.sagemath.org/ticket/22509#comment:7
<ul>
<li><strong>commit</strong>
changed from <em>8a621af7a5f3e10d1337c07b42d04edee6a8f1b8</em> to <em>1391089b36f43b45fa0eb7bf04e83fc1e351dfea</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=1391089b36f43b45fa0eb7bf04e83fc1e351dfea"><span class="icon"></span>1391089</a></td><td><code>Remove the '-v' from the 'cp' command.</code>
</td></tr></table>
TicketgitWed, 31 May 2017 13:34:09 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:8
https://trac.sagemath.org/ticket/22509#comment:8
<ul>
<li><strong>commit</strong>
changed from <em>1391089b36f43b45fa0eb7bf04e83fc1e351dfea</em> to <em>e2941230aea00481d6cf1f58992faa006a7b63f2</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=e2941230aea00481d6cf1f58992faa006a7b63f2"><span class="icon"></span>e294123</a></td><td><code>Add staged install support for atlas</code>
</td></tr></table>
TicketembrayWed, 31 May 2017 13:58:36 GMTdescription changed
https://trac.sagemath.org/ticket/22509#comment:9
https://trac.sagemath.org/ticket/22509#comment:9
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=9">diff</a>)
</li>
</ul>
TicketembrayWed, 31 May 2017 13:59:17 GMTdescription changed
https://trac.sagemath.org/ticket/22509#comment:10
https://trac.sagemath.org/ticket/22509#comment:10
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=10">diff</a>)
</li>
</ul>
TicketgitThu, 01 Jun 2017 08:15:46 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:11
https://trac.sagemath.org/ticket/22509#comment:11
<ul>
<li><strong>commit</strong>
changed from <em>e2941230aea00481d6cf1f58992faa006a7b63f2</em> to <em>00f6de8d173a25b865c26c6af3f588b49e8401eb</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=00f6de8d173a25b865c26c6af3f588b49e8401eb"><span class="icon"></span>00f6de8</a></td><td><code>This file should be deleted from the temp install dir during staged install on OSX</code>
</td></tr></table>
TicketgitFri, 02 Jun 2017 07:36:10 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:12
https://trac.sagemath.org/ticket/22509#comment:12
<ul>
<li><strong>commit</strong>
changed from <em>00f6de8d173a25b865c26c6af3f588b49e8401eb</em> to <em>48e9ffbf5fe86d3186a505ae1e9ab1b13c8195f2</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=16bc700b5cb5cb54697702154f37f1121238fb15"><span class="icon"></span>16bc700</a></td><td><code>Update freetype and pkgconf to use DESTDIR</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=eb660c7c262195f4d8910f3e8e6a1953b24e3c91"><span class="icon"></span>eb660c7</a></td><td><code>Updated various packages with non-standard installers or that are otherwise</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=8de16d2f311650cb4308b11935735350f586d54c"><span class="icon"></span>8de16d2</a></td><td><code>Update cephes to work with staged install.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=73c45e6c71706a012ec3e2209da63ca6efb4176f"><span class="icon"></span>73c45e6</a></td><td><code>Update openblas to use staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=e1ad6d5ff7b7b5e11a8de9316c2dbb4971036d65"><span class="icon"></span>e1ad6d5</a></td><td><code>Update lcalc to support staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f07e0568957fd46d4ada5f9b6b18fc04de1767b1"><span class="icon"></span>f07e056</a></td><td><code>Update zn_poly, which doesn't really have a proper installer, to use staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=9997a37138448b82eef16b34645c902ed93e03c9"><span class="icon"></span>9997a37</a></td><td><code>Updated the python2/3 pacakges to use staged install.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f9cfa6f2e806c6d809f3429429e8f05273616297"><span class="icon"></span>f9cfa6f</a></td><td><code>Remove the '-v' from the 'cp' command.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=e51f578ed9c1a31e24461ec935e043204aa0d65d"><span class="icon"></span>e51f578</a></td><td><code>Add staged install support for atlas</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=48e9ffbf5fe86d3186a505ae1e9ab1b13c8195f2"><span class="icon"></span>48e9ffb</a></td><td><code>This file should be deleted from the temp install dir during staged install on OSX</code>
</td></tr></table>
TicketjdemeyerTue, 06 Jun 2017 19:46:06 GMTstatus, milestone changed
https://trac.sagemath.org/ticket/22509#comment:13
https://trac.sagemath.org/ticket/22509#comment:13
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage-wishlist</em> to <em>sage-8.1</em>
</li>
</ul>
<p>
This already fails on the first package... <code>patch</code>:
</p>
<pre class="wiki">Making install in src
make[5]: Entering directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/src'
make[6]: Entering directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/src'
/bin/mkdir -p '/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/inst/opt/sage/sage-git/local/bin'
/usr/bin/install -c patch '/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/inst/opt/sage/sage-git/local/bin'
make[6]: Nothing to be done for `install-data-am'.
make[6]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/src'
make[5]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/src'
Making install in tests
make[5]: Entering directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/tests'
make[6]: Entering directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/tests'
make[6]: Nothing to be done for `install-exec-am'.
make[6]: Nothing to be done for `install-data-am'.
make[6]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/tests'
make[5]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src/tests'
make[5]: Entering directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src'
make[6]: Entering directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src'
make[6]: Nothing to be done for `install-exec-am'.
/bin/mkdir -p '/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/inst/opt/sage/sage-git/local/share/man/man1'
/usr/bin/install -c -m 644 'patch.man' '/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/inst/opt/sage/sage-git/local/share/man/man1/patch.1'
make[6]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src'
make[5]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src'
make[4]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src'
make[3]: Leaving directory `/opt/sage/sage-git/local/var/tmp/sage/build/patch-2.7.5/src'
Cannot find the patch program we just installed
</pre>
TicketjdemeyerTue, 06 Jun 2017 19:47:25 GMT
https://trac.sagemath.org/ticket/22509#comment:14
https://trac.sagemath.org/ticket/22509#comment:14
<p>
I see <code>${SAGE_INST_TEMP}${SAGE_LOCAL}</code> occuring in various places, maybe you should define a new variable for this.
</p>
TicketjdemeyerTue, 06 Jun 2017 19:48:25 GMT
https://trac.sagemath.org/ticket/22509#comment:15
https://trac.sagemath.org/ticket/22509#comment:15
<p>
Let me say that this a very brave patch :-)
</p>
TicketembrayWed, 07 Jun 2017 11:48:45 GMT
https://trac.sagemath.org/ticket/22509#comment:16
https://trac.sagemath.org/ticket/22509#comment:16
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:14" title="Comment 14">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
I see <code>${SAGE_INST_TEMP}${SAGE_LOCAL}</code> occuring in various places, maybe you should define a new variable for this.
</p>
</blockquote>
<p>
Yeah, I keep thinking about that. At first I was just going to redefine <code>$SAGE_LOCAL</code> to that, but that doesn't work because one needs the bare <code>$SAGE_LOCAL</code>, e.g., for the prefix. I think I just need a good name for the combined path. Maybe <code>$SAGE_LOCAL_TEMP</code>?
</p>
TicketembrayWed, 07 Jun 2017 12:01:19 GMT
https://trac.sagemath.org/ticket/22509#comment:17
https://trac.sagemath.org/ticket/22509#comment:17
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:15" title="Comment 15">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Let me say that this a very brave patch :-)
</p>
</blockquote>
<p>
Thanks! I just hope you agree with it in principle, because I really think it should be done. Open to ideas on the specific details, although I think this is already pretty close. There's a lot of things we could be doing better packaging-wise in the first place of course, but I'm still just trying to make the least disruptive changes I can.
</p>
TicketjdemeyerWed, 07 Jun 2017 12:41:08 GMT
https://trac.sagemath.org/ticket/22509#comment:18
https://trac.sagemath.org/ticket/22509#comment:18
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:16" title="Comment 16">embray</a>:
</p>
<blockquote class="citation">
<p>
Maybe <code>$SAGE_LOCAL_TEMP</code>?
</p>
</blockquote>
<p>
I would prefer <code>$SAGE_TEMP_LOCAL</code> simply because that's consistent with the order (first <code>SAGE_INST_TEMP</code>, then <code>SAGE_LOCAL</code>).
</p>
TicketembrayWed, 07 Jun 2017 14:45:56 GMT
https://trac.sagemath.org/ticket/22509#comment:19
https://trac.sagemath.org/ticket/22509#comment:19
<p>
That's fine. But before I do a mass rename on this, perhaps you (and/or others) could have a look at <a class="closed ticket" href="https://trac.sagemath.org/ticket/23160" title="enhancement: Add a library of common helper functions for use in spkg-install (closed: fixed)">#23160</a>. It would be very nice to have this, or something like it, as a dependency on this ticket so that I can make a few bulk changes once (in the case of <a class="closed ticket" href="https://trac.sagemath.org/ticket/23160" title="enhancement: Add a library of common helper functions for use in spkg-install (closed: fixed)">#23160</a> it would be replacing all calls to <code>$MAKE/make install</code> with <code>sdh_make_install</code>, and then many future tweaks can be made in one place--the definition of <code>sdh_make_install</code>).
</p>
TicketgitWed, 07 Jun 2017 14:46:53 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:20
https://trac.sagemath.org/ticket/22509#comment:20
<ul>
<li><strong>commit</strong>
changed from <em>48e9ffbf5fe86d3186a505ae1e9ab1b13c8195f2</em> to <em>7bcbccb7a1b223017eb45d334cb097f365632c48</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=d794f9701e20106fbdd412ceee990a685090d24f"><span class="icon"></span>d794f97</a></td><td><code>Updates the autotools spkg to use staged install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=7bcbccb7a1b223017eb45d334cb097f365632c48"><span class="icon"></span>7bcbccb</a></td><td><code>Fix installation of files containing spaces in their names</code>
</td></tr></table>
TicketgitWed, 07 Jun 2017 14:52:51 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:21
https://trac.sagemath.org/ticket/22509#comment:21
<ul>
<li><strong>commit</strong>
changed from <em>7bcbccb7a1b223017eb45d334cb097f365632c48</em> to <em>9eb5876c6793718ba6225aaba0e07fcf71eab432</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=9eb5876c6793718ba6225aaba0e07fcf71eab432"><span class="icon"></span>9eb5876</a></td><td><code>Remove this sanity check</code>
</td></tr></table>
TicketembrayWed, 07 Jun 2017 14:54:43 GMT
https://trac.sagemath.org/ticket/22509#comment:22
https://trac.sagemath.org/ticket/22509#comment:22
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:13" title="Comment 13">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
This already fails on the first package... <code>patch</code>:
</p>
</blockquote>
<p>
Just a peculiarity of that package's <code>spkg-install</code>. I removed the seemingly pointless check for now.
</p>
TicketgitThu, 08 Jun 2017 07:39:34 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:23
https://trac.sagemath.org/ticket/22509#comment:23
<ul>
<li><strong>commit</strong>
changed from <em>9eb5876c6793718ba6225aaba0e07fcf71eab432</em> to <em>5f7d07967411a4c2ebdf2a837f6427bc210a761e</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=5f7d07967411a4c2ebdf2a837f6427bc210a761e"><span class="icon"></span>5f7d079</a></td><td><code>Fix sagenb installation</code>
</td></tr></table>
TicketgitThu, 08 Jun 2017 08:30:44 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:24
https://trac.sagemath.org/ticket/22509#comment:24
<ul>
<li><strong>commit</strong>
changed from <em>5f7d07967411a4c2ebdf2a837f6427bc210a761e</em> to <em>23799c8b38066712b67354916bba6c265b4f15d2</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=23799c8b38066712b67354916bba6c265b4f15d2"><span class="icon"></span>23799c8</a></td><td><code>Actually fix sagenb install</code>
</td></tr></table>
TicketgitTue, 11 Jul 2017 15:21:54 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:25
https://trac.sagemath.org/ticket/22509#comment:25
<ul>
<li><strong>commit</strong>
changed from <em>23799c8b38066712b67354916bba6c265b4f15d2</em> to <em>84a6827229bbaa3a26154b1202d4f12fe4acad43</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f30f96437555e3e86fba17c028afe8f1fb72542d"><span class="icon"></span>f30f964</a></td><td><code>Remove this sanity check</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=637d13628ebd79c5f9976d2241793efd3380c6e4"><span class="icon"></span>637d136</a></td><td><code>Fix sagenb installation</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=03b50fea279268b525f46cb35961e9e7332d54f6"><span class="icon"></span>03b50fe</a></td><td><code>Actually fix sagenb install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=2095d1d5fa3fbfbc39010fa7caa061d2be1cd77c"><span class="icon"></span>2095d1d</a></td><td><code>Make sure the path these symlinks are installed in exists first.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=48304f2970132eeda72c4e3c7edd1ba7c55095fa"><span class="icon"></span>48304f2</a></td><td><code>Some more enhancements to the sage-dist-helpers:</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=5f0637dbbcdb3dd87c69e803cd694d10ed86e52d"><span class="icon"></span>5f0637d</a></td><td><code>Include the PKG_NAME, PKG_BASE, and PKG_VER variables directly in the install</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=b5633e10f551c453adf785be2614bfcf7faa6852"><span class="icon"></span>b5633e1</a></td><td><code>Now that sdh_make_install implements DESTDIR we can replace all manual</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f41ab626a625b7b544c4a0731df061c35237a365"><span class="icon"></span>f41ab62</a></td><td><code>A few more standard autotools packages that are easily updated but I missed previously</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=06e7339f535ee8fc69454b202aa17be1f466a50a"><span class="icon"></span>06e7339</a></td><td><code>Disable pip installation when installing Python 3; see https://trac.sagemath.org/ticket/23398</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=84a6827229bbaa3a26154b1202d4f12fe4acad43"><span class="icon"></span>84a6827</a></td><td><code>Don't override the valgrind suppression file from python2; add a separate one for Python 3</code>
</td></tr></table>
TicketembrayTue, 11 Jul 2017 15:23:44 GMTdependencies changed
https://trac.sagemath.org/ticket/22509#comment:26
https://trac.sagemath.org/ticket/22509#comment:26
<ul>
<li><strong>dependencies</strong>
changed from <em>#23059 #23096</em> to <em>#23059 #23096 #23160</em>
</li>
</ul>
<p>
Now depends on <a class="closed ticket" href="https://trac.sagemath.org/ticket/23160" title="enhancement: Add a library of common helper functions for use in spkg-install (closed: fixed)">#23160</a>. Updates a bunch of packages to use the helper functions--once that change is made once it will be much easier to make mass updates simply by updating the helper functions.
</p>
TicketgitTue, 11 Jul 2017 15:28:09 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:27
https://trac.sagemath.org/ticket/22509#comment:27
<ul>
<li><strong>commit</strong>
changed from <em>84a6827229bbaa3a26154b1202d4f12fe4acad43</em> to <em>0885622958923d3f9f00725076b6553e10b8de4b</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=0885622958923d3f9f00725076b6553e10b8de4b"><span class="icon"></span>0885622</a></td><td><code>Handle the 2to3 script better so that Python2 and Python3 don't overwrite each other</code>
</td></tr></table>
TicketembrayWed, 12 Jul 2017 11:15:59 GMT
https://trac.sagemath.org/ticket/22509#comment:28
https://trac.sagemath.org/ticket/22509#comment:28
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:18" title="Comment 18">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:16" title="Comment 16">embray</a>:
</p>
<blockquote class="citation">
<p>
Maybe <code>$SAGE_LOCAL_TEMP</code>?
</p>
</blockquote>
<p>
I would prefer <code>$SAGE_TEMP_LOCAL</code> simply because that's consistent with the order (first <code>SAGE_INST_TEMP</code>, then <code>SAGE_LOCAL</code>).
</p>
</blockquote>
<p>
Actually, I think I'm going to rename <code>$SAGE_INST_TEMP</code> to <code>$SAGE_DESTDIR</code> for consistency with the DESTDIR convention. I think that makes it clear what it's for. Then the <code>$SAGE_DESTDIR$SAGE_LOCAL</code> combination can be called <code>$SAGE_DESTDIR_LOCAL</code>.
</p>
TicketgitWed, 12 Jul 2017 16:07:46 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:29
https://trac.sagemath.org/ticket/22509#comment:29
<ul>
<li><strong>commit</strong>
changed from <em>0885622958923d3f9f00725076b6553e10b8de4b</em> to <em>62e916b7cda969e7798fb096672923a2c049fa10</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=57ea8c9df098426c4116d9644260ade6fff42b8f"><span class="icon"></span>57ea8c9</a></td><td><code>Added the ability for packages to have a post-install script called spkg-postinst.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=76703a467e7f4d053066c95dde6a416fb4afa3c7"><span class="icon"></span>76703a4</a></td><td><code>Add special post-install handling for the backports/__init__.py shared</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=be4f47afdf4260100fd30cf38f3317056b86ccef"><span class="icon"></span>be4f47a</a></td><td><code>Remove some no-longer used local variables</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=62e916b7cda969e7798fb096672923a2c049fa10"><span class="icon"></span>62e916b</a></td><td><code>Add handling for packages that install .info files and update share/info/dir</code>
</td></tr></table>
TicketembrayThu, 13 Jul 2017 08:21:33 GMT
https://trac.sagemath.org/ticket/22509#comment:30
https://trac.sagemath.org/ticket/22509#comment:30
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:14" title="Comment 14">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
I see <code>${SAGE_INST_TEMP}${SAGE_LOCAL}</code> occuring in various places, maybe you should define a new variable for this.
</p>
</blockquote>
<p>
Actually on second thought, there was a reason for this: The idea was that if <code>$SAGE_INST_TEMP</code> isn't set then installation straight into <code>$SAGE_LOCAL</code> will still work. The way this works currently is <code>$SAGE_INST_TEMP</code> is set by <code>sage-spkg</code> which handles the details of how to copy from the temporary install path into SAGE_LOCAL, and record the file manifest.
</p>
<p>
But if you're just running a package's spkg-install directly, then <code>$SAGE_INST_TEMP</code> is not set. With <a class="closed ticket" href="https://trac.sagemath.org/ticket/23179" title="enhancement: Automatically wrap spkg-install scripts with boilerplate (closed: fixed)">#23179</a> in place I could make it so that <code>$SAGE_INST_TEMP</code> is always set in the script wrappers. But I'm not necessarily sure that's desirable, since if you're running the script outside of sage-spkg the implementation details of staged installation are not meaningful, and it may be more useful to install directly into SAGE_LOCAL assuming it's for testing purposes. Of course, one can always manually pass in a value for <code>$SAGE_INST_TEMP</code>.
</p>
<p>
So I think it would be better *not* to combine <code>$SAGE_INST_TEMP$SAGE_LOCAL</code> into a single variable, unless others are convinced that there is no use case for installing without <code>$SAGE_INST_TEMP</code> set.
</p>
<p>
(I do still want to rename it <code>$SAGE_DESTDIR</code>--in this way it is used quite analogously to the way <code>$(DESTDIR)</code> is used in Makefiles.)
</p>
TicketembrayThu, 13 Jul 2017 08:24:36 GMT
https://trac.sagemath.org/ticket/22509#comment:31
https://trac.sagemath.org/ticket/22509#comment:31
<p>
If it helps makes things more readable I could use the curly-brace syntax like <code>${SAGE_INST_TEMP}${SAGE_LOCAL}</code> rather then running them directly together.
</p>
TicketgitThu, 13 Jul 2017 08:27:54 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:32
https://trac.sagemath.org/ticket/22509#comment:32
<ul>
<li><strong>commit</strong>
changed from <em>62e916b7cda969e7798fb096672923a2c049fa10</em> to <em>01c05f0c3bdd164d07846c72324e15f897328d6b</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=01c05f0c3bdd164d07846c72324e15f897328d6b"><span class="icon"></span>01c05f0</a></td><td><code>Rename $SAGE_INST_TEMP to $SAGE_DESTDIR</code>
</td></tr></table>
TicketembrayThu, 20 Jul 2017 16:08:33 GMTstatus changed
https://trac.sagemath.org/ticket/22509#comment:33
https://trac.sagemath.org/ticket/22509#comment:33
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
I think that, pending <a class="closed ticket" href="https://trac.sagemath.org/ticket/23160" title="enhancement: Add a library of common helper functions for use in spkg-install (closed: fixed)">#23160</a>, this is basically ready. I've tested it quite extensively myself on Linux and Cygwin and haven't encountered any more obvious problems.
</p>
<p>
The only exception would be that there are still some packages that write files that clobber each other, but they are only cases of optional/experimental packages clobbering standard packages, or optional/experimental packages clobbering each other. These include:
</p>
<ul><li>openblas / atlas
</li><li>mpir / gmp
</li><li>boost_cropped / boost
</li><li>pari_seadata_small / database_pari
</li><li>database_stein_watkins_small / database_stein_watkins
</li><li>gcc / compilerwrapper
</li></ul><p>
However, these incompatibilities were already the status quo. While it will be possible in the future to improve the status quo (<a class="new ticket" href="https://trac.sagemath.org/ticket/23465" title="enhancement: Groundwork for 'abstract' spkgs (new)">#23465</a> is work in that direction), I think these cases can be fixed independently of this ticket. This ticket otherwise fixes all examples among standard (and some optional packages) where files were overwriting each other.
</p>
TicketjhpalmieriThu, 20 Jul 2017 16:54:24 GMTstatus changed
https://trac.sagemath.org/ticket/22509#comment:34
https://trac.sagemath.org/ticket/22509#comment:34
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
On OS X, I get this with the <code>patch</code> package:
</p>
<pre class="wiki">cp: illegal option -- u
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory
cp: illegal option -- u
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory
Successfully installed patch-2.7.5
</pre><p>
A little later the same error causes yasm not to be installed, so mpir won't install:
</p>
<pre class="wiki">[mpir-3.0.0.p0] configure: error: no system-wide yasm found
</pre><p>
In addition to fixing the <code>cp</code> command in <code>sage-spkg</code>, should you also check for errors after executing commands like this so as to avoid "Successfully installed patch-2.7.5" when it didn't actually successfully install?
</p>
TicketjhpalmieriThu, 20 Jul 2017 17:00:10 GMT
https://trac.sagemath.org/ticket/22509#comment:35
https://trac.sagemath.org/ticket/22509#comment:35
<p>
See also <a class="ext-link" href="https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/cp.1.html"><span class="icon"></span>https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/cp.1.html</a> (this page says it is retired, but it agrees with the man page on my Mac) and possibly also <a class="ext-link" href="https://stackoverflow.com/questions/1132508/how-to-emulate-cp-update-behavior-on-mac-os-x"><span class="icon"></span>https://stackoverflow.com/questions/1132508/how-to-emulate-cp-update-behavior-on-mac-os-x</a>.
</p>
TicketembrayThu, 20 Jul 2017 17:01:56 GMT
https://trac.sagemath.org/ticket/22509#comment:36
https://trac.sagemath.org/ticket/22509#comment:36
<p>
Thanks for the quick feedback! I'll have to take a look at the OSX manpage 😞. This is fairly simple functionality to duplicate.
</p>
TicketembrayThu, 20 Jul 2017 17:10:59 GMT
https://trac.sagemath.org/ticket/22509#comment:37
https://trac.sagemath.org/ticket/22509#comment:37
<p>
Honestly, the <code>-u</code> isn't even necessary. In most cases it's going to be superfluous so I'd probably just remove it.
</p>
TicketgitThu, 20 Jul 2017 17:14:56 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:38
https://trac.sagemath.org/ticket/22509#comment:38
<ul>
<li><strong>commit</strong>
changed from <em>01c05f0c3bdd164d07846c72324e15f897328d6b</em> to <em>d9500366634b7eb5b24eea91dbf5fd78a50848e3</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=d9500366634b7eb5b24eea91dbf5fd78a50848e3"><span class="icon"></span>d950036</a></td><td><code>Don't use the -u option to cp. It's not portable to BSD's cp, and is</code>
</td></tr></table>
TicketembrayThu, 20 Jul 2017 17:15:24 GMTstatus changed
https://trac.sagemath.org/ticket/22509#comment:39
https://trac.sagemath.org/ticket/22509#comment:39
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
TicketjhpalmieriThu, 20 Jul 2017 21:06:22 GMTstatus changed
https://trac.sagemath.org/ticket/22509#comment:40
https://trac.sagemath.org/ticket/22509#comment:40
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
<code>ncurses</code> fails to install for me (on OS X):
</p>
<pre class="wiki">Copying package files from temporary location /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.0.rc0/local/var/tmp/sage/build/ncurses-6.0/inst to /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.0.rc0/local
cp: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.0.rc0/local/var/tmp/sage/build/ncurses-6.0/inst/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.0.rc0/local//lib/terminfo is a directory (not copied).
************************************************************************
Error copying files for ncurses.
************************************************************************
</pre>
TicketembrayFri, 21 Jul 2017 08:16:06 GMT
https://trac.sagemath.org/ticket/22509#comment:41
https://trac.sagemath.org/ticket/22509#comment:41
<p>
Do you know the correct way to copy a symbolic link without dereferencing it on OSX? It should be <code>-P</code>, but according to that man page it only respects <code>-P</code> if <code>-R</code> is used also. But this is copying files one at a time. I <em>could</em> use <code>-R</code> but would still have to loop over all the files anyways in order to build the manifest. Also that man page says:
</p>
<pre class="wiki">In -R mode, cp will continue copying even if errors are detected.
</pre><p>
which isn't necessarily what I want either.
</p>
TicketgitFri, 21 Jul 2017 08:21:57 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:42
https://trac.sagemath.org/ticket/22509#comment:42
<ul>
<li><strong>commit</strong>
changed from <em>d9500366634b7eb5b24eea91dbf5fd78a50848e3</em> to <em>d605943e4ee29256ea30180803b96aa6a2d6917c</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=d605943e4ee29256ea30180803b96aa6a2d6917c"><span class="icon"></span>d605943</a></td><td><code>Just use 'cp -a' which seems to be compatible across most cp's.</code>
</td></tr></table>
TicketembrayFri, 21 Jul 2017 08:22:12 GMTstatus changed
https://trac.sagemath.org/ticket/22509#comment:43
https://trac.sagemath.org/ticket/22509#comment:43
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
TicketembrayFri, 21 Jul 2017 12:24:44 GMT
https://trac.sagemath.org/ticket/22509#comment:44
https://trac.sagemath.org/ticket/22509#comment:44
<p>
<del>It occurs to me that the <code>pkgconf</code> package isn't being handled quite correctly. This leads to problems when re-installing / upgrading that package.</del>
</p>
<p>
Nevermind, for the purposes of <em>this</em> ticket it's fine. It's only a problem in <a class="closed ticket" href="https://trac.sagemath.org/ticket/22510" title="enhancement: Enhanced package uninstallation for Sage spkgs (closed: worksforme)">#22510</a>.
</p>
TicketjhpalmieriSat, 22 Jul 2017 05:10:59 GMT
https://trac.sagemath.org/ticket/22509#comment:45
https://trac.sagemath.org/ticket/22509#comment:45
<p>
Now python2 and python3 don't install:
</p>
<pre class="wiki">Installing valgrind suppression file...
./spkg-install: line 140: ./python: is a directory
Testing importing of various modules...
./spkg-install: line 162: ./python: is a directory
ctypes module failed to import
./spkg-install: line 162: ./python: is a directory
math module failed to import
./spkg-install: line 162: ./python: is a directory
hashlib module failed to import
./spkg-install: line 162: ./python: is a directory
crypt module failed to import
./spkg-install: line 162: ./python: is a directory
readline module failed to import
./spkg-install: line 162: ./python: is a directory
socket module failed to import
./spkg-install: line 171: ./python: is a directory
_scproxy module failed to import
Error: One or more modules failed to import.
</pre>
TicketembrayMon, 31 Jul 2017 09:11:58 GMT
https://trac.sagemath.org/ticket/22509#comment:46
https://trac.sagemath.org/ticket/22509#comment:46
<p>
Does OSX have case-insensitivity or something?
</p>
TicketembrayMon, 31 Jul 2017 10:44:31 GMT
https://trac.sagemath.org/ticket/22509#comment:47
https://trac.sagemath.org/ticket/22509#comment:47
<p>
Or in other words, if I have in the same directory an executable called <code>python</code> and a directory called <code>Python</code> (as is the case in the CPython source tree) what will happen if I run <code>./python</code> on OSX?
</p>
TicketembrayMon, 31 Jul 2017 12:21:25 GMT
https://trac.sagemath.org/ticket/22509#comment:48
https://trac.sagemath.org/ticket/22509#comment:48
<p>
Coincidentally, I just discovered that this also fixes a rare race condition that results from the fact that multiple packages write to <code>$SAGE_LOCAL/usr/share/info/dir</code>. If two packages that write to that file are installed in parallel there are probably a number of ways that can go wrong, such as (what just happened to me):
</p>
<p>
A) <code>[ ! -d $(DESTDIR)$info_dir ] ...</code><br />
B) <code>mkdir $(DESTDIR)$info_dir</code><br />
A) <code>... || mkdir $(DESTDIR)$info_dir</code><br />
</p>
<p>
resulting in directory already exists error.
</p>
TicketembrayMon, 31 Jul 2017 12:30:15 GMT
https://trac.sagemath.org/ticket/22509#comment:49
https://trac.sagemath.org/ticket/22509#comment:49
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:46" title="Comment 46">embray</a>:
</p>
<blockquote class="citation">
<p>
Does OSX have case-insensitivity or something?
</p>
</blockquote>
<p>
It turns out Python's <code>./configure</code> explicitly checks this. Does the build log say something like
</p>
<pre class="wiki">checking for case-insensitive build directory...yes
</pre><p>
?
</p>
TicketembrayMon, 31 Jul 2017 12:39:13 GMT
https://trac.sagemath.org/ticket/22509#comment:50
https://trac.sagemath.org/ticket/22509#comment:50
<p>
Indeed, it looks like on <em>any</em> OS (including OSX) if a case-insensitive filesystem is being built on it outputs the Python executable in the build dir as <code>python.exe</code> so we just need to look for that I guess.
</p>
TicketgitMon, 31 Jul 2017 12:45:10 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:51
https://trac.sagemath.org/ticket/22509#comment:51
<ul>
<li><strong>commit</strong>
changed from <em>d605943e4ee29256ea30180803b96aa6a2d6917c</em> to <em>dbdb94788f6275e19c5fd150bec8a889188f3164</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=dbdb94788f6275e19c5fd150bec8a889188f3164"><span class="icon"></span>dbdb947</a></td><td><code>Fix Python build on non-Cygwin systems (namely OSX) with case-insensitive filesystems</code>
</td></tr></table>
TicketembrayMon, 31 Jul 2017 12:45:39 GMT
https://trac.sagemath.org/ticket/22509#comment:52
https://trac.sagemath.org/ticket/22509#comment:52
<p>
Give that a try. I can't imagine there's any other reason for this.
</p>
TicketjhpalmieriMon, 31 Jul 2017 17:24:05 GMT
https://trac.sagemath.org/ticket/22509#comment:53
https://trac.sagemath.org/ticket/22509#comment:53
<p>
Yes, OS X has a type of case-insensitivity. The current branch gets a little farther, but still fails. With python2:
</p>
<pre class="wiki">Installing valgrind suppression file...
Testing importing of various modules...
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/Lib/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ImportError: dlopen(/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_ctypes.so, 2): Symbol not found: _PyUnicodeUCS4_AsEncodedString
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_ctypes.so
Expected in: flat namespace
in /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_ctypes.so
ctypes module failed to import
math module imported OK
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/Lib/hashlib.py", line 158, in <module>
import struct
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/Lib/struct.py", line 1, in <module>
from _struct import *
ImportError: dlopen(/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_struct.so, 2): Symbol not found: _PyUnicodeUCS4_AsEncodedString
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_struct.so
Expected in: flat namespace
in /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_struct.so
hashlib module failed to import
crypt module imported OK
readline module imported OK
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/Lib/socket.py", line 47, in <module>
import _socket
ImportError: dlopen(/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_socket.so, 2): Symbol not found: _PyUnicodeUCS4_AsEncodedString
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_socket.so
Expected in: flat namespace
in /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python2-2.7.13.p1/src/build/lib.macosx-10.9-x86_64-2.7/_socket.so
socket module failed to import
_scproxy module imported OK
Error: One or more modules failed to import.
</pre><p>
It's odd that some of the modules succeeded while others failed.
</p>
<p>
With python3:
</p>
<pre class="wiki">Installing valgrind suppression file...
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
Testing importing of various modules...
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 160: 84992 Abort trap: 6 $PYTHON -c "import $module"
ctypes module failed to import
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 160: 84993 Abort trap: 6 $PYTHON -c "import $module"
math module failed to import
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 160: 84995 Abort trap: 6 $PYTHON -c "import $module"
hashlib module failed to import
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 160: 84997 Abort trap: 6 $PYTHON -c "import $module"
crypt module failed to import
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 160: 84998 Abort trap: 6 $PYTHON -c "import $module"
readline module failed to import
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 160: 85000 Abort trap: 6 $PYTHON -c "import $module"
socket module failed to import
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/lib/libpython3.6m.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/python3-3.6.1/src/./python.exe
Reason: image not found
./spkg-install: line 176: 85001 Abort trap: 6 $PYTHON -c "import _scproxy"
_scproxy module failed to import
Error: One or more modules failed to import.
</pre>
TicketembrayTue, 01 Aug 2017 13:22:52 GMT
https://trac.sagemath.org/ticket/22509#comment:54
https://trac.sagemath.org/ticket/22509#comment:54
<p>
I see. This probably means that some <code>DYLD_LIBRARY_PATH</code> or similar needs to be set in order for Python being run out of the source tree to link with the correct libpython.
</p>
<p>
Thanks for your testing / patience--I need to bother nthiery to get me an OSX machine I can test on so that this will be easier in the future.
</p>
<p>
Other than this Python thing I don't anticipate too many other issues with this. The only reason Python is being stubborn is that I made a somewhat non-trivial change to how the Python self-tests are performed. Very few of the other packages otherwise have significant changes to how they're installed (though a few do).
</p>
TicketjhpalmieriTue, 01 Aug 2017 18:23:08 GMT
https://trac.sagemath.org/ticket/22509#comment:55
https://trac.sagemath.org/ticket/22509#comment:55
<p>
Python has also always been finicky to install on OS X. Anyway, imitating what was there already for linux works on this machine: adding
</p>
<pre class="wiki">if [ "$UNAME" = "Darwin" ]; then
export DYLD_LIBRARY_PATH="."
fi
</pre><p>
right before <code>echo "Testing importing of various modules..."</code>, for example, fixed both <code>python2</code> and <code>python3</code>.
</p>
<p>
Other issues: this one is strange and non-repeatable, but I saw this once with maxima:
</p>
<pre class="wiki">Found local metadata for maxima-5.39.0.p0
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
ImportError: No module named site
************************************************************************
Error downloading maxima-5.39.0.tar.gz
************************************************************************
</pre><p>
In this case, the maxima tarball had already been downloaded to the <code>upstream</code> directory. When I ran "make" again, it went smoothly. I don't know if this is worth investigating.
</p>
<p>
More importantly, I get build errors with <code>gsl</code> and <code>numpy</code>. For <code>gsl</code>:
</p>
<pre class="wiki">checking build system type... x86_64-apple-darwin16.7.0
checking host system type... x86_64-apple-darwin16.7.0
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in `/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta0/local/var/tmp/sage/build/gsl-2.3/src':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
</pre><p>
config.log says
</p>
<pre class="wiki">configure:3480: checking whether the C compiler works
configure:3502: gcc -g -O2 -L/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib conftest.c -lopenblas -lm >&5
configure:3506: $? = 0
configure:3554: result: yes
configure:3557: checking for C compiler default output file name
configure:3559: result: a.out
configure:3565: checking for suffix of executables
configure:3572: gcc -o conftest -g -O2 -L/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib conftest.c -lopenblas -lm >&5
configure:3576: $? = 0
configure:3598: result:
configure:3620: checking whether we are cross compiling
configure:3628: gcc -o conftest -g -O2 -L/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib conftest.c -lopenblas -lm >&5
configure:3632: $? = 0
configure:3639: ./conftest
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/openblas-0.2.19.p0/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/libopenblas_haswell-r0.2.19.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/gsl-2.3/src/./conftest
Reason: image not found
./configure: line 3641: 5891 Abort trap: 6 ./conftest$ac_cv_exeext
configure:3643: $? = 134
configure:3650: error: in `/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/gsl-2.3/src':
configure:3652: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
</pre><p>
I don't know why it seems to be looking for <code>libopenblas_haswell-r0.2.19.dylib</code> in the temporary directory; the command invoking <code>./configure</code> has the correct <code>--prefix</code> and <code>--libdir</code>.
</p>
<p>
For <code>numpy</code>:
</p>
<pre class="wiki">Copying numpy.egg-info to /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/numpy-1.12.1-py2.7.egg-info
running install_scripts
creating /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/bin
copying build/scripts.macosx-10.9-x86_64-2.7/f2py2 -> /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/bin
changing mode of /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/bin/f2py2 to 755
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/numpy/__init__.py", line 142, in <module>
from . import add_newdocs
File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/numpy-1.12.1/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 24, in <module>
raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control). Otherwise reinstall numpy.
</pre>
TicketembrayMon, 07 Aug 2017 08:57:33 GMT
https://trac.sagemath.org/ticket/22509#comment:56
https://trac.sagemath.org/ticket/22509#comment:56
<p>
I don't get any of these problems. I will probably need an OSX machine to investigate.
</p>
<p>
It could have something to do with the
</p>
<pre class="wiki">LIBS="`pkg-config --libs-only-l cblas` -lm"
</pre><p>
passed to <code>configure</code> in gsl's spkg-install. But that *shouldn't* be a problem.
</p>
TicketgitWed, 09 Aug 2017 13:06:59 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:57
https://trac.sagemath.org/ticket/22509#comment:57
<ul>
<li><strong>commit</strong>
changed from <em>dbdb94788f6275e19c5fd150bec8a889188f3164</em> to <em>ce38d0756f226ec9a87b55890e716edd80f0cbdc</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=4360f314520c3394f788d64b3585f391b5c6b8c7"><span class="icon"></span>4360f31</a></td><td><code>Added the ability for packages to have a post-install script called spkg-postinst.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=0972a41e1de2dfe3e9f1e74d5e46dff4df939d82"><span class="icon"></span>0972a41</a></td><td><code>Add special post-install handling for the backports/__init__.py shared</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=41c73a3b12f8096d8c734607a849d9549e029461"><span class="icon"></span>41c73a3</a></td><td><code>Remove some no-longer used local variables</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=750943a34b27fc9d134326bf85d9f0278718effe"><span class="icon"></span>750943a</a></td><td><code>Add handling for packages that install .info files and update share/info/dir</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=53a9c478704380f9cacb9e36c632b67bdee05e33"><span class="icon"></span>53a9c47</a></td><td><code>Rename $SAGE_INST_TEMP to $SAGE_DESTDIR</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=76157141ee63f260a097bc8a4fa46cf79f266322"><span class="icon"></span>7615714</a></td><td><code>Don't use the -u option to cp. It's not portable to BSD's cp, and is</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=d4bb6f7849adf520b557af4038e3c9b2d749e942"><span class="icon"></span>d4bb6f7</a></td><td><code>Just use 'cp -a' which seems to be compatible across most cp's.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=ed8957174bfc4fd7d74a113bb38cf340c9f50714"><span class="icon"></span>ed89571</a></td><td><code>Fix Python build on non-Cygwin systems (namely OSX) with case-insensitive filesystems</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=44228b281303b311abb8026429a6c2327d5a9d8b"><span class="icon"></span>44228b2</a></td><td><code>On OSX DYLD_LIBRARY_PATH should be set a la LD_LIBRARY_PATH to run Python from its build directory</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=ce38d0756f226ec9a87b55890e716edd80f0cbdc"><span class="icon"></span>ce38d07</a></td><td><code>Work around bug in OpenBLAS's Makefile with its use of DESTDIR with</code>
</td></tr></table>
TicketembrayWed, 09 Aug 2017 13:11:54 GMT
https://trac.sagemath.org/ticket/22509#comment:58
https://trac.sagemath.org/ticket/22509#comment:58
<p>
Rebased on the latest (positively reviewed) <a class="closed ticket" href="https://trac.sagemath.org/ticket/23160" title="enhancement: Add a library of common helper functions for use in spkg-install (closed: fixed)">#23160</a>.
</p>
<p>
I added the <code>DYLD_LIBRARY_PATH</code> fix needed for Python installation.
</p>
<p>
Also added an <em>untested</em> possible fix for the gsl installation issue. I think the issue here was a bug in OpenBLAS's Makefile on OSX with how it uses <code>install_name_tool</code>, and this should fix it but I'm not positive. Chances are this would also fix the problem with numpy which is probably trying to link (unsuccessfully) to openblas.
</p>
TicketjhpalmieriWed, 09 Aug 2017 17:30:11 GMT
https://trac.sagemath.org/ticket/22509#comment:59
https://trac.sagemath.org/ticket/22509#comment:59
<p>
This works now, Sage builds and all tests pass.
</p>
TicketjhpalmieriWed, 09 Aug 2017 21:33:35 GMT
https://trac.sagemath.org/ticket/22509#comment:60
https://trac.sagemath.org/ticket/22509#comment:60
<p>
Several packages fail their test suites. flint:
</p>
<pre class="wiki">gcc -fno-common -ansi -pedantic -Wall -O2 -funroll-loops -g -mpopcnt -c -o test_helpers.o test_helpers.c
make build/test/t-add_ssaaaa build/test/t-add_sssaaaaaa build/test/t-count_leading_zeros build/test/t-count_trailing_zeros build/test/t-sdiv_qrnnd build/test/t-smul_ppmm build/test/t-sub_ddmmss build/test/t-udiv_qrnnd build/test/t-udiv_qrnnd_preinv build/test/t-umul_ppmm
mkdir -p build/test
gcc -fno-common -ansi -pedantic -Wall -O2 -funroll-loops -g -mpopcnt -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib test/t-add_ssaaaa.c -o build/test/t-add_ssaaaa -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -lflint -lmpfr -lgmp -lm -lntl -lpthread
[snip]
gcc -fno-common -ansi -pedantic -Wall -O2 -funroll-loops -g -mpopcnt -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib test/t-udiv_qrnnd_preinv.c -o build/test/t-udiv_qrnnd_preinv -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -lflint -lmpfr -lgmp -lm -lntl -lpthread
gcc -fno-common -ansi -pedantic -Wall -O2 -funroll-loops -g -mpopcnt -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -I/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/include -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib test/t-umul_ppmm.c -o build/test/t-umul_ppmm -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib -lflint -lmpfr -lgmp -lm -lntl -lpthread
(BUILD_DIR=build; export BUILD_DIR; make -f Makefile.subdirs -C . check || exit$?;)
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib/libflint-13.5.2.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/flint-2.5.2.p1/src/build/test/t-add_ssaaaa
Reason: image not found
make[4]: *** [build/test/t-add_ssaaaa_RUN] Abort trap: 6
/bin/sh: exit2: command not found
make[3]: *** [check] Error 127
Error: FLINT failed to pass its test suite.
</pre><p>
arb:
</p>
<pre class="wiki">Running the test suite for arb-2.8.1.p1...
make[4]: Nothing to be done for `shared'.
make[5]: Nothing to be done for `shared'.
CC ../build/fmpr/test/t-abs_bound_le_2exp_fmpz
CC ../build/fmpr/test/t-abs_bound_lt_2exp_fmpz
CC ../build/fmpr/test/t-abs_bound_lt_2exp_si
CC ../build/fmpr/test/t-add
CC ../build/fmpr/test/t-add_naive
CC ../build/fmpr/test/t-cmp
CC ../build/fmpr/test/t-cmp_2exp_si
CC ../build/fmpr/test/t-cmpabs
CC ../build/fmpr/test/t-cmpabs_2exp_si
CC ../build/fmpr/test/t-div
CC ../build/fmpr/test/t-divappr_abs_ubound
CC ../build/fmpr/test/t-exp
CC ../build/fmpr/test/t-expm1
CC ../build/fmpr/test/t-get_d
CC ../build/fmpr/test/t-get_fmpz
CC ../build/fmpr/test/t-get_mpfr
CC ../build/fmpr/test/t-log
CC ../build/fmpr/test/t-log1p
CC ../build/fmpr/test/t-mul
CC ../build/fmpr/test/t-mul_fmpz
CC ../build/fmpr/test/t-mul_naive
CC ../build/fmpr/test/t-mul_si
CC ../build/fmpr/test/t-mul_ui
CC ../build/fmpr/test/t-normalise
CC ../build/fmpr/test/t-root
CC ../build/fmpr/test/t-rsqrt
CC ../build/fmpr/test/t-set_fmpq
CC ../build/fmpr/test/t-set_fmpz_2exp
CC ../build/fmpr/test/t-set_round_mpn
CC ../build/fmpr/test/t-set_round_ui_2exp_fmpz
CC ../build/fmpr/test/t-set_round_uiui_2exp_fmpz
CC ../build/fmpr/test/t-sqrt
CC ../build/fmpr/test/t-sub
CC ../build/fmpr/test/t-sum
CC ../build/fmpr/test/t-ulp
dyld: Library not loaded: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib/libarb-1.1.1.dylib
Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/arb-2.8.1.p1/src/fmpr/../build/fmpr/test/t-abs_bound_le_2exp_fmpz
Reason: image not found
make[4]: *** [../build/fmpr/test/t-abs_bound_le_2exp_fmpz_RUN] Abort trap: 6
make[3]: *** [check] Error 2
</pre><p>
R:
</p>
<pre class="wiki">Running the test suite for r-3.4.0.p0...
../../bin/R: line 246: /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib/R//etc/ldpaths: No such file or directory
make[6]: *** [test-Examples-Base] Error 1
make[5]: *** [test-Examples] Error 2
make[4]: *** [test-all-basics] Error 1
make[3]: *** [check] Error 2
Error while running the R testsuite ... exiting
</pre><p>
Singular:
</p>
<pre class="wiki">Running the test suite for singular-4.1.0p3.p0...
./spkg-check: line 33: Singular: command not found
Error loading the freegb library in Singular.
</pre><p>
pycrypto:
</p>
<pre class="wiki">Running the test suite for pycrypto-2.6.1.p2...
PyCrypto will now be tested
running test
Traceback (most recent call last):
File "setup.py", line 456, in <module>
core.setup(**kw)
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 321, in run
from Crypto import SelfTest
ImportError: No module named Crypto
Error: PyCrypto failed to pass its test suite.
</pre><p>
cvxopt:
</p>
<pre class="wiki">Running the test suite for cvxopt-1.1.8.p2...
Testing in /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/cvxopt-1.1.8.p2/src/examples/doc/chap10
Testing l1svc.py ...
Traceback (most recent call last):
File "l1svc.py", line 3, in <module>
from cvxopt import normal, setseed
ImportError: No module named cvxopt
Error: test /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-8.1.beta1/local/var/tmp/sage/build/cvxopt-1.1.8.p2/src/examples/doc/chap10/l1svc.py failed
</pre>
TicketembrayThu, 10 Aug 2017 18:21:49 GMT
https://trac.sagemath.org/ticket/22509#comment:61
https://trac.sagemath.org/ticket/22509#comment:61
<p>
How are you running these? I don't know why it would work any differently.
</p>
TicketjhpalmieriThu, 10 Aug 2017 19:02:16 GMT
https://trac.sagemath.org/ticket/22509#comment:62
https://trac.sagemath.org/ticket/22509#comment:62
<p>
I get failures from <code>make distclean; export SAGE_CHECK=yes; make</code>. I get some (but not all) of these failures from <code>./sage -f -c PKG_NAME</code>; this is after unsetting <code>SAGE_CHECK</code> and running <code>make</code>. But once the package has been installed successfully, then <code>./sage -f -c PKG_NAME</code> may behave differently, because it may be using installed versions of libraries, binaries, etc.; if it hasn't been installed, it won't be installed until <code>spkg-check</code> finishes successfully.
</p>
TicketjdemeyerFri, 11 Aug 2017 11:35:04 GMT
https://trac.sagemath.org/ticket/22509#comment:63
https://trac.sagemath.org/ticket/22509#comment:63
<p>
In <code>build/pkgs/widgetsnbextension/spkg-install</code> you write
</p>
<pre class="wiki"># Note: Setting PYTHONPATH here is something of a hack to support
# installation into $SAGE_DESTDIR. This might be better handled
# as a post-install step
SITE_PACKAGES="$(sage-python23 -c 'import site; print(site.getsitepackages()[0])')"
PYTHONPATH="$SAGE_DESTDIR$SITE_PACKAGES" jupyter nbextension enable --sys-prefix --py widgetsnbextension
</pre><p>
I agree that this looks more like a post-install step, so why didn't you implement it that way?
</p>
TicketjdemeyerFri, 11 Aug 2017 12:00:52 GMT
https://trac.sagemath.org/ticket/22509#comment:64
https://trac.sagemath.org/ticket/22509#comment:64
<p>
I am a bit lost where the <code>DESTDIR</code> stuff for autoconf packages is actually implemented. For Python packages, I see that you added the <code>--root</code> option to <code>PIP_INSTALL</code>.
</p>
TicketjdemeyerFri, 11 Aug 2017 12:03:58 GMT
https://trac.sagemath.org/ticket/22509#comment:65
https://trac.sagemath.org/ticket/22509#comment:65
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:33" title="Comment 33">embray</a>:
</p>
<blockquote class="citation">
<p>
This ticket otherwise fixes all examples among standard (and some optional packages) where files were overwriting each other.
</p>
</blockquote>
<p>
What is the relevance of that for this ticket?
</p>
<p>
All those packages worked before. If such clobbering is no longer allowed, I consider that a regression introduced by this ticket.
</p>
TicketembrayFri, 11 Aug 2017 12:29:18 GMT
https://trac.sagemath.org/ticket/22509#comment:66
https://trac.sagemath.org/ticket/22509#comment:66
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:63" title="Comment 63">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
I agree that this looks more like a post-install step, so why didn't you implement it that way?
</p>
</blockquote>
<p>
When I first wrote that, support for post-install scripts wasn't added yet. Now that it's there, I could probably revisit things like that. Though I'd have to review what exactly <code>jupyter nbextension enable</code> does. If it just writes some files then this might be fine.
</p>
TicketembrayFri, 11 Aug 2017 12:42:09 GMT
https://trac.sagemath.org/ticket/22509#comment:67
https://trac.sagemath.org/ticket/22509#comment:67
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:62" title="Comment 62">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
I get failures from <code>make distclean; export SAGE_CHECK=yes; make</code>. I get some (but not all) of these failures from <code>./sage -f -c PKG_NAME</code>; this is after unsetting <code>SAGE_CHECK</code> and running <code>make</code>. But once the package has been installed successfully, then <code>./sage -f -c PKG_NAME</code> may behave differently, because it may be using installed versions of libraries, binaries, etc.; if it hasn't been installed, it won't be installed until <code>spkg-check</code> finishes successfully.
</p>
</blockquote>
<p>
I'm having trouble building python3 with <code>SAGE_CHECK=yes</code>. Did that ever work in the first place?
</p>
TicketjhpalmieriFri, 11 Aug 2017 15:00:12 GMT
https://trac.sagemath.org/ticket/22509#comment:68
https://trac.sagemath.org/ticket/22509#comment:68
<p>
In my experience, neither <code>python2</code> nor <code>python3</code> passed their self tests. With <code>python2</code> at least, this was true on a range of platforms. We should probably change the default value of <code>SAGE_CHECK_PACKAGES</code> to exclude <code>python3</code>; it already excludes <code>python2</code>.
</p>
<p>
Edit: I created <a class="closed ticket" href="https://trac.sagemath.org/ticket/23612" title="defect: Include !python3 in default value for SAGE_CHECK_PACKAGES (closed: fixed)">#23612</a>.
</p>
TicketembrayMon, 21 Aug 2017 15:14:05 GMT
https://trac.sagemath.org/ticket/22509#comment:69
https://trac.sagemath.org/ticket/22509#comment:69
<p>
On the topic of the other SAGE_CHECK failures, the problem is probably similar to the problems I was having with Python: The checks are being performed before the files are installed from <code>SAGE_DESTDIR</code> into <code>SAGE_LOCAL</code>, so at the very least it might be necessary to set <code>LD_LIBRARY_PATH/DYLIB_LIBRARY_PATH</code>, but possibly other paths as well?
</p>
<p>
Rather than trying to fix every <code>spkg-check</code>, which may be difficult especially if I can't reproduce all the problems, it might be better for the sake of this ticket if I moved execution of <code>spkg-check</code> until after installation to <code>SAGE_LOCAL</code>. This has the downside that the package still gets installed even if the check failed, but that's the existing situation anyways. Once <a class="closed ticket" href="https://trac.sagemath.org/ticket/22510" title="enhancement: Enhanced package uninstallation for Sage spkgs (closed: worksforme)">#22510</a> is done (which depends on this) it will be easier to do things like make a backup of an old package--try installing the new one--and revert to the old one if the installation failed.
</p>
TicketembrayMon, 21 Aug 2017 15:19:02 GMT
https://trac.sagemath.org/ticket/22509#comment:70
https://trac.sagemath.org/ticket/22509#comment:70
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:64" title="Comment 64">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
I am a bit lost where the <code>DESTDIR</code> stuff for autoconf packages is actually implemented. For Python packages, I see that you added the <code>--root</code> option to <code>PIP_INSTALL</code>.
</p>
</blockquote>
<p>
Just realized I never replied to this. <code>DESTDIR=</code> is added in the helper function <code>sdh_make_install</code> (which is partly why I wanted that feature implemented for this ticket). Note that it assumes <code>SAGE_DESTDIR</code> is set, which it may not be if the <code>spkg-install</code> is not being run from <code>sage-spkg</code>. It might be worth noting that this is intentional--if <code>SAGE_DESTDIR</code> is left blank then direct installation to <code>SAGE_LOCAL</code> proceeds as normal, which is important and useful for testing, if nothing else.
</p>
<p>
Otherwise, <code>SAGE_DESTDIR</code> is set in <code>sage-spkg</code>, and gets passed to <code>make install</code>, in most cases, by way of <code>sdh_make_install</code>.
</p>
TicketembrayMon, 21 Aug 2017 17:26:23 GMT
https://trac.sagemath.org/ticket/22509#comment:71
https://trac.sagemath.org/ticket/22509#comment:71
<p>
For some reason on Linux I couldn't reproduce the problems running the tests for flint or arb (despite running <code>make distclean</code> first). But I did get the same problems for the other packages. As I wrote <a class="ext-link" href="https://trac.sagemath.org/ticket/22509#comment:69"><span class="icon"></span>here</a> it seems to be mostly a matter of setting the required environment variables. But I think I'll still revert to the old behavior for SAGE_CHECK for now, just to keep this simpler.
</p>
TicketgitMon, 04 Sep 2017 08:37:57 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:72
https://trac.sagemath.org/ticket/22509#comment:72
<ul>
<li><strong>commit</strong>
changed from <em>ce38d0756f226ec9a87b55890e716edd80f0cbdc</em> to <em>70013431e8c8a4ac4f49191378aea54e006b1a4b</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=ed2446b0cec371fd5fa83389f50e6b719b849467"><span class="icon"></span>ed2446b</a></td><td><code>Move spkg-check to after the package has been installed to SAGE_LOCAL.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=70013431e8c8a4ac4f49191378aea54e006b1a4b"><span class="icon"></span>7001343</a></td><td><code>Enable widgetsnbextension in the notebook's config file as a postinst step.</code>
</td></tr></table>
TicketembrayMon, 04 Sep 2017 08:38:39 GMT
https://trac.sagemath.org/ticket/22509#comment:73
https://trac.sagemath.org/ticket/22509#comment:73
<p>
Just remembered there was still unfinished work on this that I hadn't pushed yet. I think now it's addressed all the issues that have been raised so far.
</p>
TicketjdemeyerMon, 04 Sep 2017 09:31:26 GMT
https://trac.sagemath.org/ticket/22509#comment:74
https://trac.sagemath.org/ticket/22509#comment:74
<p>
One concern with this patch is that it's really a patch bomb, which might be hard to get reviewed and merged.
</p>
<p>
I don't think you have answered <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:65" title="Comment 65">65</a>
</p>
TicketembrayMon, 04 Sep 2017 10:28:02 GMT
https://trac.sagemath.org/ticket/22509#comment:75
https://trac.sagemath.org/ticket/22509#comment:75
<p>
I thought I addressed this somewhere but maybe not...
</p>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:65" title="Comment 65">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:33" title="Comment 33">embray</a>:
</p>
<blockquote class="citation">
<p>
This ticket otherwise fixes all examples among standard (and some optional packages) where files were overwriting each other.
</p>
</blockquote>
<p>
What is the relevance of that for this ticket?
</p>
<p>
All those packages worked before. If such clobbering is no longer allowed, I consider that a regression introduced by this ticket.
</p>
</blockquote>
<p>
I wouldn't consider it a regression at all. The fact that some packages overwrote files installed by other packages is a bug, just one that hasn't had very serious effects. Most cases of this were very minor (e.g. python3 overwrote the valgrind.supp installed by python2).
</p>
<p>
The goal of this ticket is, along with <a class="closed ticket" href="https://trac.sagemath.org/ticket/22510" title="enhancement: Enhanced package uninstallation for Sage spkgs (closed: worksforme)">#22510</a> (where this becomes important) is to make sagedist work a little more like a real OS packaging system in that files are belonged by packages, with any one file belonging to just one package. That way, uninstalling one package does not break another package that owns some of the same file(s).
</p>
<p>
In theory one <em>could</em> design a packaging system where a file can belong to multiple packages, and only be removed once all packages that use it are removed. But this is more complicated, and only works if that file is identical for all packages that use it (it may not be).
</p>
<p>
There is one particular example of a file shared between packages where this ticket <em>would</em> introduce a regression if it didn't carefully handle this case: There's a file <code>info/share/dir</code> that is updated by any package that installs an <code>info</code> manual. The new way packages are installed by ticket is such that each package will write a fresh <code>info/share/dir</code>, and then overwrite the existing one, rather than updating the existing one. This makes sure to always update <code>info/share/dir</code> in a post-install script for those packages.
</p>
<p>
Other than that, the other cases are mostly trivial.
</p>
TicketgitMon, 04 Sep 2017 10:34:37 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:76
https://trac.sagemath.org/ticket/22509#comment:76
<ul>
<li><strong>commit</strong>
changed from <em>70013431e8c8a4ac4f49191378aea54e006b1a4b</em> to <em>53e3dc54db7658c6eceb50d573eeb324d886a532</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=02750584ab2382f1b986e674adc7a98c0ed279bb"><span class="icon"></span>0275058</a></td><td><code>On OSX DYLD_LIBRARY_PATH should be set a la LD_LIBRARY_PATH to run Python from its build directory</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=8e193dba196ccdfe5f5bc37c462a5fa02ac7e910"><span class="icon"></span>8e193db</a></td><td><code>Move spkg-check to after the package has been installed to SAGE_LOCAL.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=53e3dc54db7658c6eceb50d573eeb324d886a532"><span class="icon"></span>53e3dc5</a></td><td><code>Enable widgetsnbextension in the notebook's config file as a postinst step.</code>
</td></tr></table>
TicketembrayMon, 04 Sep 2017 10:35:19 GMT
https://trac.sagemath.org/ticket/22509#comment:77
https://trac.sagemath.org/ticket/22509#comment:77
<p>
Had to rebase after I realized I pop'd the completely wrong stash entry into an earlier commit.
</p>
TicketembrayMon, 04 Sep 2017 10:37:38 GMT
https://trac.sagemath.org/ticket/22509#comment:78
https://trac.sagemath.org/ticket/22509#comment:78
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:74" title="Comment 74">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
One concern with this patch is that it's really a patch bomb, which might be hard to get reviewed and merged.
</p>
</blockquote>
<p>
True, but there's not much other way to handle it (I don't mind separating some changes out to separate tickets but some of them go hand-in-hand with this). John did test this extensively in OSX and got it working. The only remaining concern was that some <code>spkg-check</code> tests weren't working. I believe I've addressed that now, though it's hard to be 100% sure since some of those tests aren't working for me with or without this patch.
</p>
<blockquote class="citation">
<p>
I don't think you have answered <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:65" title="Comment 65">65</a>
</p>
</blockquote>
<p>
Done.
</p>
TicketembrayMon, 04 Sep 2017 17:18:30 GMTdependencies changed
https://trac.sagemath.org/ticket/22509#comment:79
https://trac.sagemath.org/ticket/22509#comment:79
<ul>
<li><strong>dependencies</strong>
changed from <em>#23059 #23096 #23160</em> to <em>#23059 #23096 #23160 #21726</em>
</li>
</ul>
<p>
In an effort to split this up a little bit, I'm moving some of the fixes for the Python2/3 packages to a separate ticket, since it can work independently of this ticket: <a class="closed ticket" href="https://trac.sagemath.org/ticket/23781" title="defect: python2/3: split spkg-install to spkg-build; fix various build issues (closed: fixed)">#23781</a>
</p>
TicketembrayMon, 04 Sep 2017 17:19:24 GMTdependencies changed
https://trac.sagemath.org/ticket/22509#comment:80
https://trac.sagemath.org/ticket/22509#comment:80
<ul>
<li><strong>dependencies</strong>
changed from <em>#23059 #23096 #23160 #21726</em> to <em>#23059 #23096 #23160 #23781</em>
</li>
</ul>
<p>
Wrong ticket.
</p>
TicketgitMon, 04 Sep 2017 18:02:00 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:81
https://trac.sagemath.org/ticket/22509#comment:81
<ul>
<li><strong>commit</strong>
changed from <em>53e3dc54db7658c6eceb50d573eeb324d886a532</em> to <em>f97fdf916d7a58006d1be37c419180c8c8ee1893</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=71cb02d1ae524fa40910ff9eaed5ba03f0ce289f"><span class="icon"></span>71cb02d</a></td><td><code>Added the ability for packages to have a post-install script called spkg-postinst.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=013e6f329bb215f4bd37423b1224cabbaac97ab6"><span class="icon"></span>013e6f3</a></td><td><code>Add special post-install handling for the backports/__init__.py shared</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=b8867259c68a782f235912cff35aa3108dead396"><span class="icon"></span>b886725</a></td><td><code>Remove some no-longer used local variables</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f94b3ec48660ca9a226f90074683dbf672440f1f"><span class="icon"></span>f94b3ec</a></td><td><code>Add handling for packages that install .info files and update share/info/dir</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f4960ea1e510e836c35612a1c0fc1698a7c6ade5"><span class="icon"></span>f4960ea</a></td><td><code>Rename $SAGE_INST_TEMP to $SAGE_DESTDIR</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=36ff6cac7e67438ab46b4efd9bde3955ab11fc5a"><span class="icon"></span>36ff6ca</a></td><td><code>Don't use the -u option to cp. It's not portable to BSD's cp, and is</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=25c4d01bb69035512f218029c1c6665a67704f5d"><span class="icon"></span>25c4d01</a></td><td><code>Just use 'cp -a' which seems to be compatible across most cp's.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=8c9a60f8f37eb1015f8a4cbcf0f15ad43658bba0"><span class="icon"></span>8c9a60f</a></td><td><code>Work around bug in OpenBLAS's Makefile with its use of DESTDIR with</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=1389380f87705b23698db8aea3c4476b845e9e3b"><span class="icon"></span>1389380</a></td><td><code>Move spkg-check to after the package has been installed to SAGE_LOCAL.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=f97fdf916d7a58006d1be37c419180c8c8ee1893"><span class="icon"></span>f97fdf9</a></td><td><code>Enable widgetsnbextension in the notebook's config file as a postinst step.</code>
</td></tr></table>
TicketjdemeyerTue, 05 Sep 2017 13:49:39 GMT
https://trac.sagemath.org/ticket/22509#comment:82
https://trac.sagemath.org/ticket/22509#comment:82
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:78" title="Comment 78">embray</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:74" title="Comment 74">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
One concern with this patch is that it's really a patch bomb, which might be hard to get reviewed and merged.
</p>
</blockquote>
<p>
True, but there's not much other way to handle it
</p>
</blockquote>
<p>
Why do you think that? This doesn't have to be an all-or-nothing patch: you could first implement the general framework and then gradually fix packages.
</p>
<p>
Anyway, if you find somebody else to review this, I don't mind at all. But for me, I am quite discouraged from reviewing such a big patch. And then there is also the potential for merge conflicts if you make so many changes.
</p>
TicketembrayTue, 05 Sep 2017 14:07:14 GMT
https://trac.sagemath.org/ticket/22509#comment:83
https://trac.sagemath.org/ticket/22509#comment:83
<p>
I'm not sure how you would propose breaking this up. It is pretty all-or-nothing: Either DESTDIR installation works or it doesn't.
</p>
<p>
There's no sense in "gradually fix packages" because they've already <em>all</em> been fixed here, and in a way that is likely to result in only minor conflicts at worse (no more so than in <a class="closed ticket" href="https://trac.sagemath.org/ticket/23179" title="enhancement: Automatically wrap spkg-install scripts with boilerplate (closed: fixed)">#23179</a>--the analysis in <a class="ext-link" href="https://trac.sagemath.org/ticket/23179#comment:43"><span class="icon"></span>this comment</a> still mostly applies).
</p>
<p>
I could break it up into multiple tickets--I already did in one case. But for the most part I don't see what that solves. If you want guidance on what to review, start by ignoring all the <code>build/pkgs</code> changes since the vast majority of them are completely mechanistic.
</p>
TicketjdemeyerTue, 05 Sep 2017 14:35:30 GMT
https://trac.sagemath.org/ticket/22509#comment:84
https://trac.sagemath.org/ticket/22509#comment:84
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:83" title="Comment 83">embray</a>:
</p>
<blockquote class="citation">
<p>
I'm not sure how you would propose breaking this up. It is pretty all-or-nothing: Either DESTDIR installation works or it doesn't.
</p>
</blockquote>
<p>
You could start with "<code>DESTDIR</code> works for a few selected packages" or "<code>DESTDIR</code> works for Python packages only"
</p>
<blockquote class="citation">
<p>
in a way that is likely to result in only minor conflicts at worse
</p>
</blockquote>
<p>
A merge conflict is a merge conflict. It doesn't matter how minor it is.
</p>
TicketembrayTue, 05 Sep 2017 15:42:52 GMT
https://trac.sagemath.org/ticket/22509#comment:85
https://trac.sagemath.org/ticket/22509#comment:85
<p>
Alright, that's fair. I'll see if I can break this up a bit more.
</p>
<p>
Fact remains, there's not much benefit to this unless all packages are updated. There's no real logical way I can think of to carving it up into multiple tickets, when it can all be done at once. But for the sake of making review easier I'll try out your suggestion...
</p>
TicketjdemeyerWed, 06 Sep 2017 08:21:37 GMT
https://trac.sagemath.org/ticket/22509#comment:86
https://trac.sagemath.org/ticket/22509#comment:86
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:85" title="Comment 85">embray</a>:
</p>
<blockquote class="citation">
<p>
I'll see if I can break this up a bit more.
</p>
</blockquote>
<p>
Let me mention that this is a personal "pet peeve". If somebody else (John Palmieri for example) is happy with this ticket and wants to review it, I won't object to that.
</p>
TicketjhpalmieriWed, 06 Sep 2017 14:18:54 GMT
https://trac.sagemath.org/ticket/22509#comment:87
https://trac.sagemath.org/ticket/22509#comment:87
<p>
I'm not sure I understand the build process well enough to be competent enough to review it.
</p>
TicketembrayWed, 06 Sep 2017 15:28:15 GMT
https://trac.sagemath.org/ticket/22509#comment:88
https://trac.sagemath.org/ticket/22509#comment:88
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:86" title="Comment 86">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:85" title="Comment 85">embray</a>:
</p>
<blockquote class="citation">
<p>
I'll see if I can break this up a bit more.
</p>
</blockquote>
<p>
Let me mention that this is a personal "pet peeve". If somebody else (John Palmieri for example) is happy with this ticket and wants to review it, I won't object to that.
</p>
</blockquote>
<p>
It's fine--I see your point. I think it's easier to review if one patch contains just:
</p>
<ul><li>The basic structural changes
</li><li>A few motivating examples for the different cases
</li></ul><p>
and then leave the remaining boilerplate changes to a separate ticket.
</p>
TicketjdemeyerWed, 06 Sep 2017 21:29:25 GMT
https://trac.sagemath.org/ticket/22509#comment:89
https://trac.sagemath.org/ticket/22509#comment:89
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:88" title="Comment 88">embray</a>:
</p>
<blockquote class="citation">
<p>
I think it's easier to review if one patch contains just:
</p>
<ul><li>The basic structural changes
</li><li>A few motivating examples for the different cases
</li></ul></blockquote>
<p>
Exactly. But I would prefer to finish <a class="closed ticket" href="https://trac.sagemath.org/ticket/23781" title="defect: python2/3: split spkg-install to spkg-build; fix various build issues (closed: fixed)">#23781</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/21535" title="enhancement: Make src/setup.py respect --build-base and --inplace, independent of ... (closed: fixed)">#21535</a> first.
</p>
TicketembrayThu, 07 Sep 2017 09:48:45 GMT
https://trac.sagemath.org/ticket/22509#comment:90
https://trac.sagemath.org/ticket/22509#comment:90
<p>
I'm not exactly sure what <a class="closed ticket" href="https://trac.sagemath.org/ticket/21535" title="enhancement: Make src/setup.py respect --build-base and --inplace, independent of ... (closed: fixed)">#21535</a> has to do with this.
</p>
TicketjdemeyerThu, 07 Sep 2017 10:18:05 GMT
https://trac.sagemath.org/ticket/22509#comment:91
https://trac.sagemath.org/ticket/22509#comment:91
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:90" title="Comment 90">embray</a>:
</p>
<blockquote class="citation">
<p>
I'm not exactly sure what <a class="closed ticket" href="https://trac.sagemath.org/ticket/21535" title="enhancement: Make src/setup.py respect --build-base and --inplace, independent of ... (closed: fixed)">#21535</a> has to do with this.
</p>
</blockquote>
<p>
It's unrelated. I just mentioned it because I can only keep track of a limited number of build-system tickets in my brain cache.
</p>
TicketgitWed, 11 Oct 2017 15:07:54 GMTcommit changed
https://trac.sagemath.org/ticket/22509#comment:92
https://trac.sagemath.org/ticket/22509#comment:92
<ul>
<li><strong>commit</strong>
changed from <em>f97fdf916d7a58006d1be37c419180c8c8ee1893</em> to <em>c1d5453c2735cdda78d53dc36b56fb54ec7dd336</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=be70ddb7efa5935b58bf31b5bebfb4f472b928ec"><span class="icon"></span>be70ddb</a></td><td><code>Added the ability for packages to have a post-install script called spkg-postinst.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=e246937533ebea75a42e56898d29a7c2b3cdb4b0"><span class="icon"></span>e246937</a></td><td><code>Add special post-install handling for the backports/__init__.py shared</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=87a02ac25b1b17cf09a44edcd2046f913e449164"><span class="icon"></span>87a02ac</a></td><td><code>Remove some no-longer used local variables</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=592731866e491d27f38ad65492af07f85cff2e23"><span class="icon"></span>5927318</a></td><td><code>Add handling for packages that install .info files and update share/info/dir</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=73245de422500f2649b856ee5c44837586278d46"><span class="icon"></span>73245de</a></td><td><code>Rename $SAGE_INST_TEMP to $SAGE_DESTDIR</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=a771cd891ed1478f5359f4a6611c6914fcd11f59"><span class="icon"></span>a771cd8</a></td><td><code>Don't use the -u option to cp. It's not portable to BSD's cp, and is</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=512504710450294645f1a65c67b5f6df660a534a"><span class="icon"></span>5125047</a></td><td><code>Just use 'cp -a' which seems to be compatible across most cp's.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=3e8e7b5868e4b8ff55616ba761523e974a68d608"><span class="icon"></span>3e8e7b5</a></td><td><code>Work around bug in OpenBLAS's Makefile with its use of DESTDIR with</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=58676b476bfcd1e2c636c3b97afb44ba1eb52683"><span class="icon"></span>58676b4</a></td><td><code>Move spkg-check to after the package has been installed to SAGE_LOCAL.</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit/?id=c1d5453c2735cdda78d53dc36b56fb54ec7dd336"><span class="icon"></span>c1d5453</a></td><td><code>Enable widgetsnbextension in the notebook's config file as a postinst step.</code>
</td></tr></table>
TicketembrayWed, 11 Oct 2017 15:09:29 GMT
https://trac.sagemath.org/ticket/22509#comment:93
https://trac.sagemath.org/ticket/22509#comment:93
<p>
Went ahead and rebased on current <a class="closed ticket" href="https://trac.sagemath.org/ticket/23781" title="defect: python2/3: split spkg-install to spkg-build; fix various build issues (closed: fixed)">#23781</a>. But I'm now I'm going to do some surgery to break this into some smaller tickets for which this ticket will serve as the mothership.
</p>
TicketembrayWed, 11 Oct 2017 15:14:45 GMTstatus, description changed
https://trac.sagemath.org/ticket/22509#comment:94
https://trac.sagemath.org/ticket/22509#comment:94
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=94">diff</a>)
</li>
</ul>
TicketjdemeyerFri, 20 Oct 2017 14:50:01 GMT
https://trac.sagemath.org/ticket/22509#comment:95
https://trac.sagemath.org/ticket/22509#comment:95
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:93" title="Comment 93">embray</a>:
</p>
<blockquote class="citation">
<p>
Went ahead and rebased on current <a class="closed ticket" href="https://trac.sagemath.org/ticket/23781" title="defect: python2/3: split spkg-install to spkg-build; fix various build issues (closed: fixed)">#23781</a>.
</p>
</blockquote>
<p>
This doesn't need to depend on <a class="closed ticket" href="https://trac.sagemath.org/ticket/23781" title="defect: python2/3: split spkg-install to spkg-build; fix various build issues (closed: fixed)">#23781</a>. Just leave the Python build script alone.
</p>
TicketembrayMon, 23 Oct 2017 08:47:00 GMT
https://trac.sagemath.org/ticket/22509#comment:96
https://trac.sagemath.org/ticket/22509#comment:96
<p>
Uhh, it does need to in a manner of speaking. Eventually the Python build script needs to be updated to support this. But I'm already working on completely breaking this up in to separate tickets.
</p>
TicketembrayWed, 25 Oct 2017 15:04:35 GMTdescription changed
https://trac.sagemath.org/ticket/22509#comment:97
https://trac.sagemath.org/ticket/22509#comment:97
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=97">diff</a>)
</li>
</ul>
<p>
Added <a class="closed ticket" href="https://trac.sagemath.org/ticket/24106" title="enhancement: Install packages in temporary root before copying to $SAGE_LOCAL ... (closed: fixed)">#24106</a>--a simplified ticket demonstrating the core functionality of the branch on this ticket. Will add followup tickets with additional details.
</p>
TicketembrayTue, 12 Dec 2017 08:27:24 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:98
https://trac.sagemath.org/ticket/22509#comment:98
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.1</em> to <em>sage-8.2</em>
</li>
</ul>
TicketembrayFri, 02 Feb 2018 13:04:40 GMTdescription changed
https://trac.sagemath.org/ticket/22509#comment:99
https://trac.sagemath.org/ticket/22509#comment:99
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=99">diff</a>)
</li>
</ul>
TicketembrayFri, 02 Feb 2018 13:51:05 GMTdescription changed
https://trac.sagemath.org/ticket/22509#comment:100
https://trac.sagemath.org/ticket/22509#comment:100
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/22509?action=diff&version=100">diff</a>)
</li>
</ul>
TicketjdemeyerFri, 02 Feb 2018 15:20:12 GMT
https://trac.sagemath.org/ticket/22509#comment:101
https://trac.sagemath.org/ticket/22509#comment:101
<p>
I am surprised now to see that <a class="needs_review ticket" href="https://trac.sagemath.org/ticket/22509" title="enhancement: Install packages in temporary root before copying to $SAGE_LOCAL (needs_review)">#22509</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/24106" title="enhancement: Install packages in temporary root before copying to $SAGE_LOCAL ... (closed: fixed)">#24106</a> are distinct tickets. Isn't this one a "duplicate"? There is a stale branch here, is that still relevant?
</p>
TicketembrayFri, 02 Feb 2018 15:41:45 GMT
https://trac.sagemath.org/ticket/22509#comment:102
https://trac.sagemath.org/ticket/22509#comment:102
<p>
see <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:93" title="Comment 93">93</a>
</p>
TicketjdemeyerMon, 19 Mar 2018 08:27:37 GMT
https://trac.sagemath.org/ticket/22509#comment:103
https://trac.sagemath.org/ticket/22509#comment:103
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/22509#comment:28" title="Comment 28">embray</a>:
</p>
<blockquote class="citation">
<p>
Then the <code>$SAGE_DESTDIR$SAGE_LOCAL</code> combination can be called <code>$SAGE_DESTDIR_LOCAL</code>.
</p>
</blockquote>
<p>
I don't remember if there was a reason not to do this, but I'll introduce this variable in <a class="closed ticket" href="https://trac.sagemath.org/ticket/25001" title="defect: gfortran breaks parallel build (closed: fixed)">#25001</a>.
</p>
TicketembrayMon, 19 Mar 2018 08:41:48 GMT
https://trac.sagemath.org/ticket/22509#comment:104
https://trac.sagemath.org/ticket/22509#comment:104
<p>
Yes, there was a reason: <a class="ext-link" href="https://trac.sagemath.org/ticket/22509#comment:30"><span class="icon"></span>https://trac.sagemath.org/ticket/22509#comment:30</a>
</p>
<p>
Though I suppose if <code>$SAGE_DESTDIR</code> isn't set then it's fine if <code>$SAGE_DESTDIR_LOCAL == $SAGE_LOCAL</code>.
</p>
TicketembrayThu, 26 Apr 2018 12:13:55 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:105
https://trac.sagemath.org/ticket/22509#comment:105
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.2</em> to <em>sage-8.3</em>
</li>
</ul>
TicketembrayWed, 18 Jul 2018 11:47:03 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:106
https://trac.sagemath.org/ticket/22509#comment:106
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.3</em> to <em>sage-8.4</em>
</li>
</ul>
<p>
I believe this issue can reasonably be addressed for Sage 8.4.
</p>
TicketembraySun, 28 Oct 2018 11:41:42 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:107
https://trac.sagemath.org/ticket/22509#comment:107
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.4</em> to <em>sage-8.5</em>
</li>
</ul>
TicketembrayFri, 28 Dec 2018 14:10:15 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:108
https://trac.sagemath.org/ticket/22509#comment:108
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.5</em> to <em>sage-8.7</em>
</li>
</ul>
<p>
Retargeting some of my tickets (somewhat optimistically for now).
</p>
TicketembrayMon, 25 Mar 2019 10:43:18 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:109
https://trac.sagemath.org/ticket/22509#comment:109
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.7</em> to <em>sage-8.8</em>
</li>
</ul>
<p>
Moving all my in-progress tickets to 8.8 milestone.
</p>
TicketembrayFri, 14 Jun 2019 14:50:27 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:110
https://trac.sagemath.org/ticket/22509#comment:110
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.8</em> to <em>sage-8.9</em>
</li>
</ul>
<p>
Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).
</p>
TicketembrayMon, 30 Dec 2019 14:48:17 GMTmilestone changed
https://trac.sagemath.org/ticket/22509#comment:111
https://trac.sagemath.org/ticket/22509#comment:111
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.9</em> to <em>sage-9.1</em>
</li>
</ul>
<p>
Ticket retargeted after milestone closed
</p>
TicketmkoeppeSun, 09 Feb 2020 22:34:08 GMTstatus, milestone changed
https://trac.sagemath.org/ticket/22509#comment:112
https://trac.sagemath.org/ticket/22509#comment:112
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage-9.1</em> to <em>sage-duplicate/invalid/wontfix</em>
</li>
</ul>
<p>
I suggest to close this ticket because its goals have been achieved and the remaining tasks seem to be covered by <a class="needs_work ticket" href="https://trac.sagemath.org/ticket/24024" title="task: Meta-ticket: Update other packages to using the sage-dist-helpers ... (needs_work)">#24024</a>.
</p>
Ticket