Sage: Ticket #21591: Task ticket: Replace use of SAGE_ROOT by more specific environment variables
https://trac.sagemath.org/ticket/21591
<p>
<a class="closed ticket" href="https://trac.sagemath.org/ticket/21501" title="enhancement: Allow SAGE_LOCAL to be customized (closed: fixed)">#21501</a>, <a class="new ticket" href="https://trac.sagemath.org/ticket/21534" title="defect: Allow SAGE_LOCAL to be customized - follow-up (new)">#21534</a> replace $SAGE_ROOT/local by $SAGE_LOCAL (to make SAGE_LOCAL customizable via <code>configure --prefix</code>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/21479" title="enhancement: ./configure --prefix=SAGE_LOCAL (closed: fixed)">#21479</a>).
</p>
<p>
But there are also other direct uses of <code>$SAGE_ROOT</code> that should be replaced by something more specific, in particular in <code>src/</code>.
</p>
<p>
This is preparation for <a class="needs_work ticket" href="https://trac.sagemath.org/ticket/21469" title="enhancement: Enable VPATH builds (several independent build trees connected to one ... (needs_work)">#21469</a> (Enable VPATH builds) and <a class="new ticket" href="https://trac.sagemath.org/ticket/21507" title="task: Meta-ticket: Make sagelib a pip-installable Python source package, ... (new)">#21507</a> (Make sagelib a pip-installable Python source package, listed on PyPI).
</p>
<p>
See also <a class="closed ticket" href="https://trac.sagemath.org/ticket/15105" title="enhancement: hardwired paths in src/sage (closed: fixed)">#15105</a> - hardwired paths in <code>src/sage</code>.
</p>
<p>
The following list needs review.
</p>
<pre class="wiki">find . -name local -prune -o -name _build -prune -o -name src/build -prune -o -name src/sage-7.4beta4 -prune -o -name .git -prune -o -type f -exec grep -nH -e SAGE_ROOT {} +
nstall'"
...
./build/pkgs/autotools/spkg-src:5:if [ -z "$SAGE_ROOT" ]; then
...
./build/pkgs/autotools/spkg-write-makefile:13:if [ -z "$SAGE_ROOT" ]; then
...
./build/pkgs/configure/SPKG.txt:25:$SAGE_ROOT/bootstrap -s or the $SAGE_ROOT/src/bin/sage-update-version
./build/pkgs/d3js/spkg-src:5:[ -n "${SAGE_ROOT}" ] || SAGE_ROOT="$(pwd)/../../../"
./build/pkgs/d3js/spkg-src:33:mv "d3js-${ZIP_VERSION}.tar.gz" "${SAGE_ROOT}/upstream/"
./build/pkgs/database_gap/spkg-check:5:SMALL_GROUPS=`echo "SmallGroup(13,1); quit;" | $SAGE_ROOT/sage -gap -b -T | grep "13"`
./build/pkgs/database_stein_watkins_mini/spkg-install:4:. "$SAGE_ROOT/build/pkgs/database_stein_watkins/spkg-install"
./build/pkgs/ecl/spkg-install:77:cp "$SAGE_ROOT"/config/config.* src
./build/pkgs/ecl/spkg-src:16:if [ -z "$SAGE_ROOT" ] ; then
...
./build/pkgs/ecm/spkg-install:26:cp "$SAGE_ROOT"/config/config.* .
...
./build/pkgs/fflas_ffpack/spkg-install:23:cp "$SAGE_ROOT"/config/config.* build-aux
...
./build/pkgs/gap/spkg-install:55:cp "$SAGE_ROOT"/config/config.* cnf
./build/pkgs/gap3/spkg-src:19:if [ -z "$SAGE_ROOT" ] ; then
...
./build/pkgs/gcc/spkg-install:143:touch "$SAGE_ROOT/configure"
./build/pkgs/gf2x/spkg-install:23:cp "$SAGE_ROOT"/config/config.* config
./build/pkgs/givaro/spkg-install:38:cp "$SAGE_ROOT"/config/config.* build-aux
./build/pkgs/glpk/spkg-install:24:cp "$SAGE_ROOT"/config/config.* .
./build/pkgs/gmp/spkg-install:72: # In some cases (see SAGE_ROOT/spkg/bin/sage-env), on Darwin,
./build/pkgs/gsl/SPKG.txt:99: $SAGE_ROOT/spkg/standard/deps, and an explanation why GSL does *not*
./build/pkgs/jmol/spkg-install:2:if [ "$SAGE_ROOT" = "" ]; then
...
./build/pkgs/libfplll/spkg-src:11:SPKG_ROOT="$SAGE_ROOT/build/pkgs/libfplll"
./build/pkgs/linbox/spkg-install:25:cp "$SAGE_ROOT"/config/config.* build-aux
./build/pkgs/lrcalc/spkg-install:21:cp "$SAGE_ROOT"/config/config.* .
./build/pkgs/m4ri/SPKG.txt:42: just use the version in the PATH (i.e., in SAGE_ROOT/spkg/bin)
./build/pkgs/m4rie/SPKG.txt:42: $SAGE_ROOT/spkg/bin (in the $PATH).
./build/pkgs/mathjax/spkg-src:5:[ -n "${SAGE_ROOT}" ] || SAGE_ROOT="$(pwd)/../../../"
./build/pkgs/matplotlib/spkg-src:6:[ -n "${SAGE_ROOT}" ] || SAGE_ROOT="$(readlink -f $(dirname ${0})/../../../)"
...
./build/pkgs/maxima/spkg-install:44:cp "$SAGE_ROOT"/config/config.* .
./build/pkgs/mpfi/spkg-install:24:cp "$SAGE_ROOT"/config/config.* .
./build/pkgs/mpir/spkg-install:106: # In some cases (see SAGE_ROOT/spkg/bin/sage-env), on Darwin,
./build/pkgs/ntl/spkg-src:11:SPKG_ROOT="$SAGE_ROOT/build/pkgs/ntl"
./build/pkgs/numpy/spkg-install:72:os.chdir(os.environ["SAGE_ROOT"]) # Import numpy from safe location
./build/pkgs/planarity/spkg-install:21:cp "$SAGE_ROOT"/config/config.* .
./build/pkgs/pycrypto/SPKG.txt:29: * config.patch: let pycrypto build when $SAGE_ROOT is in $PATH
./build/pkgs/python_igraph/spkg-src:3:# This script must be run from folder SAGE_ROOT. It modifies the file
./build/pkgs/python_igraph/spkg-src:8:if [ -z "$SAGE_ROOT" -o -z "$SAGE_DISTFILES" ]; then
...
./build/pkgs/qhull/patches/Makefile.patch:11:+DESTDIR = $(SAGE_ROOT)/local
</pre><p>
(The above line is <a class="closed ticket" href="https://trac.sagemath.org/ticket/21572" title="enhancement: Simplify qhull spkg patch (closed: fixed)">#21572</a>.)
</p>
<pre class="wiki">./build/pkgs/r/spkg-src:11:mv $SOURCE_TARBALL $SAGE_ROOT/upstream/$TARGET_TARBALL
./build/pkgs/sage_mode/spkg-install:56: # Trick the shell into expanding $INSTALL_DIR and $SAGE_ROOT
./build/pkgs/sage_mode/SPKG.txt:50:(setq sage-command "$SAGE_ROOT/sage")
./build/pkgs/sagetex/SPKG.txt:46:The directory `$SAGE_ROOT/local/share/doc/sagetex` contains
./build/pkgs/sagetex/SPKG.txt:48:`$SAGE_ROOT/local/share/texmf/tex/latex/sagetex` for the source code and
./build/pkgs/scipoptsuite/SPKG.txt:36:scipoptsuite-VERSION.tgz in $SAGE_ROOT/upstream
./build/pkgs/sqlite/spkg-install:24:cp "$SAGE_ROOT"/config/config.* .
./build/pkgs/symmetrica/SPKG.txt:85: * make sure SAGE_ROOT is defined
./build/pkgs/threejs/spkg-src:8:[ -n "${SAGE_ROOT}" ] || SAGE_ROOT="$(pwd)/../../../"
...
./build/pkgs/zeromq/spkg-install:21:cp "$SAGE_ROOT"/config/config.* config
./build/sage_bootstrap/creator.py:21:from sage_bootstrap.env import SAGE_ROOT
./build/sage_bootstrap/creator.py:29: self.path = os.path.join(SAGE_ROOT, 'build', 'pkgs', package_name)
./build/sage_bootstrap/env.py:8:* ``SAGE_ROOT``
...
./build/sage_bootstrap/package.py:22:from sage_bootstrap.env import SAGE_ROOT, SAGE_DISTFILES
...
./build/sage_bootstrap/updater.py:22:from sage_bootstrap.env import SAGE_ROOT
./build/test/test_package_cmdline.py:147: env=dict(SAGE_ROOT=tmp)
./build/tox.ini:9: SAGE_ROOT = {toxinidir}/..
./configure.ac:86:SAGE_ROOT=`pwd -P`
...
./Makefile:42: export SAGE_ROOT=$$(pwd) && \
...
./README.md:215: $SAGE_ROOT/local/share/doc/sage/html/
...
./sage:20:# Set SAGE_ROOT to the location of the sage install, i.e. the directory
....
./src/bin/sage-download-upstream:3:for pkg in $SAGE_ROOT/build/pkgs/*
./src/bin/sage-env:22:# from the SAGE_ROOT directory.
...
./src/bin/sage-fix-pkg-checksums:7:# $SAGE_ROOT/upstream are checksummed.
./src/bin/sage-fixdoctests:131: from sage.env import SAGE_ROOT
...
./src/bin/sage-list-packages:15:if "SAGE_ROOT" not in os.environ:
...
./src/bin/sage-location:7:SAGE_ROOT = os.path.realpath(os.environ['SAGE_ROOT'])
...
./src/bin/sage-notebook:13:from sage.env import SAGE_ROOT
./src/bin/sage-README-osx.txt:89:preferences. You can also drag a sage folder (SAGE_ROOT) onto
...
./src/bin/sage-rebase.bat:5:rem Rebase all dlls in the SAGE_ROOT directory (and its subdirectories),
...
./src/bin/sage-rebase.sh:6:# Rebase all dlls in the SAGE_ROOT directory (and its subdirectories),
...
./src/bin/sage-rebaseall.bat:6:rem Rebase all dlls in the SAGE_ROOT directory (and its subdirectories),
...
./src/bin/sage-rebaseall.sh:8:# Rebase all dlls in the SAGE_ROOT directory (and its subdirectories)
...
./src/bin/sage-rsyncdist:67:# Run this script from SAGE_ROOT
...
./src/bin/sage-sdist:26:if [ -z "$SAGE_ROOT" ]; then
...
./src/bin/sage-starts:3:# Run this test from SAGE_ROOT.
...
./src/bin/sage-sws2rst:151:file in $SAGE_ROOT/src/doc to include your file, or
./src/bin/sage-update-version:24:if [ -z "$SAGE_ROOT" ]; then
...
./src/bin/sage-upgrade:3:# This script must be run from $SAGE_ROOT and without having sourced
./src/bin/testcc.sh:68:if [ -z "$SAGE_ROOT" ]; then
...
./src/bin/testcflags.sh:79:if [ -z "$SAGE_ROOT" ]; then
...
./src/bin/testcxx.sh:68:if [ -z "$SAGE_ROOT" ]; then
...
./src/doc/common/themes/sage/static/thebe-sage.js:9: * installed in ${SAGE_ROOT}/local/share/thebe/thebe.js by Sage's thebe package.
./src/doc/.... (many)
...
./src/mac-app/AppController.m:427: // If the file is a directory, see if it's SAGE_ROOT
./src/mac-app/Makefile:16:ifdef SAGE_ROOT
...
./src/sage/all.py:76:from sage.env import SAGE_ROOT, SAGE_SRC, SAGE_DOC_SRC, SAGE_LOCAL, DOT_SAGE, SAGE_ENV
./src/sage/calculus/desolvers.py:1623: os.path.join('$SAGE_ROOT','local','lib','libTIDES.a') + ' $LDFLAGS '
...
./src/sage/doctest/control.py:635: from sage.env import SAGE_SRC, SAGE_DOC_SRC, SAGE_ROOT
...
./src/sage/env.py:92:_add_variable_or_fallback('SAGE_ROOT', None)
...
./src/sage/misc/citation.pyx:4:from sage.env import SAGE_ROOT
./src/sage/misc/citation.pyx:88: strings = [a[0].replace(SAGE_ROOT, "") + " " + a[2] for a in stats.stats.keys()]
./src/sage/misc/copying.py:7:from sage.env import SAGE_ROOT
./src/sage/misc/copying.py:17: return open(os.path.join(SAGE_ROOT,'COPYING.txt')).read()
</pre><p>
The above is <a class="new ticket" href="https://trac.sagemath.org/ticket/21571" title="enhancement: Install COPYING.txt (new)">#21571</a>.
</p>
<pre class="wiki">./src/sage/misc/cython_c.pyx:29: ``$SAGE_ROOT`` is expanded, but other environment variables
./src/sage/misc/edit_module.py:292: 'SAGE_ROOT/src'.
./src/sage/misc/inline_fortran.py:70: sage: os.chdir(SAGE_ROOT)
./src/sage/misc/inline_fortran.py:75: sage: os.getcwd() == SAGE_ROOT
./src/sage/misc/misc.py:111: sage: sage_executable = os.path.join(SAGE_ROOT, 'sage')
./src/sage/misc/package.py:46:from sage.env import SAGE_ROOT
./src/sage/misc/package.py:62:SAGE_PKGS = os.path.join(SAGE_ROOT, "build", "pkgs")
./src/sage/misc/package.py:366: package has a directory in ``SAGE_ROOT/build/pkgs/``, then
./src/sage/misc/sage_ostools.py:27: sage: have_program('sage', path=SAGE_ROOT)
./src/sage/misc/sage_ostools.py:29: sage: have_program('ls', path=SAGE_ROOT)
./src/sage/repl/ipython_kernel/install.py:13: SAGE_ROOT, SAGE_DOC, SAGE_LOCAL, SAGE_EXTCODE,
./src/sage/repl/ipython_kernel/install.py:154: os.path.join(SAGE_ROOT, 'sage'),
./src/sage/structure/sage_object.pyx:1376: that and doctest Sage, you'll find that the ``SAGE_ROOT``/tmp/
./src/sage/structure/sage_object.pyx:1386: ``SAGE_ROOT/tmp/pickle_jar``
./src/sage/structure/sage_object.pyx:1416: dir = os.environ['SAGE_ROOT'] + '/tmp/pickle_jar/'
./src/sage/structure/sage_object.pyx:1532: copying the files from ``SAGE_ROOT/tmp/pickle_jar*`` into the standard pickle
</pre>en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/21591
Trac 1.1.6mkoeppeSat, 24 Sep 2016 23:03:47 GMTdescription changed
https://trac.sagemath.org/ticket/21591#comment:1
https://trac.sagemath.org/ticket/21591#comment:1
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21591?action=diff&version=1">diff</a>)
</li>
</ul>
TicketembrayMon, 26 Sep 2016 12:07:14 GMT
https://trac.sagemath.org/ticket/21591#comment:2
https://trac.sagemath.org/ticket/21591#comment:2
<p>
Thanks--yeah I've encountered a ton of this. Likewise there are misuses of <code>$SAGE_LOCAL</code> scattered about when there are other, more specific variables that could be used. I mostly fix them when I see them, but haven't considered doing a comprehensive cleanup.
</p>
TicketmkoeppeTue, 27 Sep 2016 22:26:57 GMTcc, type, description, summary changed
https://trac.sagemath.org/ticket/21591#comment:3
https://trac.sagemath.org/ticket/21591#comment:3
<ul>
<li><strong>cc</strong>
<em>jhpalmieri</em> <em>tscrim</em> added
</li>
<li><strong>type</strong>
changed from <em>enhancement</em> to <em>task</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/21591?action=diff&version=3">diff</a>)
</li>
<li><strong>summary</strong>
changed from <em>Replace use of SAGE_ROOT by more specific environment variables</em> to <em>Task ticket: Replace use of SAGE_ROOT by more specific environment variables</em>
</li>
</ul>
TicketembrayWed, 28 Sep 2016 06:25:49 GMT
https://trac.sagemath.org/ticket/21591#comment:4
https://trac.sagemath.org/ticket/21591#comment:4
<p>
Has anyone done any work on this yet? I'm in the mood for a (somewhat) mindless and tedious task.
</p>
TicketfbisseyWed, 28 Sep 2016 06:33:30 GMT
https://trac.sagemath.org/ticket/21591#comment:5
https://trac.sagemath.org/ticket/21591#comment:5
<p>
Be my guest! I historically reduced the use of <code>SAGE_ROOT</code> in the build system and at runtime. Most of the usage in <code>src/sage</code>, with the exception of <code>env.py</code> is replaced or removed (<code>package.py</code>) one way or another in sage-on-gentoo.
</p>
TicketembrayWed, 28 Sep 2016 06:44:05 GMT
https://trac.sagemath.org/ticket/21591#comment:6
https://trac.sagemath.org/ticket/21591#comment:6
<p>
fbissey: Will improvements on this front then cut down on the amount of contortions sage-on-gentoo needs to go through?
</p>
TicketfbisseyWed, 28 Sep 2016 06:57:40 GMT
https://trac.sagemath.org/ticket/21591#comment:7
https://trac.sagemath.org/ticket/21591#comment:7
<p>
If done right, yes. Although looking at the list of files there are a variety of issues. I would move <code>package.py</code> completely to <code>sage_setup</code> so it is never called at runtime. I also moved <code>repl/ipython_kernel/install.py</code> to <code>sage_setup</code> as similarly you should only use it at build/install time. I have touched the issue with Jeroen before but he wanted to wait to see if the way to install jupyter kernel would get "formalised".
</p>
<p>
In a lot of other instances <code>SAGE_ROOT</code> is replaced by <code>SAGE_LIB</code>.
</p>
TicketmkoeppeThu, 29 Sep 2016 07:09:38 GMT
https://trac.sagemath.org/ticket/21591#comment:8
https://trac.sagemath.org/ticket/21591#comment:8
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/21591#comment:4" title="Comment 4">embray</a>:
</p>
<blockquote class="citation">
<p>
Has anyone done any work on this yet? I'm in the mood for a (somewhat) mindless and tedious task.
</p>
</blockquote>
<p>
Please go ahead!
</p>
TicketembrayThu, 29 Sep 2016 12:55:18 GMT
https://trac.sagemath.org/ticket/21591#comment:9
https://trac.sagemath.org/ticket/21591#comment:9
<p>
It looks like <a class="closed ticket" href="https://trac.sagemath.org/ticket/15105" title="enhancement: hardwired paths in src/sage (closed: fixed)">#15105</a> got a lot of this done already. I'll see if I can help there first.
</p>
<p>
Or more specifically, the work should probably be done on top of <a class="closed ticket" href="https://trac.sagemath.org/ticket/15105" title="enhancement: hardwired paths in src/sage (closed: fixed)">#15105</a> so as not to conflict with it.
</p>
TicketembrayThu, 29 Sep 2016 14:06:56 GMT
https://trac.sagemath.org/ticket/21591#comment:10
https://trac.sagemath.org/ticket/21591#comment:10
<p>
For <code>picklejar</code>, I know it's mostly a utility for testing, but <code>SAGE_ROOT</code>/tmp hardly seems the appropriate place for it by default.
</p>
<p>
I would think instead the current working directory is appropriate in general--for running the tests I would just put it in <code>SAGE_TMP</code>.
</p>
TicketembrayThu, 29 Sep 2016 14:29:36 GMTdependencies set
https://trac.sagemath.org/ticket/21591#comment:11
https://trac.sagemath.org/ticket/21591#comment:11
<ul>
<li><strong>dependencies</strong>
set to <em>#21571</em>
</li>
</ul>
<p>
I think <a class="new ticket" href="https://trac.sagemath.org/ticket/21571" title="enhancement: Install COPYING.txt (new)">#21571</a> should be considered a dependency, since it's definitely a piece of this puzzle.
</p>
TicketmkoeppeThu, 29 Sep 2016 18:12:55 GMTdependencies, description changed
https://trac.sagemath.org/ticket/21591#comment:12
https://trac.sagemath.org/ticket/21591#comment:12
<ul>
<li><strong>dependencies</strong>
changed from <em>#21571</em> to <em>#21571, #21572, #15105</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/21591?action=diff&version=12">diff</a>)
</li>
</ul>
TicketmkoeppeWed, 30 May 2018 17:31:26 GMT
https://trac.sagemath.org/ticket/21591#comment:13
https://trac.sagemath.org/ticket/21591#comment:13
<p>
See also: <a class="closed ticket" href="https://trac.sagemath.org/ticket/25150" title="defect: Eliminate dependence on VERSION.txt within Sage (closed: fixed)">#25150</a>
</p>
Ticket