Sage: Ticket #21707: Meta-ticket: Split sage-env into 5 to clean up sage configuration
https://trac.sagemath.org/ticket/21707
<p>
<code>src/bin/sage-env</code> (with generated configuration in <code>src/bin/sage-env-config</code>; installed in <code>$SAGE_LOCAL/bin</code>) is used in all of the following contexts:
</p>
<ol><li>Sage-the-distribution while building spkgs,
</li><li>Sage-the-distribution for building sagelib,
</li><li>for environment variables needed to start Sage,
</li><li>as variables by Sage at runtime (through <code>sage.env</code>), and
</li><li>for environment variables affecting subprocesses of Python invoked by sagelib modules.
</li></ol><p>
This ticket proposes to split the configuration according to these 5 contexts. This will make the installation of Sage more modular and flexible.
</p>
<p>
1./2. Create <code>build/bin/sage-build-env</code>(with generated configuration in <code>build/bin/sage-build-env-config</code>) for the build-time environment variables for sage-the-distribution (spkg and sagelib). It is not installed in $SAGE_LOCAL.
</p>
<ol class="loweralpha" start="2"><li>In <code>build/make/Makefile.in</code> and the scripts generated by <code>build/bin/sage-spkg</code>, source <code>build/bin/sage-build-env</code> in addition to <code>src/bin/sage-env</code>. This is <a class="closed ticket" href="https://trac.sagemath.org/ticket/29052" title="enhancement: Create build/bin/sage-build-env-config and call it from ... (closed: fixed)">#29052</a>.
</li><li>Gradually, we will move settings from <code>src/bin/sage-env[-config]</code> to <code>build/bin/sage-build-env[-config]</code> that are known to be only needed for 1./2. (Note that not all compiler-related environment variables can be moved exclusively to 1./2. – some need to be added to 5 as well for some some calls to compilers needed at runtime to support things like <code>%cython</code> (?) and <code>sage.calculus.desolvers.desolve_mintides</code>.) As well as users' use of <code>pip</code> to install additional Python packages!
</li><li>Eventually, we will remove the call to <code>src/bin/sage-env</code> from the build scripts.
</li></ol><p>
2./3. Obtain sagelib's build-time configuration such as <code>src/setup.py</code>'s <code>library_dirs</code> and runtime information (<code>sage.env</code>) from <code>sage-config</code> (introduced in <a class="closed ticket" href="https://trac.sagemath.org/ticket/29038" title="enhancement: Python package sage_conf: Provides optional configuration information ... (closed: fixed)">#29038</a>).
</p>
<ol class="loweralpha"><li>In <code>src/Makefile</code>, poison the <code>SAGE_LOCAL</code> variable.
<ul><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/29411" title="enhancement: make sagelib a script package (closed: fixed)">#29411</a>: make sagelib a script package
</li><li><a class="new ticket" href="https://trac.sagemath.org/ticket/29779" title="enhancement: pkg-config installed from SPKG pkgconf should not depend on ... (new)">#29779</a>: <code>pkg-config</code> installed from SPKG <code>pkgconf</code> should not depend on environment variable <code>SAGE_LOCAL</code>
</li></ul></li><li>Pass <code>SAGE_PKGS</code> (if still needed at all - see <a class="new ticket" href="https://trac.sagemath.org/ticket/28815" title="enhancement: src/setup.py: Replace use of is_package_installed_and_updated by ... (new)">#28815</a> or <a class="new ticket" href="https://trac.sagemath.org/ticket/29705" title="task: Meta-ticket: Modularize sagelib into separate distributions ... (new)">#29705</a>) via <code>sage-config</code> instead of from <code>src/Makefile</code>.
</li></ol><p>
3./4./5. Using <code>sage_conf</code> (Python module and script <code>sage-config</code>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/29038" title="enhancement: Python package sage_conf: Provides optional configuration information ... (closed: fixed)">#29038</a>), make <code>sage.all</code> fully functional when imported from a Python, without setting any environment variables (<code>sage-env</code>).
</p>
<ol class="loweralpha"><li>Initially, phase out <code>src/bin/sage-env-config</code> by using <code>sage-config</code> instead.
<ul><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/29384" title="enhancement: Clean up src/bin/sage-env-config.in: Move logic to src/bin/sage-env, ... (closed: fixed)">#29384</a>: Clean up <code>src/bin/sage-env-config.in</code>: Move logic to <code>src/bin/sage-env</code>, move non-environment configuration variables to <code>sage_conf.py</code>
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/29825" title="enhancement: Clean-up for src/bin/sage-env, move src/bin/sage-clone-source, ... (closed: fixed)">#29825</a>: Clean-up for src/bin/sage-env
</li></ul></li><li>Set <code>sage.env</code> variables via the Python module <code>sage_conf</code> instead of relying on information from environment variables. Example: <code>CYSIGNALS_CRASH_DAYS</code>
</li><li>Set environment variables that are needed only by subprocesses invoked by sagelib in the environment of these subprocesses, rather than relying on them begin set in <code>sage-env</code>. This environment could be provided by a variable <code>sage_conf.SAGE_SUBPROCESS_ENV</code> and/or more invidual variables. Example: <code>R_PROFILE</code>
<ul><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/30563" title="enhancement: Use configuration variable MAXIMA instead of hardcoding "maxima" (closed: fixed)">#30563</a>: Use configuration variable <code>MAXIMA</code> instead of hardcoding "maxima"
</li><li><a class="new ticket" href="https://trac.sagemath.org/ticket/30818" title="enhancement: Meta-ticket: Set environment for subprocesses invoked by Sage (new)">#30818</a> Set environment for subprocesses invoked by Sage
</li></ul></li><li>Finally, remove use of <code>sage-env</code> for all purposes except for <code>sage -sh</code>.
</li></ol><p>
<strong>Aspects of downstream sage packaging:</strong>
</p>
<ul><li>The goal is to enable downstream sage packagers to use a whole unmodified <code>src</code> directory to build and install sagelib.
<ul><li>more precisely, to use the pip-installable source package created by <code>build/pkgs/sagelib/spkg-src</code> (<a class="closed ticket" href="https://trac.sagemath.org/ticket/29950" title="enhancement: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add ... (closed: fixed)">#29950</a>); see also <a class="new ticket" href="https://trac.sagemath.org/ticket/30036" title="enhancement: sage-conf_pypi: API for incremental builds (new)">#30036</a> (pip-installable sage)
</li></ul></li><li>Downstream packagers could provide their own "implementation" (one Python module <a class="ext-link" href="https://git.sagemath.org/sage.git/tree/build/pkgs/sage_conf/src/sage_conf.py.in?id=1c5d011b5f05a954dd2ea810635c9d59cc48e7b5"><span class="icon"></span>sage_conf.py</a>) of the configuration module defined by <a class="closed ticket" href="https://trac.sagemath.org/ticket/29038" title="enhancement: Python package sage_conf: Provides optional configuration information ... (closed: fixed)">#29038</a>, instead of patching sources. They would not necessarily use the reference implementation in <code>build/pkgs/sage_conf</code>, which is intended for use with sage-the-distribution.
</li></ul><p>
<strong>Binary packages:</strong>
</p>
<ul><li><a class="new ticket" href="https://trac.sagemath.org/ticket/31417" title="enhancement: version of package sage_conf for relocatable binary distributions (new)">#31417</a> version of package <code>sage_conf</code> for relocatable binary distributions
</li></ul><p>
<strong>pip-installable Sage</strong>
</p>
<ul><li><a class="positive_review ticket" href="https://trac.sagemath.org/ticket/29039" title="enhancement: pip-installable version of package sage_conf - installs non-Python ... (positive_review)">#29039</a> pip-installable version of package <code>sage_conf</code> - installs non-Python bits of the Sage distribution in <code>~/.sage/</code>
</li><li><a class="needs_review ticket" href="https://trac.sagemath.org/ticket/31396" title="enhancement: relocatable wheel version of package sage_conf (needs_review)">#31396</a> relocatable wheel version of package <code>sage_conf</code>
</li></ul><p>
<strong>More related tickets:</strong>
</p>
<ul><li>cleaning of <code>src/bin</code> as described in <a class="closed ticket" href="https://trac.sagemath.org/ticket/21570" title="enhancement: Move non-scripts of src/bin/ elsewhere (and also move their install ... (closed: invalid)">#21570</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/21559" title="enhancement: Install src/bin scripts by sagelib's setup.py, not make (closed: fixed)">#21559</a>.
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/25486" title="enhancement: Discover SAGE_SCRIPTS_DIR to make $SAGE_LOCAL/bin/sage work from any ... (closed: fixed)">#25486</a>: Discover <code>SAGE_SCRIPTS_DIR</code> to make <code>$SAGE_LOCAL/bin/sage</code> work from any directory, in an environment without <code>SAGE_*</code> variables - this adds a <code>SAGE_ROOT</code> variable to <code>sage-env-config</code>
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/30724" title="enhancement: src/bin/sage-env, src/bin/sage-env-config.in: Remove PYTHON_FOR_VENV (closed: fixed)">#30724</a>: <code>src/bin/sage-env</code>, <code>src/bin/sage-env-config.in</code>: Remove PYTHON_FOR_VENV
</li><li><a class="new ticket" href="https://trac.sagemath.org/ticket/29133" title="task: META-META-TICKET: Build, packaging, testing improvements (new)">#29133</a> META-META-TICKET: Build, packaging, testing improvements
</li></ul>en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/21707
Trac 1.1.6fbisseySat, 15 Oct 2016 08:54:42 GMT
https://trac.sagemath.org/ticket/21707#comment:1
https://trac.sagemath.org/ticket/21707#comment:1
<p>
One comment. Currently <code>sage_setup</code> is installed because it needs to be installed to be doctested - like the rest of sage. But nothing in <code>sage_setup</code> is really needed at runtime and distro can choose not to install it. I don't in sage-on-gentoo. I tend to push for thing not needed at runtime to be moved in there. And that's where I would have put <code>sage-build-env</code> but I am not going to fight it going into <code>build/bin</code> although I'd like everything used by <code>setup.py</code> to be neatly under <code>src</code>.
</p>
TicketmkoeppeSat, 15 Oct 2016 18:53:56 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:2
https://trac.sagemath.org/ticket/21707#comment:2
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=2">diff</a>)
</li>
</ul>
<p>
Thanks - I've updated the description
</p>
TicketmkoeppeSat, 15 Oct 2016 18:54:23 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:3
https://trac.sagemath.org/ticket/21707#comment:3
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=3">diff</a>)
</li>
</ul>
TicketembrayMon, 17 Oct 2016 09:25:04 GMT
https://trac.sagemath.org/ticket/21707#comment:4
https://trac.sagemath.org/ticket/21707#comment:4
<p>
I agree.
</p>
<p>
Consider a slight alternative: Rewrite sage-env as a Python script. This would give it much more flexibility and probably more legibility, including some command-line options for different environments (<code>--build</code>, especially).
</p>
<p>
The output would of course be the shell commands to actually set the variables. So rather than <code>source sage-env</code> one would <code>eval $(sage-env)</code>. My one concern is that it would be slower, but ideally it's something that shouldn't happen frequently anyways, but rather only when not already in sage-env.
</p>
TicketfbisseyMon, 17 Oct 2016 09:30:20 GMT
https://trac.sagemath.org/ticket/21707#comment:5
https://trac.sagemath.org/ticket/21707#comment:5
<p>
I have a tendency to mix up <code>sage-env</code> the script and <code>sage.env.py</code>, my comment is really an example of that. However a lot is common between the two.
</p>
TicketembrayMon, 17 Oct 2016 09:39:48 GMT
https://trac.sagemath.org/ticket/21707#comment:6
https://trac.sagemath.org/ticket/21707#comment:6
<p>
Right--they could be one in the same.
</p>
TicketfbisseyMon, 17 Oct 2016 09:46:17 GMT
https://trac.sagemath.org/ticket/21707#comment:7
https://trac.sagemath.org/ticket/21707#comment:7
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/21707#comment:6" title="Comment 6">embray</a>:
</p>
<blockquote class="citation">
<p>
Right--they could be one in the same.
</p>
</blockquote>
<blockquote>
<p>
I have thought about it often. Of course that really means that you'll have to removing all the build time stuff. But in sage-on-gentoo where I ship a very simplified sage-env (in /etc as well) they are virtually doing the same thing, one in shell, the other in python.
</p>
</blockquote>
TicketembrayMon, 17 Oct 2016 09:55:32 GMT
https://trac.sagemath.org/ticket/21707#comment:8
https://trac.sagemath.org/ticket/21707#comment:8
<p>
In either case--whether making a single script, or two separate shell scripts, this would be useful for gentoo (and probably other distros as well) then right?
</p>
TicketfbisseyMon, 17 Oct 2016 09:59:20 GMT
https://trac.sagemath.org/ticket/21707#comment:9
https://trac.sagemath.org/ticket/21707#comment:9
<p>
Probably, the current <code>sage-env</code> is a forest mostly dedicated to the build system. I just discard it and bring my own, it takes less space and maintenance than a mega patch.
</p>
TicketmkoeppeMon, 17 Oct 2016 14:34:14 GMT
https://trac.sagemath.org/ticket/21707#comment:10
https://trac.sagemath.org/ticket/21707#comment:10
<p>
Ideally, the configuration for sagelib runtime should take place in a generated Python file and not involve environment variables at all.
</p>
TicketembrayMon, 17 Oct 2016 14:40:26 GMT
https://trac.sagemath.org/ticket/21707#comment:11
https://trac.sagemath.org/ticket/21707#comment:11
<p>
More likely parts of it would be static, and other parts could be generated, but yes, what I meant above in terms of rewriting <code>sage-env</code> in Python was that it would not (by default) involve environment variables at all.
</p>
<p>
But it's still useful for interacting with other tools, especially for build purposes, as well as dropping into <code>./sage -sh</code> to be able to set environment variables, which is why I suggest a mode to output a list of variables that can be read from the shell.
</p>
TicketmkoeppeMon, 20 May 2019 00:29:22 GMTbranch set
https://trac.sagemath.org/ticket/21707#comment:12
https://trac.sagemath.org/ticket/21707#comment:12
<ul>
<li><strong>branch</strong>
set to <em>u/mkoeppe/split_sage_env_into_sage_build_env_and_sage_env</em>
</li>
</ul>
TicketmkoeppeMon, 20 May 2019 00:31:50 GMTstatus, cc, milestone changed; author, dependencies, commit set
https://trac.sagemath.org/ticket/21707#comment:13
https://trac.sagemath.org/ticket/21707#comment:13
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
<li><strong>author</strong>
set to <em>Matthias Koeppe</em>
</li>
<li><strong>cc</strong>
<em>dimpase</em> added
</li>
<li><strong>dependencies</strong>
set to <em>#25130</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage-7.5</em> to <em>sage-8.8</em>
</li>
<li><strong>commit</strong>
set to <em>9189d609fa8cc90d616cb7ea05be405ce196295e</em>
</li>
</ul>
<p>
New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=7405dd5b05259028e4dadc981b5bd309a124fbfb"><span class="icon"></span>7405dd5</a></td><td><code>Move sage-dist-helpers from src/bin to build/bin</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=9189d609fa8cc90d616cb7ea05be405ce196295e"><span class="icon"></span>9189d60</a></td><td><code>Split out build/bin/sage-build-env-config from sage-env-config</code>
</td></tr></table>
TicketembrayTue, 21 May 2019 15:19:50 GMT
https://trac.sagemath.org/ticket/21707#comment:14
https://trac.sagemath.org/ticket/21707#comment:14
<p>
As-is this is going to conflict with tickets like <a class="closed ticket" href="https://trac.sagemath.org/ticket/27825" title="enhancement: spkg-configure.m4 for libgd (closed: fixed)">#27825</a> that are adding still more environment variables needed at build-time to sage-env-config.
</p>
<p>
I'm completely in favor of this separation in principle though. The question is whether we should do this first and then update tickets like <a class="closed ticket" href="https://trac.sagemath.org/ticket/27825" title="enhancement: spkg-configure.m4 for libgd (closed: fixed)">#27825</a> on top of it, or the other way around?
</p>
TicketdimpaseTue, 21 May 2019 15:25:29 GMT
https://trac.sagemath.org/ticket/21707#comment:15
https://trac.sagemath.org/ticket/21707#comment:15
<p>
I really would like to first get our existing tickets (<a class="closed ticket" href="https://trac.sagemath.org/ticket/27825" title="enhancement: spkg-configure.m4 for libgd (closed: fixed)">#27825</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/27822" title="task: Update configure with checks for mpfr, mpc, and ntl (closed: fixed)">#27822</a>) touching sage-env-config in.
</p>
TicketmkoeppeWed, 22 May 2019 20:34:34 GMT
https://trac.sagemath.org/ticket/21707#comment:16
https://trac.sagemath.org/ticket/21707#comment:16
<p>
The tickets would be trivial to rebase on top of this one.
</p>
TicketmkoeppeThu, 23 May 2019 15:08:52 GMT
https://trac.sagemath.org/ticket/21707#comment:17
https://trac.sagemath.org/ticket/21707#comment:17
<p>
Probably one should introduce a command <code>sage --buildsh</code>, which would provide the larger environment.
</p>
TicketembrayMon, 01 Jul 2019 12:44:57 GMT
https://trac.sagemath.org/ticket/21707#comment:18
https://trac.sagemath.org/ticket/21707#comment:18
<p>
For the sake of a bit more terseness I wonder if we could call it just <code>sage-env-build.in</code> and <code>sage-env-build</code>. The only reason for the <code>-config</code> in the other one is to distinguish it from plain <code>sage-env</code>.
</p>
<p>
Now that I look I'm not even entirely sure we need to separate <code>sage-env-config</code> from <code>sage-env</code>.
</p>
<p>
I think we could probably have a <code>sage-env.in</code> produce <code>sage-env</code> and get rid of the separate <code>sage-env-config</code> entirely, though it's possible there's some subtlety to that that I'm missing.
</p>
TicketembrayMon, 01 Jul 2019 12:57:45 GMTcommit, branch changed
https://trac.sagemath.org/ticket/21707#comment:19
https://trac.sagemath.org/ticket/21707#comment:19
<ul>
<li><strong>commit</strong>
changed from <em>9189d609fa8cc90d616cb7ea05be405ce196295e</em> to <em>ff0712125a3d05500a1f77e62c371d695d1a9794</em>
</li>
<li><strong>branch</strong>
changed from <em>u/mkoeppe/split_sage_env_into_sage_build_env_and_sage_env</em> to <em>public/split_sage_env_into_sage_build_env_and_sage_env</em>
</li>
</ul>
<p>
Went ahead and rebased on current develop so it could incorporate all the additional variables we've since added.
</p>
<p>
I didn't change the name of the script yet; I would still prefer to shorten it but if anyone disagrees we can leave it as-is.
</p>
<hr />
<p>
New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=a34a9f1c60297944ef119faafe65342cb9cc6ca2"><span class="icon"></span>a34a9f1</a></td><td><code>Move sage-dist-helpers from src/bin to build/bin</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=ff0712125a3d05500a1f77e62c371d695d1a9794"><span class="icon"></span>ff07121</a></td><td><code>Split out build/bin/sage-build-env-config from sage-env-config</code>
</td></tr></table>
TicketfbisseyMon, 01 Jul 2019 20:45:04 GMT
https://trac.sagemath.org/ticket/21707#comment:20
https://trac.sagemath.org/ticket/21707#comment:20
<p>
The only subtility to <code>sage-env-config</code> in my mind, is that you can touch it or even regenerate it without touching <code>sage-env</code>. But I am sure you don't need a separate file to achieve your objectives in either case.
</p>
TicketembrayTue, 02 Jul 2019 12:02:33 GMT
https://trac.sagemath.org/ticket/21707#comment:21
https://trac.sagemath.org/ticket/21707#comment:21
<p>
No, probably not. I might try it, though we'll leave that as a separate exercise from this ticket.
</p>
TicketembrayWed, 03 Jul 2019 11:37:56 GMTmilestone changed
https://trac.sagemath.org/ticket/21707#comment:22
https://trac.sagemath.org/ticket/21707#comment:22
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.8</em> to <em>sage-8.9</em>
</li>
</ul>
<p>
Moving tickets from the Sage 8.8 milestone that have been actively worked on in the last six months to the next release milestone (optimistically).
</p>
TicketembrayMon, 30 Dec 2019 14:48:17 GMTmilestone changed
https://trac.sagemath.org/ticket/21707#comment:23
https://trac.sagemath.org/ticket/21707#comment:23
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.9</em> to <em>sage-9.1</em>
</li>
</ul>
<p>
Ticket retargeted after milestone closed
</p>
TicketmkoeppeWed, 15 Jan 2020 22:55:43 GMTstatus changed
https://trac.sagemath.org/ticket/21707#comment:24
https://trac.sagemath.org/ticket/21707#comment:24
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
TicketmkoeppeWed, 15 Jan 2020 23:36:44 GMT
https://trac.sagemath.org/ticket/21707#comment:25
https://trac.sagemath.org/ticket/21707#comment:25
<p>
The run-time environment variables of sage could as well be configured by a Python module <code>src/sage/env_config.py</code> - see <a class="closed ticket" href="https://trac.sagemath.org/ticket/29022" title="enhancement: Make sagelib installation directory more flexible by creating a module ... (closed: wontfix)">#29022</a>.
</p>
TicketmkoeppeWed, 15 Jan 2020 23:55:47 GMT
https://trac.sagemath.org/ticket/21707#comment:26
https://trac.sagemath.org/ticket/21707#comment:26
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/21707#comment:17" title="Comment 17">mkoeppe</a>:
</p>
<blockquote class="citation">
<p>
Probably one should introduce a command <code>sage --buildsh</code>, which would provide the larger environment.
</p>
</blockquote>
<p>
As an extension of this, there could be <code>sage --buildsh SPKG</code>, which would set up the environment in which <code>spkg-install</code> runs. This could be useful for testing new versions of packages.
</p>
TicketmkoeppeSat, 18 Jan 2020 19:20:14 GMT
https://trac.sagemath.org/ticket/21707#comment:27
https://trac.sagemath.org/ticket/21707#comment:27
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/21707#comment:25" title="Comment 25">mkoeppe</a>:
</p>
<blockquote class="citation">
<p>
The run-time environment variables of sage could as well be configured by a Python module <code>src/sage/env_config.py</code> - see <a class="closed ticket" href="https://trac.sagemath.org/ticket/29022" title="enhancement: Make sagelib installation directory more flexible by creating a module ... (closed: wontfix)">#29022</a>.
</p>
</blockquote>
<p>
Or <a class="closed ticket" href="https://trac.sagemath.org/ticket/29038" title="enhancement: Python package sage_conf: Provides optional configuration information ... (closed: fixed)">#29038</a> - Python package sage_conf: Provides optional configuration information for sagelib
</p>
TicketmkoeppeSun, 19 Jan 2020 17:12:56 GMTstatus, description, cc, summary, dependencies, type changed; branch, commit deleted
https://trac.sagemath.org/ticket/21707#comment:28
https://trac.sagemath.org/ticket/21707#comment:28
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_info</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=28">diff</a>)
</li>
<li><strong>cc</strong>
<em>isuruf</em> <em>arojas</em> <em>infinity0</em> <em>gh-timokau</em> added
</li>
<li><strong>summary</strong>
changed from <em>Split sage-env into sage-build-env and sage-env</em> to <em>Split sage-env into 5</em>
</li>
<li><strong>dependencies</strong>
changed from <em>#25130</em> to <em>#29038</em>
</li>
<li><strong>branch</strong>
<em>public/split_sage_env_into_sage_build_env_and_sage_env</em> deleted
</li>
<li><strong>commit</strong>
<em>ff0712125a3d05500a1f77e62c371d695d1a9794</em> deleted
</li>
<li><strong>type</strong>
changed from <em>enhancement</em> to <em>task</em>
</li>
</ul>
<p>
Hoping for input from people involved in packaging.
</p>
TicketmkoeppeMon, 20 Jan 2020 14:20:34 GMTstatus changed
https://trac.sagemath.org/ticket/21707#comment:29
https://trac.sagemath.org/ticket/21707#comment:29
<ul>
<li><strong>status</strong>
changed from <em>needs_info</em> to <em>needs_review</em>
</li>
</ul>
TicketmkoeppeMon, 20 Jan 2020 14:51:35 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:30
https://trac.sagemath.org/ticket/21707#comment:30
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=30">diff</a>)
</li>
</ul>
TicketmkoeppeMon, 20 Jan 2020 15:15:55 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:31
https://trac.sagemath.org/ticket/21707#comment:31
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=31">diff</a>)
</li>
</ul>
TicketmkoeppeMon, 20 Jan 2020 17:30:54 GMTdependencies changed
https://trac.sagemath.org/ticket/21707#comment:32
https://trac.sagemath.org/ticket/21707#comment:32
<ul>
<li><strong>dependencies</strong>
changed from <em>#29038</em> to <em>#29038, #29052</em>
</li>
</ul>
TicketmkoeppeFri, 31 Jan 2020 14:05:36 GMTsummary changed
https://trac.sagemath.org/ticket/21707#comment:33
https://trac.sagemath.org/ticket/21707#comment:33
<ul>
<li><strong>summary</strong>
changed from <em>Split sage-env into 5</em> to <em>Meta-ticket: Split sage-env into 5 to clean up sage configuration</em>
</li>
</ul>
TicketmkoeppeFri, 31 Jan 2020 21:02:59 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:34
https://trac.sagemath.org/ticket/21707#comment:34
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=34">diff</a>)
</li>
</ul>
TicketmkoeppeFri, 31 Jan 2020 23:49:09 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:35
https://trac.sagemath.org/ticket/21707#comment:35
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=35">diff</a>)
</li>
</ul>
TicketmkoeppeSat, 01 Feb 2020 00:49:55 GMTcc changed
https://trac.sagemath.org/ticket/21707#comment:36
https://trac.sagemath.org/ticket/21707#comment:36
<ul>
<li><strong>cc</strong>
<em>mjo</em> added
</li>
</ul>
TicketmkoeppeSun, 29 Mar 2020 00:49:46 GMTstatus, description changed
https://trac.sagemath.org/ticket/21707#comment:37
https://trac.sagemath.org/ticket/21707#comment:37
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=37">diff</a>)
</li>
</ul>
TicketmkoeppeFri, 10 Apr 2020 22:52:15 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:38
https://trac.sagemath.org/ticket/21707#comment:38
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=38">diff</a>)
</li>
</ul>
TicketmkoeppeSat, 18 Apr 2020 16:54:14 GMTmilestone changed
https://trac.sagemath.org/ticket/21707#comment:39
https://trac.sagemath.org/ticket/21707#comment:39
<ul>
<li><strong>milestone</strong>
changed from <em>sage-9.1</em> to <em>sage-9.2</em>
</li>
</ul>
TicketmkoeppeFri, 01 May 2020 17:44:03 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:40
https://trac.sagemath.org/ticket/21707#comment:40
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=40">diff</a>)
</li>
</ul>
TicketmkoeppeFri, 01 May 2020 17:44:25 GMTcc changed
https://trac.sagemath.org/ticket/21707#comment:41
https://trac.sagemath.org/ticket/21707#comment:41
<ul>
<li><strong>cc</strong>
<em>jhpalmieri</em> added
</li>
</ul>
TicketmkoeppeWed, 27 May 2020 15:55:55 GMTkeywords set
https://trac.sagemath.org/ticket/21707#comment:42
https://trac.sagemath.org/ticket/21707#comment:42
<ul>
<li><strong>keywords</strong>
<em>sd109</em> added
</li>
</ul>
TicketmkoeppeSun, 07 Jun 2020 21:58:42 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:43
https://trac.sagemath.org/ticket/21707#comment:43
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=43">diff</a>)
</li>
</ul>
TicketmkoeppeTue, 09 Jun 2020 05:52:46 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:44
https://trac.sagemath.org/ticket/21707#comment:44
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=44">diff</a>)
</li>
</ul>
TicketmkoeppeMon, 13 Jul 2020 18:48:35 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:45
https://trac.sagemath.org/ticket/21707#comment:45
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=45">diff</a>)
</li>
</ul>
TicketmkoeppeMon, 13 Jul 2020 19:21:26 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:46
https://trac.sagemath.org/ticket/21707#comment:46
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=46">diff</a>)
</li>
</ul>
TicketmkoeppeTue, 06 Oct 2020 18:35:33 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:47
https://trac.sagemath.org/ticket/21707#comment:47
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=47">diff</a>)
</li>
</ul>
TicketmkoeppeFri, 23 Oct 2020 21:15:55 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:48
https://trac.sagemath.org/ticket/21707#comment:48
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=48">diff</a>)
</li>
</ul>
Ticketgh-tobiasdiezFri, 23 Oct 2020 22:22:07 GMT
https://trac.sagemath.org/ticket/21707#comment:49
https://trac.sagemath.org/ticket/21707#comment:49
<p>
Not sure if this is the right place, but I would suggest to use the refactoring process to also convert the environment variables to configuration files.
</p>
<p>
The idea would be to have a <code>toml</code> file (or <code>yaml</code> or whatever you prefer) in some predefined folder (say <code>src</code> for the start). This file could be generated e.g by the <code>configure</code> script. This config file is then parsed and wrapped by a python class (natural candidate would be the current <code>sage_conf.py</code>). The python class also provides reasonable default values in case the config file doesn't exist.
</p>
<p>
This has several advantages in my opinion:
</p>
<ul><li>More transparent, as one sees on one glance which variables influence sage.
</li><li>Easily editable by the user/developer.
</li><li>Local (e.g. think about multiple virtual environments with different configs).
</li><li>Type-safe, i.e, it is ensured when parsing the toml file that the types of the config variables is correct.
</li><li>Easier for downstream package managers, since they only need to provide a proper config file.
</li><li>(Higher security as one cannot inject arbitrary python code, which would be the case if <code>sage_conf.py</code> should be edited.)
</li></ul><p>
What do you think?
</p>
TicketmkoeppeFri, 23 Oct 2020 22:37:07 GMT
https://trac.sagemath.org/ticket/21707#comment:50
https://trac.sagemath.org/ticket/21707#comment:50
<p>
<code>sage_conf</code> defines an interface already - and the plan is indeed to get rid of dependencies on environment variables using this interface.
</p>
<p>
How <code>sage_conf</code> obtains the configuration is orthogonal to the defined interface.
</p>
<p>
I am not interested in defining configuration files at the moment. There is just too much potential on wasting energy on discussions regarding the "best" config file format.
</p>
TicketmkoeppeSat, 24 Oct 2020 20:15:01 GMTmilestone changed
https://trac.sagemath.org/ticket/21707#comment:51
https://trac.sagemath.org/ticket/21707#comment:51
<ul>
<li><strong>milestone</strong>
changed from <em>sage-9.2</em> to <em>sage-9.3</em>
</li>
</ul>
Ticketgh-tobiasdiezSun, 25 Oct 2020 10:37:39 GMT
https://trac.sagemath.org/ticket/21707#comment:52
https://trac.sagemath.org/ticket/21707#comment:52
<p>
Well, you are actually proposing to add a configuration file in the form of a python file (<code>sage_conf.py</code>). One could easily bypass an extensive discussion and follow the arguments of <a class="ext-link" href="https://www.python.org/dev/peps/pep-0518/#overview-of-file-formats-considered"><span class="icon"></span>https://www.python.org/dev/peps/pep-0518/#overview-of-file-formats-considered</a>.
</p>
<p>
But I agree that my suggestion concerning a config file is somewhat out of line with the goal of this ticket. I would still suggest to keep this in mind while designing the interface for <code>sage_conf</code>, so that it might be easy in the future to switch to a design based on a config file. In particular, I would replace the module wide variables (like <code>VERSION</code>) with methods in a config class (like <code>Config.get_version</code>) that can also provide suitable defaults and do slight post-processing.
</p>
TicketmkoeppeSun, 25 Oct 2020 15:15:54 GMT
https://trac.sagemath.org/ticket/21707#comment:53
https://trac.sagemath.org/ticket/21707#comment:53
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/21707#comment:52" title="Comment 52">gh-tobiasdiez</a>:
</p>
<blockquote class="citation">
<p>
Well, you are actually proposing to add a configuration file in the form of a python file (<code>sage_conf.py</code>).
</p>
</blockquote>
<p>
Note that <code>sage_conf.py</code> is already in use since Sage 9.1
</p>
TicketmkoeppeSun, 25 Oct 2020 15:22:39 GMT
https://trac.sagemath.org/ticket/21707#comment:54
https://trac.sagemath.org/ticket/21707#comment:54
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/21707#comment:52" title="Comment 52">gh-tobiasdiez</a>:
</p>
<blockquote class="citation">
<p>
I would still suggest to keep this in mind while designing the interface for <code>sage_conf</code>, so that it might be easy in the future to switch to a design based on a config file. In particular, I would replace the module wide variables (like <code>VERSION</code>) with methods in a config class (like <code>Config.get_version</code>) that can also provide suitable defaults and do slight post-processing.
</p>
</blockquote>
<p>
All of this discussion is certainly valid and I am sure that you (and others) have valuable insights there, but now is simply not the right time for it. To get the design right, one needs to know about objectives and constraints. We are in the process of expanding the ways of Sage installation/deployment into several directions, including pip-installability and modularization; and also binary packaging will need an overhaul.
</p>
TicketmkoeppeTue, 15 Dec 2020 01:41:59 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:55
https://trac.sagemath.org/ticket/21707#comment:55
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=55">diff</a>)
</li>
</ul>
TicketmkoeppeSat, 13 Feb 2021 20:51:01 GMTmilestone changed
https://trac.sagemath.org/ticket/21707#comment:56
https://trac.sagemath.org/ticket/21707#comment:56
<ul>
<li><strong>milestone</strong>
changed from <em>sage-9.3</em> to <em>sage-9.4</em>
</li>
</ul>
<p>
Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
</p>
TicketmkoeppeSat, 20 Feb 2021 03:03:30 GMTdescription changed
https://trac.sagemath.org/ticket/21707#comment:57
https://trac.sagemath.org/ticket/21707#comment:57
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/21707?action=diff&version=57">diff</a>)
</li>
</ul>
TicketmkoeppeMon, 19 Jul 2021 00:44:56 GMTmilestone changed
https://trac.sagemath.org/ticket/21707#comment:58
https://trac.sagemath.org/ticket/21707#comment:58
<ul>
<li><strong>milestone</strong>
changed from <em>sage-9.4</em> to <em>sage-9.5</em>
</li>
</ul>
<p>
Setting a new milestone for this ticket based on a cursory review.
</p>
Ticket