Opened 4 years ago
Last modified 14 months ago
#22652 new enhancement
Add a plain-text config file that feeds into sage.env
Reported by: | embray | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-wishlist |
Component: | porting | Keywords: | |
Cc: | fbissey, jdemeyer, mkoeppe, gh-timokau | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #27040 | Stopgaps: |
Description
Issues like #22628 remind me of an idea I've had before for improving the situation with the otherwise difficult to maintain and unwieldy sage.env
module.
ISTM that almsot all the data in that file could be read from a plain text file format (either an ini-like file or JSON, for example). Furthermore, this file could be generated, or updated, at install time for spkgs (including Sage itself). This would help make the growing number of (sometimes contentious) spkg-specific variables (CONWAY_POLYNOMIALS_DATA_DIR
, GRAPHS_DATA_DIR
, GAP_ROOT_DIR
, SINGULAR_SO
, etc.) more manageable.
If each spkg is allowed its own "section" in this config file, the installers for those spkgs could be allowed to manage the data in that section as well, which is sometimes platform-specific. It is also data that is usually more accessible at install-time of that package than it is to Sage at runtime. For example, #22628 raises a tricky issue about how to set SINGULAR_SO
. If the correct path to SINGULAR_SO
were written to a config file at install time, that would push all platform-specific logic about how to set that path to the spkg-install
script, and out of sage.env
. For system packagers, such as for Gentoo or Debian, they would be free to write whatever they want in this config file, and it would free them having to do as much patching to sage.env
and possibly other files.
I have some idea how this might work, so I'm willing to cook up a prototype if it doesn't sound like a terrible idea.
Change History (21)
comment:1 Changed 4 years ago by
- Dependencies set to #21479
comment:2 Changed 4 years ago by
I agree--one file per package sounds better, and will also have fewer problems in parallel builds.
I need to take a closer look but my hope is that such a config file could reduce complexity in both sage-env
and sage.env
.
The only thing with sage-env
is that it should be easy to read/parse the file without, say, invoking a Python interpreter. That's another good argument for one per-package config files, as that would reduce the complexity of the format. Probably a <key>=<value>
format would be good enough.
comment:3 Changed 4 years ago by
I would concur with Jeroen for a design with a directory but I am thinking of something like /etc/{env.d,profile.d,sudoers.d}
and the list goes on.
Bonus point if both sage-env
and env.py
can feed from it avoiding duplication.
comment:4 Changed 4 years ago by
To be clear, I would still allow all these options to be read from environment variables too, and environment variables would override config files. But this might also reduce the number of environment variables that need to be set.
Another possible advantage is the ability for per-user config files that override the globals, but I would leave that as a separate task.
comment:5 Changed 4 years ago by
- Cc mkoeppe added
comment:6 Changed 4 years ago by
See also #15087.
comment:7 follow-up: ↓ 10 Changed 4 years ago by
#15087 appears to be much more limited in scope and is slightly different, though not incompatible with this idea. #15087 is specifically for modifying the environment, but I'm hoping with this to actually modify the environment less, since options can instead be read from a config file. Good to know it exists though.
comment:8 follow-up: ↓ 9 Changed 4 years ago by
One thing I'm struggling with a bit on this is, if this is implemented, what the upgrade path should look like.
The idea is to generate these config files at build time. So how will they be generated for packages that are already built? Maybe I should just increase the patch levels on those packages? I think the only packages that might get their own config files are currently:
- conway_polynomials
- graphs
- elliptic_curves
- polytopes_db
- gap
- thebe
- singular
plus sagelib itself. Any others? I determined this (rough) list based on those packages that require any settings in sage.env
.
There are also packages that require special environment variables to be set in sage-env
. Those include:
- jupyter
- matplotlib
- r
- maxima
- ecl
- ncurses
This might be a job for #15087 like functionality.
comment:9 in reply to: ↑ 8 Changed 4 years ago by
comment:10 in reply to: ↑ 7 Changed 4 years ago by
Replying to embray:
#15087 appears to be much more limited in scope and is slightly different, though not incompatible with this idea. #15087 is specifically for modifying the environment, but I'm hoping with this to actually modify the environment less, since options can instead be read from a config file. Good to know it exists though.
I don't see any difference between #15087 and this ticket, except that #15087 might be slightly more focused on the shell, while ticket is slightly more focused on the Sage library.
comment:11 Changed 4 years ago by
I agree, that's the only difference. But it is a substantive one. We have two cases here: settings that must be reflected in the environment, and settings that would only be read by the Sage library, and not set as environment variables. Some packages could have settings that fall into both categories too (though I don't think there are any such examples currently).
It might be possible to kill two birds with one stone here, but there needs to be a way to make this distinction.
comment:12 follow-up: ↓ 13 Changed 4 years ago by
Maybe a simple way to kill two birds: Just a simple script that sets variables and possibly exports them. If the line starts with export then naturally it will set an environment variable when sourced. Otherwise it will just set a shell variable but not export it to the environment. When sage.env
reads these files it can parse it the same way.
comment:13 in reply to: ↑ 12 Changed 4 years ago by
Replying to embray:
Maybe a simple way to kill two birds: Just a simple script that sets variables and possibly exports them. If the line starts with export then naturally it will set an environment variable when sourced. Otherwise it will just set a shell variable but not export it to the environment. When
sage.env
reads these files it can parse it the same way.
This sounds like a good solution. Note src/bin/sage-env-config
is already in a simple format, and we should keep it in this form (see discussion on #22646).
comment:14 Changed 4 years ago by
I nearly forgot, I was working on implementing this last week. I think it was working, but I wasn't sure because there were some test failures (but it seems these failures are unrelated).
comment:15 Changed 4 years ago by
I'm reconsidering this now that I've gotten a little deeper into implementation. There are some cases that are somewhat non-trivial, such as the environment settings for R (which, for consistency's sake, would be nice to move entirely to its own file).
A related, but alternative approach I'm considering is that rather than sourcing these files directly, they should be executable scripts that do nothing but print out the environment variables that they should set. So, for example, rather than source $SAGE_LOCAL/etc/sage/env.d/pkgname.sh
, one would eval `$SAGE_LOAL/etc/sage/env.d/pkgname.sh`
. Or something like that.
comment:16 Changed 3 years ago by
- Cc gh-timokau added
comment:17 Changed 2 years ago by
- Milestone changed from sage-8.0 to sage-wishlist
comment:18 Changed 2 years ago by
[ Deleted this comment which was meant for #26968. ]
comment:19 Changed 2 years ago by
- Dependencies changed from #21479 to #27040
comment:20 Changed 2 years ago by
My last comment was intended for #26968. Sorry for the mixup.
comment:21 Changed 14 months ago by
Instead of a plain-text config file, in #29022 I generate a Python module.
The issue with the library dir configuration (SINGULAR_SO
) mentioned in the ticket description has again come up in #29013 (venv).
It should indeed be fixed by just writing out the configured library path to the new configuration file.
Instead of one file, having one directory where each package can write a file seems like a much better idea. Think of
pkgconfig
: we don't have onepkgconfig
file, we have a directorylocal/lib/pkgconfig
with one file per package.It's not clear to me if you plan to change only the Python module
sage.env
or also the shell scriptsrc/bin/sage-env
. Ideally, your idea should also reduce duplication/complexity insage-env
.I'm adding #21479 as dependency because of potential conflicts.