Opened 7 years ago
Closed 15 months ago
#13426 closed enhancement (fixed)
Improve gap_packages
Reported by:  vbraun  Owned by:  tbd 

Priority:  major  Milestone:  sage8.3 
Component:  packages: optional  Keywords:  days85 
Cc:  dimpase, mmarco, vbraun  Merged in:  
Authors:  Travis Scrimshaw  Reviewers:  Dima Pasechnik 
Report Upstream:  N/A  Work issues:  
Branch:  951e7e7 (Commits)  Commit:  951e7e7345103ff6836f379131afa1a5aedcd838 
Dependencies:  Stopgaps: 
Description (last modified by )
We should include more packages into GAP. This ticket adds the following:
 cohomolo
 CoReLG
 GBNP
 hecke
 liealgdb
 LiePRing
 LieRing
 loops
 mapclass
 qpa
 quagroup
 repsn
 sla
Change History (36)
comment:1 Changed 7 years ago by
 Description modified (diff)
comment:2 Changed 6 years ago by
 Milestone changed from sage5.11 to sage5.12
comment:3 Changed 6 years ago by
 Milestone changed from sage6.1 to sage6.2
comment:4 Changed 5 years ago by
 Milestone changed from sage6.2 to sage6.3
comment:5 Changed 5 years ago by
 Milestone changed from sage6.3 to sage6.4
comment:6 Changed 2 years ago by
 Milestone changed from sage6.4 to sage7.6
comment:7 Changed 2 years ago by
 Branch set to public/packages/increase_gap_packages22244
 Commit set to 951e7e7345103ff6836f379131afa1a5aedcd838
 Description modified (diff)
 Status changed from new to needs_review
New commits:
951e7e7  Added new packages to gap_packages.

comment:8 Changed 2 years ago by
 Description modified (diff)
 Keywords days85 added
comment:9 Changed 2 years ago by
to quote email by Alex K. on a GAP developers list:
In pull request https://github.com/gapsystem/gap/pull/1135 I have attempted to reduce a list of packages loaded by default to [ "autpgrp", "ctbllib", "factint", "fga", "irredsol", "tomlib" ], With dependencies, this results in the following set of packages being loaded at startup: AtlasRep 1.5.1, AutPGrp 1.6, Browse 1.8.6, CRISP 1.4.4, CTblLib 1.2.2, FactInt 1.5.3, FGA 1.3.1, GAPDoc 1.5.1, IO 4.4.6, IRREDSOL 1.3.1, SpinSym 1.5, TomLib 1.2.6
so we should aim to have as many from the latter list as possible. (packages that need GAP kernel extensions, like IO, are not possible to support ATM, as this will need changes in libGAP).
comment:10 Changed 2 years ago by
AtlasRep
is problematic in global installs. Debian has a package for it that does something but I may miss some bits that they put in gap itself. The problem is that AtlasRep
fetch data from the net and tries to dump it where the package is installed. If you are a regular user and gap is installed in location belonging to root you are in trouble.
AtlasRep
really need to dump data in ~/.gap
.
comment:11 Changed 2 years ago by
To note, we already have (some form of) AtlasRep
, AutPGrp
, CTblLib
, FactInt
included in gap_packages
.
The changes to libGAP that will allow IO should be handled in #22626 (or at least be doable afterwards). However, I don't want to hold up the addition of some other packages, a number of which are related to my research (I'm already using QuaGroup
in #22623). So I'd appreciate it if we could push off making gap_packages
"perfect" to a later ticket.
comment:12 Changed 2 years ago by
 Milestone changed from sage7.6 to sage8.1
Let me reformulate that better as a question: What would you want to see included that currently is not?
comment:13 Changed 2 years ago by
I will take a more provocative stance.
You can install gap packages for personal use or testing in ~/.gap/pkg
and they will be picked up automatically. This may need to be better documented but it means that we only need to care about packages that are difficult to install or require building against a library or executable found under SAGE_LOCAL
. The rest is just a matter of unpacking a tarball in the right place. It is even easier than installing a R package.
comment:14 Changed 2 years ago by
If I understand you correctly, you want to restrict gap_packages
to be just those packages and are "difficult," and let the "easy" packages be left to the individual user. I have no philosophical objections to this, but there are a few technical things that need to be addressed. (Documentation will, of course, depend on precisely what we do).
How do we want to treat the GAP packages wrt doctesting? Will they be treated as optional (i.e., automatically tested against if installed), experimental, or something else? #22623 might be a good test case (I need to add more doctests anyways).
Do we want a hook as part of sage i
or some other optional sage
argument to help ease installation? Or will the "official" way to install a GAP package is to manually extract the tarball that a user downloads?
What will we do about the packages that are previously installed by gap_packages
that would no longer be the case? I know this will not affect people with them currently installed, but imagine someone builds a new version from scratch (or after a make distclean
). Then suddenly a particular optional GAP package does not work. So it becomes backwards incompatible in a way (unless we have Sage automatically download it when trying to call said package, but there might be a legal issue here too).
comment:15 Changed 2 years ago by
Note that I preceded my comment with the word "provocative" (instant success). More seriously:
 there is a small list of package we may want to provide by default, we sort of already do. It is impossible to start gap without the gapdoc package being available for example.
 if a gap package is essential for some sage functionality we should ship it as standard.
 The spkg we currently have for gap packages and database are for convenience and are already optional. A ticket like this one wants to basically extend the convenience. Mainly in my opinion because people do not know how to install gap packages on their own. Gap upstream is not particularly verbose on that matter. They ship all the packages to you after all.
Documentation alone won't make the issue go away. Raising awareness of what you can do may achieve something. The mechanism to install R packages is well know. While debian seems to want to package CRAN we don't. People using R know what to do when they need a R packages. Gap users don't and I think that's the point that really need to be addressed.
comment:16 followup: ↓ 17 Changed 2 years ago by
The bigger thing for me right now is QuaGroup
. I got an email where I could point towards #22623, but had some issues with telling them how to get the QuaGroup
in a robust way.
I agree completely with points 1 and 2. For point 3, I feel that gap_packages
is the closest thing that Sage has to an "official" way to install optional GAP pacakges because I think Sage tells users " you do not have to know anything about the underlying pacakges". So this would be a change to the official way, but I think in a good way.
Some other technical questions:
 What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?
 On my system, I do not have a
~/.gap
folder (of course, I can create one), but I do not have a nonSage copy of GAP. Do we want this in~/.gap
or some subfolder of~/.sage
(in particular,~/.sage/gap
)? At least, I don't think your proposal will work because:sage: gap.console() gap> Print( GAPInfo.RootPaths, "\n"); [ "/home/travis/sagebuild/local/gap/latest/" ]
 How do we get users to run
gap_reset_workspace()
(IIRC it is necessary)?
comment:17 in reply to: ↑ 16 ; followup: ↓ 18 Changed 2 years ago by
Replying to tscrim:
The bigger thing for me right now is
QuaGroup
. I got an email where I could point towards #22623, but had some issues with telling them how to get theQuaGroup
in a robust way.I agree completely with points 1 and 2. For point 3, I feel that
gap_packages
is the closest thing that Sage has to an "official" way to install optional GAP pacakges because I think Sage tells users " you do not have to know anything about the underlying pacakges". So this would be a change to the official way, but I think in a good way.Some other technical questions:
 What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?
I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.
 On my system, I do not have a
~/.gap
folder (of course, I can create one), but I do not have a nonSage copy of GAP. Do we want this in~/.gap
or some subfolder of~/.sage
(in particular,~/.sage/gap
)? At least, I don't think your proposal will work because:sage: gap.console() gap> Print( GAPInfo.RootPaths, "\n"); [ "/home/travis/sagebuild/local/gap/latest/" ]
After inspection, it seems to be related to how gap is started from sage. That will need to be inspected.
fbissey@moonloop ~/sandbox/gitfork/sage $ ./sage sh Starting subshell with Sage environment variables set. Don't forget to exit when you are done. Beware: * Do not do anything with other copies of Sage on your system. * Do not use this for installing Sage packages using "sage i" or for running "make" at Sage's root directory. These should be done outside the Sage shell. Bypassing shell configuration files... Note: SAGE_ROOT=/home/fbissey/sandbox/gitfork/sage (sagesh) fbissey@moonloop:sage$ gap ┌───────┐ GAP 4.8.6, 12Nov2016, build of 20170706 10:42:48 (NZST) │ GAP │ http://www.gapsystem.org └───────┘ Architecture: x86_64pclinuxgnugccdefault64 Libs used: gmp, readline Loading the library and packages ... Packages: GAPDoc 1.5.1 Try '??help' for help. See also '?copyright', '?cite' and '?authors' gap> Print( GAPInfo.RootPaths, "\n"); [ "/home/fbissey/.gap/", "/home/fbissey/sandbox/gitfork/sage/local/gap/latest/" ]
Related to your previous question, RootPaths? can be customised (and obviously it is done in sage already), so you could imagine adding a unique location for that version of sage.
 How do we get users to run
gap_reset_workspace()
(IIRC it is necessary)?
I am aware of that problem and I am not sure. I wonder if upstream sees this as a problem or if it is particular to the way sage interface with gap.
comment:18 in reply to: ↑ 17 ; followups: ↓ 19 ↓ 22 Changed 2 years ago by
Replying to fbissey:
Replying to tscrim:
 What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?
I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.
I guess we already have this sort of problem with multiple Sage versions and init.sage
being in ~/.sage
. So putting something in ~/.sage
would not exacerbate the problem.
 On my system, I do not have a
~/.gap
folder (of course, I can create one), but I do not have a nonSage copy of GAP. Do we want this in~/.gap
or some subfolder of~/.sage
(in particular,~/.sage/gap
)? At least, I don't think your proposal will work because:sage: gap.console() gap> Print( GAPInfo.RootPaths, "\n"); [ "/home/travis/sagebuild/local/gap/latest/" ]After inspection, it seems to be related to how gap is started from sage. That will need to be inspected.
fbissey@moonloop ~/sandbox/gitfork/sage $ ./sage sh [snip] gap> Print( GAPInfo.RootPaths, "\n"); [ "/home/fbissey/.gap/", "/home/fbissey/sandbox/gitfork/sage/local/gap/latest/" ]
Doing this, I get:
gap> Print( GAPInfo.RootPaths, "\n"); [ "/home/travis/.gap/", "/home/travis/sagebuild/local/gap/latest/" ]
However, via gap.console()
might be considered a special way of starting gap, but it is something I think we will need to address because
sage: %gap > Switching to Gap < gap: Print( GAPInfo.RootPaths, "\n"); [ "/home/travis/sagebuild/local/gap/latest/" ] sage: gap('GAPInfo.RootPaths') [ "/home/travis/sagebuild/local/gap/latest/" ]
So within Sage calls to gap, it seems to use that version.
Related to your previous question, RootPaths? can be customised (and obviously it is done in sage already), so you could imagine adding a unique location for that version of sage.
I propose ~/.sage/gap
.
I'm guessing anything installed as proposed would also be available via libgap. At least via gap_packages
in #22623 I was able to make calls via libgap. (Granted, I think this will be made moot when we go to the next gap version).
 How do we get users to run
gap_reset_workspace()
(IIRC it is necessary)?I am aware of that problem and I am not sure. I wonder if upstream sees this as a problem or if it is particular to the way sage interface with gap.
I have no idea. At least if we had some way to install this using something like sage i
, we could run it as part of our installation process.
comment:19 in reply to: ↑ 18 ; followup: ↓ 20 Changed 2 years ago by
Replying to tscrim:
Replying to fbissey:
Replying to tscrim:
 What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?
I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.
I guess we already have this sort of problem with multiple Sage versions and
init.sage
being in~/.sage
. So putting something in~/.sage
would not exacerbate the problem.
I may have misunderstood you. You meant a gap package only available to sage, while with your wording I understood sageX.Y. Is it correct?
So far I haven't been able to dig where sage fiddle with the gap RootPaths?.
comment:20 in reply to: ↑ 19 Changed 2 years ago by
Replying to fbissey:
Replying to tscrim:
Replying to fbissey:
Replying to tscrim:
 What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?
I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.
I guess we already have this sort of problem with multiple Sage versions and
init.sage
being in~/.sage
. So putting something in~/.sage
would not exacerbate the problem.I may have misunderstood you. You meant a gap package only available to sage, while with your wording I understood sageX.Y. Is it correct?
Let me try to be more precise. Suppose I have two versions of sage, X and Y. I want to be able to install gap packages only available to X but not to Y.
Actually, thinking about this more, I don't think this would be a problem as if a gap package doesn't work with a particular version of gap, then it will fail normally. Plus there are probably better ways to compare two versions of a package. So I don't think this is really necessary.
comment:21 Changed 2 years ago by
A clean way to resolve this seems to be to pass the token upstream: ask them for a variant of gap i
facility to install GAPs packages.
I have opened up an issue #1499 on GAP's github repo.
comment:22 in reply to: ↑ 18 Changed 2 years ago by
Replying to tscrim:
At least if we had some way to install this using something like
sage i
, we could run it as part of our installation process.
1 to use sage i
to install packages in ~/.sage/gap
or some variant of that. I believe that there should be a clear difference between installing packages inside Sage itself and inside some user directory.
comment:23 Changed 2 years ago by
OK after some rather intricate tracking I found why RootPaths
is different in sage from outside sage: https://github.com/sagemath/sage/blob/master/src/sage/interfaces/gap.py#L200
The "r" parameter passed to gap
has the effect to make it ignore any .gaprc
file and set IgnoreGapRC
. But a side effect is that if IgnoreGapRC
is set
src/system.c: if (!IgnoreGapRC) { src/system.c SySetGapRootPath(DotGapPath); src/system.c }
.gap
is not added to RootPaths
.
comment:24 followups: ↓ 25 ↓ 27 Changed 2 years ago by
What we would be doing is setting a custom place for the Sage version of gap packages, and and so we would need a custom root path passed in via gap l <path>
.
@jdemeyer I somewhat see this as installing a Sage package, just within the gap part of Sage. However, I am not really convinced of the feasibility of sage i
as we would need all of the metadata for the gap packages and mirrors and other stuff that (IMO) is not really worth dealing with. I can deal with installing it in a specific folder (or making that the official installation method). Yet I see ~/.sage
as being something (mildly) hidden, and so we shouldn't need to tell users to go into there. IDK. What would be your preference?
Upstream does seem amenable to having gap i
, but not enough people with the time and/or skill to properly implement it. Also, we would want to have gap install it to a specified place, which would not be where gap is installed for Sage as they would get removed on upgrade. I would like to have something more workable in the near future.
comment:25 in reply to: ↑ 24 ; followups: ↓ 26 ↓ 28 Changed 2 years ago by
Replying to tscrim:
What would be your preference?
What's wrong with keeping the current $SAGE_LOCAL/gap/latest
? We install every Sage package in $SAGE_LOCAL
, why would GAP packages be an exception?
comment:26 in reply to: ↑ 25 Changed 2 years ago by
comment:27 in reply to: ↑ 24 ; followup: ↓ 29 Changed 2 years ago by
Replying to tscrim:
Upstream does seem amenable to having
gap i
, but not enough people with the time and/or skill to properly implement it. Also, we would want to have gap install it to a specified place, which would not be where gap is installed for Sage as they would get removed on upgrade. I would like to have something more workable in the near future.
I'm going to work on gap i
; it appears to align well with the current ODK grant goals.
comment:28 in reply to: ↑ 25 Changed 2 years ago by
Replying to jdemeyer:
Replying to tscrim:
What would be your preference?
What's wrong with keeping the current
$SAGE_LOCAL/gap/latest
? We install every Sage package in$SAGE_LOCAL
, why would GAP packages be an exception?
The problem as I understand it is that $SAGE_LOCAL/gap/latest
is just a symlink and changes on each upgrade. So suppose you install a package P
in gapX
, then you upgrade (in the process of upgrading Sage) to gapY
. Then $SAGE_LOCAL/gap/latest
points to gapY
, which does not contain P
, and your code that uses the package suddenly doesn't work. With the current gap_packages
, this is not a problem because our current framework automatically updates that as well. Or am I misunderstanding something?
However, I was asking about your preference for how to tell users to install optional gap packages? I guess your proposal it to tell them untar in $SAGE_LOCAL/gap/latest
and make that the official way (rather than on a mostly hidden sage wiki page).
comment:29 in reply to: ↑ 27 Changed 2 years ago by
Replying to dimpase:
Replying to tscrim:
Upstream does seem amenable to having
gap i
, but not enough people with the time and/or skill to properly implement it. Also, we would want to have gap install it to a specified place, which would not be where gap is installed for Sage as they would get removed on upgrade. I would like to have something more workable in the near future.I'm going to work on
gap i
; it appears to align well with the current ODK grant goals.
Great. I would offer to help, but I have neither the time nor expertise to contribute much. However, let me know if there is anything I can do. I guess if that is in the works, then we can simply tell users to install in a specific place until that method is realized as a workaround.
comment:30 Changed 16 months ago by
What is the status of this ticket? It appears to work, and I'd like to have qpa easily accessible. Can I have some cake?
comment:31 Changed 16 months ago by
 Milestone changed from sage8.1 to sage8.3
 Reviewers set to Dima Pasechnik
So these added packages don't do any GAP kernel modules, right? Just checkingotherwise it looks good.
comment:32 Changed 16 months ago by
I believe so. The only thing that needs to be compiled is cohomolo (https://www.gapsystem.org/Packages/cohomolo.html), but that is because of some extra C bindings IIRC.
comment:33 Changed 16 months ago by
 Status changed from needs_review to positive_review
OK, send it to the bots then. I hope they can get the tar file.
comment:34 Changed 15 months ago by
 Cc vbraun added
Volker, is there a reason why this ticket did not get merged in the past beta?
comment:35 Changed 15 months ago by
 Dependencies #13211 deleted
comment:36 Changed 15 months ago by
 Branch changed from public/packages/increase_gap_packages22244 to 951e7e7345103ff6836f379131afa1a5aedcd838
 Resolution set to fixed
 Status changed from positive_review to closed
See also #22190 and #22244.