#31498 closed enhancement (fixed)
upgrade GAP to 4.11.1 or newer
Reported by:  Dima Pasechnik  Owned by:  

Priority:  major  Milestone:  sage9.4 
Component:  packages: standard  Keywords:  upgrade, gap 
Cc:  Antonio Rojas, Erik Bray, François Bissey, Matthias Köppe, Samuel Lelièvre, Tobias Hansen, Vincent Delecroix  Merged in:  
Authors:  Dima Pasechnik  Reviewers:  Volker Braun, Travis Scrimshaw 
Report Upstream:  N/A  Work issues:  
Branch:  82f3fa7 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
the current GAP release is 4.11.1. Time to upgrade  previous upgrade was in #29314.
Here we also upgrade libsemigroups, in lockstep with GAP, to 1.1.0.
GAP tarball is now hosted on GitHub; we also need to install more GAP packages to satsify needs of updated GAP package digraph and semigroups.
Finally, we fix a bug in TestSuite? for abelian groups
Change History (32)
comment:1 Changed 21 months ago by
Cc:  Antonio Rojas Erik Bray François Bissey Matthias Köppe Samuel Lelièvre added 

Keywords:  upgrade gap added 
comment:2 Changed 19 months ago by
Authors:  → Dima Pasechnik 

Branch:  → u/dimpase/packages/gap4111 
Commit:  → 37466dd9a122c4aade0f412313f53c59151e82c5 
comment:3 Changed 19 months ago by
Commit:  37466dd9a122c4aade0f412313f53c59151e82c5 → 8367f8e8422c2859418e084ff70f05b0f5f7bd97 

Branch pushed to git repo; I updated commit sha1. New commits:
8367f8e  ctbl versions now numbered

comment:4 Changed 19 months ago by
Commit:  8367f8e8422c2859418e084ff70f05b0f5f7bd97 → 246664d79f4267f2ee9566bd25f26da772a3d4db 

comment:5 followup: 6 Changed 19 months ago by
what remains is to fix this weird error  something is emitting that string of *
whcih overrides the test output. Not reproducible at sage:
prompt:
File "src/sage/tests/gap_packages.py", line 8, in sage.tests.gap_packages Failed example: test_packages(pkgs, only_failures=True) # optional  gap_packages Expected: ... Status Package GAP Output ++++ Got: Status Package GAP Output ++++ **********************************************************************
but
sage: from sage.tests.gap_packages import all_installed_packages, test_packages sage: pkgs = all_installed_packages(ignore_dot_gap=True) sage: test_packages(pkgs, only_failures=True) Status Package GAP Output ++++ sage:
comment:6 Changed 19 months ago by
Replying to dimpase:
what remains is to fix this weird error  something is emitting that string of
*
whcih overrides the test output.
Isn't that just the usual test failure output? It seems to me that the actual issue is the "..." wildcard on the first line  there is no newline before the "Status" line so it doesn't match.
comment:7 Changed 19 months ago by
Commit:  246664d79f4267f2ee9566bd25f26da772a3d4db → 717019729bb9e8a713b57fcdeb18420ec838a7f3 

Branch pushed to git repo; I updated commit sha1. New commits:
7170197  modify the test so that Sage's "table" is not used

comment:8 Changed 19 months ago by
another test to fix:
sage t warnlong 50.1 randomseed=0 src/sage/groups/abelian_gps/abelian_group_gap.py ********************************************************************** File "src/sage/groups/abelian_gps/abelian_group_gap.py", line 201, in sage.groups.abelian_gps.abelian_group_gap.AbelianGroupElement_polycyclic Failed example: TestSuite(G.an_element()).run() # optional  gap_packages Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/sitepackages/sage/misc/sage_unittest.py", line 297, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 647, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:5023) tester.assertEqual(loads(dumps(self)), self) File "sage/misc/persist.pyx", line 935, in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7536) return unpickler.load() File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9335) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4622) raise File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4514) return C._element_constructor(x) File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/sitepackages/sage/groups/abelian_gps/abelian_group_gap.py", line 368, in _element_constructor_ x *= gens_gap[i]**(exp[i] % orders[i]) File "sage/rings/integer.pyx", line 3413, in sage.rings.integer.Integer.__mod__ (build/cythonized/sage/rings/integer.c:22616) raise ZeroDivisionError("Integer modulo by zero") ZeroDivisionError: Integer modulo by zero  The following tests failed: _test_pickling **********************************************************************
comment:9 Changed 19 months ago by
Commit:  717019729bb9e8a713b57fcdeb18420ec838a7f3 → 5c885dd62363bbc63e88c2dc2473a783bf2c4035 

Branch pushed to git repo; I updated commit sha1. New commits:
5c885dd  do not take "mod 0", skip "mod"

comment:10 Changed 19 months ago by
Description:  modified (diff) 

Status:  new → needs_review 
comment:12 Changed 19 months ago by
I won't lie, I am not quite ready for this in sageongentoo. I have been under rather heavy stress so far this year and I am not doing as much as I want.
comment:13 Changed 19 months ago by
Status:  needs_review → positive_review 

Tell me about it ;)
But I think we should have people bang away at this asap and not later in the release cycle.
comment:15 Changed 19 months ago by
Reviewers:  → Volker Braun 

comment:16 followup: 17 Changed 19 months ago by
Status:  positive_review → needs_work 

There is a small merge conflict with #31489, and a failure
sage t long warnlong 43.1 randomseed=0 src/sage/tests/gap_packages.py ********************************************************************** File "src/sage/tests/gap_packages.py", line 9, in sage.tests.gap_packages Failed example: for p in pkgs: pkg = p.split('')[0] if not libgap.LoadPackage(pkg): raise Exception raised: Traceback (most recent call last): File "/home/release/Sage/local/lib64/python3.9/sitepackages/sage/doctest/forker.py", line 714, in _run self.compile_and_execute(example, compiler, test.globs) File "/home/release/Sage/local/lib64/python3.9/sitepackages/sage/doctest/forker.py", line 1133, in compile_and_execute exec(compiled, globs) File "<doctest sage.tests.gap_packages[3]>", line 4, in <module> raise RuntimeError: No active exception to reraise ********************************************************************** 1 item had failures: 1 of 6 in sage.tests.gap_packages [12 tests, 1 failure, 0.26 s]  sage t long warnlong 43.1 randomseed=0 src/sage/tests/gap_packages.py # 1 doctest failed 
comment:17 Changed 19 months ago by
Replying to vbraun:
There is a small merge conflict with #31489, and a failure
sage t long warnlong 43.1 randomseed=0 src/sage/tests/gap_packages.py ...
I only see this if I manually uninstall gap_packages. But this meant to be with them installed?
96396e238d4 (Volker Braun 20130722 17:05:24 0400 2) Test the optional GAP packages
GAP doesn't have a clean way to uninstall packages.
I can rebase over #31489 if it helps.
comment:18 Changed 19 months ago by
these test_packages stuff is a mess. We have two different functions
that both claim to Return list of all installed packages.
, test_packages()
, and all_installed_packages()
which differ in the output format  the former outputs table
(sic!)
and the latter does not check whether these packages actually may be loaded (so it's largely meaningless).
May we do it on another ticket?
comment:19 Changed 19 months ago by
Commit:  5c885dd62363bbc63e88c2dc2473a783bf2c4035 → 7ee7b5801bb4e4fe29b22de4acf28b2f8b01ab02 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
a9ac506  GAP version bump to 4.11.1

f8c87d4  ctbl versions now numbered

a2bc010  update libsemigroup

40e6105  more GAP packages are deps

8d44ad0  modify the test so that Sage's "table" is not used

05b434a  do not take "mod 0", skip "mod"

7ee7b58  better fix for the test

comment:20 Changed 19 months ago by
Status:  needs_work → needs_review 

I believe the test is fixed now.
comment:21 Changed 19 months ago by
Dependencies:  → #31489 

comment:22 Changed 19 months ago by
Commit:  7ee7b5801bb4e4fe29b22de4acf28b2f8b01ab02 → 101bd7451120e1a7972433f1c968ad62e5b266dc 

Branch pushed to git repo; I updated commit sha1. New commits:
101bd74  use the change from #31489

comment:23 Changed 19 months ago by
OK, now it depends on #31489 and should cleanly merge (the only change from #31489 is done in comment:22
comment:25 Changed 18 months ago by
Commit:  101bd7451120e1a7972433f1c968ad62e5b266dc → 82f3fa7bcc0e73c85690a0886a67c5d8cd2e5c2b 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
96df458  GAP version bump to 4.11.1

91a695f  ctbl versions now numbered

79b3bde  update libsemigroup

1116073  more GAP packages are deps

7469042  modify the test so that Sage's "table" is not used

82f3fa7  better fix for the test

comment:27 Changed 18 months ago by
Dependencies:  #31489 

comment:28 Changed 18 months ago by
Reviewers:  Volker Braun → Volker Braun, Travis Scrimshaw 

Status:  needs_review → positive_review 
Let's try to get this into the next beta.
comment:29 Changed 18 months ago by
Branch:  u/dimpase/packages/gap4111 → 82f3fa7bcc0e73c85690a0886a67c5d8cd2e5c2b 

Resolution:  → fixed 
Status:  positive_review → closed 
comment:30 Changed 18 months ago by
Commit:  82f3fa7bcc0e73c85690a0886a67c5d8cd2e5c2b 

I am a bit dumbfounded by this upgrade in a way. There are strange particulars to it in sageongentoo. I cannot build the doc.
[categorie] /var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups1 [categorie] from /var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/doc/en/reference/categories/sage/categories/finite_coxeter_groups.rst: [categorie] Traceback (most recent call last): [categorie] File "/usr/lib/python3.9/sitepackages/matplotlib/sphinxext/plot_directive.py", line 483, in run_code [categorie] exec(code, ns) [categorie] File "<string>", line 1, in <module> [categorie] File "sage/misc/lazy_import.pyx", line 362, in sage.misc.lazy_import.LazyImport.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/lazy_import.c:4041) [categorie] return self.get_object()(*args, **kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/combinat/root_system/coxeter_group.py", line 135, in CoxeterGroup [categorie] return CoxeterMatrixGroup(cartan_type, base_ring, index_set) [categorie] File "sage/misc/classcall_metaclass.pyx", line 322, in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/classcall_metaclass.c:1743) [categorie] return cls.classcall(cls, *args, **kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 230, in __classcall_private__ [categorie] return super(CoxeterMatrixGroup, cls).__classcall__(cls, [categorie] File "sage/misc/cachefunc.pyx", line 1001, in sage.misc.cachefunc.CachedFunction.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:6087) [categorie] w = self.f(*args, **kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/structure/unique_representation.py", line 1007, in __classcall__ [categorie] instance = typecall(cls, *args, **options) [categorie] File "sage/misc/classcall_metaclass.pyx", line 486, in sage.misc.classcall_metaclass.typecall (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/classcall_metaclass.c:2207) [categorie] return (<PyTypeObject*>type).tp_call(cls, args, kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 273, in __init__ [categorie] one = MS.one() [categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:12918) [categorie] self.cache = f(self._instance) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/matrix/matrix_space.py", line 1608, in identity_matrix [categorie] A = self.zero_matrix().__copy__() [categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:12918) [categorie] self.cache = f(self._instance) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/matrix/matrix_space.py", line 1766, in zero_matrix [categorie] zero = self.base_ring().zero() [categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:12918) [categorie] self.cache = f(self._instance) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/rings/universal_cyclotomic_field.py", line 1397, in zero [categorie] return self.element_class(self, libgap.zero()) [categorie] File "sage/libs/gap/libgap.pyx", line 618, in sage.libs.gap.libgap.Gap.zero (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/libgap.c:6277) [categorie] return self(0) [categorie] File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/structure/parent.c:9342) [categorie] return mor._call_(x) [categorie] File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/structure/coerce_maps.c:4628) [categorie] raise [categorie] File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/structure/coerce_maps.c:4520) [categorie] return C._element_constructor(x) [categorie] File "sage/libs/gap/libgap.pyx", line 298, in sage.libs.gap.libgap.Gap._element_constructor_ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/libgap.c:3391) [categorie] initialize() [categorie] File "sage/libs/gap/util.pyx", line 314, in sage.libs.gap.util.initialize (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/util.c:5897) [categorie] with atomic_write(workspace) as f: [categorie] File "sage/libs/gap/util.pyx", line 316, in sage.libs.gap.util.initialize (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/util.c:5852) [categorie] gap_eval('SaveWorkspace("{0}")'.format(f.name)) [categorie] File "sage/libs/gap/util.pyx", line 398, in sage.libs.gap.util.gap_eval (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/util.c:6068) [categorie] GAP_Enter() [categorie] sage.libs.gap.util.GAPError: Error, reached the preset memory limit [categorie] (change it with the o command line option) [graphs ] The inventory files are in ../../build_doc/inventory/en/reference/graphs. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/graphs [functions] building [inventory]: targets for 20 source files that are out of date [functions] updating environment: [new config] 20 added, 0 changed, 0 removed [finite_ri] The inventory files are in ../../build_doc/inventory/en/reference/finite_rings. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/finite_rings [function_] The inventory files are in ../../build_doc/inventory/en/reference/function_fields. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/function_fields [game_theo] building [inventory]: targets for 6 source files that are out of date [game_theo] updating environment: [new config] 6 added, 0 changed, 0 removed [games ] building [inventory]: targets for 5 source files that are out of date [games ] updating environment: [new config] 5 added, 0 changed, 0 removed [game_theo] The inventory files are in ../../build_doc/inventory/en/reference/game_theory. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/game_theory [categorie] The inventory files are in ../../build_doc/inventory/en/reference/categories. Error building the documentation. Traceback (most recent call last): File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__main__.py", line 2, in <module> main() File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 1813, in main builder() File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 345, in _wrapper getattr(get_builder(document), 'inventory')(*args, **kwds) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 582, in _wrapper self._build_everything_except_bibliography(format, *args, **kwds) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 565, in _build_everything_except_bibliography build_many(build_ref_doc, non_references) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 297, in build_many _build_many(target, args, processes=processes) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/utils.py", line 289, in build_many raise worker_exc.original_exception OSError: /var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups1
In sageongentoo, I do separate gap packages and the source of error is more precisely ctbllib1.3.1+ (I say + because there is a 1.3.2 now in its upstream and the result is the same with it). If I downgrade ctbllib to 1.2_p2 the documentation builds.
The error message suggests to increase the memory limit. This is set somewhere in sage/interface/gap.py
and I pushed it from the current 400MB
to 2000MB
without improvement.
I am probably missing something but what?
comment:31 followup: 32 Changed 18 months ago by
Whats the output of _get_gap_memory_pool_size_MB
?
GAP needs swap due to how they reserve the memory pool.
comment:32 Changed 18 months ago by
Replying to vbraun:
Whats the output of
_get_gap_memory_pool_size_MB
?GAP needs swap due to how they reserve the memory pool.
568MB if I leave it to the default. With the changes for my last attempt, it should be 2001MB. But what puzzles me is the influence of ctbllib
. So, I am wondering if the message is in fact unhelpful for the issue at hand.
New commits:
GAP version bump to 4.11.1