#31498 closed enhancement (fixed)
upgrade GAP to 4.11.1 or newer
Reported by:  dimpase  Owned by:  

Priority:  major  Milestone:  sage9.4 
Component:  packages: standard  Keywords:  upgrade, gap 
Cc:  arojas, embray, fbissey, mkoeppe, slelievre, thansen, vdelecroix  Merged in:  
Authors:  Dima Pasechnik  Reviewers:  Volker Braun, Travis Scrimshaw 
Report Upstream:  N/A  Work issues:  
Branch:  82f3fa7 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
the current GAP release is 4.11.1. Time to upgrade  previous upgrade was in #29314.
Here we also upgrade libsemigroups, in lockstep with GAP, to 1.1.0.
GAP tarball is now hosted on GitHub; we also need to install more GAP packages to satsify needs of updated GAP package digraph and semigroups.
Finally, we fix a bug in TestSuite? for abelian groups
Change History (32)
comment:1 Changed 11 months ago by
 Cc arojas embray fbissey mkoeppe slelievre added
 Keywords upgrade gap added
comment:2 Changed 9 months ago by
 Branch set to u/dimpase/packages/gap4111
 Commit set to 37466dd9a122c4aade0f412313f53c59151e82c5
comment:3 Changed 9 months ago by
 Commit changed from 37466dd9a122c4aade0f412313f53c59151e82c5 to 8367f8e8422c2859418e084ff70f05b0f5f7bd97
Branch pushed to git repo; I updated commit sha1. New commits:
8367f8e  ctbl versions now numbered

comment:4 Changed 9 months ago by
 Commit changed from 8367f8e8422c2859418e084ff70f05b0f5f7bd97 to 246664d79f4267f2ee9566bd25f26da772a3d4db
comment:5 followup: ↓ 6 Changed 9 months ago by
what remains is to fix this weird error  something is emitting that string of *
whcih overrides the test output. Not reproducible at sage:
prompt:
File "src/sage/tests/gap_packages.py", line 8, in sage.tests.gap_packages Failed example: test_packages(pkgs, only_failures=True) # optional  gap_packages Expected: ... Status Package GAP Output ++++ Got: Status Package GAP Output ++++ **********************************************************************
but
sage: from sage.tests.gap_packages import all_installed_packages, test_packages sage: pkgs = all_installed_packages(ignore_dot_gap=True) sage: test_packages(pkgs, only_failures=True) Status Package GAP Output ++++ sage:
comment:6 in reply to: ↑ 5 Changed 9 months ago by
Replying to dimpase:
what remains is to fix this weird error  something is emitting that string of
*
whcih overrides the test output.
Isn't that just the usual test failure output? It seems to me that the actual issue is the "..." wildcard on the first line  there is no newline before the "Status" line so it doesn't match.
comment:7 Changed 9 months ago by
 Commit changed from 246664d79f4267f2ee9566bd25f26da772a3d4db to 717019729bb9e8a713b57fcdeb18420ec838a7f3
Branch pushed to git repo; I updated commit sha1. New commits:
7170197  modify the test so that Sage's "table" is not used

comment:8 Changed 9 months ago by
another test to fix:
sage t warnlong 50.1 randomseed=0 src/sage/groups/abelian_gps/abelian_group_gap.py ********************************************************************** File "src/sage/groups/abelian_gps/abelian_group_gap.py", line 201, in sage.groups.abelian_gps.abelian_group_gap.AbelianGroupElement_polycyclic Failed example: TestSuite(G.an_element()).run() # optional  gap_packages Expected nothing Got: Failure in _test_pickling: Traceback (most recent call last): File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/sitepackages/sage/misc/sage_unittest.py", line 297, in run test_method(tester=tester) File "sage/structure/sage_object.pyx", line 647, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:5023) tester.assertEqual(loads(dumps(self)), self) File "sage/misc/persist.pyx", line 935, in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7536) return unpickler.load() File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9335) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4622) raise File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4514) return C._element_constructor(x) File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/sitepackages/sage/groups/abelian_gps/abelian_group_gap.py", line 368, in _element_constructor_ x *= gens_gap[i]**(exp[i] % orders[i]) File "sage/rings/integer.pyx", line 3413, in sage.rings.integer.Integer.__mod__ (build/cythonized/sage/rings/integer.c:22616) raise ZeroDivisionError("Integer modulo by zero") ZeroDivisionError: Integer modulo by zero  The following tests failed: _test_pickling **********************************************************************
comment:9 Changed 9 months ago by
 Commit changed from 717019729bb9e8a713b57fcdeb18420ec838a7f3 to 5c885dd62363bbc63e88c2dc2473a783bf2c4035
Branch pushed to git repo; I updated commit sha1. New commits:
5c885dd  do not take "mod 0", skip "mod"

comment:10 Changed 9 months ago by
 Description modified (diff)
 Status changed from new to needs_review
comment:12 Changed 8 months ago by
I won't lie, I am not quite ready for this in sageongentoo. I have been under rather heavy stress so far this year and I am not doing as much as I want.
comment:13 Changed 8 months ago by
 Status changed from needs_review to positive_review
Tell me about it ;)
But I think we should have people bang away at this asap and not later in the release cycle.
comment:14 Changed 8 months ago by
Thanks!
comment:15 Changed 8 months ago by
 Reviewers set to Volker Braun
comment:16 followup: ↓ 17 Changed 8 months ago by
 Status changed from positive_review to needs_work
There is a small merge conflict with #31489, and a failure
sage t long warnlong 43.1 randomseed=0 src/sage/tests/gap_packages.py ********************************************************************** File "src/sage/tests/gap_packages.py", line 9, in sage.tests.gap_packages Failed example: for p in pkgs: pkg = p.split('')[0] if not libgap.LoadPackage(pkg): raise Exception raised: Traceback (most recent call last): File "/home/release/Sage/local/lib64/python3.9/sitepackages/sage/doctest/forker.py", line 714, in _run self.compile_and_execute(example, compiler, test.globs) File "/home/release/Sage/local/lib64/python3.9/sitepackages/sage/doctest/forker.py", line 1133, in compile_and_execute exec(compiled, globs) File "<doctest sage.tests.gap_packages[3]>", line 4, in <module> raise RuntimeError: No active exception to reraise ********************************************************************** 1 item had failures: 1 of 6 in sage.tests.gap_packages [12 tests, 1 failure, 0.26 s]  sage t long warnlong 43.1 randomseed=0 src/sage/tests/gap_packages.py # 1 doctest failed 
comment:17 in reply to: ↑ 16 Changed 8 months ago by
Replying to vbraun:
There is a small merge conflict with #31489, and a failure
sage t long warnlong 43.1 randomseed=0 src/sage/tests/gap_packages.py ...
I only see this if I manually uninstall gap_packages. But this meant to be with them installed?
96396e238d4 (Volker Braun 20130722 17:05:24 0400 2) Test the optional GAP packages
GAP doesn't have a clean way to uninstall packages.
I can rebase over #31489 if it helps.
comment:18 Changed 8 months ago by
these test_packages stuff is a mess. We have two different functions
that both claim to Return list of all installed packages.
, test_packages()
, and all_installed_packages()
which differ in the output format  the former outputs table
(sic!)
and the latter does not check whether these packages actually may be loaded (so it's largely meaningless).
May we do it on another ticket?
comment:19 Changed 8 months ago by
 Commit changed from 5c885dd62363bbc63e88c2dc2473a783bf2c4035 to 7ee7b5801bb4e4fe29b22de4acf28b2f8b01ab02
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
a9ac506  GAP version bump to 4.11.1

f8c87d4  ctbl versions now numbered

a2bc010  update libsemigroup

40e6105  more GAP packages are deps

8d44ad0  modify the test so that Sage's "table" is not used

05b434a  do not take "mod 0", skip "mod"

7ee7b58  better fix for the test

comment:20 Changed 8 months ago by
 Status changed from needs_work to needs_review
I believe the test is fixed now.
comment:21 Changed 8 months ago by
 Dependencies set to #31489
comment:22 Changed 8 months ago by
 Commit changed from 7ee7b5801bb4e4fe29b22de4acf28b2f8b01ab02 to 101bd7451120e1a7972433f1c968ad62e5b266dc
Branch pushed to git repo; I updated commit sha1. New commits:
101bd74  use the change from #31489

comment:23 Changed 8 months ago by
OK, now it depends on #31489 and should cleanly merge (the only change from #31489 is done in comment:22
comment:24 Changed 8 months ago by
I have not merged the full branch of #31489 here, though.
comment:25 Changed 8 months ago by
 Commit changed from 101bd7451120e1a7972433f1c968ad62e5b266dc to 82f3fa7bcc0e73c85690a0886a67c5d8cd2e5c2b
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
96df458  GAP version bump to 4.11.1

91a695f  ctbl versions now numbered

79b3bde  update libsemigroup

1116073  more GAP packages are deps

7469042  modify the test so that Sage's "table" is not used

82f3fa7  better fix for the test

comment:26 Changed 8 months ago by
rebased on 9.4.beta1
comment:27 Changed 8 months ago by
 Dependencies #31489 deleted
comment:28 Changed 8 months ago by
 Reviewers changed from Volker Braun to Volker Braun, Travis Scrimshaw
 Status changed from needs_review to positive_review
Let's try to get this into the next beta.
comment:29 Changed 7 months ago by
 Branch changed from u/dimpase/packages/gap4111 to 82f3fa7bcc0e73c85690a0886a67c5d8cd2e5c2b
 Resolution set to fixed
 Status changed from positive_review to closed
comment:30 Changed 7 months ago by
 Commit 82f3fa7bcc0e73c85690a0886a67c5d8cd2e5c2b deleted
I am a bit dumbfounded by this upgrade in a way. There are strange particulars to it in sageongentoo. I cannot build the doc.
[categorie] /var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups1 [categorie] from /var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/doc/en/reference/categories/sage/categories/finite_coxeter_groups.rst: [categorie] Traceback (most recent call last): [categorie] File "/usr/lib/python3.9/sitepackages/matplotlib/sphinxext/plot_directive.py", line 483, in run_code [categorie] exec(code, ns) [categorie] File "<string>", line 1, in <module> [categorie] File "sage/misc/lazy_import.pyx", line 362, in sage.misc.lazy_import.LazyImport.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/lazy_import.c:4041) [categorie] return self.get_object()(*args, **kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/combinat/root_system/coxeter_group.py", line 135, in CoxeterGroup [categorie] return CoxeterMatrixGroup(cartan_type, base_ring, index_set) [categorie] File "sage/misc/classcall_metaclass.pyx", line 322, in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/classcall_metaclass.c:1743) [categorie] return cls.classcall(cls, *args, **kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 230, in __classcall_private__ [categorie] return super(CoxeterMatrixGroup, cls).__classcall__(cls, [categorie] File "sage/misc/cachefunc.pyx", line 1001, in sage.misc.cachefunc.CachedFunction.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:6087) [categorie] w = self.f(*args, **kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/structure/unique_representation.py", line 1007, in __classcall__ [categorie] instance = typecall(cls, *args, **options) [categorie] File "sage/misc/classcall_metaclass.pyx", line 486, in sage.misc.classcall_metaclass.typecall (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/classcall_metaclass.c:2207) [categorie] return (<PyTypeObject*>type).tp_call(cls, args, kwds) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 273, in __init__ [categorie] one = MS.one() [categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:12918) [categorie] self.cache = f(self._instance) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/matrix/matrix_space.py", line 1608, in identity_matrix [categorie] A = self.zero_matrix().__copy__() [categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:12918) [categorie] self.cache = f(self._instance) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/matrix/matrix_space.py", line 1766, in zero_matrix [categorie] zero = self.base_ring().zero() [categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/misc/cachefunc.c:12918) [categorie] self.cache = f(self._instance) [categorie] File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/rings/universal_cyclotomic_field.py", line 1397, in zero [categorie] return self.element_class(self, libgap.zero()) [categorie] File "sage/libs/gap/libgap.pyx", line 618, in sage.libs.gap.libgap.Gap.zero (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/libgap.c:6277) [categorie] return self(0) [categorie] File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/structure/parent.c:9342) [categorie] return mor._call_(x) [categorie] File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/structure/coerce_maps.c:4628) [categorie] raise [categorie] File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/structure/coerce_maps.c:4520) [categorie] return C._element_constructor(x) [categorie] File "sage/libs/gap/libgap.pyx", line 298, in sage.libs.gap.libgap.Gap._element_constructor_ (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/libgap.c:3391) [categorie] initialize() [categorie] File "sage/libs/gap/util.pyx", line 314, in sage.libs.gap.util.initialize (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/util.c:5897) [categorie] with atomic_write(workspace) as f: [categorie] File "sage/libs/gap/util.pyx", line 316, in sage.libs.gap.util.initialize (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/util.c:5852) [categorie] gap_eval('SaveWorkspace("{0}")'.format(f.name)) [categorie] File "sage/libs/gap/util.pyx", line 398, in sage.libs.gap.util.gap_eval (/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/cythonized/sage/libs/gap/util.c:6068) [categorie] GAP_Enter() [categorie] sage.libs.gap.util.GAPError: Error, reached the preset memory limit [categorie] (change it with the o command line option) [graphs ] The inventory files are in ../../build_doc/inventory/en/reference/graphs. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/graphs [functions] building [inventory]: targets for 20 source files that are out of date [functions] updating environment: [new config] 20 added, 0 changed, 0 removed [finite_ri] The inventory files are in ../../build_doc/inventory/en/reference/finite_rings. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/finite_rings [function_] The inventory files are in ../../build_doc/inventory/en/reference/function_fields. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/function_fields [game_theo] building [inventory]: targets for 6 source files that are out of date [game_theo] updating environment: [new config] 6 added, 0 changed, 0 removed [games ] building [inventory]: targets for 5 source files that are out of date [games ] updating environment: [new config] 5 added, 0 changed, 0 removed [game_theo] The inventory files are in ../../build_doc/inventory/en/reference/game_theory. Build finished. The built documents can be found in /var/tmp/portage/scimathematics/sage9999/work/build_doc/inventory/en/reference/game_theory [categorie] The inventory files are in ../../build_doc/inventory/en/reference/categories. Error building the documentation. Traceback (most recent call last): File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__main__.py", line 2, in <module> main() File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 1813, in main builder() File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 345, in _wrapper getattr(get_builder(document), 'inventory')(*args, **kwds) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 582, in _wrapper self._build_everything_except_bibliography(format, *args, **kwds) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 565, in _build_everything_except_bibliography build_many(build_ref_doc, non_references) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/__init__.py", line 297, in build_many _build_many(target, args, processes=processes) File "/var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/sage_docbuild/utils.py", line 289, in build_many raise worker_exc.original_exception OSError: /var/tmp/portage/scimathematics/sage9999/work/sage9999/srcpython3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups1
In sageongentoo, I do separate gap packages and the source of error is more precisely ctbllib1.3.1+ (I say + because there is a 1.3.2 now in its upstream and the result is the same with it). If I downgrade ctbllib to 1.2_p2 the documentation builds.
The error message suggests to increase the memory limit. This is set somewhere in sage/interface/gap.py
and I pushed it from the current 400MB
to 2000MB
without improvement.
I am probably missing something but what?
comment:31 followup: ↓ 32 Changed 7 months ago by
Whats the output of _get_gap_memory_pool_size_MB
?
GAP needs swap due to how they reserve the memory pool.
comment:32 in reply to: ↑ 31 Changed 7 months ago by
Replying to vbraun:
Whats the output of
_get_gap_memory_pool_size_MB
?GAP needs swap due to how they reserve the memory pool.
568MB if I leave it to the default. With the changes for my last attempt, it should be 2001MB. But what puzzles me is the influence of ctbllib
. So, I am wondering if the message is in fact unhelpful for the issue at hand.
New commits:
GAP version bump to 4.11.1