Opened 2 years ago

Closed 18 months ago

#29314 closed enhancement (fixed)

update GAP to version 4.11, libsemigroup to 1.0.9

Reported by: dimpase Owned by:
Priority: major Milestone: sage-9.3
Component: packages: standard Keywords:
Cc: arojas, gh-timokau, soehms, gh-tobiasdiez Merged in:
Authors: Dima Pasechnik Reviewers: Matthias Koeppe, Tobias Diez
Report Upstream: N/A Work issues:
Branch: a28138c (Commits, GitHub, GitLab) Commit: a28138c2fc1396171734b12fdbe5904b79bf1ec1
Dependencies: Stopgaps:

Status badges

Description (last modified by dimpase)

GAP 4.11.0 and libsemigroup 1.3.1 have been released. (we pick libsemigroup 1.0.9 as the one compatible with GAP's 4.11 semigroups package)

tarball: see upstream_url fields

Attachments (1)

libgap_wrapper_test.log (71.5 KB) - added by soehms 20 months ago.
Output of doctest of libgap_wrapper.pyx on LinuxMint? 18

Download all attachments as: .zip

Change History (111)

comment:1 Changed 2 years ago by arojas

  • Cc arojas added

comment:2 Changed 2 years ago by gh-timokau

  • Cc gh-timokau added

comment:3 Changed 2 years ago by mkoeppe

Does the upgrade fix #28728?

comment:4 Changed 2 years ago by arojas

Besides changes in tests output, there's only one small change needed due to the VERSION alias removal. Patch at https://aur.archlinux.org/cgit/aur.git/tree/sagemath-gap-4.11.patch?h=sagemath-git

I haven't tested the gap-packages update

comment:5 Changed 2 years ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

comment:6 Changed 21 months ago by dimpase

  • Authors set to Dima Pasechnik
  • Branch set to u/dimpase/packages/gap411
  • Commit set to 1b80d95959282e2b20331721fae67f4d276217d1
  • Description modified (diff)
  • Summary changed from update GAP to version 4.11 to update GAP to version 4.11, libsemigroup to 1.3.1

New commits:

47c42d0version bumps, checksums, patch removal, packages: updated names
1b80d95libsemigroups update

comment:7 Changed 21 months ago by git

  • Commit changed from 1b80d95959282e2b20331721fae67f4d276217d1 to f361b88fa462485cd7f03725845ae7c741d9d011

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f361b88libsemigroups update

comment:8 Changed 21 months ago by dimpase

  • Description modified (diff)
  • Summary changed from update GAP to version 4.11, libsemigroup to 1.3.1 to update GAP to version 4.11, libsemigroup to 1.0.9
sage -t --warn-long 105.7 --random-seed=0 src/sage/libs/gap/element.pyx  # 6 doctests failed
sage -t --warn-long 105.7 --random-seed=0 src/sage/libs/gap/libgap.pyx  # 1 doctest failed

something to be fixed 1st

comment:9 Changed 21 months ago by dimpase

  • Dependencies set to #30496

comment:10 Changed 21 months ago by git

  • Commit changed from f361b88fa462485cd7f03725845ae7c741d9d011 to 8429a9c6e3f03a94003c32536541f08f0b38528a

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

82191cafix conversion from gap polynomial with valence > 0
c582e80better syntax for libgap calls
bbd7419version bumps, checksums, patch removal, packages: updated names
87c6a6blibsemigroups update
8429a9cpatches from Arch (cf #29314)

comment:11 Changed 21 months ago by git

  • Commit changed from 8429a9c6e3f03a94003c32536541f08f0b38528a to 08193ae145223f5f8b25f0e44bb11096f37c6548

Branch pushed to git repo; I updated commit sha1. New commits:

08193aetagged #random tests with output depending upon gap_packages installed

comment:12 Changed 21 months ago by git

  • Commit changed from 08193ae145223f5f8b25f0e44bb11096f37c6548 to c90360ba618188c4b8874f8f56d1759b2bd898f1

Branch pushed to git repo; I updated commit sha1. New commits:

c90360bone more doctest fix

comment:13 Changed 21 months ago by git

  • Commit changed from c90360ba618188c4b8874f8f56d1759b2bd898f1 to 21211413f3399e93051696a149bcff0e5d4c936e

Branch pushed to git repo; I updated commit sha1. New commits:

2121141more robust test, so that GAP's packages don't matter

comment:14 Changed 21 months ago by dimpase

  • Status changed from new to needs_review

there seems to be one failing doctest that is macOS-specific, in src/sage/groups/cubic_braid.py, but I still think we have to proceed (it's probably a platform-specific GAP bug).

comment:15 Changed 21 months ago by dimpase

on macOS 10.15.6 with and without gap_packages installed

sage -t --warn-long 70.9 --random-seed=0 src/sage/groups/cubic_braid.py
**********************************************************************
File "src/sage/groups/cubic_braid.py", line 1454, in sage.groups.cubic_braid.CubicBraidGroup.as_matrix_group
Failed example:
    m_back = C5(m)
Exception raised:
    Traceback (most recent call last):
      File "/Users/dima/software/sagetrac-mirror/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 715, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/dima/software/sagetrac-mirror/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1139, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.groups.cubic_braid.CubicBraidGroup.as_matrix_group[4]>", line 1, in <module>
        m_back = C5(m)
      File "sage/structure/parent.pyx", line 900, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9336)
        return mor._call_(x)
      File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4623)
        raise
      File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4515)
        return C._element_constructor(x)
      File "/Users/dima/software/sagetrac-mirror/local/lib/python3.7/site-packages/sage/groups/cubic_braid.py", line 1314, in _element_constructor_
        return map_to.lift(x)
      File "/Users/dima/software/sagetrac-mirror/local/lib/python3.7/site-packages/sage/groups/libgap_morphism.py", line 514, in lift
        if h.gap() not in phi.Image():
      File "sage/libs/gap/element.pyx", line 656, in sage.libs.gap.element.GapElement.__contains__ (build/cythonized/sage/libs/gap/element.c:7753)
        return GAP_IN(other, self).sage()
      File "sage/libs/gap/element.pyx", line 2525, in sage.libs.gap.element.GapElement_Function.__call__ (build/cythonized/sage/libs/gap/element.c:19780)
        sig_on()
    sage.libs.gap.util.GAPError: Error, reached the pre-set memory limit
    (change it with the -o command line option)

can't reproduce this on Linux

comment:16 follow-up: Changed 21 months ago by dimpase

  • Cc soehms added

Sebastian, could you reproduce this computation in GAP? I suspect it's a "pure" GAP bug.

comment:17 follow-up: Changed 21 months ago by arojas

It's not macOS-specific, I've been seeing this on Arch since I updated GAP. It only happens when running the full test suite, testing only the groups/cubic_braid.py file or even the entire groups module works.

comment:18 in reply to: ↑ 17 Changed 21 months ago by dimpase

Replying to arojas:

It's not macOS-specific, I've been seeing this on Arch since I updated GAP. It only happens when running the full test suite, testing only the groups/cubic_braid.py file or even the entire groups module works.

On macOS it's reproducible at Sage prompt.

comment:19 Changed 21 months ago by dimpase

I'll just do

  • src/sage/interfaces/gap.py

    a b def get_gap_memory_pool_size(): 
    277277    # Don't eat all address space if the user set ulimit -v
    278278    suggested_size = min(suggested_size, vmax // 10)
    279279    # ~220MB is the minimum for long doctests
    280     suggested_size = max(suggested_size, 250 * 1024**2)
     280    suggested_size = max(suggested_size, 350 * 1024**2)
    281281    return suggested_size

this limit has not been chnaged for 7 years, so that's OK :-)

comment:20 Changed 21 months ago by git

  • Commit changed from 21211413f3399e93051696a149bcff0e5d4c936e to f02f6e1d67dfbff7a4d1f96533ea14be765c418a

Branch pushed to git repo; I updated commit sha1. New commits:

f02f6e1increase memory pool for libgap

comment:21 Changed 21 months ago by dimpase

made it 400m

comment:23 Changed 21 months ago by mkoeppe

https://github.com/sagemath/sage/actions just needs some housekeeping - there are lots of workflowed from the WSL pull request that are no longer needed

comment:24 Changed 21 months ago by dimpase

ping?

comment:25 Changed 21 months ago by mkoeppe

  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

The updated libsemigroup fails to build on the outdated platforms ubuntu-trusty, debian-jessie, linuxmint-17. This is a regression regarding platform support, but I think that's fine for an optional package (also normaliz has the same problem).

Otherwise looks fine.

comment:26 Changed 21 months ago by mkoeppe

Does the upgrade fix something "critical"?

comment:27 Changed 21 months ago by dimpase

it removes a lot of patches (in fact, all the patches we have for gap and gap_packages), for one, which is always good. It incorporates numerous upstream bug fixes too, and it's a bit overdue, as 4.11 was released 6 months ago.

Last edited 21 months ago by dimpase (previous) (diff)

comment:28 in reply to: ↑ 16 Changed 21 months ago by soehms

Replying to dimpase:

Sebastian, could you reproduce this computation in GAP? I suspect it's a "pure" GAP bug.

Sorry, that I didn't see that earlier, since I'm on vacation until next weekend! But it seems that you've solved that already.

Do you know the reason why the domains of permutation groups of matrix groups have increased in the new version (see the differences of the doctests of permgroup.py at line 850 and 899)? On a first impression I wouldn't take that for an improvement. Anyway, I would break these long doctests lines.

Sebastian.

comment:29 Changed 21 months ago by vbraun

  • Status changed from positive_review to needs_work

Docbuild hangs forever on buildbots, a strace ends with

$ strace -ff ./sage --docbuild --verbose 99 --no-pdf-links all html
[...]
[pid 106971] access("/var/lib/buildbot/slave/sage_git/build/local/share/gap/pkg/resclasses-4.7.2/init.g", F_OK) = 0
[pid 106971] access("/var/lib/buildbot/slave/sage_git/build/local/share/gap/pkg/resclasses-4.7.2/read.g", F_OK) = 0
[pid 106971] access("/var/lib/buildbot/slave/sage_git/build/local/share/gap/pkg/polenta-1.3.9/doc/manual.six", F_OK) = 0
[pid 106971] access("/var/lib/buildbot/slave/sage_git/build/local/share/gap/pkg/resclasses-4.7.2/doc/manual.six", F_OK) = 0
[pid 106971] access("/var/lib/buildbot/.gap/gaprc", R_OK) = -1 ENOENT (No such file or directory)
[pid 106971] access("/var/lib/buildbot/.gap/gaprc.gz", R_OK) = -1 ENOENT (No such file or directory)
[pid 106971] access("/var/lib/buildbot/slave/sage_git/build/local/share/gap/gaprc", R_OK) = -1 ENOENT (No such file or directory)
[pid 106971] access("/var/lib/buildbot/slave/sage_git/build/local/share/gap/gaprc.gz", R_OK) = -1 ENOENT (No such file or directory)
[pid 106971] access("/var/lib/buildbot/.gap", F_OK) = -1 ENOENT (No such file or directory)
[pid 106971] open("/var/lib/buildbot/.gaprc", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 106971] access("/var/lib/buildbot/.gaprc.gz", R_OK) = -1 ENOENT (No such file or directory)
[pid 106971] access("/var/lib/buildbot/.gaprc.gz.gz", R_OK) = -1 ENOENT (No such file or directory)
[pid 106971] open("/var/lib/buildbot/slave/sage_git/build/local/lib/python3.8/site-packages/sage/libs/gap/sage.gaprc", O_RDONLY) = 12
[pid 106971] read(12, "# This file is run by Sage when "..., 20000) = 567
[pid 106971] read(12, "", 20000)        = 0
[pid 106971] close(12)                  = 0
[pid 106971] rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART, 0x7f2bc20154b0}, {0x7f0bc9679880, [CHLD], SA_RESTORER|SA_RESTART, 0x7f2bc20154b0}, 8) = 0
[pid 106971] stat("/var/lib/buildbot/slave/sage_git/build/local/lib/python3.8/site-packages/sage/matrix", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
[pid 106971] stat("/var/lib/buildbot/slave/sage_git/build/local/lib/python3.8/site-packages/sage/matrix/action.cpython-38-x86_64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=1110904, ...}) = 0
[pid 106971] open("/var/lib/buildbot/slave/sage_git/build/local/lib/python3.8/site-packages/sage/matrix/action.cpython-38-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 12
[pid 106971] read(12, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300?\0\0\0\0\0\0"..., 832) = 832
[pid 106971] fstat(12, {st_mode=S_IFREG|0755, st_size=1110904, ...}) = 0
[pid 106971] mmap(NULL, 2244824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 12, 0) = 0x7f0babc8a000
[pid 106971] mprotect(0x7f0babca9000, 2093056, PROT_NONE) = 0
[pid 106971] mmap(0x7f0babea8000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 12, 0x1e000) = 0x7f0babea8000
[pid 106971] mmap(0x7f0babeae000, 216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0babeae000
[pid 106971] close(12)                  = 0
[pid 106971] mprotect(0x7f0babea8000, 4096, PROT_READ) = 0
[pid 106971] mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0babc4a000
[pid 106971] munmap(0x7f0babc4a000, 262144) = 0
[pid 106971] mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0babc4a000
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={16, 968000}, ru_stime={0, 556000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={16, 968000}, ru_stime={0, 556000}, ...}) = 0
[pid 106971] madvise(0x100004a00000, 7344128, MADV_DONTNEED) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={17, 532000}, ru_stime={0, 556000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={17, 532000}, ru_stime={0, 556000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={18, 76000}, ru_stime={0, 556000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={18, 76000}, ru_stime={0, 556000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={18, 624000}, ru_stime={0, 564000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={18, 624000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={19, 172000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={19, 172000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={19, 712000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={19, 712000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={20, 252000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={20, 252000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={20, 788000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={20, 792000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={21, 328000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={21, 328000}, ru_stime={0, 568000}, ...}) = 0
[pid 106971] getrusage(RUSAGE_SELF, {ru_utime={21, 872000}, ru_stime={0, 568000}, ...}) = 0
[... repeat ad infinitum ...]

but not when building locally. I tried to delete the content of sage.gaprc but that didn't help.

comment:30 Changed 21 months ago by mkoeppe

  • Branch changed from u/dimpase/packages/gap411 to u/mkoeppe/packages/gap411

comment:31 Changed 21 months ago by dimpase

  • Commit changed from f02f6e1d67dfbff7a4d1f96533ea14be765c418a to 35649994b528588b33968f860530ad8fef5c38c7

Absolutely no idea what sort of fresh hell this might be.

Time to get rid of buildbots, maybe...


New commits:

3564999build/pkgs/gap_packages: Use symlinks to refer to gap's tarball

comment:32 Changed 21 months ago by dimpase

could it be something like running out of file descriptors?

comment:33 Changed 21 months ago by vbraun

Unlikely, it happened also on kucalc which is rather beefy

comment:34 Changed 21 months ago by dimpase

OS has a hard limit on the number of them, beefy or not, and if you leak them, the god help you.

could also be some parallelisation crap. Do bildbots try to over-parallelise things, with setting MAKE to something etc?

Could also be something similar to https://trac.sagemath.org/ticket/30351 - which deals with a hanging docbuild on macOS - maybe this desease is now infecting buildbots too?

Version 0, edited 21 months ago by dimpase (next)

comment:35 Changed 21 months ago by vbraun

There are multiple physical hosts, its not a resource limit (or at least one thats set up by the docbuilder)

Last edited 21 months ago by vbraun (previous) (diff)

comment:36 Changed 21 months ago by dimpase

any idea at what point of docbuild it hangs?

comment:37 Changed 21 months ago by vbraun

cd ../.. && sage-logger -p './sage --docbuild --no-pdf-links all html -j ' logs/dochtml.log
[dochtml] 
[dochtml] Building reference manual, first pass.
[dochtml] 
[dochtml] [reference] building [inventory]: targets for 1 source files that are out of date
[dochtml] [reference] updating environment: [new config] 1 added, 0 changed, 0 removed
[dochtml] [reference] The inventory files are in local/share/doc/sage/inventory/en/reference/references.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/references
[dochtml] [spkg     ] building [inventory]: targets for 271 source files that are out of date
[dochtml] [spkg     ] updating environment: [new config] 271 added, 0 changed, 0 removed
[dochtml] [spkg     ] The inventory files are in local/share/doc/sage/inventory/en/reference/spkg.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/spkg
[dochtml] [manifolds] building [inventory]: targets for 78 source files that are out of date
[dochtml] [manifolds] updating environment: [new config] 78 added, 0 changed, 0 removed
[dochtml] [manifolds] The inventory files are in local/share/doc/sage/inventory/en/reference/manifolds.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/manifolds
[dochtml] [polynomia] building [inventory]: targets for 62 source files that are out of date
[dochtml] [polynomia] updating environment: [new config] 62 added, 0 changed, 0 removed
[dochtml] [polynomia] The inventory files are in local/share/doc/sage/inventory/en/reference/polynomial_rings.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/polynomial_rings
[dochtml] [repl     ] building [inventory]: targets for 35 source files that are out of date
[dochtml] [repl     ] updating environment: [new config] 35 added, 0 changed, 0 removed
[dochtml] [repl     ] The inventory files are in local/share/doc/sage/inventory/en/reference/repl.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/repl
[dochtml] [algebras ] building [inventory]: targets for 95 source files that are out of date
[dochtml] [algebras ] updating environment: [new config] 95 added, 0 changed, 0 removed
[dochtml] [algebras ] The inventory files are in local/share/doc/sage/inventory/en/reference/algebras.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/algebras
[dochtml] [tensor_fr] building [inventory]: targets for 19 source files that are out of date
[dochtml] [tensor_fr] updating environment: [new config] 19 added, 0 changed, 0 removed
[dochtml] [tensor_fr] The inventory files are in local/share/doc/sage/inventory/en/reference/tensor_free_modules.
[dochtml] Build finished. The built documents can be found in /home/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/tensor_free_modules
[dochtml] Warning: Missing title for sage.combinat.posets.mobile
[dochtml] [combinat ] building [inventory]: targets for 378 source files that are out of date
[dochtml] [combinat ] updating environment: [new config] 378 added, 0 changed, 0 removed
command timed out: 1200 seconds without output running ['make', 'doc-html-mathjax'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=1349.554977

@mkoeppe: can you try to reproduce it on a CI build?

comment:38 Changed 21 months ago by dimpase

What is the meaning of -j option in ./sage --docbuild --no-pdf-links all html -j ?

Is it documented? If it's akin to -j for make, then, hmm, all bets are off.

comment:39 Changed 21 months ago by dimpase

Ah, ok, -j is for MathJax. Is MathJax locally installed?

Last edited 21 months ago by dimpase (previous) (diff)

comment:40 follow-up: Changed 21 months ago by vbraun

Definitely not on all buildbots...

comment:41 in reply to: ↑ 40 Changed 20 months ago by dimpase

Replying to vbraun:

Definitely not on all buildbots...

does it only hang on buildbots without locally installed MathJax ?

Last edited 20 months ago by dimpase (previous) (diff)

comment:42 follow-up: Changed 20 months ago by soehms

Here an observation which could help (done under WSL Ubuntu):

If I checkout this ticket on top of 9.2.beta12 and do

make SAGE_SPKG="sage-spkg -o" gap_packages-clean build

then I get:

sage: B = BraidGroup(8)
Traceback (most recent call last):
...
GAPError: Error, no method found! Error, no 2nd choice method found for `InverseSameMutability' on 1 arguments

I came accross that while looking at the docbuild failure noted in #30346 (which seems to be caused by this ticket as well).

Last edited 20 months ago by soehms (previous) (diff)

comment:43 in reply to: ↑ 42 ; follow-up: Changed 20 months ago by dimpase

Replying to soehms:

Here an observation which could help (done under WSL Ubuntu):

If I checkout this ticket on top of 9.2.beta12 and do

make SAGE_SPKG="sage-spkg -o" gap_packages-clean build

then I get:

sage: B = BraidGroup(8)
Traceback (most recent call last):
...
GAPError: Error, no method found! Error, no 2nd choice method found for `InverseSameMutability' on 1 arguments

I came accross that while looking at the docbuild failure noted in #30346 (which seems to be caused by this ticket as well).

I tried B = BraidGroup(8) on this branch, without and with gap_packages installed, and it works for me. (On Gentoo Linux - so this might be an WSL problem).

Last edited 20 months ago by dimpase (previous) (diff)

comment:44 Changed 20 months ago by dimpase

comment:45 Changed 20 months ago by mkoeppe

Can this upstream patch be applied to the existing gap version (4.10.2) as well? Then that should probably be done on a separate ticket

comment:46 Changed 20 months ago by dimpase

  • Branch changed from u/mkoeppe/packages/gap411 to u/dimpase/packages/gap411
  • Commit changed from 35649994b528588b33968f860530ad8fef5c38c7 to 3435c24a53490f484ff28267b61498282c8afbbb
  • Status changed from needs_work to needs_review

added a patch needed for gap_packages on XCode 12


New commits:

b0d8fc8version bumps, checksums, patch removal, packages: updated names
c0b35f9libsemigroups update
6a6c8cepatches from Arch (cf #29314)
3c45832tagged #random tests with output depending upon gap_packages installed
07968ebone more doctest fix
2728be6more robust test, so that GAP's packages don't matter
b2fa6d8increase memory pool for libgap
dd39f33build/pkgs/gap_packages: Use symlinks to refer to gap's tarball
3435c24clang 12 (macOS XCode 12) needs extra includes

comment:47 Changed 20 months ago by git

  • Commit changed from 3435c24a53490f484ff28267b61498282c8afbbb to ca2c40a570cb0cd03fc110298c2dc436370cf846

Branch pushed to git repo; I updated commit sha1. New commits:

ca2c40aversion bump

comment:48 Changed 20 months ago by mkoeppe

  • Reviewers changed from Matthias Koeppe to Matthias Koeppe, https://github.com/mkoeppe/sage/actions/runs/270864243

comment:49 Changed 20 months ago by git

  • Commit changed from ca2c40a570cb0cd03fc110298c2dc436370cf846 to 5549ee60a915dbcb01bdbce9bdae9d6555b1fda9

Branch pushed to git repo; I updated commit sha1. New commits:

5549ee6fix cohomolo pkg code to allow build with gcc10

comment:50 Changed 20 months ago by dimpase

added forgotten gcc-10 patch

comment:51 in reply to: ↑ 43 ; follow-up: Changed 20 months ago by soehms

Replying to dimpase:

Replying to soehms:

Here an observation which could help (done under WSL Ubuntu):

....

I tried B = BraidGroup(8) on this branch, without and with gap_packages installed, and it works for me. (On Gentoo Linux - so this might be an WSL problem).

Dima, I'm having the same problem on LinuxMint 19.1. So, its probably not a WSL issue but maybe a problem with the person in front of the computer. Can you please post how I have to build Sage after checking out the branch?

comment:52 in reply to: ↑ 51 ; follow-up: Changed 20 months ago by dimpase

Replying to soehms:

Dima, I'm having the same problem on LinuxMint 19.1. So, its probably not a WSL issue but maybe a problem with the person in front of the computer. Can you please post how I have to build Sage after checking out the branch?

  1. pull the branch
  2. ./sage -f gap
  3. ./sage -f gap_packages # (optional)
  4. make build
Last edited 20 months ago by dimpase (previous) (diff)

comment:53 in reply to: ↑ 52 Changed 20 months ago by soehms

Replying to dimpase:

Replying to soehms:

Dima, I'm having the same problem on LinuxMint 19.1. So, its probably not a WSL issue but maybe a problem with the person in front of the computer. Can you please post how I have to build Sage after checking out the branch?

  1. pull the branch
  2. ./sage -f gap
  3. ./sage -f gap_packages # (optional)
  4. make build

Thank you! Unfortunately that didn't solve the problem. Is it possible that there is a dependency missing in the ticket description?

The list of doctest failures under groups I'm getting under WSL and LinuxMint looks like this:

...
sage -t --random-seed=0 src/sage/groups/braid.py  # 289 doctests failed
sage -t --random-seed=0 src/sage/groups/raag.py  # 145 doctests failed
sage -t --random-seed=0 src/sage/groups/libgap_wrapper.pyx  # 13 doctests failed
sage -t --random-seed=0 src/sage/groups/free_group.py  # Killed due to segmentation fault
sage -t --random-seed=0 src/sage/groups/finitely_presented.py  # 251 doctests failed
...

In the case of finitely_presented.py I also got longjmp error messages:

*** longjmp causes uninitialized stack frame ***: /home/sebastian/develop/sage/local/bin/python3 terminated
*** longjmp causes uninitialized stack frame ***: /home/sebastian/develop/sage/local/bin/python3 terminated

comment:54 follow-up: Changed 20 months ago by dimpase

  1. try removing ~/.sage/gap/, or even ~/.sage/.
  1. Could it be you have GAP installed systemwide, and you get an interference? Or perhaps something in your gap.ini or gaprc files, cf. https://www.gap-system.org/Manuals/doc/ref/chap3.html#X7FD66F977A3B02DF ?
  1. Test the Sage's GAP 4.11 installation, e.g. by
    $ SAGE_CHECK=yes ./sage -f gap
    
  1. Try building this branch from scratch.

comment:55 Changed 20 months ago by git

  • Commit changed from 5549ee60a915dbcb01bdbce9bdae9d6555b1fda9 to 40a2d0f2109daf56293c51e153747cc5ca3b25ff

Branch pushed to git repo; I updated commit sha1. New commits:

40a2d0fremove old workaround from #22626

comment:56 Changed 20 months ago by dimpase

  • Status changed from needs_review to positive_review

Volker, could we try this again? I gather that the commit in comment:55 should change the behaviour of buildbots in comment:29, where crunch seems to happen after reading sage.gaprc (and this commit removes an overlooked GAP 4.10-specific workaround in sage.gaprc)

Last edited 20 months ago by dimpase (previous) (diff)

comment:57 in reply to: ↑ 54 ; follow-up: Changed 20 months ago by soehms

Replying to dimpase:

Thanks for the advices! Here my results:

  1. try removing ~/.sage/gap/, or even ~/.sage/.

no succsess

  1. Could it be you have GAP installed systemwide, and you get an interference? Or perhaps something in your gap.ini or gaprc files ...

yes, on both of the platforms there is gap 4.8.8, but I was not aware of that. I once installed Sage via apt which is an old version (8.1). My guess is that the systemwide gap came along with that (possible?).

But anyway, I have an older computer running under LinuxMint 18 without a systemwide gap installed and the bug shows there, as well, even after your latest commit. Furthermore I didn't find gap.ini and gaprc below local/share/gap.

  1. Test the Sage's GAP 4.11 installation, e.g. by
    $ SAGE_CHECK=yes ./sage -f gap
    

The gap testsuite passed through successfully (in all cases).

  1. Try building this branch from scratch.

On the WSL machine I build from a fresh git clone. The bug still shows.

The error output on the older machine produced back-trace information for the longjmp errors. Shall I upload a log-file?

Last edited 20 months ago by soehms (previous) (diff)

comment:58 Changed 20 months ago by git

  • Commit changed from 40a2d0f2109daf56293c51e153747cc5ca3b25ff to 2ffc24af85679499aff4846b1f135f8cea22b8c0
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

2ffc24athis file should exist - leave just comments there

comment:59 in reply to: ↑ 57 ; follow-up: Changed 20 months ago by dimpase

Replying to soehms:

Replying to dimpase:

Thanks for the advices! Here my results:

  1. try removing ~/.sage/gap/, or even ~/.sage/.

no succsess

  1. Could it be you have GAP installed systemwide, and you get an interference? Or perhaps something in your gap.ini or gaprc files ...

yes, on both of the platforms there is gap 4.8.8, but I was not aware of that. I once installed Sage via apt which is an old version (8.1). My guess is that the systemwide gap came along with that (possible?).

Could it be that you have an old libgap too, somewhere, from Sage 8.1?

libgap and GAP share stuff under Sage, and if something came from GAP 4.8.8 then certainly there is a high chance that things break. Could you remove the old GAP and old Sage and try again?

But anyway, I have an older computer running under LinuxMint 18 without a systemwide gap installed and the bug shows there, as well, even after your latest commit. Furthermore I didn't find gap.ini and gaprc below local/share/gap.

Please make sure there no traces of other GAP installations, .gaprc etc on that other machine.

  1. Test the Sage's GAP 4.11 installation, e.g. by
    $ SAGE_CHECK=yes ./sage -f gap
    

The gap testsuite passed through successfully (in all cases).

  1. Try building this branch from scratch.

On the WSL machine I build from a fresh git clone. The bug still shows.

The error output on the older machine produced back-trace information for the longjmp errors. Shall I upload a log-file?

Hopefully it's just stale bits that cause this error. Could you post the linkage of libgap Python module, i.e. the output of

ldd local/lib/python3*/site-packages/sage/libs/gap/libgap.*.so

comment:60 in reply to: ↑ 59 Changed 20 months ago by soehms

Replying to dimpase:

Replying to soehms:

Replying to dimpase:

......

Hopefully it's just stale bits that cause this error. Could you post the linkage of libgap Python module, i.e. the output of

ldd local/lib/python3*/site-packages/sage/libs/gap/libgap.*.so

Here the one from the LinuxMint 18

local/lib/python3.7/site-packages/sage/libs/gap/libgap.cpython-37m-x86_64-linux-gnu.so:
	linux-vdso.so.1 =>  (0x00007ffd3a776000)
	libgap.so.0 => /home/sebastian/develop/sage/local/lib/libgap.so.0 (0x00007efc33b3c000)
	libpython3.7m.so.1.0 => not found
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efc3391f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efc33555000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007efc332d5000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efc330bb000)
	libreadline.so.6 => /home/sebastian/develop/sage/local/lib/libreadline.so.6 (0x00007efc32e71000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efc32b68000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efc32964000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007efc32761000)
	/lib64/ld-linux-x86-64.so.2 (0x00007efc34a8c000)
	libtinfo.so.6 => /home/sebastian/develop/sage/local/lib/libtinfo.so.6 (0x00007efc32528000)
local/lib/python3.8/site-packages/sage/libs/gap/libgap.cpython-38-x86_64-linux-gnu.so:
	linux-vdso.so.1 =>  (0x00007fff44b9d000)
	libgap.so.0 => /home/sebastian/develop/sage/local/lib/libgap.so.0 (0x00007fdca3ad8000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdca38bb000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdca34f1000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fdca3271000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fdca3057000)
	libreadline.so.6 => /home/sebastian/develop/sage/local/lib/libreadline.so.6 (0x00007fdca2e0d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdca2b04000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fdca2900000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fdca26fd000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fdca4a29000)
	libtinfo.so.6 => /home/sebastian/develop/sage/local/lib/libtinfo.so.6 (0x00007fdca24c4000)

Here from WSL case

        linux-vdso.so.1 (0x00007ffff0e43000)
        libgap.so.0 => /home/sebastian/devel/sage/local/lib/libgap.so.0 (0x00007f53a2a65000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f53a2830000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f53a2430000)
        libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f53a21a0000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f53a1f80000)
        libreadline.so.7 => /lib/x86_64-linux-gnu/libreadline.so.7 (0x00007f53a1d30000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f53a1980000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f53a1770000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f53a1560000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f53a3600000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f53a1330000)

comment:61 follow-up: Changed 20 months ago by dimpase

Linux Mint case looks a bit funny, as you have both python 3.7 and 3.8 in one installation? Or they are different?

Could you also post the errors you get in src/sage/groups/libgap_wrapper.pyx ? (which is the root cause of the other errors, I gather)

Anyhow, I'm now testing on Linux Mint 19 (in an lxc container), let's see if this is reproducible.

Changed 20 months ago by soehms

Output of doctest of libgap_wrapper.pyx on LinuxMint? 18

comment:62 in reply to: ↑ 61 Changed 20 months ago by soehms

Replying to dimpase:

Could you also post the errors you get in src/sage/groups/libgap_wrapper.pyx ? (which is the root cause of the other errors, I gather)

See the attached file!

comment:63 follow-up: Changed 20 months ago by dimpase

I've installed Linux Mint Tara (i.e. 19) in an xlc container, built this branch there, and ran tests - and everything worked. Here is what I have in the apt sources list

deb http://packages.linuxmint.com tara main upstream import backport
deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu bionic-security main restricted universe multiverse

the kernel is

Linux m1 5.4.48-06434-gd868196d13af #1 SMP PREEMPT Wed Jul 29 00:32:34 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux

So it looks as if everything should run just fine - I guess something is wrong with the particular box, sorry.

comment:64 Changed 20 months ago by dimpase

  • Status changed from needs_review to positive_review

comment:65 Changed 20 months ago by mkoeppe

  • Reviewers changed from Matthias Koeppe, https://github.com/mkoeppe/sage/actions/runs/270864243 to Matthias Koeppe

comment:66 in reply to: ↑ 63 ; follow-up: Changed 20 months ago by soehms

Replying to dimpase:

So it looks as if everything should run just fine - I guess something is wrong with the particular box, sorry.

Dima, one thing common to my platforms is, that I have gap3 installed (from https://webusers.imj-prg.fr/~jean.michel/gap3/ according to the advices given in src/sage/interfaces/gap3.py, and I'm using it inside Sage). Did you test with gap3, as well?

BTW: removing gap 4.8.8 did not help!

comment:67 in reply to: ↑ 66 ; follow-up: Changed 20 months ago by dimpase

Replying to soehms:

Replying to dimpase:

So it looks as if everything should run just fine - I guess something is wrong with the particular box, sorry.

Dima, one thing common to my platforms is, that I have gap3 installed (from https://webusers.imj-prg.fr/~jean.michel/gap3/ according to the advices given in src/sage/interfaces/gap3.py, and I'm using it inside Sage). Did you test with gap3, as well?

I just tried installing gap3 experimental package and running tests, and nothing is broken for me (in Linux Mint container, too).

I don't know the advantage of following the manual instructions in src/sage/interfaces/gap3.py instead -- are there any?

Perhaps they should be marked as obsolete, or just removed. In any event, WARNING about broken pexpect there should go.

Needless to say, if you mangle your Sage installation manually in some way, then all the bets are off, we can't say that Sage has a bug then. :-)

comment:68 Changed 20 months ago by dimpase

for the latter, see #30686.

comment:69 in reply to: ↑ 67 ; follow-up: Changed 20 months ago by soehms

Replying to dimpase:

Replying to soehms: ...

I just tried installing gap3 experimental package and running tests, and nothing is broken for me (in Linux Mint container, too).

Thanks for your help! In the meantime I realized that gap3 isn't the reason for my desaster, either (since against my expectation it was not present on the WSL).

I don't know the advantage of following the manual instructions in src/sage/interfaces/gap3.py instead -- are there any?

I don't know. I did that years ago when I was new to Sage, and probably didn't know it better.

Perhaps they should be marked as obsolete, or just removed. In any event, WARNING about broken pexpect there should go.

Needless to say, if you mangle your Sage installation manually in some way, then all the bets are off, we can't say that Sage has a bug then. :-)

I agree! But that was not my point. I have to fix my facilities until this ticket is released (or completely switch to working in docker containers).

comment:70 in reply to: ↑ 69 ; follow-up: Changed 20 months ago by dimpase

Replying to soehms:

do you have a particular "interesting" hardware for WSL and Mint - is it the same box? Unusual toolchain?

Could you maybe spin a WSL somewhere else and try to reproduce?

I agree! But that was not my point. I have to fix my facilities until this ticket is released (or completely switch to working in docker containers).

any test errors in src/sage/libs/gap/ ? in src/sage/groups/matrix_gps/ ? Or is your problem specifically related to free groups and finitely presented groups?

comment:71 in reply to: ↑ 70 Changed 20 months ago by soehms

Replying to dimpase:

Replying to soehms:

Thanks again for your hints!

do you have a particular "interesting" hardware for WSL and Mint - is it the same box? Unusual toolchain?

The three cases above where all Lenovo ThinkPads (L560 WSL, X240 LinuxMint 19.1, L520 LinuxMint 18, all running on Intel i5). In addition I have now a fourth test case: SonyVaio (on Intel i5, as well, and LinuxMint 17.3).

On the X240 I created a Docker image for the ticket using the $SAGE_ROOT/docker/Dockerfile. The bug does not show in the corresponding container. So, I think the issue has nothing to do with the hardware.

Could you maybe spin a WSL somewhere else and try to reproduce?

I will check that out.

any test errors in src/sage/libs/gap/ ? in src/sage/groups/matrix_gps/ ? Or is your problem specifically related to free groups and finitely presented groups?

Yes:

File "src/sage/libs/gap/element.pyx", line 1335, in sage.libs.gap.element.GapElement.sage
File "src/sage/libs/gap/element.pyx", line 1336, in sage.libs.gap.element.GapElement.sage
File "src/sage/libs/gap/element.pyx", line 1338, in sage.libs.gap.element.GapElement.sage
File "src/sage/libs/gap/element.pyx", line 1347, in sage.libs.gap.element.GapElement.sage
File "src/sage/libs/gap/element.pyx", line 1348, in sage.libs.gap.element.GapElement.sage
File "src/sage/libs/gap/element.pyx", line 1350, in sage.libs.gap.element.GapElement.sage
File "src/sage/libs/gap/libgap.pyx", line 351, in sage.libs.gap.libgap.Gap._construct_matrix

sage -t --warn-long 46.3 --random-seed=0 src/sage/libs/gap/element.pyx  # 6 doctests failed
sage -t --warn-long 46.3 --random-seed=0 src/sage/libs/gap/libgap.pyx  # 1 doctest failed

Under src/sage/groups there are in sum 1182 failing doctests, 78 of them under sage.groups.matrix_gps. Free groups and finitely presented groups are just the top scores. Probably they all have the same origin.

comment:72 follow-up: Changed 20 months ago by dimpase

By the way, do you by any chances have any stray/old patches in build/pkgs/gap/patches or in build/pkgs/gap_packages/patches ?

Certainly, errors in src/sage/libs/gap/element.pyx and src/sage/libs/gap/libgap.pyx are the root of the problem, and should be relatively easy to debug, as it's more or less direct interaction with libgap, and well-localised - everything has to do with dealing with a GAP variable x.

So what happens if you run the stuff around line 1335 from libs/gap/element.pyx?

sage: x = libgap.Integers.Indeterminate("x")
sage: x
sage: x+1
sage: x^2
sage: p = x^2 - 2*x + 3

comment:73 follow-up: Changed 20 months ago by vbraun

  • Status changed from positive_review to needs_work

Still hangs the buildbot

[dochtml] [algebras ] building [inventory]: targets for 95 source files that are out of date
[dochtml] [algebras ] updating environment: [new config] 95 added, 0 changed, 0 removed
[dochtml] [algebras ] The inventory files are in local/share/doc/sage/inventory/en/reference/algebras.
[dochtml] Build finished. The built documents can be found in /var/lib/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/algebras
[dochtml] [tensor_fr] building [inventory]: targets for 19 source files that are out of date
[dochtml] [tensor_fr] updating environment: [new config] 19 added, 0 changed, 0 removed
[dochtml] [tensor_fr] The inventory files are in local/share/doc/sage/inventory/en/reference/tensor_free_modules.
[dochtml] Build finished. The built documents can be found in /var/lib/buildbot/slave/sage_git/build/local/share/doc/sage/inventory/en/reference/tensor_free_modules
[dochtml] Warning: Missing title for sage.combinat.posets.mobile
[dochtml] [combinat ] building [inventory]: targets for 379 source files that are out of date
[dochtml] [combinat ] updating environment: [new config] 379 added, 0 changed, 0 removed
command timed out: 1200 seconds without output running ['make', 'doc-html-mathjax'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=1394.804628

comment:74 Changed 20 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

How about backporting the patch added for Xcode 12 and leaving the gap upgrade to the 9.3 series.

comment:75 Changed 20 months ago by mkoeppe

Created #30720 for this

comment:76 in reply to: ↑ 72 ; follow-up: Changed 20 months ago by soehms

Replying to dimpase:

By the way, do you by any chances have any stray/old patches in build/pkgs/gap/patches or in build/pkgs/gap_packages/patches ?

No!

Certainly, errors in src/sage/libs/gap/element.pyx and src/sage/libs/gap/libgap.pyx are the root of the problem, and should be relatively easy to debug, as it's more or less direct interaction with libgap, and well-localised - everything has to do with dealing with a GAP variable x.

So what happens if you run the stuff around line 1335 from libs/gap/element.pyx?

sage: x = libgap.Integers.Indeterminate("x")
sage: x
sage: x+1
sage: x^2
sage: p = x^2 - 2*x + 3

The crash already occured at the addition x+1, more precisely in local/lib/python3.8/site-packages/cysignals/macros.h. It jumped to line 134 even though cysigs.sig_on_count was zero.

(gdb) n
__pyx_f_4sage_4libs_3gap_7element_10GapElement__add_ (__pyx_v_self=0x7f5e227f1840, __pyx_v_right=0x7f5e2281eb00, __pyx_skip_dispatch=<optimized out>)
    at build/cythonized/sage/libs/gap/element.c:10206
10206       __pyx_t_5 = sig_on(); if (unlikely(__pyx_t_5 == ((int)0))) __PYX_ERR(0, 1007, __pyx_L4_error)
(gdb) s
_sig_on_prejmp (message=0x0, file=0x7f5e31f0c728 "build/cythonized/sage/libs/gap/element.c", line=10206)
    at /home/sebastian/develop/sage/local/lib/python3.8/site-packages/cysignals/macros.h:117
117         cysigs.s = message;
(gdb) s
132         if (cysigs.sig_on_count > 0)
(gdb) p cysigs.sig_on_count
$2 = 0
(gdb) l
127             fprintf(stderr, "\n*** ERROR *** sig_on() with sig_on_count = %i, block_sigint = %i\n",
128                     (int)cysigs.sig_on_count, (int)cysigs.block_sigint);
129             print_backtrace();
130         }
131     #endif
132         if (cysigs.sig_on_count > 0)
133         {
134             cysigs.sig_on_count++;
135             return 1;
136         }
(gdb) s
134             cysigs.sig_on_count++;
(gdb) p cysigs.sig_on_count
$3 = 0

Therefore, I ran

sage -f cysignals
make build

and that solved the problem!

But, why did I have to do that even in the case where I build fro scratch? Anyway, thanks a lot for your patience!

comment:77 in reply to: ↑ 76 ; follow-up: Changed 20 months ago by dimpase

Replying to soehms: [...]

Therefore, I ran

sage -f cysignals
make build

and that solved the problem!

Great!

But, why did I have to do that even in the case where I build fro scratch? Anyway, thanks a lot for your patience!

Could be #30687 (with the original suject: "local/lib/python3*" must be removed) - unless you did run make distclean which removes local/ completely anyway...

comment:78 in reply to: ↑ 73 Changed 20 months ago by dimpase

Replying to vbraun:

Still hangs the buildbot

Someone has to debug this somehow, I have no clue what's going on there. Could you do another strace -ff ... as in comment:29 ?

comment:79 Changed 20 months ago by mkoeppe

  • Work issues set to rebase on top of #30720

comment:80 Changed 20 months ago by git

  • Commit changed from 2ffc24af85679499aff4846b1f135f8cea22b8c0 to 649bc8defe5c4f58da7582b1f7bfa10f1c3f793e

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

2033e4alibsemigroups update
540d761patches from Arch (cf #29314)
98267a5tagged #random tests with output depending upon gap_packages installed
aec0937one more doctest fix
8a2251emore robust test, so that GAP's packages don't matter
d2fb822increase memory pool for libgap
22c7420build/pkgs/gap_packages: Use symlinks to refer to gap's tarball
b54ec20version bump
3b22dd9remove old workaround from #22626
649bc8dthis file should exist - leave just comments there

comment:81 Changed 20 months ago by dimpase

  • Dependencies changed from #30496 to #30720
  • Status changed from needs_work to needs_review
  • Work issues rebase on top of #30720 deleted

rebased over #30720

comment:82 Changed 20 months ago by jhpalmieri

The patches for gap_packages don't work:

Applying patches from ../patches...
Applying ../patches/cohomolo-gcc10.patch
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/pkg/cohomolo-1.6.7/standalone/progs.d/crp1.c b/pkg/cohomolo-1.6.7/standalone/progs.d/crp1.c
|index 3bbdc45..7df699f 100644
|--- a/pkg/cohomolo-1.6.7/standalone/progs.d/crp1.c
|+++ b/pkg/cohomolo-1.6.7/standalone/progs.d/crp1.c
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
4 out of 4 hunks ignored
can't find file to patch at input line 54
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/pkg/cohomolo-1.6.7/standalone/progs.d/nq+chfns.c b/pkg/cohomolo-1.6.7/standalone/progs.d/nq+chfns.c
|index a7396b2..658496f 100644
|--- a/pkg/cohomolo-1.6.7/standalone/progs.d/nq+chfns.c
|+++ b/pkg/cohomolo-1.6.7/standalone/progs.d/nq+chfns.c
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
4 out of 4 hunks ignored
can't find file to patch at input line 98
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/pkg/cohomolo-1.6.7/standalone/progs.d/nqmfns.c b/pkg/cohomolo-1.6.7/standalone/progs.d/nqmfns.c
|index 0896551..6841bc8 100644
|--- a/pkg/cohomolo-1.6.7/standalone/progs.d/nqmfns.c
|+++ b/pkg/cohomolo-1.6.7/standalone/progs.d/nqmfns.c
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
4 out of 4 hunks ignored
can't find file to patch at input line 135
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/pkg/cohomolo-1.6.7/standalone/progs.d/nqmp.c b/pkg/cohomolo-1.6.7/standalone/progs.d/nqmp.c
|index 01cf914..0144883 100644
|--- a/pkg/cohomolo-1.6.7/standalone/progs.d/nqmp.c
|+++ b/pkg/cohomolo-1.6.7/standalone/progs.d/nqmp.c
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
2 out of 2 hunks ignored
Error applying '../patches/cohomolo-gcc10.patch'
************************************************************************
Error applying patches
************************************************************************

comment:83 Changed 20 months ago by jhpalmieri

I see: the versions are wrong. Here's a fix.

comment:84 Changed 20 months ago by jhpalmieri

  • Branch changed from u/dimpase/packages/gap411 to u/jhpalmieri/packages/gap411

comment:85 follow-up: Changed 20 months ago by jhpalmieri

  • Commit changed from 649bc8defe5c4f58da7582b1f7bfa10f1c3f793e to 47fcc169297fdf7cc7b58c8f454255bb29f955c5

This (including libsemigroups and gap_packages) builds for me on OS X with Xcode 12.


New commits:

47fcc16trac 29314: fix version numbers in patches for gap_packages

comment:86 Changed 20 months ago by jhpalmieri

Building the documentation, I see

Error, reached the pre-set memory limit
(change it with the -o command line option)

I don't remember ever seeing this before. (The documentation build continues, but the error message was strange.)

comment:87 follow-up: Changed 20 months ago by mkoeppe

Then patching the failing packages in #30729 using the diff to GAP 4.11 would be a way forward

comment:88 Changed 20 months ago by jhpalmieri

Doctests pass for me (with libsemigroups and gap_packages installed).

comment:89 in reply to: ↑ 87 Changed 20 months ago by jhpalmieri

Replying to mkoeppe:

Then patching the failing packages in #30729 using the diff to GAP 4.11 would be a way forward

Or just merge this ticket instead.

Last edited 20 months ago by jhpalmieri (previous) (diff)

comment:90 follow-up: Changed 20 months ago by mkoeppe

There are unresolved problems with this upgrade

comment:91 in reply to: ↑ 85 ; follow-up: Changed 20 months ago by dimpase

Replying to jhpalmieri:

This (including libsemigroups and gap_packages) builds for me on OS X with Xcode 12.

thanks for fixing this. What XCode 12 are you on? 12.0? 12.2?

comment:92 in reply to: ↑ 90 Changed 20 months ago by dimpase

Replying to mkoeppe:

There are unresolved problems with this upgrade

and a problem I cannot debug, as it's essentially a privately run network of buildbots which (in part?) seem to hang on our damn docbuild :-(

comment:93 Changed 20 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:94 Changed 20 months ago by mkoeppe

It does not seem realistic to get this ticket done for 9.2. So let's do #30729

comment:95 in reply to: ↑ 91 Changed 20 months ago by jhpalmieri

Replying to dimpase:

Replying to jhpalmieri:

This (including libsemigroups and gap_packages) builds for me on OS X with Xcode 12.

thanks for fixing this. What XCode 12 are you on? 12.0? 12.2?

12.0.1

comment:96 in reply to: ↑ 77 Changed 20 months ago by soehms

Replying to dimpase:

Replying to soehms: [...]

Therefore, I ran

sage -f cysignals
make build

and that solved the problem!

Great!

But, why did I have to do that even in the case where I build fro scratch? Anyway, thanks a lot for your patience!

Could be #30687 (with the original suject: "local/lib/python3*" must be removed) - unless you did run make distclean which removes local/ completely anyway...

It must be an independent problem. As it appears now on all of my systems (by whatever reason) I have to rebuild cysignals whenever I switch back and forth between gap 4.10 and 4.11. This is regardless of whether I merged #30687 in or not.

comment:97 Changed 20 months ago by dimpase

perhaps buildbot hanging is a multithreading issue - libgap in not multithreaded, calling it from several threads is asking for trouble: https://github.com/gap-system/gap/issues/2893#issuecomment-707700408

comment:98 follow-up: Changed 19 months ago by dimpase

we should move PackageManager GAP package from gap_packages to gap - it is small and useful.

comment:99 Changed 19 months ago by dimpase

  • Status changed from needs_work to needs_review

The ball here is in Volker's court. I have no idea why his buildbots hang.

comment:100 Changed 19 months ago by mkoeppe

  • Reviewers changed from Matthias Koeppe to Matthias Koeppe, ...

comment:101 in reply to: ↑ 98 ; follow-up: Changed 19 months ago by slelievre

Replying to dimpase:

we should move PackageManager GAP package from gap_packages to gap - it is small and useful.

+1 -- separate ticket or here?

comment:102 in reply to: ↑ 101 Changed 19 months ago by dimpase

Replying to slelievre:

Replying to dimpase:

we should move PackageManager GAP package from gap_packages to gap - it is small and useful.

+1 -- separate ticket or here?

I'll do it here - it's still "needs review" after all.

comment:103 Changed 19 months ago by dimpase

  • Branch changed from u/jhpalmieri/packages/gap411 to u/dimpase/packages/gap411
  • Commit changed from 47fcc169297fdf7cc7b58c8f454255bb29f955c5 to 9836f794b7bf90f9c3b6c1b2928f032f4a7bd093
  • Dependencies #30720 deleted

comment:104 Changed 19 months ago by dimpase

rebased over the latest beta, moved PackageManager GAP package as discussed.

comment:105 follow-up: Changed 18 months ago by gh-tobiasdiez

  • Cc gh-tobiasdiez added
  • Reviewers changed from Matthias Koeppe, ... to Matthias Koeppe, Tobias Diez, ...

I've used this ticket in #30371, and it runs well on the github action (https://github.com/tobiasdiez/sage/runs/1413136300?check_suite_focus=true). In particular, there was no "hanging" or multithreading problem.

The only gap related doc tests that fail are:

2020-11-17T17:27:35.4675866Z sage -t --random-seed=0 /__w/sage/sage/src/sage/interfaces/gap_workspace.py
2020-11-17T17:27:35.4676910Z **********************************************************************
2020-11-17T17:27:35.4677821Z File "/__w/sage/sage/src/sage/interfaces/gap_workspace.py", line 55, in sage.interfaces.gap_workspace.gap_workspace_file
2020-11-17T17:27:35.4681873Z Failed example:
2020-11-17T17:27:35.4683784Z     name1 = Popen([sys.executable, '-c', cmd], stdout=PIPE).communicate()[0]
2020-11-17T17:27:35.4684739Z Expected nothing
2020-11-17T17:27:35.4685235Z Got:
2020-11-17T17:27:35.4686344Z     /__w/sage/sage/src/sage/calculus/functional.py:57: DeprecationWarning: invalid escape sequence \d
2020-11-17T17:27:35.4687468Z       """
2020-11-17T17:27:39.4060681Z **********************************************************************
2020-11-17T17:27:39.4062343Z File "/__w/sage/sage/src/sage/interfaces/gap_workspace.py", line 56, in sage.interfaces.gap_workspace.gap_workspace_file
2020-11-17T17:27:39.4063371Z Failed example:
2020-11-17T17:27:39.4065030Z     name2 = Popen([sys.executable, '-c', cmd], stdout=PIPE).communicate()[0]
2020-11-17T17:27:39.4065813Z Expected nothing
2020-11-17T17:27:39.4066315Z Got:
2020-11-17T17:27:39.4067147Z     /__w/sage/sage/src/sage/calculus/functional.py:57: DeprecationWarning: invalid escape sequence \d
2020-11-17T17:27:39.4067993Z       """
2020-11-17T17:27:39.4129689Z **********************************************************************

But that could also very well be a result of the somewhat unusual setup of #30371. So from my side this looks good.

comment:106 in reply to: ↑ 105 Changed 18 months ago by dimpase

Replying to gh-tobiasdiez:

I've used this ticket in #30371, and it runs well on the github action (https://github.com/tobiasdiez/sage/runs/1413136300?check_suite_focus=true). In particular, there was no "hanging" or multithreading problem.

The only gap related doc tests that fail are:

Thanks. I can't reproduce these, they don't seem to originate from this ticket.

comment:107 Changed 18 months ago by mkoeppe

  • Branch changed from u/dimpase/packages/gap411 to u/mkoeppe/packages/gap411

comment:108 Changed 18 months ago by mkoeppe

  • Commit changed from 9836f794b7bf90f9c3b6c1b2928f032f4a7bd093 to a28138c2fc1396171734b12fdbe5904b79bf1ec1
  • Reviewers changed from Matthias Koeppe, Tobias Diez, ... to Matthias Koeppe, Tobias Diez, https://github.com/mkoeppe/sage/actions/runs/409420323

New commits:

a28138cMerge tag '9.3.beta3' into t/29314/packages/gap411

comment:109 Changed 18 months ago by mkoeppe

  • Reviewers changed from Matthias Koeppe, Tobias Diez, https://github.com/mkoeppe/sage/actions/runs/409420323 to Matthias Koeppe, Tobias Diez
  • Status changed from needs_review to positive_review

Looks all fine on GH Actions. (comment:25 of course still applies)

comment:110 Changed 18 months ago by vbraun

  • Branch changed from u/mkoeppe/packages/gap411 to a28138c2fc1396171734b12fdbe5904b79bf1ec1
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.