libGAP messes with Python subprocesses

[jdemeyer]

I haven't figured out when/why, but this might happen:

sage -t devel/sage/sage/homology/simplicial_complex.py
**********************************************************************
File "devel/sage/sage/homology/simplicial_complex.py", line 3134, in sage.homology.simplicial_complex.SimplicialComplex.automorphism_group
Failed example:
    S.automorphism_group().is_isomorphic(SymmetricGroup(4))
Exception raised:
    Traceback (most recent call last):
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 455, in _run
        self.execute(example, compiled, test.globs)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 811, in execute
        exec compiled in globs
      File "<doctest sage.homology.simplicial_complex.SimplicialComplex.automorphism_group[1]>", line 1, in <module>
        S.automorphism_group().is_isomorphic(SymmetricGroup(Integer(4)))
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/homology/simplicial_complex.py", line 3155, in automorphism_group
        [f.tuple() for f in self.facets()]])
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/graphs/generic_graph.py", line 16414, in automorphism_group
        output.append(PermutationGroup([perm_group_elt(aa) for aa in a]))
      File "refinement_graphs.pyx", line 941, in sage.groups.perm_gps.partn_ref.refinement_graphs.perm_group_elt (sage/groups/perm_gps/partn_ref/refinement_graphs.c:32796)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup.py", line 653, in __call__
        return self._element_class()(x, self, check=check)
      File "permgroup_element.pyx", line 452, in sage.groups.perm_gps.permgroup_element.PermutationGroupElement.__init__ (sage/groups/perm_gps/permgroup_element.c:4490)
      File "sage_object.pyx", line 474, in sage.structure.sage_object.SageObject._gap_ (sage/structure/sage_object.c:4544)
      File "sage_object.pyx", line 450, in sage.structure.sage_object.SageObject._interface_ (sage/structure/sage_object.c:4144)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 199, in __call__
        return cls(self, x, name=name)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/expect.py", line 1280, in __init__
        self._name = parent._create(value, name=name)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 389, in _create
        self.set(name, value)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1322, in set
        out = self._eval_line(cmd, allow_use_file=True)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 730, in _eval_line
        self._start()
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1170, in _start
        gap_reset_workspace(verbose=False)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1496, in gap_reset_workspace
        g = Gap(use_workspace_cache=False, max_workspace_size=None)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1084, in __init__
        max_workspace_size = _get_gap_memory_pool_size_MB()
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 292, in _get_gap_memory_pool_size_MB
        pool = get_gap_memory_pool_size()
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 268, in get_gap_memory_pool_size
        suggested_size = min(suggested_size, int(mem.virtual_memory_limit()/10))
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/misc/memory_info.py", line 131, in virtual_memory_limit
        if platform.architecture()[0] == '32bit':
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python/platform.py", line 1084, in architecture
        output = _syscmd_file(executable, '')
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python/platform.py", line 1032, in _syscmd_file
        rc = f.close()
    IOError: [Errno 10] No child processes
**********************************************************************

Looking at a strace, this is because libGAP installs a SIGCHLD signal handler which interferes with Python's multiprocessing.

spkg: ​http://boxen.math.washington.edu/home/jdemeyer/spkg/libgap-4.5.7.p2.spkg

(signal handler fix only, see ​https://bitbucket.org/vbraun/libgap/commits/3f4ebd64fc1219858802173cd86b7910040fef36)

[jdemeyer]

Does Sage support libGAP child processes? Otherwise, that feature could simply be removed from libGAP.

[vbraun]

I removed the SIGCHLD handler already, you just need to review #14039.

Close this ticket as duplicate.

[jdemeyer]

Replying to vbraun:

Close this ticket as duplicate.

Not really. I'd like to fix this as soon as possible, which means without waiting for #14187+#14014+#14039.

[vbraun]

Replying to jdemeyer:

Not really. I'd like to fix this as soon as possible, which means without waiting for #14187+#14014+#14039.

Well tough, I would like those tickets reviewed as soon as possible because its an immense maintenance burden for me to carry around a patch that rebases all matrix groups.

[jdemeyer]

Volker: you are missing an entry in SPKG.txt for this version. Should I add it manually or will you do it using your scripts?

[vbraun]

I've added the SPKG entry in ​http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.6.2.p0.spkg

[jdemeyer]

Replying to vbraun:

I've added the SPKG entry in ​http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.6.2.p0.spkg

...which is the wrong version. This ticket is about libgap-4.5.7.p2.

[vbraun]

Then I suggest we close this one as superseded by #14039.

[jdemeyer]

Volker, I don't understand what you're trying to accomplish here. Acting like you're doing now is not helping. As I already said, this is an important issue which cannot wait for #14039.

I can totally understand your frustration that nobody is reviewing #14014, but this ticket shouldn't suffer from that. If you want reviewers for #14014, you could either ask David Roe (which is listed as reviewer) or ask sage-devel.

[vbraun]

I don't understand why it is an issue to fix this blocker before sage-5.9 is released. We are currently at beta0, so it seems there will be at least 1-2 months until this needs to be fixed.

[vbraun]

Sorry, wanted to say: I don't understand why it is an issue to fix this blocker long before sage-5.9 is released. We are currently at beta0, so it seems there will be at least 1-2 months until this needs to be fixed. If #14039 isn't reviewed by rc0 we can talk.

[jdemeyer]

Why wait for rc0 to fix this? We know the problem and we know the fix.

Imagine we don't fix this before sage-5.9.rc0. The problem on this ticket is causing random strange failures, so for every multiprocessing-related problem that people report, we would need to check whether this ticket fixes it.

[vbraun]

Replying to jdemeyer:

Why wait for rc0 to fix this? We know the problem and we know the fix.

And the fix is at #14039.

Imagine we don't fix this before sage-5.9.rc0. The problem on this ticket is causing random strange failures, so for every multiprocessing-related problem that people report, we would need to check whether this ticket fixes it.

I'll be happy to tell every one of them that this is their own fault for not reviewing #14039.

Maybe David can tell us how long it'll take...

[jdemeyer]

Fixed SPKG.txt.

[leif]

FWIW, I've built and tested Sage 5.9.beta0 from scratch with this spkg without issues, but I didn't have problems before either. (Ubuntu 10.04.4 LTS x86_64)