Opened 7 years ago
Closed 7 years ago
#14323 closed defect (fixed)
libGAP messes with Python subprocesses
Reported by: | jdemeyer | Owned by: | was |
---|---|---|---|
Priority: | blocker | Milestone: | sage-5.9 |
Component: | interfaces | Keywords: | child processes |
Cc: | vbraun, leif, roed | Merged in: | sage-5.9.beta2 |
Authors: | Volker Braun | Reviewers: | Jeroen Demeyer |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
I haven't figured out when/why, but this might happen:
sage -t devel/sage/sage/homology/simplicial_complex.py ********************************************************************** File "devel/sage/sage/homology/simplicial_complex.py", line 3134, in sage.homology.simplicial_complex.SimplicialComplex.automorphism_group Failed example: S.automorphism_group().is_isomorphic(SymmetricGroup(4)) Exception raised: Traceback (most recent call last): File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 455, in _run self.execute(example, compiled, test.globs) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 811, in execute exec compiled in globs File "<doctest sage.homology.simplicial_complex.SimplicialComplex.automorphism_group[1]>", line 1, in <module> S.automorphism_group().is_isomorphic(SymmetricGroup(Integer(4))) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/homology/simplicial_complex.py", line 3155, in automorphism_group [f.tuple() for f in self.facets()]]) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/graphs/generic_graph.py", line 16414, in automorphism_group output.append(PermutationGroup([perm_group_elt(aa) for aa in a])) File "refinement_graphs.pyx", line 941, in sage.groups.perm_gps.partn_ref.refinement_graphs.perm_group_elt (sage/groups/perm_gps/partn_ref/refinement_graphs.c:32796) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup.py", line 653, in __call__ return self._element_class()(x, self, check=check) File "permgroup_element.pyx", line 452, in sage.groups.perm_gps.permgroup_element.PermutationGroupElement.__init__ (sage/groups/perm_gps/permgroup_element.c:4490) File "sage_object.pyx", line 474, in sage.structure.sage_object.SageObject._gap_ (sage/structure/sage_object.c:4544) File "sage_object.pyx", line 450, in sage.structure.sage_object.SageObject._interface_ (sage/structure/sage_object.c:4144) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 199, in __call__ return cls(self, x, name=name) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/expect.py", line 1280, in __init__ self._name = parent._create(value, name=name) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 389, in _create self.set(name, value) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1322, in set out = self._eval_line(cmd, allow_use_file=True) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 730, in _eval_line self._start() File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1170, in _start gap_reset_workspace(verbose=False) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1496, in gap_reset_workspace g = Gap(use_workspace_cache=False, max_workspace_size=None) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1084, in __init__ max_workspace_size = _get_gap_memory_pool_size_MB() File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 292, in _get_gap_memory_pool_size_MB pool = get_gap_memory_pool_size() File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 268, in get_gap_memory_pool_size suggested_size = min(suggested_size, int(mem.virtual_memory_limit()/10)) File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/misc/memory_info.py", line 131, in virtual_memory_limit if platform.architecture()[0] == '32bit': File "/padic/release/merger/sage-5.9.beta0/local/lib/python/platform.py", line 1084, in architecture output = _syscmd_file(executable, '') File "/padic/release/merger/sage-5.9.beta0/local/lib/python/platform.py", line 1032, in _syscmd_file rc = f.close() IOError: [Errno 10] No child processes **********************************************************************
Looking at a strace
, this is because libGAP installs a SIGCHLD
signal handler which interferes with Python's multiprocessing.
spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/libgap-4.5.7.p2.spkg
(signal handler fix only, see https://bitbucket.org/vbraun/libgap/commits/3f4ebd64fc1219858802173cd86b7910040fef36)
Change History (19)
comment:1 Changed 7 years ago by
- Summary changed from libGAP messes with Python multiprocessing to libGAP messes with Python subprocesses
comment:2 Changed 7 years ago by
comment:3 Changed 7 years ago by
- Cc leif added
- Keywords child processes added
comment:4 follow-up: ↓ 5 Changed 7 years ago by
- Milestone changed from sage-5.9 to sage-duplicate/invalid/wontfix
- Status changed from new to needs_review
I removed the SIGCHLD handler already, you just need to review #14039.
Close this ticket as duplicate.
comment:5 in reply to: ↑ 4 ; follow-up: ↓ 6 Changed 7 years ago by
- Milestone changed from sage-duplicate/invalid/wontfix to sage-5.9
- Status changed from needs_review to needs_work
comment:6 in reply to: ↑ 5 Changed 7 years ago by
Replying to jdemeyer:
Not really. I'd like to fix this as soon as possible, which means without waiting for #14187+#14014+#14039.
Well tough, I would like those tickets reviewed as soon as possible because its an immense maintenance burden for me to carry around a patch that rebases all matrix groups.
comment:7 Changed 7 years ago by
- Description modified (diff)
- Reviewers set to Jeroen Demeyer
- Status changed from needs_work to positive_review
comment:8 Changed 7 years ago by
- Status changed from positive_review to needs_work
- Work issues set to SPKG.txt
Volker: you are missing an entry in SPKG.txt
for this version. Should I add it manually or will you do it using your scripts?
comment:9 follow-up: ↓ 10 Changed 7 years ago by
I've added the SPKG entry in http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.6.2.p0.spkg
comment:10 in reply to: ↑ 9 Changed 7 years ago by
Replying to vbraun:
I've added the SPKG entry in http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.6.2.p0.spkg
...which is the wrong version. This ticket is about libgap-4.5.7.p2.
comment:11 Changed 7 years ago by
Then I suggest we close this one as superseded by #14039.
comment:12 Changed 7 years ago by
Volker, I don't understand what you're trying to accomplish here. Acting like you're doing now is not helping. As I already said, this is an important issue which cannot wait for #14039.
I can totally understand your frustration that nobody is reviewing #14014, but this ticket shouldn't suffer from that. If you want reviewers for #14014, you could either ask David Roe (which is listed as reviewer) or ask sage-devel.
comment:13 follow-up: ↓ 14 Changed 7 years ago by
I don't understand why it is an issue to fix this blocker before sage-5.9 is released. We are currently at beta0, so it seems there will be at least 1-2 months until this needs to be fixed.
comment:14 in reply to: ↑ 13 Changed 7 years ago by
Sorry, wanted to say: I don't understand why it is an issue to fix this blocker long before sage-5.9 is released. We are currently at beta0, so it seems there will be at least 1-2 months until this needs to be fixed. If #14039 isn't reviewed by rc0 we can talk.
comment:15 follow-up: ↓ 16 Changed 7 years ago by
Why wait for rc0 to fix this? We know the problem and we know the fix.
Imagine we don't fix this before sage-5.9.rc0. The problem on this ticket is causing random strange failures, so for every multiprocessing-related problem that people report, we would need to check whether this ticket fixes it.
comment:16 in reply to: ↑ 15 Changed 7 years ago by
- Cc roed added
Replying to jdemeyer:
Why wait for rc0 to fix this? We know the problem and we know the fix.
And the fix is at #14039.
Imagine we don't fix this before sage-5.9.rc0. The problem on this ticket is causing random strange failures, so for every multiprocessing-related problem that people report, we would need to check whether this ticket fixes it.
I'll be happy to tell every one of them that this is their own fault for not reviewing #14039.
Maybe David can tell us how long it'll take...
comment:17 Changed 7 years ago by
- Description modified (diff)
- Status changed from needs_work to positive_review
- Work issues SPKG.txt deleted
Fixed SPKG.txt
.
comment:18 Changed 7 years ago by
FWIW, I've built and tested Sage 5.9.beta0 from scratch with this spkg without issues, but I didn't have problems before either. (Ubuntu 10.04.4 LTS x86_64)
comment:19 Changed 7 years ago by
- Merged in set to sage-5.9.beta2
- Resolution set to fixed
- Status changed from positive_review to closed
Does Sage support libGAP child processes? Otherwise, that feature could simply be removed from libGAP.