#26083 closed defect (duplicate)
Race condition when upgrading python
Reported by: | embray | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | build | Keywords: | |
Cc: | jdemeyer | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
I've encountered a race condition a couple times when doing a build on Cygwin where, for one reason or other, the python2 package gets upgraded (or python3 if --with-python=3
). The error occurs in another package that is being built at the same time, I think when it tries to run the sage-flock
command (which invokes /usr/bin/env python
). That's also why I've only seen this on Cygwin--it's only on Cygwin that sage-flock
is being invoked at the end of each package install.
Update: I have also now seen this on Linux too. It can also happen because sage-spkg-uninstall
and sage-download-file
use Python, among others. Any program use by sage-spkg
that uses Python can break if it's called in the middle of upgrading or installing the Python in Sage.
The error ends up looking something like this:
[python2-2.7.15.p0] Copying package files from temporary location /home/embray/src/sagemath/sage/local/var/tmp/sage/build/python2-2.7.15.p0/inst to /home/embray/src/sagemath/sage/local [ppl-1.2.p1] mkdir -p -- /home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst/home/embray/src/sagemath/sage/local/share/doc/ppl/ppl-user-c-interface-1.2-html [ppl-1.2.p1] make[6]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/doc' [ppl-1.2.p1] make[5]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/doc' [ppl-1.2.p1] Making install in m4 [ppl-1.2.p1] make[5]: Entering directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4' [ppl-1.2.p1] make[6]: Entering directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4' [ppl-1.2.p1] make[6]: Nothing to be done for 'install-exec-am'. [ppl-1.2.p1] /usr/bin/mkdir -p '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst/home/embray/src/sagemath/sage/local/share/aclocal' [ppl-1.2.p1] /usr/bin/install -c -m 644 ppl.m4 ppl_c.m4 '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst/home/embray/src/sagemath/sage/local/share/aclocal' [ppl-1.2.p1] make[6]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4' [ppl-1.2.p1] make[5]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4' [ppl-1.2.p1] make[4]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src' [ppl-1.2.p1] [ppl-1.2.p1] real 14m49.948s [ppl-1.2.p1] user 11m49.638s [ppl-1.2.p1] sys 6m34.550s [ppl-1.2.p1] Copying package files from temporary location /home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst to /home/embray/src/sagemath/sage/local [ppl-1.2.p1] Could not find platform independent libraries <prefix> [ppl-1.2.p1] Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] [ppl-1.2.p1] ImportError: No module named site [ppl-1.2.p1] Waiting for rebase lock [ppl-1.2.p1] Successfully installed ppl-1.2.p1
It says "Successfully installed" but then make exits with an error shortly thereafter.
The problem is that although ppl itself does not have a dependency on Python, sage-spkg
itself does. But the python2 files are still in the process of being copied over. The bin/python
has been copied, but not all of the stdlib, so that bin/python
is broken.
I believe there are a few things that should be done about this. First of all, putting a lock around all modifications to $SAGE_LOCAL
, as discussed in https://trac.sagemath.org/ticket/26018#comment:10 might help.
But I've mentioned in the context of other tickets that the build system itself should always stick to using the system python explicitly, and that Sage's Python should only be invoked explicitly in specific spkg-installs
that need to build against the Sage Python.
Change History (7)
comment:1 Changed 3 years ago by
- Cc jdemeyer added
comment:2 Changed 3 years ago by
- Description modified (diff)
- Summary changed from Race condition on Cygwin when upgrading python to Race condition when upgrading python
comment:3 Changed 3 years ago by
Duplicate of #18438?
comment:4 Changed 3 years ago by
I'll have to check out Volker's branch there. Something like sage-system-python
is one possible solution I considered (on that I'm still not entirely happy with but it's better than nothing).
comment:5 Changed 3 years ago by
I'm seeing this rather frequently now. The race was there for a long time but its getting more likely to hit.
comment:6 Changed 3 years ago by
- Milestone changed from sage-8.4 to sage-duplicate/invalid/wontfix
- Resolution set to duplicate
- Status changed from new to closed
comment:7 Changed 3 years ago by
IF we're going to close this ticket (which is fine) the milestone / priority on the existing ticket should also be updated, and maybe link to this ticket which I believe adds more context.
Opinions?