id,summary,reporter,owner,description,type,status,priority,milestone,component,resolution,keywords,cc,merged,author,reviewer,upstream,work_issues,branch,commit,dependencies,stopgaps
26083,Race condition when upgrading python,embray,,"I've encountered a race condition a couple times when doing a build on Cygwin where, for one reason or other, the python2 package gets upgraded (or python3 if `--with-python=3`). The error occurs in another package that is being built at the same time, I think when it tries to run the `sage-flock` command (which invokes `/usr/bin/env python`). That's also why I've only seen this on Cygwin--it's only on Cygwin that `sage-flock` is being invoked at the end of each package install.
**Update:** I have also now seen this on Linux too. It can also happen because `sage-spkg-uninstall` and `sage-download-file` use Python, among others. ''Any program'' use by `sage-spkg` that uses Python can break if it's called in the middle of upgrading or installing the Python in Sage.
The error ends up looking something like this:
{{{
[python2-2.7.15.p0] Copying package files from temporary location /home/embray/src/sagemath/sage/local/var/tmp/sage/build/python2-2.7.15.p0/inst to /home/embray/src/sagemath/sage/local
[ppl-1.2.p1] mkdir -p -- /home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst/home/embray/src/sagemath/sage/local/share/doc/ppl/ppl-user-c-interface-1.2-html
[ppl-1.2.p1] make[6]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/doc'
[ppl-1.2.p1] make[5]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/doc'
[ppl-1.2.p1] Making install in m4
[ppl-1.2.p1] make[5]: Entering directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4'
[ppl-1.2.p1] make[6]: Entering directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4'
[ppl-1.2.p1] make[6]: Nothing to be done for 'install-exec-am'.
[ppl-1.2.p1] /usr/bin/mkdir -p '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst/home/embray/src/sagemath/sage/local/share/aclocal'
[ppl-1.2.p1] /usr/bin/install -c -m 644 ppl.m4 ppl_c.m4 '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst/home/embray/src/sagemath/sage/local/share/aclocal'
[ppl-1.2.p1] make[6]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4'
[ppl-1.2.p1] make[5]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src/m4'
[ppl-1.2.p1] make[4]: Leaving directory '/home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/src'
[ppl-1.2.p1]
[ppl-1.2.p1] real 14m49.948s
[ppl-1.2.p1] user 11m49.638s
[ppl-1.2.p1] sys 6m34.550s
[ppl-1.2.p1] Copying package files from temporary location /home/embray/src/sagemath/sage/local/var/tmp/sage/build/ppl-1.2.p1/inst to /home/embray/src/sagemath/sage/local
[ppl-1.2.p1] Could not find platform independent libraries
[ppl-1.2.p1] Consider setting $PYTHONHOME to [:]
[ppl-1.2.p1] ImportError: No module named site
[ppl-1.2.p1] Waiting for rebase lock
[ppl-1.2.p1] Successfully installed ppl-1.2.p1
}}}
It says ""Successfully installed"" but then make exits with an error shortly thereafter.
The problem is that although ppl itself does not have a dependency on Python, `sage-spkg` itself does. But the python2 files are still in the process of being copied over. The `bin/python` has been copied, but not all of the stdlib, so that `bin/python` is broken.
I believe there are a few things that should be done about this. First of all, putting a lock around all modifications to `$SAGE_LOCAL`, as discussed in https://trac.sagemath.org/ticket/26018#comment:10 might help.
But I've mentioned in the context of other tickets that the build system itself should always stick to using the system python explicitly, and that Sage's Python should only be invoked explicitly in specific `spkg-installs` that need to build against the Sage Python.",defect,closed,major,sage-duplicate/invalid/wontfix,build,duplicate,,jdemeyer,,,,N/A,,,,,