Opened 7 months ago

Closed 7 months ago

#26990 closed defect (duplicate)

Race when copying package files

Reported by: vbraun Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: build Keywords:
Cc: embray Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by vbraun)

There seems to be a race when copying files as cp is not atomic. I'm not sure if the copying holds a lock. Either the copying or the *.pyc file generation by the Python interpreter in another process conflicted here:

[future-0.17.1] Successfully installed future-0.17.1
[future-0.17.1] Cleaning up...
[future-0.17.1] Removed build tracker '/tmp/pip-req-tracker-ruDJcT'
[future-0.17.1] 
[future-0.17.1] real	0m4.907s
[future-0.17.1] user	0m2.226s
[future-0.17.1] sys	0m0.382s
[future-0.17.1] Copying package files from temporary location /home/chapoton/sage/local/var/tmp/sage/build/future-0.17.1/inst to /home/chapoton/sage/local
[future-0.17.1] cp: cannot create regular file '/home/chapoton/sage/local/./lib/python2.7/site-packages/future/__init__.pyc': File exists
[future-0.17.1] ************************************************************************
[future-0.17.1] Error copying files for future-0.17.1.
[future-0.17.1] ************************************************************************
[future-0.17.1] Please email sage-devel (http://groups.google.com/group/sage-devel)
[future-0.17.1] explaining the problem and including the log file
[future-0.17.1]   /home/chapoton/sage/logs/pkgs/future-0.17.1.log
[future-0.17.1] Describe your computer, operating system, etc.
[future-0.17.1] ************************************************************************

See also: https://unix.stackexchange.com/questions/116280/cannot-create-regular-file-filename-file-exists

Change History (8)

comment:1 Changed 7 months ago by vbraun

  • Description modified (diff)

comment:2 Changed 7 months ago by embray

  • Milestone changed from sage-8.6 to sage-duplicate/invalid/wontfix
  • Resolution set to duplicate
  • Status changed from new to closed

This is basically a duplicate of #26018, particularly comment:4:ticket:26018, though I'm not positive if the branch there fixes it for all cases.

comment:3 Changed 7 months ago by embray

Actually I'm not sure this is #26018, but it still looks like a fluke. How could you get:

Error copying files for future-0.17.1.

after

Successfully installed future-0.17.1

unless somehow the same package was being installed twice simultaneously?

Last edited 7 months ago by embray (previous) (diff)

comment:4 Changed 7 months ago by vbraun

  • Resolution duplicate deleted
  • Status changed from closed to new

The Error copying files for future-0.17.1. is just the error branch in build/bin/sage-spkg

The success message is written to stdout, and the error message to stderr. Both are independently buffered streams, it is to be expected that they are out of order.

Last edited 7 months ago by vbraun (previous) (diff)

comment:5 Changed 7 months ago by embray

I understand that, but the error case is:

 903     $SAGE_SUDO cp -Rp "$PREFIX/." "$SAGE_LOCAL"
 904     if [ $? -ne 0 ]; then
 905         error_msg "Error copying files for $PKG_NAME."
 906         exit 1
 907     fi

i.e. it should have exited the script before the success message was ever echoed at all.

comment:6 Changed 7 months ago by vbraun

The first 'Successfully installed' is printed by pip, which is also why its before the 'Removed build tracker...'

comment:7 Changed 7 months ago by embray

Yes, I just realized that. It's the same exact message so it's confusing. In that case I do believe this is a duplicate.

comment:8 Changed 7 months ago by embray

  • Resolution set to duplicate
  • Status changed from new to closed

So this is still a duplicate of #26018. The problem with the future package in particular is that it (confusingly) provides stand-ins for several packages and modules in the Python 3 stdlib, such that they end up getting imported during the build process by other packages (such as pip) that support Python 3.

So I need to update #26018 to include future in the list of "python toolchain packages" that are not installed in parallel.

Note: See TracTickets for help on using tickets.