Opened 5 months ago

Closed 5 months ago

#26990 closed defect (duplicate)

Race when copying package files

Reported by: vbraun Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: build Keywords:
Cc: embray Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by vbraun)

There seems to be a race when copying files as cp is not atomic. I'm not sure if the copying holds a lock. Either the copying or the *.pyc file generation by the Python interpreter in another process conflicted here:

[future-0.17.1] Successfully installed future-0.17.1
[future-0.17.1] Cleaning up...
[future-0.17.1] Removed build tracker '/tmp/pip-req-tracker-ruDJcT'
[future-0.17.1] 
[future-0.17.1] real	0m4.907s
[future-0.17.1] user	0m2.226s
[future-0.17.1] sys	0m0.382s
[future-0.17.1] Copying package files from temporary location /home/chapoton/sage/local/var/tmp/sage/build/future-0.17.1/inst to /home/chapoton/sage/local
[future-0.17.1] cp: cannot create regular file '/home/chapoton/sage/local/./lib/python2.7/site-packages/future/__init__.pyc': File exists
[future-0.17.1] ************************************************************************
[future-0.17.1] Error copying files for future-0.17.1.
[future-0.17.1] ************************************************************************
[future-0.17.1] Please email sage-devel (http://groups.google.com/group/sage-devel)
[future-0.17.1] explaining the problem and including the log file
[future-0.17.1]   /home/chapoton/sage/logs/pkgs/future-0.17.1.log
[future-0.17.1] Describe your computer, operating system, etc.
[future-0.17.1] ************************************************************************

See also: https://unix.stackexchange.com/questions/116280/cannot-create-regular-file-filename-file-exists

Change History (8)

comment:1 Changed 5 months ago by vbraun

  • Description modified (diff)

comment:2 Changed 5 months ago by embray

  • Milestone changed from sage-8.6 to sage-duplicate/invalid/wontfix
  • Resolution set to duplicate
  • Status changed from new to closed

This is basically a duplicate of #26018, particularly comment:4:ticket:26018, though I'm not positive if the branch there fixes it for all cases.

comment:3 Changed 5 months ago by embray

Actually I'm not sure this is #26018, but it still looks like a fluke. How could you get:

Error copying files for future-0.17.1.

after

Successfully installed future-0.17.1

unless somehow the same package was being installed twice simultaneously?

Last edited 5 months ago by embray (previous) (diff)

comment:4 Changed 5 months ago by vbraun

  • Resolution duplicate deleted
  • Status changed from closed to new

The Error copying files for future-0.17.1. is just the error branch in build/bin/sage-spkg

Version 0, edited 5 months ago by vbraun (next)

comment:5 Changed 5 months ago by embray

I understand that, but the error case is:

 903     $SAGE_SUDO cp -Rp "$PREFIX/." "$SAGE_LOCAL"
 904     if [ $? -ne 0 ]; then
 905         error_msg "Error copying files for $PKG_NAME."
 906         exit 1
 907     fi

i.e. it should have exited the script before the success message was ever echoed at all.

comment:6 Changed 5 months ago by vbraun

The first 'Successfully installed' is printed by pip, which is also why its before the 'Removed build tracker...'

comment:7 Changed 5 months ago by embray

Yes, I just realized that. It's the same exact message so it's confusing. In that case I do believe this is a duplicate.

comment:8 Changed 5 months ago by embray

  • Resolution set to duplicate
  • Status changed from new to closed

So this is still a duplicate of #26018. The problem with the future package in particular is that it (confusingly) provides stand-ins for several packages and modules in the Python 3 stdlib, such that they end up getting imported during the build process by other packages (such as pip) that support Python 3.

So I need to update #26018 to include future in the list of "python toolchain packages" that are not installed in parallel.

Note: See TracTickets for help on using tickets.