Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#14166 closed defect (fixed)

Use "tar", not "cp -pr", to copy files in spkg-install

Reported by: jhpalmieri Owned by: jdemeyer
Priority: blocker Milestone: sage-5.8
Component: scripts Keywords: cp symlink OSX Solaris
Cc: Merged in: sage-5.8.beta2
Authors: John Palmieri Reviewers: Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by jhpalmieri)

According to http://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.68/html_node/Limitations-of-Usual-Tools.html#Limitations-of-Usual-Tools, we should avoid the "-r" flag for cp. On linux, using "-r" or "-R" doesn't make a difference, but on OS X, the man page for cp says that with the -r flag, "it does not correctly copy special files, symbolic links, or fifo's."

As a result, after building from scratch using #6495 on OS X or OpenSolaris, the various files doc/en/reference/MODULE/conf.py, which are all supposed to by symlinks to doc/en/reference/conf_sub.py, are copies of that file instead, and the hg repo is not clean. So I'm marking this as a blocker. The only other instances of "cp -pr" in any spkg are in R spkg and in sage-make_devel_packages, neither of which causes problems, so we shouldn't worry about them.


Apply trac_14166-sage-tar.patch.

Attachments (4)

trac_14166-scripts.patch (690 bytes) - added by jhpalmieri 8 years ago.
scripts repo
trac_14166-sage.2.patch (691 bytes) - added by jhpalmieri 8 years ago.
trac_14166-sage.patch (691 bytes) - added by jhpalmieri 8 years ago.
Sage library
trac_14166-sage-tar.patch (662 bytes) - added by jhpalmieri 8 years ago.
Sage library

Download all attachments as: .zip

Change History (17)

Changed 8 years ago by jhpalmieri

scripts repo

comment:1 Changed 8 years ago by jhpalmieri

  • Status changed from new to needs_review

comment:2 Changed 8 years ago by jhpalmieri

  • Description modified (diff)

comment:3 Changed 8 years ago by jhpalmieri

  • Description modified (diff)
  • Keywords OSX Solaris added

On Solaris, things are more complicated. If we use cp -pr, then it behaves like OS X, i.e., badly. If we use cp -pR, then it still doesn't work; instead, cp -pPR almost works. The only problem is that file permissions can get messed up, at least when a file in one directory is a link pointing to another directory. Consider this directory TEST:

$ ls -l TEST/
total 6
drwxr-xr-x   2 palmieri other          3 Feb 25 04:07 dir/
-rw-r--r--   1 palmieri other         79 Feb 24 21:13 FILE
lrwxrwxrwx   1 palmieri other          4 Feb 25 04:06 link1 -> FILE
$ ls -l TEST/dir/
total 1
lrwxrwxrwx   1 palmieri other          4 Feb 25 04:07 link2 -> FILE

Now copy it:

$ cp -pPR TEST X
$ ls -l X
total 6
drwxr-xr-x   2 palmieri other          3 Feb 25 04:07 dir/
-rwxrwxrwx   1 palmieri other         79 Feb 25 04:06 FILE*
lrwxrwxrwx   1 palmieri other          4 Feb 25 04:08 link1 -> FILE*
$ ls -l X/dir
total 1
lrwxrwxrwx   1 palmieri other          4 Feb 25 04:08 link2 -> FILE

Note the permissions on FILE. The way around this seems to be to use /usr/xpg4/bin/cp -pPR@ SOURCE TARGET, naturally.

Let's ignore the scripts patch and just focus on the patch for the Sage library.

Changed 8 years ago by jhpalmieri

Changed 8 years ago by jhpalmieri

Sage library

comment:4 Changed 8 years ago by jhpalmieri

  • Description modified (diff)

comment:5 Changed 8 years ago by jhpalmieri

There are at least two good ways to test this: start with a fresh Sage 5.8.beta1 tarball, unpack the sage library spkg (spkg/standard/sage-5.8.beta1.spkg), apply this patch, package it back up (with sage -pkg sage-5.8.beta1). Then build Sage. Or do something similar, but start with a built version of Sage. Delete devel/sage and devel/sage-main. Delete spkg/installed/sage-5.8.beta1 and anything more recent. Create a patched version of the Sage spkg, as above. Run make ptestlong.

comment:6 Changed 8 years ago by jdemeyer

Alternatively, tar is more portable than cp and usually deals with symbolic links correctly. I know for example PARI use tar during the installer for this reason. I don't know tar well enough, but something along the lines of

# tar is the most portable way to copy files
tar c * .hg* | (cd "$DEVEL/sage-main/" && tar x)

comment:7 Changed 8 years ago by jdemeyer

  • Summary changed from use "cp -pR", not "cp -pr" to Use "cp -pR", not "cp -pr" in spkg-install

comment:8 Changed 8 years ago by jdemeyer

  • Owner changed from leif to jdemeyer

If you decide to stick with the current approach, please use

[ -x /usr/xpg4/bin/cp ]

instead of

[ $UNAME = "SunOS" ]

comment:9 Changed 8 years ago by jhpalmieri

  • Description modified (diff)

I like the tar approach. Here's a patch.

Changed 8 years ago by jhpalmieri

Sage library

comment:10 Changed 8 years ago by jhpalmieri

  • Summary changed from Use "cp -pR", not "cp -pr" in spkg-install to Use "tar", not "cp -pr", to copy files in spkg-install

comment:11 Changed 8 years ago by jhpalmieri

With this patch, everything seems to work on sage.math, a few other linux machines on skynet, OpenSolaris (hawk), Solaris (mark), and a few OS X 10.8 boxes I have access to.

comment:12 Changed 8 years ago by jdemeyer

  • Merged in set to sage-5.8.beta2
  • Resolution set to fixed
  • Reviewers set to Jeroen Demeyer
  • Status changed from needs_review to closed

Works fine!

comment:13 Changed 8 years ago by jdemeyer

See also #14236.

Note: See TracTickets for help on using tickets.