Ticket #5350 (closed enhancement: fixed)

Opened 19 months ago

Last modified 14 months ago

[with patch, positive review] sage-clone should use hard links for the build directory

Reported by: burcin Owned by: burcin
Priority: major Milestone: sage-4.1
Component: misc Keywords:
Cc: cwitty, robertwb, ralf@… Author(s): Burcin Erocal
Report Upstream: Reviewer(s): Robert Miller
Merged in: sage-4.1.alpha3 Work issues:

Description

We copy the build directory when we're cloning the tree. This wastes disk space, and makes switching between branches slow, since new files need to be loaded from disk while the previous ones might already be in the cache.

Attached patch to the scripts repository changes the sage-clone script to hard link the build directory. On my laptop this saves >450 MB per clone.

burcin@karr ~/sage/sage-3.3/devel $ du -sh sage-*
593M    sage-hl
125M    sage-hl1
557M    sage-main

Also the time to clone on my laptop goes from:

real    0m14.709s
user    0m4.640s
sys     0m1.924s

to

real    0m6.100s
user    0m4.712s
sys     0m0.928s

about 2.8 seconds of which is spent in the sage -b step.

Unfortunately, hard linking the .c, .cpp, and .h files doesn't work. This might be a problem with how cython handles its output when the output file is already present. This would save another ~100MB if it works.

Attachments

trac_5350-clone_hardlink.patch Download (1.7 KB) - added by burcin 19 months ago.
make sage-clone use hard links
trac_5350-clone_hardlink-take2.patch Download (1.9 KB) - added by burcin 15 months ago.
rebased for 4.0.1

Change History

Changed 19 months ago by burcin

make sage-clone use hard links

Changed 19 months ago by mabshoff

  • milestone changed from sage-3.4 to sage-3.4.1

I don't think we want this in a critical stable release :)

Cheers,

Michael

Changed 18 months ago by robertwb

Cython 0.11 will make hard links on .c and .cpp files work too.

 http://trac.cython.org/cython_trac/ticket/220 and #4987

Changed 15 months ago by burcin

rebased for 4.0.1

Changed 15 months ago by burcin

  • status changed from new to assigned

attachment:trac_5350-clone_hardlink-take2.patch Download is a new version of the patch. It is rebased to 4.0.1. We now use hard links for autogenerated cython files and documentation output too.

Changed 15 months ago by rlm

  • summary changed from [with patch, needs review] sage-clone should use hard links for the build directory to [with patch, positive review] sage-clone should use hard links for the build directory
  • milestone changed from sage-4.1 to sage-4.1.1

Everything works fine, and the speedup is impressive, but I agree that a two-digit release might not be appropriate for something so invasive.

Changed 14 months ago by boothby

  • status changed from assigned to closed
  • resolution set to fixed
  • merged set to sage-4.1.alpha2

Changed 14 months ago by mvngu

  • reviewer set to Robert Miller
  • milestone changed from sage-4.1.1 to sage-4.1
  • author set to Burcin Erocal

Changed 14 months ago by rlm

  • merged changed from sage-4.1.alpha2 to sage-4.1.alpha3
Note: See TracTickets for help on using tickets.