Opened 9 years ago
Last modified 9 years ago
#9433 closed enhancement
Put more files under revision control. — at Version 22
Reported by: | jhpalmieri | Owned by: | tbd |
---|---|---|---|
Priority: | blocker | Milestone: | sage-4.7 |
Component: | distribution | Keywords: | |
Cc: | was, ddrake, kcrisman, leif | Merged in: | |
Authors: | John Palmieri | Reviewers: | |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
Put the text files in $SAGE_ROOT, and also the text files in spkg, under revision control. (See the discussion at the end of #9351.)
Here are the instructions:
- apply the patches trac_9433-sage-repo.patch and trac_9433-scripts.patch
- move the attached file "hgignore" to SAGE_ROOT/.hgignore
- move the attached file "root-spkg-install" to SAGE_ROOT/spkg/root-spkg-install
Then from $SAGE_ROOT, run the attached script "hg_script" to create the Mercurial repository.
Change History (26)
comment:1 Changed 9 years ago by
- Status changed from new to needs_review
comment:2 Changed 9 years ago by
A little explanation: this patch creates a directory "other-scripts" in SAGE_ROOT/local/bin. This new directory contains a brief README.txt and also subdirectories "root" and "spkg". "root" contains the text files from SAGE_ROOT. The only one with any changes is README.txt which explains how these files are under revision control. Similarly, "spkg" contains various text files from SAGE_ROOT/spkg, and the only one with any changes is README.txt.
comment:3 Changed 9 years ago by
This probably needs to be rebased. When people are ready to look at it, let me know and I'll see what I can do.
comment:4 Changed 9 years ago by
New approach, after a discussion on sage-devel: create a new repo at the top level tracking the appropriate files. I'm attaching a new version of the patch for the scripts repo. Someone -- the release manager, I guess -- also needs to create the top level repo, because I don't know how to do this in such a way that it can be posted on a ticket. Here are the instructions:
- move the attached file "hgignore" to SAGE_ROOT/.hgignore
- cd $SAGE_ROOT
- hg init .
- hg add .hgignore COPYING.txt README.txt makefile sage sage-python
- cd spkg
- hg add README.txt gen_html install pipestatus
- cd standard
- hg add README.txt deps libdist_filelist newest_version
- hg add notes.txt numeric-24.2.txt
(I don't know if we really need these last two files, but this is probably not the ticket for making such decisions.) Finally, do
- hg commit
When you run "sage -sdist ..." it should add a tag for the new version of Sage.
This does not create a new spkg for the files in SAGE_ROOT, since those files have to be in place when you unpack the sage tar file. But it creates the repository so that people can post patches to the trac server, etc.
comment:5 follow-up: ↓ 6 Changed 9 years ago by
- Cc was added
Looking with my eyes, this looks good. I don't have time to test right now. The test would be to take a clean Sage, do the above, then do "sage -sdist ..." and make sure that in the sdist the above is all still there.
comment:6 in reply to: ↑ 5 Changed 9 years ago by
Replying to was:
The test would be to take a clean Sage, do the above, then do "sage -sdist ..." and make sure that in the sdist the above is all still there.
This works for me, but other people should definitely look at it carefully.
comment:7 Changed 9 years ago by
This probably needs work: how will it work with "sage -upgrade"?
comment:8 Changed 9 years ago by
- Description modified (diff)
Here's a new version of the patch for the scripts repo. I think this should deal with upgrading: the script "sage-upgrade" now runs "sage --hg branch" from SAGE_ROOT, and if this fails, it assumes that there is no Mercurial repository and creates it.
comment:9 Changed 9 years ago by
- Cc ddrake added
comment:10 Changed 9 years ago by
comment:12 Changed 9 years ago by
In sage-sdist, where you have # copy sage root repo over
, why not just clone the repo? That will take care of copying all the necessary files, and if we add or remove files tracked by the repo, we won't need to mess with sage-sdist. I'm thinking that something like
cd $SAGE_ROOT hg clone --pull . DEST_DIR
Using --pull means that it doesn't use hardlinks in the clone; I *think* there would be no problem with using hardlinks, but it's unlikely to make a big difference. The clone will include a hgrc file that points to where it came from: it would look something like this:
[paths] default = /home/foo/sage-whatever
We could simply delete the file, or just leave it, since it would not negatively affect anything (except running hg pull
from SAGE_ROOT, which you wouldn't do anyway).
So, all the cp -p
lines could be just
hg clone --pull . $TMP rm $TMP/.hg/hgrc
and files added or removed to the repo would get copied correctly without changing any scripts.
comment:13 Changed 9 years ago by
Note that if you use "hg clone ..." to copy the repo, you have to do it earlier in the process: it apparently needs to be done with an empty directory, so in sage-sdist it is now done right after creating $TMP.
comment:14 Changed 9 years ago by
- Cc kcrisman added
comment:15 follow-up: ↓ 17 Changed 9 years ago by
- Milestone changed from sage-4.5.2 to sage-4.5.3
- Status changed from needs_review to needs_work
If I
- Build the forthcoming Sage 4.5.2 (which is just 4.5.2.rc1 + #9226) from source.
- Follow the steps in the description.
./sage -sdist 4.5.99
hg log
in the "root" repo now gives
changeset: 1:0fea58e94942 tag: tip user: Mitesh Patel <qed777@gmail.com> date: Fri Aug 06 21:40:23 2010 -0700 summary: Added tag 4.5.99 for changeset 4c1f4320f743 changeset: 0:4c1f4320f743 tag: 4.5.99 user: Mitesh Patel <qed777@gmail.com> date: Fri Aug 06 21:33:45 2010 -0700 summary: Initial Sage "root" repository
The new 4.5.99 builds successfully from source and passes the long doctests. But hg log
in the root repo for 4.5.99 lists just revision 0, and the root repo is missing from a binary distribution made with ./sage -bdist 4.5.99
.
comment:16 Changed 9 years ago by
I also noticed
SAGE_ROOT/ipython
andSAGE_ROOT/sage-README-osx.txt
are missing from the new source and binary distributions.- An extra
SAGE_ROOT/sage-python
after unpackingsage-4.5.99.tar
.
comment:17 in reply to: ↑ 15 ; follow-up: ↓ 19 Changed 9 years ago by
- Status changed from needs_work to needs_review
Replying to mpatel:
The new 4.5.99 builds successfully from source and passes the long doctests. But
hg log
in the root repo for 4.5.99 lists just revision 0, and the root repo is missing from a binary distribution made with./sage -bdist 4.5.99
.
Right, I didn't do this right. In the new patch, the root repo is not modified at all by sage-make_devel_packages; instead, sage-sdist clones it, tags it, and commits the new tag, while sage-bdist just clones it.
I also noticed
- SAGE_ROOT/ipython and SAGE_ROOT/sage-README-osx.txt are missing from the new source and binary distributions.
The missing ipython directory was an oversight. I think I've fixed it. The missing sage-README-osx.txt was intentional: this should only be included for binary distributions on OS X, and its presence there is taken care of by sage-bdist:
if [ "$UNAME" = "Darwin" ]; then ... cp sage/local/bin/sage-README-osx.txt README.txt ...
Perhaps we can close #6938 if this gets merged?
- An extra SAGE_ROOT/sage-python after unpacking sage-4.5.99.tar.
This was intentional. Before this, the file sage-python was stored in the scripts repo and then unpacked to the top level. I'm not sure why this was done, but I wanted to keep the end result as close as possible to what it was before.
comment:18 Changed 9 years ago by
- Description modified (diff)
comment:19 in reply to: ↑ 17 Changed 9 years ago by
I also noticed
- SAGE_ROOT/ipython and SAGE_ROOT/sage-README-osx.txt are missing from the new source and binary distributions.
The missing ipython directory was an oversight. I think I've fixed it. The missing sage-README-osx.txt was intentional: this should only be included for binary distributions on OS X, and its presence there is taken care of by sage-bdist:
if [ "$UNAME" = "Darwin" ]; then ... cp sage/local/bin/sage-README-osx.txt README.txt ...Perhaps we can close #6938 if this gets merged?
As one of the people involved on that ticket, that is fine. The problem is that #6938 does not currently have positive review! So I think that would be necessary first, or something else indicating that the solution proposed there is correct. Maybe 'merge' that ticket at the same time as this one, for whatever it's worth.
Sounds like you agree :) In fact, notice that once that is removed, that file will only appear ABOVE the SAGE_ROOT directory, in the place a normal README would occur in a dmg or bundle, so it does work properly (I've tested this numerous times
comment:20 Changed 9 years ago by
- Description modified (diff)
I realized there was another problem with the previous patch: while it would work when upgrading from a Sage build with no root repo, it wouldn't do anything when upgrading a Sage build with an existing root repo. The new patch constructs a sage_root spkg file, but this file only gets used during upgrading. So it is installed in the script "sage-upgrade", but it does not appear in spkg/standard/deps.
comment:21 Changed 9 years ago by
- Status changed from needs_review to needs_work
This still doesn't deal with upgrading very well. I need to think about how to do this and work on it for a while in a place where I have a better internet connection. I may have something in a few days. Meanwhile, if anyone has suggestions for how to deal with upgrades, I would like to hear them. The issues:
- currently, I don't even know what happens with "sage -upgrade ..." if SAGE_ROOT/makefile is changed, for example. Does anything happen?
- does it matter whether we install the new sage_root spkg before or after the sage_scripts spkg? The patch here affects the sage-upgrade script, but the new version won't get run during an upgrade anyway.
- is it good enough to just install the new sage_root spkg? I think it might be, but I might be confused. Anyway, I think I need time to play with it and test it out.
comment:22 Changed 9 years ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
This seems to work for me, with one slight glitch: if I run "sage -upgrade" on a copy of sage which does not yet include the root repo, it lists the spkg's to be upgraded and asks me "do you really want to continue", then it does some stuff, and then it lists just the root_repo spkg and asks me again if I want to continue. The issue is that, before it has installed some of the upgraded packages, it doesn't know what to do with the root_repo spkg, so it doesn't get installed the first time through. I don't see any way around this, but it's a one-time problem.
I'm marking this as "needs review", but consider this patch experimental.