Sage: Ticket #9433: Put more files under revision control.
https://trac.sagemath.org/ticket/9433
<p>
Put the text files in <code>$SAGE_ROOT</code>, and also the text files in spkg, under revision control. (See the discussion at the end of <a class="closed ticket" href="https://trac.sagemath.org/ticket/9351" title="defect: deps for sagetex with SAGE_CHECK='yes' (closed: fixed)">#9351</a>.)
</p>
<p>
Here are the instructions:
</p>
<ul><li>apply the patches <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/trac_9433sagerepo.2.patch" title="Attachment 'trac_9433sagerepo.2.patch' in Ticket #9433">trac_9433sagerepo.2.patch</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/trac_9433sagerepo.2.patch" title="Download"></a> and <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/trac_9433scripts.v6.patch" title="Attachment 'trac_9433scripts.v6.patch' in Ticket #9433">trac_9433scripts.v6.patch</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/trac_9433scripts.v6.patch" title="Download"></a>
</li><li>move the attached file <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/hgignore" title="Attachment 'hgignore' in Ticket #9433">hgignore</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/hgignore" title="Download"></a> to <code>$SAGE_ROOT/.hgignore</code> (note that this is a <em>new</em> file)
</li><li>move the attached file <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/rootspkginstall.v2" title="Attachment 'rootspkginstall.v2' in Ticket #9433">rootspkginstall.v2</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/rootspkginstall.v2" title="Download"></a> to <code>$SAGE_ROOT/spkg/rootspkginstall</code> (note that this is a <em>new</em> file)
</li><li>apply <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/9433_install.diff" title="Attachment '9433_install.diff' in Ticket #9433">9433_install.diff</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/9433_install.diff" title="Download"></a> to <code>$SAGE_ROOT/spkg/install</code>
</li><li>apply <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/9433_deps.diff" title="Attachment '9433_deps.diff' in Ticket #9433">9433_deps.diff</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/9433_deps.diff" title="Download"></a> to <code>$SAGE_ROOT/spkg/standard/deps</code>
</li></ul><p>
Then <strong>from $SAGE_ROOT</strong>, run the attached script <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/9433_hg_script.sh" title="Attachment '9433_hg_script.sh' in Ticket #9433">9433_hg_script.sh</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/9433_hg_script.sh" title="Download"></a> to create the Mercurial repository.
</p>
enusSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/9433
Trac 1.1.6jhpalmieriTue, 06 Jul 2010 00:03:54 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:1
https://trac.sagemath.org/ticket/9433#comment:1
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
</ul>
<p>
I'm marking this as "needs review", but consider this patch experimental.
</p>
TicketjhpalmieriTue, 06 Jul 2010 00:19:41 GMT
https://trac.sagemath.org/ticket/9433#comment:2
https://trac.sagemath.org/ticket/9433#comment:2
<p>
A little explanation: this patch creates a directory "otherscripts" in SAGE_ROOT/local/bin. This new directory contains a brief README.txt and also subdirectories "root" and "spkg". "root" contains the text files from SAGE_ROOT. The only one with any changes is README.txt which explains how these files are under revision control. Similarly, "spkg" contains various text files from SAGE_ROOT/spkg, and the only one with any changes is README.txt.
</p>
TicketjhpalmieriTue, 06 Jul 2010 03:27:14 GMT
https://trac.sagemath.org/ticket/9433#comment:3
https://trac.sagemath.org/ticket/9433#comment:3
<p>
This probably needs to be rebased. When people are ready to look at it, let me know and I'll see what I can do.
</p>
TicketjhpalmieriThu, 08 Jul 2010 15:46:52 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.2.patch</em>
</li>
</ul>
<p>
rebased against 4.5.alpha4 + <a class="closed ticket" href="https://trac.sagemath.org/ticket/9456" title="defect: zlib should be a prerequisite for libpng (closed: fixed)">#9456</a>
</p>
TicketjhpalmieriThu, 08 Jul 2010 22:17:57 GMT
https://trac.sagemath.org/ticket/9433#comment:4
https://trac.sagemath.org/ticket/9433#comment:4
<p>
New approach, after a <a class="extlink" href="http://groups.google.com/group/sagedevel/t/3ba410046ae641f8"><span class="icon"></span>discussion on sagedevel</a>: create a new repo at the top level tracking the appropriate files. I'm attaching a new version of the patch for the scripts repo. Someone  the release manager, I guess  also needs to create the top level repo, because I don't know how to do this in such a way that it can be posted on a ticket. Here are the instructions:
</p>
<ul><li>move the attached file "hgignore" to SAGE_ROOT/.hgignore
</li><li>cd $SAGE_ROOT
</li><li>hg init .
</li><li>hg add .hgignore COPYING.txt README.txt makefile sage sagepython
</li><li>cd spkg
</li><li>hg add README.txt gen_html install pipestatus
</li><li>cd standard
</li><li>hg add README.txt deps libdist_filelist newest_version
</li><li>hg add notes.txt numeric24.2.txt
</li></ul><p>
(I don't know if we really need these last two files, but this is probably not the ticket for making such decisions.) Finally, do
</p>
<ul><li>hg commit
</li></ul><p>
When you run "sage sdist ..." it should add a tag for the new version of Sage.
</p>
<p>
This does not create a new spkg for the files in SAGE_ROOT, since those files have to be in place when you unpack the sage tar file. But it creates the repository so that people can post patches to the trac server, etc.
</p>
TicketwasThu, 08 Jul 2010 22:25:13 GMTcc set
https://trac.sagemath.org/ticket/9433#comment:5
https://trac.sagemath.org/ticket/9433#comment:5
<ul>
<li><strong>cc</strong>
<em>was</em> added
</li>
</ul>
<p>
Looking with my eyes, this looks good. I don't have time to test right now. The test would be to take a clean Sage, do the above, then do "sage sdist ..." and make sure that in the sdist the above is all still there.
</p>
TicketjhpalmieriFri, 09 Jul 2010 00:43:54 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433sagerepo.patch</em>
</li>
</ul>
<p>
main repo: add "hg_root" command to Sage
</p>
TicketjhpalmieriFri, 09 Jul 2010 04:42:31 GMT
https://trac.sagemath.org/ticket/9433#comment:6
https://trac.sagemath.org/ticket/9433#comment:6
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:5" title="Comment 5">was</a>:
</p>
<blockquote class="citation">
<p>
The test would be to take a clean Sage, do the above, then do "sage sdist ..." and make sure that in the sdist the above is all still there.
</p>
</blockquote>
<p>
This works for me, but other people should definitely look at it carefully.
</p>
TicketjhpalmieriThu, 22 Jul 2010 04:14:06 GMT
https://trac.sagemath.org/ticket/9433#comment:7
https://trac.sagemath.org/ticket/9433#comment:7
<p>
This probably needs work: how will it work with "sage upgrade"?
</p>
TicketjhpalmieriMon, 26 Jul 2010 20:47:51 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:8
https://trac.sagemath.org/ticket/9433#comment:8
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=8">diff</a>)
</li>
</ul>
<p>
Here's a new version of the patch for the scripts repo. I think this should deal with upgrading: the script "sageupgrade" now runs "sage hg branch" from SAGE_ROOT, and if this fails, it assumes that there is no Mercurial repository and creates it.
</p>
TicketddrakeTue, 27 Jul 2010 13:42:21 GMTcc changed
https://trac.sagemath.org/ticket/9433#comment:9
https://trac.sagemath.org/ticket/9433#comment:9
<ul>
<li><strong>cc</strong>
<em>ddrake</em> added
</li>
</ul>
TicketjhpalmieriTue, 27 Jul 2010 14:49:36 GMT
https://trac.sagemath.org/ticket/9433#comment:10
https://trac.sagemath.org/ticket/9433#comment:10
<p>
This may need to be changed if <a class="closed ticket" href="https://trac.sagemath.org/ticket/9609" title="defect: Remove unnecessary files from spkg/standard (closed: fixed)">#9609</a> is merged: the directions here, and also the modified "sageupgrade" script, refer to files which would be deleted by <a class="closed ticket" href="https://trac.sagemath.org/ticket/9609" title="defect: Remove unnecessary files from spkg/standard (closed: fixed)">#9609</a>.
</p>
TicketjhpalmieriWed, 28 Jul 2010 02:25:03 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:11
https://trac.sagemath.org/ticket/9433#comment:11
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=11">diff</a>)
</li>
</ul>
<p>
New version, rebased against <a class="closed ticket" href="https://trac.sagemath.org/ticket/9609" title="defect: Remove unnecessary files from spkg/standard (closed: fixed)">#9609</a>.
</p>
TicketddrakeWed, 28 Jul 2010 06:24:24 GMT
https://trac.sagemath.org/ticket/9433#comment:12
https://trac.sagemath.org/ticket/9433#comment:12
<p>
In sagesdist, where you have <code># copy sage root repo over</code>, why not just clone the repo? That will take care of copying all the necessary files, and if we add or remove files tracked by the repo, we won't need to mess with sagesdist. I'm thinking that something like
</p>
<pre class="wiki">cd $SAGE_ROOT
hg clone pull . DEST_DIR
</pre><p>
Using pull means that it doesn't use hardlinks in the clone; I *think* there would be no problem with using hardlinks, but it's unlikely to make a big difference. The clone will include a hgrc file that points to where it came from: it would look something like this:
</p>
<pre class="wiki">[paths]
default = /home/foo/sagewhatever
</pre><p>
We could simply delete the file, or just leave it, since it would not negatively affect anything (except running <code>hg pull</code> from SAGE_ROOT, which you wouldn't do anyway).
</p>
<p>
So, all the <code>cp p</code> lines could be just
</p>
<pre class="wiki">hg clone pull . $TMP
rm $TMP/.hg/hgrc
</pre><p>
and files added or removed to the repo would get copied correctly without changing any scripts.
</p>
TicketjhpalmieriWed, 28 Jul 2010 15:05:28 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.3.patch</em>
</li>
</ul>
<p>
new version, using "hg clone". apply to scripts repo.
</p>
TicketjhpalmieriWed, 28 Jul 2010 17:38:26 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.patch</em>
</li>
</ul>
<p>
new version, using "hg clone". apply to scripts repo.
</p>
TicketjhpalmieriWed, 28 Jul 2010 17:39:55 GMT
https://trac.sagemath.org/ticket/9433#comment:13
https://trac.sagemath.org/ticket/9433#comment:13
<p>
Note that if you use "hg clone ..." to copy the repo, you have to do it earlier in the process: it apparently needs to be done with an empty directory, so in sagesdist it is now done right after creating $TMP.
</p>
TicketkcrismanFri, 06 Aug 2010 19:31:51 GMTcc changed
https://trac.sagemath.org/ticket/9433#comment:14
https://trac.sagemath.org/ticket/9433#comment:14
<ul>
<li><strong>cc</strong>
<em>kcrisman</em> added
</li>
</ul>
TicketmpatelSat, 07 Aug 2010 06:18:27 GMTstatus, milestone changed; author set
https://trac.sagemath.org/ticket/9433#comment:15
https://trac.sagemath.org/ticket/9433#comment:15
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage4.5.2</em> to <em>sage4.5.3</em>
</li>
<li><strong>author</strong>
set to <em>John Palmieri</em>
</li>
</ul>
<p>
If I
</p>
<ul><li>Build the forthcoming Sage 4.5.2 (which is just 4.5.2.rc1 + <a class="closed ticket" href="https://trac.sagemath.org/ticket/9226" title="defect: README.txt says " Sage builds with GCC >= 3.x" but it does NOT (closed: fixed)">#9226</a>) from source.
</li><li>Follow the steps in the description.
</li><li><code>./sage sdist 4.5.99</code>
</li></ul><p>
<code>hg log</code> in the "root" repo now gives
</p>
<pre class="wiki">changeset: 1:0fea58e94942
tag: tip
user: Mitesh Patel <qed777@gmail.com>
date: Fri Aug 06 21:40:23 2010 0700
summary: Added tag 4.5.99 for changeset 4c1f4320f743
changeset: 0:4c1f4320f743
tag: 4.5.99
user: Mitesh Patel <qed777@gmail.com>
date: Fri Aug 06 21:33:45 2010 0700
summary: Initial Sage "root" repository
</pre><p>
The new 4.5.99 builds successfully from source and passes the long doctests. But <code>hg log</code> in the root repo for 4.5.99 lists just revision 0, and the root repo is missing from a binary distribution made with <code>./sage bdist 4.5.99</code>.
</p>
TicketmpatelSat, 07 Aug 2010 07:13:00 GMT
https://trac.sagemath.org/ticket/9433#comment:16
https://trac.sagemath.org/ticket/9433#comment:16
<p>
I also noticed
</p>
<ul><li><code>SAGE_ROOT/ipython</code> and <code>SAGE_ROOT/sageREADMEosx.txt</code> are missing from the new source and binary distributions.
</li><li>An extra <code>SAGE_ROOT/sagepython</code> after unpacking <code>sage4.5.99.tar</code>.
</li></ul>
TicketjhpalmieriSun, 08 Aug 2010 22:38:42 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:17
https://trac.sagemath.org/ticket/9433#comment:17
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:15" title="Comment 15">mpatel</a>:
</p>
<blockquote class="citation">
<p>
The new 4.5.99 builds successfully from source and passes the long doctests. But <code>hg log</code> in the root repo for 4.5.99 lists just revision 0, and the root repo is missing from a binary distribution made with <code>./sage bdist 4.5.99</code>.
</p>
</blockquote>
<p>
Right, I didn't do this right. In the new patch, the root repo is not modified at all by sagemake_devel_packages; instead, sagesdist clones it, tags it, and commits the new tag, while sagebdist just clones it.
</p>
<blockquote class="citation">
<p>
I also noticed
</p>
<ul><li>SAGE_ROOT/ipython and SAGE_ROOT/sageREADMEosx.txt are missing from the new source and binary distributions.
</li></ul></blockquote>
<p>
The missing ipython directory was an oversight. I think I've fixed it. The missing sageREADMEosx.txt was intentional: this should only be included for binary distributions on OS X, and its presence there is taken care of by sagebdist:
</p>
<pre class="wiki">if [ "$UNAME" = "Darwin" ]; then
...
cp sage/local/bin/sageREADMEosx.txt README.txt
...
</pre><p>
Perhaps we can close <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> if this gets merged?
</p>
<blockquote class="citation">
<ul><li>An extra SAGE_ROOT/sagepython after unpacking sage4.5.99.tar.
</li></ul></blockquote>
<p>
This was intentional. Before this, the file sagepython was stored in the scripts repo and then unpacked to the top level. I'm not sure why this was done, but I wanted to keep the end result as close as possible to what it was before.
</p>
TicketjhpalmieriSun, 08 Aug 2010 22:41:36 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:18
https://trac.sagemath.org/ticket/9433#comment:18
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=18">diff</a>)
</li>
</ul>
TicketkcrismanMon, 09 Aug 2010 13:03:12 GMT
https://trac.sagemath.org/ticket/9433#comment:19
https://trac.sagemath.org/ticket/9433#comment:19
<blockquote class="citation">
<blockquote class="citation">
<p>
I also noticed
</p>
<ul><li>SAGE_ROOT/ipython and SAGE_ROOT/sageREADMEosx.txt are missing from the new source and binary distributions.
</li></ul></blockquote>
<p>
The missing ipython directory was an oversight. I think I've fixed it. The missing sageREADMEosx.txt was intentional: this should only be included for binary distributions on OS X, and its presence there is taken care of by sagebdist:
</p>
<pre class="wiki">if [ "$UNAME" = "Darwin" ]; then
...
cp sage/local/bin/sageREADMEosx.txt README.txt
...
</pre><p>
Perhaps we can close <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> if this gets merged?
</p>
</blockquote>
<p>
As one of the people involved on that ticket, that is fine. The problem is that <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> does not currently have positive review! So I think that would be necessary first, or something else indicating that the solution proposed there is correct. Maybe 'merge' that ticket at the same time as this one, for whatever it's worth.
</p>
<p>
Sounds like you agree :) In fact, notice that once that is removed, that file will only appear ABOVE the SAGE_ROOT directory, in the place a normal README would occur in a dmg or bundle, so it does work properly (I've tested this numerous times
</p>
TicketjhpalmieriMon, 09 Aug 2010 14:33:18 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:20
https://trac.sagemath.org/ticket/9433#comment:20
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=20">diff</a>)
</li>
</ul>
<p>
I realized there was another problem with the previous patch: while it would work when upgrading from a Sage build with no root repo, it wouldn't do anything when upgrading a Sage build with an existing root repo. The new patch constructs a sage_root spkg file, but this file only gets used during upgrading. So it is installed in the script "sageupgrade", but it does not appear in spkg/standard/deps.
</p>
TicketjhpalmieriTue, 10 Aug 2010 02:57:08 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:21
https://trac.sagemath.org/ticket/9433#comment:21
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
This still doesn't deal with upgrading very well. I need to think about how to do this and work on it for a while in a place where I have a better internet connection. I may have something in a few days. Meanwhile, if anyone has suggestions for how to deal with upgrades, I would like to hear them. The issues:
</p>
<ul><li>currently, I don't even know what happens with "sage upgrade ..." if SAGE_ROOT/makefile is changed, for example. Does anything happen?
</li></ul><ul><li>does it matter whether we install the new sage_root spkg before or after the sage_scripts spkg? The patch here affects the sageupgrade script, but the new version won't get run during an upgrade anyway.
</li></ul><ul><li>is it good enough to just install the new sage_root spkg? I think it might be, but I might be confused. Anyway, I think I need time to play with it and test it out.
</li></ul>
TicketjhpalmieriWed, 11 Aug 2010 19:31:26 GMTstatus, description changed
https://trac.sagemath.org/ticket/9433#comment:22
https://trac.sagemath.org/ticket/9433#comment:22
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=22">diff</a>)
</li>
</ul>
<p>
This seems to work for me, with one slight glitch: if I run "sage upgrade" on a copy of sage which does not yet include the root repo, it lists the spkg's to be upgraded and asks me "do you really want to continue", then it does some stuff, and then it lists just the root_repo spkg and asks me again if I want to continue. The issue is that, before it has installed some of the upgraded packages, it doesn't know what to do with the root_repo spkg, so it doesn't get installed the first time through. I don't see any way around this, but it's a onetime problem.
</p>
TicketleifMon, 13 Sep 2010 08:02:46 GMTcc changed
https://trac.sagemath.org/ticket/9433#comment:23
https://trac.sagemath.org/ticket/9433#comment:23
<ul>
<li><strong>cc</strong>
<em>leif</em> added
</li>
</ul>
<p>
Ooops, I once had been aware of this ticket, but forgot to cc me...
</p>
<p>
Before doing this, can we rename <code>makefile</code> to <code>Makefile</code>?
</p>
TicketjhpalmieriMon, 13 Sep 2010 18:11:42 GMT
https://trac.sagemath.org/ticket/9433#comment:24
https://trac.sagemath.org/ticket/9433#comment:24
<blockquote class="citation">
<p>
Before doing this, can we rename makefile to Makefile?
</p>
</blockquote>
<p>
I have no problems with that. Are there ever any (good) reasons for using <code>makefile</code> instead of <code>Makefile</code>?
</p>
TicketleifMon, 13 Sep 2010 19:17:01 GMT
https://trac.sagemath.org/ticket/9433#comment:25
https://trac.sagemath.org/ticket/9433#comment:25
<p>
Should <code>sageREADMEosx.txt</code> be ignored (i.e. <strong>not</strong> be under revision control)?
</p>
<p>
Also a candidate for renaming (<code>README.MacOS_X.txt</code> or alike).
</p>
<p>
From the attached <code>hg_script</code>:
</p>
<div class="wikicode"><div class="code"><pre>...
<span class="nv">$hg</span> add .hgignore .hgtags COPYING.txt README.txt makefile sage sagepython
<span class="nb">cd </span>ipython
<span class="nv">$hg</span> add *.py ipythonrc*
...
</pre></div></div><p>
<code>sagepython</code> and <code>ipython/</code> should IMHO be in <code>.hgignore</code>, since these are not in <code>$SAGE_ROOT</code> until Sage is built.
</p>
TicketjhpalmieriMon, 13 Sep 2010 19:53:13 GMT
https://trac.sagemath.org/ticket/9433#comment:26
https://trac.sagemath.org/ticket/9433#comment:26
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:25" title="Comment 25">leif</a>:
</p>
<blockquote class="citation">
<p>
Should <code>sageREADMEosx.txt</code> be ignored (i.e. <strong>not</strong> be under revision control)?
</p>
</blockquote>
<p>
It shouldn't be present at all, and with the patches here, it isn't. See <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> and also <a class="extlink" href="http://trac.sagemath.org/sage_trac/ticket/9433?replyto=25#comment:17"><span class="icon"></span>my comment above</a>. (It should only appear in the <strong>parent</strong> directory of SAGE_ROOT when you make a dmg file on a Mac.)
</p>
<blockquote class="citation">
<p>
<code>sagepython</code> and <code>ipython/</code> should IMHO be in <code>.hgignore</code>, since these are not in <code>$SAGE_ROOT</code> until Sage is built.
</p>
</blockquote>
<p>
This is another change on this ticket: these files are now tracked in this repo, not in sage_scripts anymore. Actually, I don't know if ipython was tracked anywhere, but now it is. I don't know who chose to include sagepython in the toplevel directory, but since it's there, it should be tracked properly. (It's a simple enough script I don't mind having two copies of it.) I personally don't see the point of it and think it should be removed, but that should probably happen on another ticket. Or maybe it should be a link to local/bin/sagepython?
</p>
TicketkcrismanMon, 13 Sep 2010 20:20:53 GMT
https://trac.sagemath.org/ticket/9433#comment:27
https://trac.sagemath.org/ticket/9433#comment:27
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:26" title="Comment 26">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:25" title="Comment 25">leif</a>:
</p>
<blockquote class="citation">
<p>
Should <code>sageREADMEosx.txt</code> be ignored (i.e. <strong>not</strong> be under revision control)?
</p>
</blockquote>
<p>
It shouldn't be present at all, and with the patches here, it isn't. See <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> and also <a class="extlink" href="http://trac.sagemath.org/sage_trac/ticket/9433?replyto=25#comment:17"><span class="icon"></span>my comment above</a>. (It should only appear in the <strong>parent</strong> directory of SAGE_ROOT when you make a dmg file on a Mac.)
</p>
</blockquote>
<p>
Correct.
</p>
TicketleifMon, 13 Sep 2010 20:27:37 GMT
https://trac.sagemath.org/ticket/9433#comment:28
https://trac.sagemath.org/ticket/9433#comment:28
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:26" title="Comment 26">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:25" title="Comment 25">leif</a>:
</p>
<blockquote class="citation">
<p>
Should <code>sageREADMEosx.txt</code> be ignored (i.e. <strong>not</strong> be under revision control)?
</p>
</blockquote>
<p>
It shouldn't be present at all, and with the patches here, it isn't. See <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> and also <a class="extlink" href="http://trac.sagemath.org/sage_trac/ticket/9433?replyto=25#comment:17"><span class="icon"></span>my comment above</a>. (It should only appear in the <strong>parent</strong> directory of SAGE_ROOT when you make a dmg file on a Mac.)
</p>
</blockquote>
<p>
Missed that.
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
<code>sagepython</code> and <code>ipython/</code> should IMHO be in <code>.hgignore</code>, since these are not in <code>$SAGE_ROOT</code> until Sage is built.
</p>
</blockquote>
<p>
This is another change on this ticket: these files are now tracked in this repo, not in sage_scripts anymore.
</p>
</blockquote>
<p>
That, too... :/
</p>
<blockquote class="citation">
<p>
Actually, I don't know if ipython was tracked anywhere, but now it is. I don't know who chose to include sagepython in the toplevel directory, but since it's there, it should be tracked properly. (It's a simple enough script I don't mind having two copies of it.) I personally don't see the point of it and think it should be removed, but that should probably happen on another ticket. Or maybe it should be a link to local/bin/sagepython?
</p>
</blockquote>
<p>
Perhaps...
</p>
<p>
Sorry for the noise.
</p>
TicketleifThu, 16 Sep 2010 12:34:36 GMT
https://trac.sagemath.org/ticket/9433#comment:29
https://trac.sagemath.org/ticket/9433#comment:29
<p>
I'd also like to have a <code>VERSION.txt</code> in <code>SAGE_ROOT/</code>, best containing just the "plain" version number (perhaps only with also the release date added); cf. <a class="closed ticket" href="https://trac.sagemath.org/ticket/9434" title="defect: Stop greping for a nonexistent sagebanner (closed: fixed)">#9434</a> (the number is a coincidence!).
</p>
<p>
What about release notes (in the toplevel directory)?
</p>
TicketkcrismanThu, 16 Sep 2010 13:09:28 GMT
https://trac.sagemath.org/ticket/9433#comment:30
https://trac.sagemath.org/ticket/9433#comment:30
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:29" title="Comment 29">leif</a>:
</p>
<blockquote class="citation">
<p>
I'd also like to have a <code>VERSION.txt</code> in <code>SAGE_ROOT/</code>, best containing just the "plain" version number (perhaps only with also the release date added); cf. <a class="closed ticket" href="https://trac.sagemath.org/ticket/9434" title="defect: Stop greping for a nonexistent sagebanner (closed: fixed)">#9434</a> (the number is a coincidence!).
</p>
</blockquote>
<p>
That's a good idea. Be sure not to hold this one up too long ;)
</p>
<blockquote class="citation">
<p>
What about release notes (in the toplevel directory)?
</p>
</blockquote>
<p>
That definitely used to be in this directory, filename <code>HISTORY.txt</code>. Maybe it was getting long? It certainly got out of date quickly. It might be an onus on the release manager to create the notes *before* the release is made, though  usually that takes a little time, and then it somehow gets added to the official version at sagemath.org (and a few other places?). The problem is that this isn't fully automated yet.
</p>
TicketleifThu, 16 Sep 2010 13:32:02 GMT
https://trac.sagemath.org/ticket/9433#comment:31
https://trac.sagemath.org/ticket/9433#comment:31
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:30" title="Comment 30">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:29" title="Comment 29">leif</a>:
</p>
<blockquote class="citation">
<p>
I'd also like to have a <code>VERSION.txt</code> in <code>SAGE_ROOT/</code>, best containing just the "plain" version number (perhaps only with also the release date added); cf. <a class="closed ticket" href="https://trac.sagemath.org/ticket/9434" title="defect: Stop greping for a nonexistent sagebanner (closed: fixed)">#9434</a> (the number is a coincidence!).
</p>
</blockquote>
<p>
That's a good idea. Be sure not to hold this one up too long ;)
</p>
</blockquote>
<p>
I don't think adding this needs great effort... We already have <code>devel/sage/sage/version.py</code>, which looks like this:
</p>
<div class="wikicode"><div class="code"><pre><span class="sd">"""nodoctests"""</span>
version<span class="o">=</span><span class="s">'4.6.alpha1'</span><span class="p">;</span> date<span class="o">=</span><span class="s">'20100915'</span>
</pre></div></div><blockquote class="citation">
<blockquote class="citation">
<p>
What about release notes (in the toplevel directory)?
</p>
</blockquote>
<p>
That definitely used to be in this directory, filename <code>HISTORY.txt</code>. Maybe it was getting long? It certainly got out of date quickly. It might be an onus on the release manager to create the notes *before* the release is made, though
</p>
</blockquote>
<p>
Doesn't have to be under revision control (i.e. could be in <code>.hgignore</code>); it IMHO shouldn't be too lengthy, perhaps just contain the most recent changes (e.g. tickets merged since last final, with a reference to a complete version elsewhere).
</p>
<blockquote class="citation">
<ul><li>usually that takes a little time, and then it somehow gets added to the official version at sagemath.org (and a few other places?). The problem is that this isn't fully automated yet.
</li></ul></blockquote>
<p>
I think the release notes as in the announcements are meanwhile fully automatically generated by a script. I just wonder if these are really "human"(means user)readable, if we intend that.
</p>
TicketkcrismanThu, 16 Sep 2010 13:42:08 GMT
https://trac.sagemath.org/ticket/9433#comment:32
https://trac.sagemath.org/ticket/9433#comment:32
<p>
Those ideas seem reasonable, but probably I'm not the one to make the call, since I'm not doing the work.
</p>
<blockquote class="citation">
<blockquote class="citation">
<ul><li>usually that takes a little time, and then it somehow gets added to the official version at sagemath.org (and a few other places?). The problem is that this isn't fully automated yet.
</li></ul></blockquote>
<p>
I think the release notes as in the announcements are meanwhile fully automatically generated by a script. I just wonder if these are really "human"(means user)readable, if we intend that.
</p>
</blockquote>
<p>
The hard part is, but at least in theory the "known issues" and "new features" sections and things like that are supposed to be humangenerated. mvngu used to make great categorized ones, but likely hasn't had the time lately. I think they are more humanreadable than some I've seen in other programs, though :)
</p>
TicketjhpalmieriThu, 16 Sep 2010 15:18:40 GMT
https://trac.sagemath.org/ticket/9433#comment:33
https://trac.sagemath.org/ticket/9433#comment:33
<p>
I think that for release notes, we could just have a document which says something like "Please see <a class="extlink" href="http://sagemath.org/mirror/src/changelogs/sage$VERSION.txt"><span class="icon"></span>http://sagemath.org/mirror/src/changelogs/sage$VERSION.txt</a> for a list of recent changes." Or we could use the link "<a class="extlink" href="http://sagemath.org/mirror/src/changelogs/"><span class="icon"></span>http://sagemath.org/mirror/src/changelogs/</a>" and then we wouldn't have to update it. This also provides easy access to older changelogs (which the $VERSION.txt doesn't, and notice it's just a text file  no links to the parent directory or older changelogs). Opinions? We could also add the link "<a class="extlink" href="http://wiki.sagemath.org/ReleaseTours/"><span class="icon"></span>http://wiki.sagemath.org/ReleaseTours/</a>", although this hasn't been updated in a while.
</p>
<p>
I like the generic option better. But notice that this file won't be autogenerated by "sage sdist" or any other script, and it should probably be under revision control. This is not the right ticket for adding new files to the top directory, or for modifying existing files, so I think this should go elsewhere.
</p>
<p>
I also agree that a VERSION.txt file is a good idea. Since we can automatically generate this, and since it shouldn't be under revision control, I think we can do it on this ticket. It should just require modifying sagesdist and sagebdist. I'll try towork on this some time soon.
</p>
TicketleifThu, 16 Sep 2010 15:34:07 GMT
https://trac.sagemath.org/ticket/9433#comment:34
https://trac.sagemath.org/ticket/9433#comment:34
<p>
As a first step, just add <code>VERSION.txt</code> to <code>.hgignore</code> (here). ;)
</p>
<p>
My intention (perhaps as a developer) regarding "release notes" (whatever the file is called) also was <em>not</em> to have to search trac or some web page(s) (or query Mercurial) just to see which tickets have recently been merged...
</p>
<p>
For users, of course a more abstract description would be better (bugs fixed, packages newly included or upgraded, new features etc.)
</p>
TicketjhpalmieriThu, 16 Sep 2010 21:00:50 GMT
https://trac.sagemath.org/ticket/9433#comment:35
https://trac.sagemath.org/ticket/9433#comment:35
<p>
I've opened up <a class="new ticket" href="https://trac.sagemath.org/ticket/9922" title="enhancement: Add release notes to SAGE_ROOT (new)">#9922</a> for adding release notes. For VERSION.txt, I think we (meaning me) should do the whole thing on this ticket: add it to <code>.hgignore</code>, create it in <code>sagesdist</code>, and make sure it gets copied by <code>sagebdist</code>, or else the whole thing should go on another ticket. If I don't get to it on this one, then I'll open a ticket describing the steps (including adding it to <code>.hgignore</code>)  I don't really see the point in doing one piece of it here.
</p>
TicketmpatelThu, 16 Sep 2010 22:09:15 GMT
https://trac.sagemath.org/ticket/9433#comment:36
https://trac.sagemath.org/ticket/9433#comment:36
<p>
Should we update <code>VERSION.txt</code> after successful upgrades? Or maybe have separate fields for the original version (and whether it was a binary) and the current version? Or even keep a brief upgrade history?
</p>
TicketjhpalmieriFri, 17 Sep 2010 01:13:58 GMT
https://trac.sagemath.org/ticket/9433#comment:37
https://trac.sagemath.org/ticket/9433#comment:37
<p>
I've attached a new version of hg_script which rename 'makefile' to 'Makefile' if it hasn't already been done. (I thought I also had to modify rootspkginstall, but I don't think I do: once the repo has been made via hg_script, it will have 'Makefile' in it, not 'makefile'.)
</p>
<p>
I also think mpatel has a good point about VERSION.txt. I think there are possible design decisions to be made about how to create that file, what should go in it, how to modify it, etc., so I think it should go another ticket. It could piggyback on <a class="new ticket" href="https://trac.sagemath.org/ticket/9922" title="enhancement: Add release notes to SAGE_ROOT (new)">#9922</a>, I think. I'm going to change the description of that ticket to include this.
</p>
TicketddrakeFri, 17 Sep 2010 03:42:50 GMT
https://trac.sagemath.org/ticket/9433#comment:38
https://trac.sagemath.org/ticket/9433#comment:38
<p>
Regarding all this VERSION.txt stuff, I think we should stay on task here. This ticket is about putting existing files under version control. Adding new files "while we're at it" is, IMHO, an unnecessary distraction.
</p>
<p>
Once SAGE_ROOT is in a Mercurial repo, we can add new files by just creating a ticket, attaching a patch, and getting a positive review. Let's get that repo there first!
</p>
TicketjhpalmieriFri, 24 Sep 2010 17:13:20 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v2.patch</em>
</li>
</ul>
<p>
fixes problems found by mpatel. apply to scripts repo
</p>
TicketjhpalmieriFri, 24 Sep 2010 17:14:21 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:39
https://trac.sagemath.org/ticket/9433#comment:39
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=39">diff</a>)
</li>
</ul>
TicketleifFri, 24 Sep 2010 17:54:47 GMT
https://trac.sagemath.org/ticket/9433#comment:40
https://trac.sagemath.org/ticket/9433#comment:40
<p>
A lot of environment variables should be quoted (<code>$SAGE_ROOT</code>, <code>$TMP</code>, <code>$OPT</code> etc. in most commands).
</p>
<p>
Needs rebasing in case <a class="closed ticket" href="https://trac.sagemath.org/ticket/9896" title="defect: Upgrading from 4.5.3 to 4.6.alpha* can fail (not limited to MacOS X) (closed: fixed)">#9896</a> gets merged... (or the other way around)
</p>
TicketleifFri, 24 Sep 2010 18:03:29 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:41
https://trac.sagemath.org/ticket/9433#comment:41
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
<code>rootspkginstall</code> also needs work:
</p>
<div class="wikicode"><div class="code"><pre>...
<span class="k">if</span> <span class="o">[</span> e makefile <span class="o">]</span>; <span class="k">then
</span>cp makefile <span class="s2">"$TARGET"</span>/Makefile
<span class="k">else
</span>cp Makefile <span class="s2">"$TARGET"</span>
<span class="k">fi
if</span> <span class="o">[</span> ! d <span class="s2">"$TARGET/skpg"</span> <span class="o">]</span>; <span class="k">then
</span>mkdir <span class="s2">"$TARGET/spkg"</span>
<span class="k">fi
if</span> <span class="o">[</span> ! d <span class="s2">"$TARGET/skpg/standard"</span> <span class="o">]</span>; <span class="k">then
</span>mkdir <span class="s2">"$TARGET/spkg/standard"</span>
...
</pre></div></div><p>
We'd better use <code>f</code> than <code>e</code>, <code>s/skpg/spkg/g</code>.
</p>
TicketleifFri, 24 Sep 2010 18:07:27 GMT
https://trac.sagemath.org/ticket/9433#comment:42
https://trac.sagemath.org/ticket/9433#comment:42
<p>
P.S.: Instead of
</p>
<div class="wikicode"><div class="code"><pre><span class="k">if</span> <span class="o">[</span> ! d foo <span class="o">]</span>; <span class="k">then
</span>mkdir foo
<span class="k">fi</span>
</pre></div></div><p>
you can simply do
</p>
<div class="wikicode"><div class="code"><pre>mkdir p foo
</pre></div></div>
TicketjhpalmieriFri, 24 Sep 2010 18:38:27 GMTstatus, description changed
https://trac.sagemath.org/ticket/9433#comment:43
https://trac.sagemath.org/ticket/9433#comment:43
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=43">diff</a>)
</li>
</ul>
<p>
Okay, I've fixed rootspkginstall, although I could probably just have one line
</p>
<pre class="wiki">mkdir p "$TARGET/spkg/standard"
</pre><p>
instead of two
</p>
<pre class="wiki">mkdir p "$TARGET/spkg"
mkdir p "$TARGET/spkg/standard"
</pre><p>
I've also quoted every variable in sight in the scripts patch.
</p>
TicketleifFri, 24 Sep 2010 19:28:05 GMT
https://trac.sagemath.org/ticket/9433#comment:44
https://trac.sagemath.org/ticket/9433#comment:44
<p>
Okay, <code>test e</code> shouldn't be a problem with <code>bash</code>, only (despite being POSIX) with some Solaris' <code>/bin/sh</code>.
</p>
TicketleifFri, 24 Sep 2010 19:51:24 GMT
https://trac.sagemath.org/ticket/9433#comment:45
https://trac.sagemath.org/ticket/9433#comment:45
<p>
You don't have to quote variable expansions in normal assignments like <code>foo=$BAR</code> or <code>echo</code>s, but never mind.
</p>
<p>
Also, <code>test d foo</code> is superfluous if you <code>rm rf foo</code>. ("force" also implies not raising an error if the directory doesn't exist.)
</p>
<p>
I'll have to take a closer look regarding (quoting) <code>$OPT</code>; this might be wrong in some cases.
</p>
TicketleifFri, 24 Sep 2010 20:13:06 GMT
https://trac.sagemath.org/ticket/9433#comment:46
https://trac.sagemath.org/ticket/9433#comment:46
<p>
There are some quotes missing at the end of <code>sagemake_devel_packages</code> (in the newly included part.)
</p>
<p>
Quoting <code>$OPT</code> <strong>currently</strong> works, since it is
</p>
<div class="wikicode"><div class="code"><pre><span class="nv">OPT</span><span class="o">=</span><span class="s2">"pPR"</span>
</pre></div></div><p>
but it is a bad idea to omit the dash(es) in <code>OPT</code> and prepend it to the expansion.
</p>
<p>
I.e., it should be
</p>
<div class="wikicode"><div class="code"><pre><span class="nv">OPT</span><span class="o">=</span><span class="s2">"pPR"</span>
...
cp <span class="nv">$OPT</span> ... <span class="c"># NOT quoted
</span>
...
cp L <span class="nv">$OPT</span> ... <span class="c"># also NOT quoted
</span></pre></div></div><p>
And I'd suggest renaming <code>OPT</code> to <code>CP_OPTS</code>.
</p>
TicketjhpalmieriFri, 24 Sep 2010 21:54:17 GMT
https://trac.sagemath.org/ticket/9433#comment:47
https://trac.sagemath.org/ticket/9433#comment:47
<blockquote class="citation">
<p>
There are some quotes missing at the end of sagemake_devel_packages (in the newly included part.)
</p>
</blockquote>
<p>
Oops, I don't know how I missed those. Anyway, here are new versions. This changes "OPT" to "CP_OPT" and includes the hyphen.
</p>
TicketjhpalmieriFri, 24 Sep 2010 22:20:28 GMT
https://trac.sagemath.org/ticket/9433#comment:48
https://trac.sagemath.org/ticket/9433#comment:48
<p>
I'm attaching new versions of <code>rootspkginstall</code> and <code>hg_script</code> which omit the file <code>SAGE_ROOT/sagepython</code>. I asked about this <a class="extlink" href="http://groups.google.com/group/sagedevel/browse_thread/thread/13b4632773e3122c#"><span class="icon"></span>on sagedevel</a> and no one demanded that it be kept.
</p>
TicketjhpalmieriFri, 24 Sep 2010 22:20:51 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>hg_script</em>
</li>
</ul>
<p>
script to initialize repository. for use by the release manager.
</p>
TicketjhpalmieriFri, 24 Sep 2010 22:21:10 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>rootspkginstall</em>
</li>
</ul>
<p>
the file SAGE_ROOT/spkg/rootspkginstall
</p>
TicketleifSat, 25 Sep 2010 01:03:14 GMT
https://trac.sagemath.org/ticket/9433#comment:49
https://trac.sagemath.org/ticket/9433#comment:49
<p>
Ok, except that <code>hg_script</code> won't work if <code>$SAGE_ROOT</code> contains spaces, and "sage" should be "Sage" in the messages, the patches and attached files now look fine (with the one exception below).
</p>
<p>
In my opinion more exit codes should be checked (of <code>hg</code>, <code>tar</code> and <code>python</code>), but most of these omissions have been in before, so it is at least "consistent". ;) (And some <code>tar</code> operations are verbose, while others are not. I also think the release date should be UTC or at least contain the [time and] time zone / UTC offset.)
</p>
<p>
But <code>sageupgrade</code> should in any case check that
</p>
<div class="wikicode"><div class="code"><pre>./pipestatus <span class="s2">"sagespkg $ROOT_REPO 2>&1"</span> <span class="s2">"tee a $SAGE_ROOT/spkg/logs/$ROOT_REPO.log"</span>
</pre></div></div><p>
worked before calling <code>./install</code>.
</p>
<p>
I also wonder if this shouldn't (yet) be
</p>
<div class="wikicode"><div class="code"><pre>./pipestatus <span class="s2">"sagespkg $ROOT_REPO 2>&1"</span> <span class="s2">"tee a \"$SAGE_ROOT\"/spkg/logs/$ROOT_REPO.log"</span>
</pre></div></div><p>
(Not tested; the side effects of <code>pipestatus</code> are quite weird.)
</p>
<p>
I haven't yet applied the patches or fully checked the functionality; at least I didn't find errors in the latest attachments. :)
</p>
<p>
Btw, why aren't base packages subject to upgrading? (I would have expected the root spkg there.)
</p>
TicketleifSat, 25 Sep 2010 01:14:37 GMT
https://trac.sagemath.org/ticket/9433#comment:50
https://trac.sagemath.org/ticket/9433#comment:50
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:49" title="Comment 49">leif</a>:
</p>
<blockquote class="citation">
<p>
I also wonder if this shouldn't (yet) be
</p>
</blockquote>
<div class="wikicode"><div class="code"><pre>./pipestatus <span class="s2">"sagespkg $ROOT_REPO 2>&1"</span> <span class="s2">"tee a \"$SAGE_ROOT\"/spkg/logs/$ROOT_REPO.log"</span>
</pre></div></div><p>
(Not tested; the side effects of <code>pipestatus</code> are quite weird.)
</p>
<p>
The above doesn't work; it should be
</p>
<div class="wikicode"><div class="code"><pre>./pipestatus <span class="s2">"sagespkg $ROOT_REPO 2>&1"</span> <span class="s2">"tee a $$SAGE_ROOT/spkg/logs/$ROOT_REPO.log"</span>
</pre></div></div><p>
(since <code>SAGE_ROOT</code> is in the environment anyway).
</p>
TicketleifSat, 25 Sep 2010 01:17:40 GMT
https://trac.sagemath.org/ticket/9433#comment:51
https://trac.sagemath.org/ticket/9433#comment:51
<p>
Ouch. Forget my last comment... (i.e. the "solution") :D
</p>
TicketleifSat, 25 Sep 2010 01:32:40 GMT
https://trac.sagemath.org/ticket/9433#comment:52
https://trac.sagemath.org/ticket/9433#comment:52
<p>
Finally :) it should be:
</p>
<div class="wikicode"><div class="code"><pre>./pipestatus <span class="s2">"sagespkg $ROOT_REPO 2>&1"</span> <span class="s2">"tee a '$SAGE_ROOT'/spkg/logs/$ROOT_REPO.log"</span>
</pre></div></div>
TicketjhpalmieriSat, 25 Sep 2010 01:33:30 GMT
https://trac.sagemath.org/ticket/9433#comment:53
https://trac.sagemath.org/ticket/9433#comment:53
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:49" title="Comment 49">leif</a>:
</p>
<blockquote class="citation">
<p>
Ok, except that <code>hg_script</code> won't work if <code>$SAGE_ROOT</code> contains spaces,
</p>
</blockquote>
<p>
Since this script only gets run once, by the release manager when this ticket is merged, and since it doesn't get distributed with Sage, I'm not going to worry about it. When Sage works with spaces in the path, if this hasn't been merged yet, I can fix it then.
</p>
<blockquote class="citation">
<p>
and "sage" should be "Sage" in the messages,
</p>
</blockquote>
<p>
I'll fix that. You mean messages like "Error building the sage source code package" or "Error copying sage root repository", right?
</p>
<blockquote class="citation">
<p>
In my opinion more exit codes should be checked (of <code>hg</code>, <code>tar</code> and <code>python</code>), but most of these omissions have been in before, so it is at least "consistent". ;) (And some <code>tar</code> operations are verbose, while others are not. I also think the release date should be UTC or at least contain the [time and] time zone / UTC offset.)
</p>
<p>
But <code>sageupgrade</code> should in any case check that
</p>
</blockquote>
<div class="wikicode"><div class="code"><pre>./pipestatus <span class="s2">"sagespkg $ROOT_REPO 2>&1"</span> <span class="s2">"tee a $SAGE_ROOT/spkg/logs/$ROOT_REPO.log"</span>
</pre></div></div><blockquote class="citation">
<p>
worked before calling <code>./install</code>.
</p>
</blockquote>
<p>
You're right, I thought about this before, but didn't do it.
</p>
<blockquote class="citation">
<p>
Btw, why aren't base packages subject to upgrading? (I would have expected the root spkg there.)
</p>
</blockquote>
<p>
I'm not sure what you mean by "base packages". The <code>sageupgrade</code> script will certainly upgrade the sageroot spkg. Let's see, the sageupdate script will download any new spkg, including any sageroot spkg. Then it gets installed in sageupgrade before the other spkgs, in case it changes deps or spkg/install.
</p>
<p>
Should I create an upgrade path on sage.math? I may wait a day or two for the disk situation to settle down...
</p>
<p>
Re the "pipestatus" command, I'm going to leave it as is and see what happens during testing. (When I tested a month ago or so, the sageroot spkg got upgraded, and I vaguely remember checking that the log file had the right name, but I'll test again.)
</p>
TicketleifSat, 25 Sep 2010 01:50:52 GMT
https://trac.sagemath.org/ticket/9433#comment:54
https://trac.sagemath.org/ticket/9433#comment:54
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:53" title="Comment 53">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:49" title="Comment 49">leif</a>:
</p>
<blockquote class="citation">
<p>
Ok, except that <code>hg_script</code> won't work if <code>$SAGE_ROOT</code> contains spaces,
</p>
</blockquote>
<p>
Since this script only gets run once, by the release manager when this ticket is merged, and since it doesn't get distributed with Sage, I'm not going to worry about it.
</p>
</blockquote>
<p>
Yes, this is (and was meant) a nonissue, I only mentioned it for completeness. ;)
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
and "sage" should be "Sage" in the messages,
</p>
</blockquote>
<p>
I'll fix that. You mean messages like "Error building the sage source code package" or "Error copying sage root repository", right?
</p>
</blockquote>
<p>
Yes.
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
Btw, why aren't base packages subject to upgrading? (I would have expected the root spkg there.)
</p>
</blockquote>
<p>
I'm not sure what you mean by "base packages".
</p>
</blockquote>
<p>
The packages /files in <code>spkg/base</code>, also referred to by <code>$(BASE)</code> in the Makefile (<code>deps</code>).
</p>
<blockquote class="citation">
<p>
The <code>sageupgrade</code> script will certainly upgrade the sageroot spkg.
</p>
</blockquote>
<p>
Yes, of course. (<code>sageupdate</code> currently doesn't attempt to upgrade anything from <code>spkg/base</code>.)
</p>
<blockquote class="citation">
<p>
Should I create an upgrade path on sage.math? I may wait a day or two for the disk situation to settle down...
</p>
</blockquote>
<p>
I have no opinion on that.
</p>
<blockquote class="citation">
<p>
Re the "pipestatus" command, I'm going to leave it as is and see what happens during testing. (When I tested a month ago or so, the sageroot spkg got upgraded, and I vaguely remember checking that the log file had the right name, but I'll test again.)
</p>
</blockquote>
<p>
This again just referred to <em>spaces in SAGE_ROOT</em>. (My last suggestions would work with such.)
</p>
TicketjhpalmieriSat, 25 Sep 2010 02:17:32 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v3.patch</em>
</li>
</ul>
<p>
apply to scripts repo
</p>
TicketjhpalmieriSat, 25 Sep 2010 02:17:51 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v2tov3.patch</em>
</li>
</ul>
<p>
for reference only: diff between v2 and v3 patches.
</p>
TicketjhpalmieriSat, 25 Sep 2010 02:21:05 GMT
https://trac.sagemath.org/ticket/9433#comment:55
https://trac.sagemath.org/ticket/9433#comment:55
<p>
Here are new versions; the only difference with any content (i.e., other than capitalization, I think) is from sageupgrade:
</p>
<div class="wikicode"><div class="code"><pre><span class="gd">./pipestatus "sagespkg $ROOT_REPO 2>&1" "tee a $SAGE_ROOT/spkg/logs/$ROOT_REPO.log"
</span><span class="gi">+./pipestatus "sagespkg $ROOT_REPO 2>&1" "tee a '$SAGE_ROOT'/spkg/logs/$ROOT_REPO.log"
+
+if [ $? ne 0 ]; then
+ echo "Error installing Sage root repository."
+ exit 1
+fi
</span></pre></div></div><p>
I don't know why base packages don't get upgraded. That's for another ticket.
</p>
<blockquote class="citation">
<p>
Should I create an upgrade path on sage.math?
</p>
</blockquote>
<p>
I meant so that people could test "sage upgrade". I'll do this pretty soon.
</p>
TicketjdemeyerSat, 23 Oct 2010 21:31:03 GMTmilestone changed; reviewer set
https://trac.sagemath.org/ticket/9433#comment:56
https://trac.sagemath.org/ticket/9433#comment:56
<ul>
<li><strong>reviewer</strong>
set to <em>Leif Leonhardy</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage4.6</em> to <em>sage4.6.1</em>
</li>
</ul>
TicketddrakeFri, 05 Nov 2010 11:24:51 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433sagerepo.2.patch</em>
</li>
</ul>
<p>
rebased for 4.6.1.alpha0
</p>
TicketddrakeFri, 05 Nov 2010 11:25:15 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v3rebased.patch</em>
</li>
</ul>
<p>
rebased for 4.6.1.alpha0
</p>
TicketddrakeFri, 05 Nov 2010 11:28:59 GMT
https://trac.sagemath.org/ticket/9433#comment:57
https://trac.sagemath.org/ticket/9433#comment:57
<p>
I rebased the above two patches. The main Sage repo one is fine, but the scripts patch had rejects on some huge hunks and it took a while to manually put them in. Someone should check over the "rebased" patch and make sure I did everything correctly.
</p>
TicketjhpalmieriFri, 05 Nov 2010 17:51:21 GMT
https://trac.sagemath.org/ticket/9433#comment:58
https://trac.sagemath.org/ticket/9433#comment:58
<p>
Hi Dan,
</p>
<p>
Thanks for working on this. The Sage repo patch looks good. Skimming the scripts patch, the only issue I see with the rebasing is that I think $PKGDIR should be quoted in the following (this is lines 3941 in sagesdist):
</p>
<pre class="wiki">cp "$SAGE_ROOT"/local/bin/sagespkg $PKGDIR/base/
cp "$SAGE_ROOT"/local/bin/sageenv $PKGDIR/base/
cp "$SAGE_ROOT"/local/bin/sagemake_relative $PKGDIR/base/
</pre><p>
I'll look at this more carefully when I have time.
</p>
TicketjhpalmieriFri, 05 Nov 2010 21:52:47 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:59
https://trac.sagemath.org/ticket/9433#comment:59
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=59">diff</a>)
</li>
</ul>
<p>
I also found the following corrections to be added to the part of the patch for sagebdist:
</p>
<div class="wikicode"><div xmlns="http://www.w3.org/1999/xhtml" class="diff">
<ul class="entries">
<li class="entry">
<h2>
<a>sagebdist</a>
</h2>
<pre>diff r fa7bc24587ef sagebdist</pre>
<table class="tracdiff inline" summary="Differences" cellspacing="0">
<colgroup><col class="lineno" /><col class="lineno" /><col class="content" /></colgroup>
<thead>
<tr>
<th title="File a/sagebdist Fri Sep 24 19:13:24 2010 0700">
a
</th>
<th title="File b/sagebdist Fri Nov 05 14:45:36 2010 0700">
b
</th>
<td><em></em> </td>
</tr>
</thead>
<tbody class="unmod">
<tr>
<th>46</th><th>46</th><td class="l"><span>if [ $? ne 0 ]; then</span></td>
</tr><tr>
<th>47</th><th>47</th><td class="l"><span> echo "Error copying Sage root repository."</span></td>
</tr><tr>
<th>48</th><th>48</th><td class="l"><span> exit 1</span></td>
</tr>
</tbody><tbody class="add">
<tr class="last first">
<th> </th><th>49</th><td class="r"><ins>fi</ins></td>
</tr>
</tbody><tbody class="unmod">
<tr>
<th>49</th><th>50</th><td class="l"><span></span></td>
</tr><tr>
<th>50</th><th>51</th><td class="l"><span>rm "$TMP"/.hg/hgrc</span></td>
</tr><tr>
<th>51</th><th>52</th><td class="l"><span>echo "Done copying root repository."</span></td>
</tr><tr>
<th>52</th><th>53</th><td class="l"><span></span></td>
</tr>
</tbody><tbody class="rem">
<tr class="first">
<th>53</th><th> </th><td class="l"><del>mkdir "$TMP"</del></td>
</tr><tr class="last">
<th>54</th><th> </th><td class="l"><del></del></td>
</tr>
</tbody><tbody class="unmod">
<tr>
<th>55</th><th>54</th><td class="l"><span>cd "$SAGE_ROOT"</span></td>
</tr><tr>
<th>56</th><th>55</th><td class="l"><span></span></td>
</tr><tr>
<th>57</th><th>56</th><td class="l"><span>echo "Copying files over to tmp directory"</span></td>
</tr>
</tbody><tbody class="mod">
<tr class="first">
<th>58</th><th> </th><td class="l"><span>cp <del></del>$CP_OPT examples local data "$TMP"/</span></td>
</tr>
<tr class="last">
<th> </th><th>57</th><td class="r"><span>cp <ins></ins>$CP_OPT examples local data "$TMP"/</span></td>
</tr>
</tbody><tbody class="unmod">
<tr>
<th>59</th><th>58</th><td class="l"><span></span></td>
</tr><tr>
<th>60</th><th>59</th><td class="l"><span>if [ d devel/sagemain ]; then</span></td>
</tr><tr>
<th>61</th><th>60</th><td class="l"><span> echo "Copying Sage library over"</span></td>
</tr>
</tbody>
<tbody class="skipped">
<tr>
<th><a href="#L63">…</a></th>
<th><a href="#L62">…</a></th>
<td><em></em> </td>
</tr>
</tbody>
<tbody class="unmod">
<tr>
<th>63</th><th>62</th><td class="l"><span> cp L $CP_OPT devel/sagenbmain "$TMP"/devel/sagenbmain</span></td>
</tr><tr>
<th>64</th><th>63</th><td class="l"><span> cp L $CP_OPT devel/sagemain "$TMP"/devel/sagemain</span></td>
</tr><tr>
<th>65</th><th>64</th><td class="l"><span> cd "$TMP"/devel</span></td>
</tr>
</tbody><tbody class="rem">
<tr class="last first">
<th>66</th><th> </th><td class="l"><del> cd $TMP/devel</del></td>
</tr>
</tbody><tbody class="unmod">
<tr>
<th>67</th><th>65</th><td class="l"><span> ln s sagemain sage</span></td>
</tr><tr>
<th>68</th><th>66</th><td class="l"><span> ln s sagenbmain sagenb</span></td>
</tr><tr>
<th>69</th><th>67</th><td class="l"><span> cd sage</span></td>
</tr>
</tbody>
</table>
</li>
</ul>
</div></div><p>
I'm posting a "v4" patch including this and the above change in sagesdist (quoting "$PKGDIR").
</p>
TicketjhpalmieriFri, 05 Nov 2010 21:53:11 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v4.patch</em>
</li>
</ul>
<p>
rebased for 4.6.1.alpha0
</p>
TicketddrakeFri, 05 Nov 2010 22:16:22 GMT
https://trac.sagemath.org/ticket/9433#comment:60
https://trac.sagemath.org/ticket/9433#comment:60
<p>
I used my rebased patches and 4.6.1.alpha0 to initialize the root repo; I sdisted that and the resulting thing has the correct root repo and built fine  so everything here seems to work "going forward"; I haven't tested any upgrading yet.
</p>
TicketjhpalmieriFri, 05 Nov 2010 22:28:26 GMT
https://trac.sagemath.org/ticket/9433#comment:61
https://trac.sagemath.org/ticket/9433#comment:61
<p>
I'm working on providing an upgrade path (or two) for testing. I'll post here when I have links.
</p>
TicketjhpalmieriSat, 06 Nov 2010 00:06:51 GMT
https://trac.sagemath.org/ticket/9433#comment:62
https://trac.sagemath.org/ticket/9433#comment:62
<p>
Okay for testing upgrading:
</p>
<pre class="wiki">./sage upgrade http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.1.9433.alpha0/
</pre><p>
This first upgrade path is just a vanilla version of 4.6.1.alpha0, plus the new root repo.
</p>
<pre class="wiki">./sage upgrade http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.1.9433.alpha1/
</pre><p>
The second upgrade path contains everything from the first one, plus a change to the toplevel README.txt file. This is so we can test upgrading to a plain root repo, then from there to a modified version. So upgrading to "...9433.alpha0" followed by "...9433.alpha1" should work, as well as just upgrading straight to "...9433.alpha1".
</p>
<p>
I think that if you upgrade from a version with a Sage root repo to either of these, you may be asked twice if you want to upgrade. I mentioned this issue <a class="extlink" href="http://trac.sagemath.org/sage_trac/ticket/9433#comment:22"><span class="icon"></span>above</a>.
</p>
TicketjhpalmieriSat, 06 Nov 2010 03:49:34 GMT
https://trac.sagemath.org/ticket/9433#comment:63
https://trac.sagemath.org/ticket/9433#comment:63
<p>
One issue: if you upgrade a version of Sage containing "makefile", then you end up with both "makefile" and "Makefile" when you're done. We can just delete "$SAGE_ROOT/makefile" when running the rootspkginstall file. That's easy enough to do; is it the right solution?
</p>
TicketleifSat, 06 Nov 2010 04:04:35 GMT
https://trac.sagemath.org/ticket/9433#comment:64
https://trac.sagemath.org/ticket/9433#comment:64
<p>
Hmmm, this doesn't work for upgrades from older versions (<=4.5), and as you mentioned, behaves strange on newer versions >=4.5.1.
</p>
<p>
We <strong>have to</strong> download <code>spkg/install</code> (and I'd prefer the rest, too) in <code>sageupdate</code> (the Python script).
</p>
<p>
What's currently newly made in <code>sageupgrade</code> should be moved to that file, which is in any case the fresh, downloaded one. <em>There</em> we have to make a distinction on if we are upgrading.
</p>
<p>
If so, <code>spkg/install</code> should then install the root repo (preferably without overwriting itself, but the root repo should, i.e. must, contain exactly the same file), since our "real" Makefile <code>spkg/standard/deps</code> doesn't (though I don't know why you don't want to put the root spkg there).
</p>
<p>
Btw, it would be better to also have <code>sagespkg</code> in the root rather than the scripts repo; or download an identical copy (from the scripts repo) along with <code>install</code>, <code>deps</code> etc.
</p>
<p>
This way we would always do the whole build with the new one, and not switch the version during the installation (after the new scripts spkg has been installed).
</p>
<p>
Another reason to keep the repos separate, and not as Robert B. suggested on sagedevel, merge them all into an even larger, monolithic one.
</p>
<p>
(To avoid trouble with getting overwritten, a script can always clone itself and then <code>exec</code> this copy, passing it a parameter such that it knows if it's the clone or the original, similar to <code>fork()</code>. We should IMHO do this for a few scripts involved in upgrading and the build process. Therefore, the chain or stack of called rather than <code>exec</code>'ed scripts shouldn't be deep.)
</p>
TicketleifSat, 06 Nov 2010 04:06:08 GMT
https://trac.sagemath.org/ticket/9433#comment:65
https://trac.sagemath.org/ticket/9433#comment:65
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:63" title="Comment 63">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
One issue: if you upgrade a version of Sage containing "makefile", then you end up with both "makefile" and "Makefile" when you're done. We can just delete "$SAGE_ROOT/makefile" when running the rootspkginstall file. That's easy enough to do; is it the right solution?
</p>
</blockquote>
<p>
Rename it to <code>Makefile.old</code>?
</p>
TicketleifSat, 06 Nov 2010 04:09:07 GMT
https://trac.sagemath.org/ticket/9433#comment:66
https://trac.sagemath.org/ticket/9433#comment:66
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:62" title="Comment 62">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
I think that if you upgrade from a version with a Sage root repo to either of these, you may be asked twice if you want to upgrade. I mentioned this issue <a class="extlink" href="http://trac.sagemath.org/sage_trac/ticket/9433#comment:22"><span class="icon"></span>above</a>.
</p>
</blockquote>
<p>
According to the comment you refer to, this should read <em>"upgrade from a version <strong>without</strong> a Sage root repo"</em>.
</p>
TicketjhpalmieriSat, 06 Nov 2010 06:11:03 GMT
https://trac.sagemath.org/ticket/9433#comment:67
https://trac.sagemath.org/ticket/9433#comment:67
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:64" title="Comment 64">leif</a>:
</p>
<blockquote class="citation">
<p>
We <strong>have to</strong> download <code>spkg/install</code> (and I'd prefer the rest, too) in <code>sageupdate</code> (the Python script).
</p>
</blockquote>
<p>
I'm not sure what "the rest" refers to here.
</p>
<blockquote class="citation">
<p>
What's currently newly made in <code>sageupgrade</code> should be moved to that file, which is in any case the fresh, downloaded one. <em>There</em> we have to make a distinction on if we are upgrading.
</p>
</blockquote>
<p>
Okay, I think I can do that.
</p>
<blockquote class="citation">
<p>
If so, <code>spkg/install</code> should then install the root repo (preferably without overwriting itself, but the root repo should, i.e. must, contain exactly the same file), since our "real" Makefile <code>spkg/standard/deps</code> doesn't (though I don't know why you don't want to put the root spkg there).
</p>
</blockquote>
<p>
I don't want to put it there because in a nonupgrade, it's already installed as part of the download.
</p>
<blockquote class="citation">
<p>
Btw, it would be better to also have <code>sagespkg</code> in the root rather than the scripts repo; or download an identical copy (from the scripts repo) along with <code>install</code>, <code>deps</code> etc.
</p>
</blockquote>
<p>
That's not a bad idea, but it should go on another ticket. What about the other scripts in spkg/base, for example sageenv?
</p>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:65" title="Comment 65">leif</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:63" title="Comment 63">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
One issue: if you upgrade a version of Sage containing "makefile", then you end up with both "makefile" and "Makefile" when you're done. We can just delete "$SAGE_ROOT/makefile" when running the rootspkginstall file. That's easy enough to do; is it the right solution?
</p>
</blockquote>
<p>
Rename it to <code>Makefile.old</code>?
</p>
</blockquote>
<p>
That sounds good to me.
</p>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:66" title="Comment 66">leif</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:62" title="Comment 62">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
I think that if you upgrade from a version with a Sage root repo to either of these, you may be asked twice if you want to upgrade. I mentioned this issue <a class="extlink" href="http://trac.sagemath.org/sage_trac/ticket/9433#comment:22"><span class="icon"></span>above</a>.
</p>
</blockquote>
<p>
According to the comment you refer to, this should read <em>"upgrade from a version <strong>without</strong> a Sage root repo"</em>.
</p>
</blockquote>
<p>
Yes, that's what I meant to say. Sorry for any confusion.
</p>
TicketleifSat, 06 Nov 2010 08:40:27 GMT
https://trac.sagemath.org/ticket/9433#comment:68
https://trac.sagemath.org/ticket/9433#comment:68
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:67" title="Comment 67">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:64" title="Comment 64">leif</a>:
</p>
<blockquote class="citation">
<p>
We <strong>have to</strong> download <code>spkg/install</code> (and I'd prefer the rest, too) in <code>sageupdate</code> (the Python script).
</p>
</blockquote>
<p>
I'm not sure what "the rest" refers to here.
</p>
</blockquote>
<p>
Well, the files that were previously downloaded (<code>deps</code> and <code>newest_version</code>, too). But also downloading <code>sageenv</code> (and e.g. <code>sagespkg</code>) is not a bad idea.
</p>
<p>
(I also plan to download <code>upgradenotes.txt</code> and <code>preupgradescript.sh</code> first, such that the user (and we) can make an informed choice.)
</p>
<p>
<br />
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
What's currently newly made in <code>sageupgrade</code> should be moved to that file, which is in any case the fresh, downloaded one. <em>There</em> we have to make a distinction on if we are upgrading.
</p>
</blockquote>
<p>
Okay, I think I can do that.
</p>
<blockquote class="citation">
<p>
If so, <code>spkg/install</code> should then install the root repo (preferably without overwriting itself, but the root repo should, i.e. must, contain exactly the same file), since our "real" Makefile <code>spkg/standard/deps</code> doesn't (though I don't know why you don't want to put the root spkg there).
</p>
</blockquote>
<p>
I don't want to put it there because in a nonupgrade, it's already installed as part of the download.
</p>
</blockquote>
<p>
Well, then <code>hg pull</code> would simply be a noop. But we should check the exit codes of the commands in <code>rootspkginstall</code> (<code>hg</code> and <code>cp</code>) anyway.
</p>
<p>
<code>hg incoming <source repo></code> returns 1 if there's nothing to pull, 0 if there are changes to pull, other values on errors, so you could change it to:
</p>
<div class="wikicode"><div class="code"><pre>...
<span class="k">if</span> <span class="o">[</span> d <span class="s2">"$TARGET"</span>/.hg <span class="o">]</span>; <span class="k">then</span>
<span class="c"># Merge the repository, rather than overwrite changes that the
</span> <span class="c"># user may have made.
</span> <span class="nb">cd</span> <span class="s2">"$TARGET"</span>
hg ci <span class="c"># Don't know if we should check in unconditionally,
</span> <span class="c"># so perhaps move this below the ifeliffi.
</span> <span class="c"># should check for errors here
</span>
hg incoming <span class="s2">"$CUR"</span> <span class="c"># perhaps redirect output
</span> <span class="k">if</span> <span class="o">[</span> <span class="nv">$?</span> eq 1 <span class="o">]</span>; <span class="k">then</span>
<span class="c"># No changes to pull
</span> <span class="nb">exit </span>0
<span class="k">elif</span> <span class="o">[</span> <span class="nv">$?</span> ne 0 <span class="o">]</span>; <span class="k">then</span>
<span class="c"># Some error, report...
</span> <span class="nb">exit </span>1
<span class="k">fi</span>
<span class="c"># $? = 0: Changes to pull...
</span> hg pull <span class="s2">"$CUR"</span>
hg merge tip
hg ci m <span class="s2">"Checkin during upgrade of Sage."</span>
hg update
<span class="c"># Add error checks above, too.
</span>
<span class="k">else</span>
...
</pre></div></div><p>
(Or you could add a rule to <code>deps</code> that tests its presence / the need to pull first, before calling <code>sagespkg</code>. But if we one day generate <code>deps</code>, this would be less optimal.)
</p>
<p>
<br />
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
Btw, it would be better to also have <code>sagespkg</code> in the root rather than the scripts repo; or download an identical copy (from the scripts repo) along with <code>install</code>, <code>deps</code> etc.
</p>
</blockquote>
<p>
That's not a bad idea, but it should go on another ticket. What about the other scripts in spkg/base, for example sageenv?
</p>
</blockquote>
<p>
Yes, see above. Once we have this ticket in, it will be easier (or at least safer) to make such changes. :)
</p>
<p>
P.S.: I wonder if the presence of <code>$SAGE_ROOT/.hg</code> guarantees us a functional Mercurial... ;)
</p>
TicketjhpalmieriSat, 06 Nov 2010 15:33:30 GMT
https://trac.sagemath.org/ticket/9433#comment:69
https://trac.sagemath.org/ticket/9433#comment:69
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:68" title="Comment 68">leif</a>:
</p>
<p>
Downloading these various files in sageupdate seems to at least partly defeat the purpose of the new repo, but anyway.
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:67" title="Comment 67">jhpalmieri</a>:
</p>
</blockquote>
<blockquote class="citation">
<blockquote class="citation">
<p>
I don't want to put it [deps] because in a nonupgrade, it's already installed as part of the download.
</p>
</blockquote>
<p>
Well, then <code>hg pull</code> would simply be a noop. But we should check the exit codes of the commands in <code>rootspkginstall</code> (<code>hg</code> and <code>cp</code>) anyway.
</p>
</blockquote>
<p>
If we put it in deps, I guess it gets installed at the end? I was tempted to make it part of BASE, but then during an upgrade, any changes to the Sage root repo would mean that everything would be rebuilt, which would be annoying. The only issue is if there is an upgrade to a script like "pipestatus" which is in the root repo, is used in the upgrade process, and is not downloaded in sageupdate.
</p>
<blockquote class="citation">
<p>
P.S.: I wonder if the presence of <code>$SAGE_ROOT/.hg</code> guarantees us a functional Mercurial... ;)
</p>
</blockquote>
<p>
I think I'll switch to running "hg verify".
</p>
TicketjhpalmieriSat, 06 Nov 2010 17:52:40 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:70
https://trac.sagemath.org/ticket/9433#comment:70
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=70">diff</a>)
</li>
</ul>
<p>
Okay, here's a new scripts patch (with no changes to sageupdate or sageupgrade), along with new "deps" and "install". I've updated the upgrade paths I listed above; I'm adding them to the ticket description. I am still building Sage 4.4; when that's done, I'll test upgrading it.
</p>
TicketjhpalmieriSat, 06 Nov 2010 17:52:53 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:71
https://trac.sagemath.org/ticket/9433#comment:71
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=71">diff</a>)
</li>
</ul>
TicketjhpalmieriSat, 06 Nov 2010 17:55:17 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>install</em>
</li>
</ul>
<p>
the file SAGE_ROOT/spkg/install
</p>
TicketjhpalmieriSat, 06 Nov 2010 17:55:53 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>install.patch</em>
</li>
</ul>
<p>
diff between current install and new one; for reference only
</p>
TicketjhpalmieriSat, 06 Nov 2010 17:57:26 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>rootspkginstall.v2</em>
</li>
</ul>
<p>
the file SAGE_ROOT/spkg/rootspkginstall
</p>
TicketjhpalmieriSat, 06 Nov 2010 17:58:05 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v5.patch</em>
</li>
</ul>
<p>
patch for scripts repo
</p>
TicketjhpalmieriSat, 06 Nov 2010 17:59:34 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:72
https://trac.sagemath.org/ticket/9433#comment:72
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=72">diff</a>)
</li>
</ul>
TicketjhpalmieriSat, 06 Nov 2010 18:47:30 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>deps</em>
</li>
</ul>
<p>
the file SAGE_ROOT/spkg/standard/deps
</p>
TicketjhpalmieriSat, 06 Nov 2010 18:47:42 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>deps.patch</em>
</li>
</ul>
<p>
diff between current deps and new one; for reference only
</p>
TicketjhpalmieriSun, 07 Nov 2010 00:03:30 GMT
https://trac.sagemath.org/ticket/9433#comment:73
https://trac.sagemath.org/ticket/9433#comment:73
<p>
For what it's worth, I've done the following successfully with the current versions:
</p>
<ul><li>build from scratch ("./sage sdist ..." produced the tar file <a class="extlink" href="http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.1.9433.alpha0.tar"><span class="icon"></span>http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.1.9433.alpha0.tar</a>)
</li><li>upgrade from 4.6.1.alpha0, and then upgrade again to the version with a modified root repo
</li><li>same, but started from 4.4
</li></ul><p>
This was all on sage.math. I also tested 4.6.1.alpha0 on OS X 10.6 with no problems.
</p>
<p>
I'm testing "./sage bdist ...", which I forgot to do earlier. I'm also testing a build from scratch on another platform.
</p>
TicketjdemeyerSun, 07 Nov 2010 17:47:23 GMT
https://trac.sagemath.org/ticket/9433#comment:74
https://trac.sagemath.org/ticket/9433#comment:74
<p>
Is this ticket now really ready for review? (if not, please change status to needs_work).
</p>
TicketjdemeyerSun, 07 Nov 2010 17:51:15 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:75
https://trac.sagemath.org/ticket/9433#comment:75
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
Please also add a <code>rootspkginstall.patch</code>. I prefer using the patches, not the patched files.
</p>
TicketjdemeyerSun, 07 Nov 2010 17:57:40 GMT
https://trac.sagemath.org/ticket/9433#comment:76
https://trac.sagemath.org/ticket/9433#comment:76
<p>
Can the people involved in this ticket give their opinion about <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a>? (informally, the patch is not yet ready for review) If you agree with <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a>, we need to coordinate this ticket with it.
</p>
TicketjhpalmieriSun, 07 Nov 2010 18:52:56 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:77
https://trac.sagemath.org/ticket/9433#comment:77
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:75" title="Comment 75">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Please also add a <code>rootspkginstall.patch</code>. I prefer using the patches, not the patched files.
</p>
</blockquote>
<p>
I'm not sure what you mean: the file "rootspkginstall" is completely new to this ticket, and is not part of any preexisting repo. So there's nothing to patch. I'm flipping this back to "needs review".
</p>
<p>
Regarding <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a>, I'd like to see more discussion on sagedevel about it. I know that leif disagrees, but I'm also interested in the idea of combining the various nonroot repos (scripts, examples, extcode, and the main Sage library) into one. It seems like the Sage community should decide about both of these issues in sagedevel rather than on a trac ticket.
</p>
TicketjdemeyerSun, 07 Nov 2010 20:17:56 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:78
https://trac.sagemath.org/ticket/9433#comment:78
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=78">diff</a>)
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:77" title="Comment 77">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:75" title="Comment 75">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Please also add a <code>rootspkginstall.patch</code>. I prefer using the patches, not the patched files.
</p>
</blockquote>
<p>
I'm not sure what you mean: the file "rootspkginstall" is completely new to this ticket, and is not part of any preexisting repo.
</p>
</blockquote>
<p>
In that case, sorry for the noise.
</p>
TicketjdemeyerSun, 07 Nov 2010 20:26:57 GMT
https://trac.sagemath.org/ticket/9433#comment:79
https://trac.sagemath.org/ticket/9433#comment:79
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:77" title="Comment 77">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Regarding <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a>, I'd like to see more discussion on sagedevel about it. I know that leif disagrees, but I'm also interested in the idea of combining the various nonroot repos (scripts, examples, extcode, and the main Sage library) into one. It seems like the Sage community should decide about both of these issues in sagedevel rather than on a trac ticket.
</p>
</blockquote>
<p>
Well, there was some discussion at <a class="extlink" href="http://groups.google.com/group/sagedevel/browse_thread/thread/cc30eaf87b283881/c2290991827fec9c?hl=en_US&#c2290991827fec9c"><span class="icon"></span>http://groups.google.com/group/sagedevel/browse_thread/thread/cc30eaf87b283881/c2290991827fec9c?hl=en_US&#c2290991827fec9c</a>.
</p>
<p>
Personally, I simply don't see the need for merging sage, sage_scripts, extcode, examples,... into one big spkg. So there is some disagreement on this. But this disagreement should not stop us from doing something to which most people seemed to agree: not repackaging extcode and examples with every new Sage version (which is exactly what I want to accomplish with <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a>).
</p>
TicketjhpalmieriSun, 07 Nov 2010 23:33:15 GMT
https://trac.sagemath.org/ticket/9433#comment:80
https://trac.sagemath.org/ticket/9433#comment:80
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:79" title="Comment 79">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Well, there was some discussion at <a class="extlink" href="http://groups.google.com/group/sagedevel/browse_thread/thread/cc30eaf87b283881/c2290991827fec9c?hl=en_US&#c2290991827fec9c"><span class="icon"></span>http://groups.google.com/group/sagedevel/browse_thread/thread/cc30eaf87b283881/c2290991827fec9c?hl=en_US&#c2290991827fec9c</a>.
</p>
</blockquote>
<p>
Not too much discussion of that proposal. You suggested it, and a few people liked the idea. On the other hand, a few people made the counterproposal to merge all of the repositories. If we end up merging everything, then doing <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a> first will make more work: whatever is involved in that, and then the merge, as opposed to just the merge. So I'd like to see more of a consensus about which direction to go before putting much effort into <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a>.
</p>
<p>
This ticket seems like a separate issue, and regardless of everything else, the Sage root repo should remain separate from the others, because it should already be installed when you unpack the Sage tar ball. Perhaps its role should increase, including files like sageenv and sagespkg, but that's for another ticket. I hope this ticket can get a positive review soon, and then if anyone feels like working on <a class="closed ticket" href="https://trac.sagemath.org/ticket/10231" title="enhancement: Don't repackage examples and extcode on sage*dist (closed: invalid)">#10231</a> (or on a possible merge of the repos), they can base their work on this ticket and other tickets which touch sagesdist and related files.
</p>
TicketddrakeTue, 09 Nov 2010 01:44:15 GMT
https://trac.sagemath.org/ticket/9433#comment:81
https://trac.sagemath.org/ticket/9433#comment:81
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:62" title="Comment 62">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Okay for testing upgrading:
</p>
<pre class="wiki">./sage upgrade http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.1.9433.alpha0/
</pre></blockquote>
<p>
I've tried twice with 4.6 using the above command, and despite the upgrade appearing to work, I get this when I start Sage:
</p>
<pre class="wiki">
 Sage Version 4.6.1.9433.alpha0, Release Date: 20101106 
 Type notebook() for the GUI, and license() for information. 

**********************************************************************
* *
* Warning: this is a prerelease version, and it may be unstable. *
* *
**********************************************************************
/home/drake/s/9433/sage4.6/local/lib/python2.6/sitepackages/sage/plot/plot3d/implicit_plot3d.py:5: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
from implicit_surface import ImplicitSurface
/home/drake/s/9433/sage4.6/local/lib/python2.6/sitepackages/sage/plot/plot3d/implicit_plot3d.py:5: RuntimeWarning: numpy.flatiter size changed, may indicate binary incompatibility
from implicit_surface import ImplicitSurface
/home/drake/s/9433/sage4.6/local/lib/python2.6/sitepackages/sage/calculus/all.py:20: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
from interpolators import polygon_spline, complex_cubic_spline
/home/drake/s/9433/sage4.6/local/lib/python2.6/sitepackages/sage/calculus/all.py:20: RuntimeWarning: numpy.flatiter size changed, may indicate binary incompatibility
from interpolators import polygon_spline, complex_cubic_spline
/home/drake/s/9433/sage4.6/local/lib/python2.6/sitepackages/sage/stats/hmm/all.py:8: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
from hmm import DiscreteHiddenMarkovModel
/home/drake/s/9433/sage4.6/local/lib/python2.6/sitepackages/sage/stats/hmm/all.py:8: RuntimeWarning: numpy.flatiter size changed, may indicate binary incompatibility
from hmm import DiscreteHiddenMarkovModel
sage:
</pre>
TicketkcrismanTue, 09 Nov 2010 01:47:39 GMT
https://trac.sagemath.org/ticket/9433#comment:82
https://trac.sagemath.org/ticket/9433#comment:82
<p>
Dan, you have to touch the Cython include file and then do <code>./sage b</code> to build the dependencies  see <a class="closed ticket" href="https://trac.sagemath.org/ticket/9808" title="task: Upgrade numpy to 1.5.0 and scipy to 0.8 (closed: fixed)">#9808</a> for details. Or you could do <code>./sage ba</code> and wait a long time.
</p>
TicketddrakeTue, 09 Nov 2010 02:50:57 GMT
https://trac.sagemath.org/ticket/9433#comment:83
https://trac.sagemath.org/ticket/9433#comment:83
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:82" title="Comment 82">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
Dan, you have to touch the Cython include file and then do <code>./sage b</code> to build the dependencies  see <a class="closed ticket" href="https://trac.sagemath.org/ticket/9808" title="task: Upgrade numpy to 1.5.0 and scipy to 0.8 (closed: fixed)">#9808</a> for details. Or you could do <code>./sage ba</code> and wait a long time.
</p>
</blockquote>
<p>
That fixed it! Thanks. The first upgrade works fine now; I'm testing the second.
</p>
TicketddrakeWed, 10 Nov 2010 01:27:52 GMT
https://trac.sagemath.org/ticket/9433#comment:84
https://trac.sagemath.org/ticket/9433#comment:84
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:62" title="Comment 62">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Okay for testing upgrading:
</p>
</blockquote>
<p>
I started with 4.6, and both upgrade paths work fine: first to "9433.alpha0", then to 9433.alpha1, and also directly to 9433.alpha1. I'm currently testing with upgrades from 4.3.5.
</p>
TicketddrakeWed, 10 Nov 2010 07:08:19 GMT
https://trac.sagemath.org/ticket/9433#comment:85
https://trac.sagemath.org/ticket/9433#comment:85
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:84" title="Comment 84">ddrake</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:62" title="Comment 62">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Okay for testing upgrading:
</p>
</blockquote>
<p>
I started with 4.6, and both upgrade paths work fine: first to "9433.alpha0", then to 9433.alpha1, and also directly to 9433.alpha1. I'm currently testing with upgrades from 4.3.5.
</p>
</blockquote>
<p>
Both upgrade paths also work when upgrading from 4.3.5. (I'm using 64bit Ubuntu 10.04.)
</p>
TicketjhpalmieriThu, 11 Nov 2010 21:05:55 GMT
https://trac.sagemath.org/ticket/9433#comment:86
https://trac.sagemath.org/ticket/9433#comment:86
<p>
I just added "VERSION.txt" to the .hgignore file, for compatibility with <a class="closed ticket" href="https://trac.sagemath.org/ticket/9434" title="defect: Stop greping for a nonexistent sagebanner (closed: fixed)">#9434</a>.
</p>
TicketvbraunThu, 13 Jan 2011 07:13:22 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>install.2</em>
</li>
</ul>
<p>
Updated spkg/install
</p>
TicketvbraunThu, 13 Jan 2011 07:19:18 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>deps.2</em>
</li>
</ul>
<p>
Updated spkg/standard/deps
</p>
TicketvbraunThu, 13 Jan 2011 07:38:33 GMTstatus, reviewer, description, milestone changed
https://trac.sagemath.org/ticket/9433#comment:87
https://trac.sagemath.org/ticket/9433#comment:87
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
<li><strong>reviewer</strong>
changed from <em>Leif Leonhardy</em> to <em>Leif Leonhardy, Volker Braun</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=87">diff</a>)
</li>
<li><strong>milestone</strong>
changed from <em>sage4.6.1</em> to <em>sage4.6.2</em>
</li>
</ul>
<p>
After some rediffing I built it successfully on top of Sage4.6.1.rc1 (identical to the Sage4.6.1 release) with the following updated files:
</p>
<ul><li><code>trac_9433scripts.v5.patch</code> > <code>trac_9433scripts.v5.2.patch</code>
</li><li><code>install</code> > <code>install.2</code>
</li><li><code>deps</code> > <code>deps.2</code>
</li></ul><p>
I've changed the main ticket documentation accordingly.
</p>
<p>
For reference, here is a list of files in the root repository:
</p>
<pre class="wiki">[vbraun@volkertwo sage4.6.1.vb2]$ hg st all  grep v '^I'
C .hgignore
C .hgtags
C COPYING.txt
C Makefile
C README.txt
C ipython/ipy_profile_sh.py
C ipython/ipy_user_conf.py
C ipython/ipythonrc
C ipython/ipythonrcmath
C ipython/ipythonrcnumeric
C ipython/ipythonrcphysics
C ipython/ipythonrcpysh
C ipython/ipythonrcscipy
C ipython/ipythonrctutorial
C sage
C spkg/README.txt
C spkg/gen_html
C spkg/install
C spkg/pipestatus
C spkg/rootspkginstall
C spkg/standard/README.txt
C spkg/standard/deps
C spkg/standard/libdist_filelist
C spkg/standard/newest_version
</pre><p>
Really, any kind of root repository would be better than the caveman technology we have in place right now. I read through all the scripts and they do make sense to me. I built my private release using <code>sage sdist <version></code> and it compiled fine. You guys put a lot of effort into this ticket to make sure that nothing breaks, and I think it really is the time to integrate this with Sage.
</p>
<p>
Positive review.
</p>
<p>
Jeroen, can you plug this into 4.6.2.alpha as soon as possible to give it as much exposure as possible?
</p>
TicketjdemeyerWed, 19 Jan 2011 08:58:05 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:88
https://trac.sagemath.org/ticket/9433#comment:88
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_info</em>
</li>
</ul>
<p>
Why do we need a <code>sage_root.spkg</code>? I don't see a reason for a "root repo" to ever exist in tarball form.
</p>
TicketjdemeyerWed, 19 Jan 2011 08:59:56 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:89
https://trac.sagemath.org/ticket/9433#comment:89
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=89">diff</a>)
</li>
</ul>
TicketjdemeyerWed, 19 Jan 2011 09:02:08 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>9433_install.diff</em>
</li>
</ul>
TicketjdemeyerWed, 19 Jan 2011 09:02:35 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:90
https://trac.sagemath.org/ticket/9433#comment:90
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=90">diff</a>)
</li>
</ul>
TicketjdemeyerWed, 19 Jan 2011 09:06:50 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:91
https://trac.sagemath.org/ticket/9433#comment:91
<ul>
<li><strong>status</strong>
changed from <em>needs_info</em> to <em>needs_work</em>
</li>
</ul>
<p>
If we do this, we really should also take care of the following <strong>duplicate</strong> files (files both in sage_scripts and in another repo):
</p>
<pre class="wiki">README.txt
spkg/install
spkg/base/sagespkg
spkg/base/sageenv
spkg/base/sagemake_relative
spkg/base/sagecheck64 (added by the notyetmerged #10303)
</pre><p>
Also, I think we should get rid of the <code>spkg/base</code> repo and merge it with the new root repo.
</p>
TicketvbraunWed, 19 Jan 2011 17:05:23 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:92
https://trac.sagemath.org/ticket/9433#comment:92
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:88" title="Comment 88">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Why do we need a <code>sage_root.spkg</code>? I don't see a reason for a "root repo" to ever exist in tarball form.
</p>
</blockquote>
<p>
The way things are set up right now, if you upgrade an existing sage installation then the updated <code>SAGE_ROOT_REPO</code> will merge updates to the <code>$SAGE_ROOT</code> repository. As long as we don't have an official online repository to pull changes from I don't see any better way to do the upgrade. Am I missing something?
</p>
<p>
I agree that we need to disentangle the <code>sage_root</code> repository more from <code>sage_scripts</code>. Basically, everything that <code>sage_scripts/spkginstall</code> manually copies into <code>$SAGE_ROOT</code> should be part of the root repo, like <code>README.txt</code>. But I think this can wait until we actually do have a <code>sage_root</code> repository. Then it'll be easy to write complimentary patches for the two repositories that clean this up.
</p>
<p>
I'm also totally in favour of merging the spkg/base repo. Since that has only 26 log entries (with the most recent one from July), I think we can live without preserving its history. Right now we don't use this repository during upgrades as far as I know.
</p>
<p>
If you agree with this then I'll make a followup ticket...
</p>
TicketjdemeyerWed, 19 Jan 2011 20:31:53 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:93
https://trac.sagemath.org/ticket/9433#comment:93
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:92" title="Comment 92">vbraun</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:88" title="Comment 88">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Why do we need a <code>sage_root.spkg</code>? I don't see a reason for a "root repo" to ever exist in tarball form.
</p>
</blockquote>
<p>
The way things are set up right now, if you upgrade an existing sage installation then the updated <code>SAGE_ROOT_REPO</code> will merge updates to the <code>$SAGE_ROOT</code> repository. As long as we don't have an official online repository to pull changes from I don't see any better way to do the upgrade.
</p>
</blockquote>
<p>
I have to admit I know nothing about this. If you really think we need a <code>sage_root.spkg</code> then I believe you...
</p>
<blockquote class="citation">
<p>
I agree that we need to disentangle the <code>sage_root</code> repository more from <code>sage_scripts</code>. Basically, everything that <code>sage_scripts/spkginstall</code> manually copies into <code>$SAGE_ROOT</code> should be part of the root repo, like <code>README.txt</code>. But I think this can wait until we actually do have a <code>sage_root</code> repository. Then it'll be easy to write complimentary patches for the two repositories that clean this up.
</p>
</blockquote>
<p>
Personally, I would rather like to clean this up as part of <em>this</em> ticket. Keep in mind that adding or changing the working of a SAGE_ROOT repo will require some changes to the process of merging Sage releases. I would prefer to have to do this only once, not once for this ticket and once for every followup ticket.
</p>
<blockquote class="citation">
<p>
I'm also totally in favour of merging the spkg/base repo. Since that has only 26 log entries (with the most recent one from July), I think we can live without preserving its history. Right now we don't use this repository during upgrades as far as I know.
</p>
<p>
If you agree with this then I'll make a followup ticket...
</p>
</blockquote>
<p>
Same answer as before: I prefer to do it in <em>this</em> ticket.
</p>
TicketjhpalmieriWed, 19 Jan 2011 20:42:37 GMT
https://trac.sagemath.org/ticket/9433#comment:94
https://trac.sagemath.org/ticket/9433#comment:94
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:93" title="Comment 93">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:92" title="Comment 92">vbraun</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:88" title="Comment 88">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Why do we need a <code>sage_root.spkg</code>? I don't see a reason for a "root repo" to ever exist in tarball form.
</p>
</blockquote>
<p>
The way things are set up right now, if you upgrade an existing sage installation then the updated <code>SAGE_ROOT_REPO</code> will merge updates to the <code>$SAGE_ROOT</code> repository. As long as we don't have an official online repository to pull changes from I don't see any better way to do the upgrade.
</p>
</blockquote>
<p>
I have to admit I know nothing about this. If you really think we need a <code>sage_root.spkg</code> then I believe you...
</p>
</blockquote>
<p>
Yes, that's the reason for its existence.
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
I agree that we need to disentangle the <code>sage_root</code> repository more from <code>sage_scripts</code>. Basically, everything that <code>sage_scripts/spkginstall</code> manually copies into <code>$SAGE_ROOT</code> should be part of the root repo, like <code>README.txt</code>. But I think this can wait until we actually do have a <code>sage_root</code> repository. Then it'll be easy to write complimentary patches for the two repositories that clean this up.
</p>
</blockquote>
<p>
Personally, I would rather like to clean this up as part of <em>this</em> ticket. Keep in mind that adding or changing the working of a SAGE_ROOT repo will require some changes to the process of merging Sage releases. I would prefer to have to do this only once, not once for this ticket and once for every followup ticket.
</p>
</blockquote>
<p>
I think I don't understand. Once there is a SAGE_ROOT repo, then any followup ticket can just have a patch which needs to be applied to that repo, as opposed to manually patching <code>spkg/install</code> or <code>Makefile</code> or whatever.
</p>
<p>
For the particular change you have in mind, it requires modifying various scripts (like spkginstall, but maybe others?) in local/bin, and it could possibly be complicated. It seems safer to do things incrementally, first setting up the new repo, then on a later ticket making further changes.
</p>
<p>
(I also have a heavy teaching load right now, so I won't have much time to work on this. I can try to fix bugs with the current implementation when anyone finds them, but I don't think I can work on any major modifications, like dealing with the spkg/base repo.)
</p>
TicketvbraunWed, 19 Jan 2011 20:50:39 GMT
https://trac.sagemath.org/ticket/9433#comment:95
https://trac.sagemath.org/ticket/9433#comment:95
<p>
I'm with John here: Its easy to generate a patch that adds a new file to the <code>sage_root</code> repository. You don't have to do anything else for updates, the change will automatically make it into the next <code>sage_root</code> package. No more manually digging around to make a new source distribution :)
</p>
TicketkcrismanWed, 19 Jan 2011 20:53:54 GMT
https://trac.sagemath.org/ticket/9433#comment:96
https://trac.sagemath.org/ticket/9433#comment:96
<blockquote class="citation">
<p>
For the particular change you have in mind, it requires modifying various scripts (like spkginstall, but maybe others?) in local/bin, and it could possibly be complicated. It seems safer to do things incrementally, first setting up the new repo, then on a later ticket making further changes.
</p>
</blockquote>
<p>
And indeed, this is the Sage way of doing things  perfect being the enemy of things ever happening in the open development, open source model.
</p>
<p>
I'd also like to put in a plug for sageREADMEosx.txt finally being removed from $SAGE_ROOT. Apparently <a class="closed ticket" href="https://trac.sagemath.org/ticket/6938" title="defect: sageREADMEosx.txt is nonsensical for a source distribution. (closed: fixed)">#6938</a> never got 'merged'...> (I also have a heavy teaching load right now, so I won't have much time to work on this. I can try to fix bugs with the current implementation when anyone finds them, but I don't think I can work on any major modifications, like dealing with the spkg/base repo.)
</p>
TicketjdemeyerWed, 19 Jan 2011 21:08:05 GMT
https://trac.sagemath.org/ticket/9433#comment:97
https://trac.sagemath.org/ticket/9433#comment:97
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:96" title="Comment 96">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
And indeed, this is the Sage way of doing things  perfect being the enemy of things ever happening in the open development, open source model.
</p>
</blockquote>
<p>
In general, I completely agree with your sentiment: too many times a patch has not been finished because it was not the "optimal" solution. However, this only applies if the halfwayimplemented patch makes the situation better.
</p>
<p>
For this particular ticket, I feel that having a halfwayimplemented sage_root repository is strictly worse than having no sage_root repository at all. The situation for merging SAGE_ROOT patches is already complicated enough (but I can manage) and I'm afraid the current patch on this ticket will make it only worse.
</p>
<p>
This is not a ticket I want to "rush" into Sage.
</p>
TicketvbraunWed, 19 Jan 2011 21:18:40 GMT
https://trac.sagemath.org/ticket/9433#comment:98
https://trac.sagemath.org/ticket/9433#comment:98
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:97" title="Comment 97">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
This is not a ticket I want to "rush" into Sage.
</p>
</blockquote>
<p>
This ticket has been worked on for 7 months and has almost 100 comments ;)
</p>
<p>
And every time I want to change something on this ticket I need half a day to rebuild sage, manually make half a dozen changes, test the source distribution.
</p>
<p>
I can make the changes you wanted in <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:91" title="Comment 91">comment:91</a> but there are a bunch of other files I want to move to the <code>sage_root</code> repository as well. Some of them might need some discussion on sagedevel first. How many more months should we wait until the main makefile is under revision control?
</p>
TicketddrakeThu, 20 Jan 2011 04:41:23 GMT
https://trac.sagemath.org/ticket/9433#comment:99
https://trac.sagemath.org/ticket/9433#comment:99
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:97" title="Comment 97">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
For this particular ticket, I feel that having a halfwayimplemented sage_root repository is strictly worse than having no sage_root repository at all.
</p>
</blockquote>
<p>
In what way is the SAGE_ROOT repo "halfwayimplemented"? With these scripts and patches, the repo will exist and work for upgrades and new builds.
</p>
<blockquote class="citation">
<p>
The situation for merging SAGE_ROOT patches is already complicated enough (but I can manage)
</p>
</blockquote>
<p>
Maybe you can manage  but there's plenty of evidence that other release managers can't. Changing files in SAGE_ROOT currently takes a long time and is an extremely brittle process; why continue in this way?
</p>
<p>
And even if there's no trouble actually changing files, we still have no version control for our basic makefile, README, and executable script. The only way to track changes is to download tarballs, unpack them, and compare files. Think about it: neither the basic file with which we build Sage (the makefile) nor the basic file with which one runs Sage (SAGE_ROOT/sage) are under any version control! That is crazy. Imagine if Firefox shipped with their "firefox" script not under version control.
</p>
<blockquote class="citation">
<p>
and I'm afraid the current patch on this ticket will make it only worse.
</p>
</blockquote>
<p>
Right now to change the README:
</p>
<ul><li>make a backup copy
</li><li>make your changes
</li><li>manually run <code>diff u</code> to produce a patch
</li><li>upload <em>both</em> the new README and the patch to a ticket
</li><li>upon positive review, hope that the release manager replaces the README file correctly and does <code>sage sdist</code> in the right directory
</li></ul><p>
With a SAGE_ROOT repo:
</p>
<ul><li>use the same procedure that one uses in the Sage library to produce a patch.
</li><li>release manager uses same procedure as for the Sage library to merge the patch.
</li></ul><p>
Perhaps it's only me, but the current situation seems far worse.
</p>
<p>
Please, let's create the SAGE_ROOT repo and argue about merging repos after they actually exist.
</p>
TicketjdemeyerThu, 20 Jan 2011 08:50:23 GMT
https://trac.sagemath.org/ticket/9433#comment:100
https://trac.sagemath.org/ticket/9433#comment:100
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:99" title="Comment 99">ddrake</a>:
</p>
<blockquote class="citation">
<ul><li>release manager uses same procedure as for the Sage library to merge the patch.
</li></ul></blockquote>
<p>
Wrong, because the <code>README</code> is actually already in a different repository, namely <code>sage_scripts</code>. This is the issue of duplicate files that I mentioned above.
</p>
TicketjhpalmieriThu, 20 Jan 2011 15:36:53 GMT
https://trac.sagemath.org/ticket/9433#comment:101
https://trac.sagemath.org/ticket/9433#comment:101
<p>
After applying the patches here, README.txt will be in the SAGE_ROOT repo, not the scripts repo. The full list of files which will be tracked:
</p>
<blockquote>
<p>
.hgignore .hgtags COPYING.txt README.txt Makefile sage
</p>
</blockquote>
<ul><li>in the ipython directory: *.py, ipythonrc*
</li><li>in the spkg directory: README.txt gen_html install pipestatus rootspkginstall
</li><li>in the spkg/standard directory: README.txt deps libdist_filelist newest_version
</li></ul>
TicketjdemeyerThu, 20 Jan 2011 16:19:28 GMT
https://trac.sagemath.org/ticket/9433#comment:102
https://trac.sagemath.org/ticket/9433#comment:102
<p>
The <code>SAGE_ROOT/ipython</code> directory is <strong>created</strong> during the Sage build process, so I don't think it should be part of the <code>SAGE_ROOT</code> repository.
</p>
TicketjdemeyerThu, 20 Jan 2011 16:20:03 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:103
https://trac.sagemath.org/ticket/9433#comment:103
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=103">diff</a>)
</li>
</ul>
TicketjdemeyerThu, 20 Jan 2011 16:36:52 GMT
https://trac.sagemath.org/ticket/9433#comment:104
https://trac.sagemath.org/ticket/9433#comment:104
<p>
What's the reason for removing the quoting of <code>SAGE_ROOT</code> in <code>sagespkginstall</code>?
</p>
TicketjhpalmieriThu, 20 Jan 2011 16:48:29 GMT
https://trac.sagemath.org/ticket/9433#comment:105
https://trac.sagemath.org/ticket/9433#comment:105
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:102" title="Comment 102">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
The <code>SAGE_ROOT/ipython</code> directory is <strong>created</strong> during the Sage build process, so I don't think it should be part of the <code>SAGE_ROOT</code> repository.
</p>
</blockquote>
<p>
It's just copied from whatever was there before. Why not track those files in a repository? Otherwise they're not tracked anywhere: what if we want to change any of them? Also, if you decide to not include them in the repo, then you'll need to work on the scripts patch: put back the parts (in sagesdist for example) which create those files.
</p>
<blockquote class="citation">
<p>
What's the reason for removing the quoting of SAGE_ROOT in sagespkginstall?
</p>
</blockquote>
<p>
That was presumably just a mistake.
</p>
TicketvbraunThu, 20 Jan 2011 16:50:55 GMT
https://trac.sagemath.org/ticket/9433#comment:106
https://trac.sagemath.org/ticket/9433#comment:106
<p>
The ipython directory is in the <code>sage_scripts</code> repository and <code>sage_scripts/spkginstall</code> manually copies it into <code>$SAGE_ROOT</code>. The patch on this ticket removes that part from <code>sage_scripts/spkginstall</code>, so it will no longer be copied over.
</p>
<p>
I'm not sure if the removal of the quotes has any deeper meaning. But righthandsides of variable assignments need not be quoted in shell script:
</p>
<pre class="wiki">[vbraun@volkertwo ~]$ x="a b"
[vbraun@volkertwo ~]$ y=$x/c
[vbraun@volkertwo ~]$ echo $y
a b/c
</pre>
TicketjdemeyerThu, 20 Jan 2011 16:55:38 GMT
https://trac.sagemath.org/ticket/9433#comment:107
https://trac.sagemath.org/ticket/9433#comment:107
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:106" title="Comment 106">vbraun</a>:
</p>
<blockquote class="citation">
<p>
The ipython directory is in the <code>sage_scripts</code> repository and <code>sage_scripts/spkginstall</code> manually copies it into <code>$SAGE_ROOT</code>. The patch on this ticket removes that part from <code>sage_scripts/spkginstall</code>, so it will no longer be copied over.
</p>
</blockquote>
<p>
Okay, I see. I got confused by the renaming of <code>sagespkginstall</code> to <code>spkginstall</code>, so I didn't realize what the file <code>sagespkginstall</code> was all about.
</p>
<blockquote class="citation">
<p>
I'm not sure if the removal of the quotes has any deeper meaning. But righthandsides of variable assignments need not be quoted in shell script:
</p>
<pre class="wiki">[vbraun@volkertwo ~]$ x="a b"
[vbraun@volkertwo ~]$ y=$x/c
[vbraun@volkertwo ~]$ echo $y
a b/c
</pre></blockquote>
<p>
I agree, but I think that having the quotes is clearer anyway.
</p>
<p>
This is most certainly a bug (in <code>sagemake_devel_packages</code>):
</p>
<pre class="wiki">+if [ ! f "$PKG/sage_scripts$SAGE_VERSION.spkg" ]; then
</pre>
TicketvbraunThu, 20 Jan 2011 17:00:14 GMT
https://trac.sagemath.org/ticket/9433#comment:108
https://trac.sagemath.org/ticket/9433#comment:108
<p>
Upon closer investigation I found that the ipython directory is in the <code>sage_scripts</code> spkg, but ignored in the <code>sage_scripts</code> mercurial repository. So its even worse :)
</p>
TicketvbraunThu, 20 Jan 2011 17:07:39 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v5.2.patch</em>
</li>
</ul>
<p>
Updated patch
</p>
TicketjdemeyerThu, 20 Jan 2011 17:07:43 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:109
https://trac.sagemath.org/ticket/9433#comment:109
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=109">diff</a>)
</li>
</ul>
TicketjdemeyerThu, 20 Jan 2011 17:08:18 GMT
https://trac.sagemath.org/ticket/9433#comment:110
https://trac.sagemath.org/ticket/9433#comment:110
<p>
Ha, I beat you by 13 seconds! :)
</p>
TicketjdemeyerThu, 20 Jan 2011 17:08:42 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:111
https://trac.sagemath.org/ticket/9433#comment:111
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=111">diff</a>)
</li>
</ul>
TicketvbraunThu, 20 Jan 2011 17:11:32 GMT
https://trac.sagemath.org/ticket/9433#comment:112
https://trac.sagemath.org/ticket/9433#comment:112
<p>
That was probably my typo in rebasing the patch to Sage4.6.1 ;)
</p>
TicketjdemeyerThu, 20 Jan 2011 17:19:40 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:113
https://trac.sagemath.org/ticket/9433#comment:113
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=113">diff</a>)
</li>
</ul>
TicketjdemeyerThu, 20 Jan 2011 17:23:07 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v6.patch</em>
</li>
</ul>
TicketjdemeyerThu, 20 Jan 2011 17:25:31 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>9433_hg_script.sh</em>
</li>
</ul>
TicketvbraunThu, 20 Jan 2011 23:20:37 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:114
https://trac.sagemath.org/ticket/9433#comment:114
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>positive_review</em>
</li>
</ul>
<p>
I'm fine with all changes.
</p>
TicketjdemeyerFri, 21 Jan 2011 03:18:26 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:115
https://trac.sagemath.org/ticket/9433#comment:115
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
Upgrading from earlier versions of Sage doesn't work because <code>spkg/standard/VERSION.txt</code> is gone:
</p>
<pre class="wiki">$ ./sage upgrade http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root/
Downloading packages from 'http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg'.
Reading package lists... Done!
The following packages will be upgraded:
examples4.6.2.sage_root extcode4.6.2.sage_root
sage4.6.2.sage_root sage_root4.6.2.sage_root
sage_scripts4.6.2.sage_root
** WARNING: This is a sourcebased upgrade, which could take hours,
** fail, and render your Sage install useless!!
Do you want to continue [y/N]? y
http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/examples4.6.2.sage_root.spkg > examples4.6.2.sage_root.spkg [..................................................]
Deleting old spkg 'examples4.6.1.spkg'...
http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/extcode4.6.2.sage_root.spkg > extcode4.6.2.sage_root.spkg [..................................................]
Deleting old spkg 'extcode4.6.1.spkg'...
http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/sage4.6.2.sage_root.spkg > sage4.6.2.sage_root.spkg [..................................................]
Deleting old spkg 'sage4.6.1.spkg'...
http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/sage_root4.6.2.sage_root.spkg > sage_root4.6.2.sage_root.spkg [..............]
http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/sage_scripts4.6.2.sage_root.spkg > sage_scripts4.6.2.sage_root.spkg [..................................................]
Deleting old spkg 'sage_scripts4.6.1.spkg'...
http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/VERSION.txt > VERSION.txt [.]
Failed to download 'http://sage.math.washington.edu/home/jdemeyer/release/sage4.6.2.sage_root/sage4.6.2.sage_root//spkg/standard/VERSION.txt'.
Abort.
</pre>
TicketjhpalmieriFri, 21 Jan 2011 05:04:23 GMT
https://trac.sagemath.org/ticket/9433#comment:116
https://trac.sagemath.org/ticket/9433#comment:116
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:115" title="Comment 115">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Upgrading from earlier versions of Sage doesn't work because <code>spkg/standard/VERSION.txt</code> is gone:
</p>
</blockquote>
<p>
That's because when the scripts patch was rebased, lines like these were removed:
</p>
<pre class="wiki"># Put VERSION.txt in a directory available for download during the
# update process. (See sageupdate.)
cp p VERSION.txt $TMP/$PKGDIR/$STD/
</pre><p>
I think that just restoring these lines should fix it .
</p>
TicketjhpalmieriFri, 21 Jan 2011 05:31:58 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v7.patch</em>
</li>
</ul>
<p>
patch for scripts repo
</p>
TicketjhpalmieriFri, 21 Jan 2011 05:33:47 GMTstatus, description changed
https://trac.sagemath.org/ticket/9433#comment:117
https://trac.sagemath.org/ticket/9433#comment:117
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=117">diff</a>)
</li>
</ul>
<p>
Here's a new patch for the scripts repo.
</p>
TicketvbraunFri, 21 Jan 2011 19:16:37 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:118
https://trac.sagemath.org/ticket/9433#comment:118
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
</ul>
<p>
Yes that fixes it!
</p>
TicketvbraunSat, 22 Jan 2011 04:00:16 GMT
https://trac.sagemath.org/ticket/9433#comment:119
https://trac.sagemath.org/ticket/9433#comment:119
<p>
For the record: I successfully upgraded a Sage4.6.1 installation to Sage4.6.2.alpha1.
</p>
TicketjdemeyerSat, 22 Jan 2011 19:45:30 GMTmilestone changed
https://trac.sagemath.org/ticket/9433#comment:120
https://trac.sagemath.org/ticket/9433#comment:120
<ul>
<li><strong>milestone</strong>
changed from <em>sage4.6.2</em> to <em>sage4.7</em>
</li>
</ul>
TicketjdemeyerSat, 22 Jan 2011 20:07:46 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>9433_testing.patch</em>
</li>
</ul>
<p>
SAGEROOT patch for testing, DO NOT APPLY
</p>
TicketjdemeyerSun, 23 Jan 2011 13:15:07 GMTpriority changed
https://trac.sagemath.org/ticket/9433#comment:121
https://trac.sagemath.org/ticket/9433#comment:121
<ul>
<li><strong>priority</strong>
changed from <em>major</em> to <em>blocker</em>
</li>
</ul>
TicketjdemeyerSun, 23 Jan 2011 19:32:48 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:122
https://trac.sagemath.org/ticket/9433#comment:122
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=122">diff</a>)
</li>
</ul>
TicketjdemeyerWed, 26 Jan 2011 22:07:20 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>9433_deps.diff</em>
</li>
</ul>
TicketjdemeyerTue, 15 Feb 2011 16:25:15 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:123
https://trac.sagemath.org/ticket/9433#comment:123
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=123">diff</a>)
</li>
</ul>
TicketjdemeyerWed, 16 Feb 2011 08:59:24 GMTstatus, description changed
https://trac.sagemath.org/ticket/9433#comment:124
https://trac.sagemath.org/ticket/9433#comment:124
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_work</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=124">diff</a>)
</li>
</ul>
TicketjdemeyerWed, 16 Feb 2011 08:59:50 GMT
https://trac.sagemath.org/ticket/9433#comment:125
https://trac.sagemath.org/ticket/9433#comment:125
<p>
Testing changes to rootspkginstall...
</p>
TicketjdemeyerWed, 16 Feb 2011 09:06:40 GMT
https://trac.sagemath.org/ticket/9433#comment:126
https://trac.sagemath.org/ticket/9433#comment:126
<p>
It seems that the <code>SAGE_ROOT</code> repository requires a very recent of Mercurial. With Mercurial version 1.6.4, I get
</p>
<pre class="wiki">$ hg version
Mercurial Distributed SCM (version 1.6.4)
Copyright (C) 20052010 Matt Mackall <mpm@selenic.com> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ hg commit
abort: requirement 'dotencode' not supported!
</pre><p>
I agree that a sufficiently recent version of Mercurial is shipped with Sage, but unless there is a good reason for this <code>dotencode</code> requirement, I would prefer if the root repo worked with Mercurial 1.3.1 like the other repos.
</p>
TicketjdemeyerWed, 16 Feb 2011 09:09:09 GMT
https://trac.sagemath.org/ticket/9433#comment:127
https://trac.sagemath.org/ticket/9433#comment:127
<p>
The file <code>spkginstall</code> from the root repo is not under revision control. Is there a reason for this?
</p>
TicketjdemeyerWed, 16 Feb 2011 09:18:30 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>rootspkginstall.v3</em>
</li>
</ul>
<p>
The file $SAGE_ROOT/spkg/rootspkginstall
</p>
TicketvbraunWed, 16 Feb 2011 10:04:32 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>9433_hg_script.2.sh</em>
</li>
</ul>
<p>
create root repository without dotencode
</p>
TicketvbraunWed, 16 Feb 2011 10:08:29 GMT
https://trac.sagemath.org/ticket/9433#comment:128
https://trac.sagemath.org/ticket/9433#comment:128
<p>
I'm pretty sure that the <code>rootspkginstall</code> ought to be under revision control. I'm travelling right now, so I can't test it out myself.
</p>
TicketjhpalmieriWed, 16 Feb 2011 15:35:10 GMT
https://trac.sagemath.org/ticket/9433#comment:129
https://trac.sagemath.org/ticket/9433#comment:129
<p>
I don't think there is anything in the repository which should require a new version of Mercurial. According to <a class="extlink" href="http://mercurial.selenic.com/wiki/RequiresFile"><span class="icon"></span>this page from the Mercurial wiki</a>, if you create a repo with a newer version and then try to access it with an older version, you can see this message. So I don't think it's anything specific about the root repo.
</p>
<p>
I'll try to look into the rootspkginstall file issue.
</p>
TicketjhpalmieriWed, 16 Feb 2011 15:38:46 GMT
https://trac.sagemath.org/ticket/9433#comment:130
https://trac.sagemath.org/ticket/9433#comment:130
<p>
Okay, line 24 in <code>9433_hg_script.sh</code> is
</p>
<pre class="wiki">( cd spkg && hg add README.txt gen_html install pipestatus rootspkginstall )
</pre><p>
So it looks like rootspkginstall should be tracked. Can you explain why you think it's not?
</p>
<p>
Also, I see that vbraun has a new version of this script which might avoid the dotencode issue...
</p>
TicketvbraunWed, 16 Feb 2011 16:02:47 GMT
https://trac.sagemath.org/ticket/9433#comment:131
https://trac.sagemath.org/ticket/9433#comment:131
<p>
Yes, I tripped over the dotencode thing before when working with mercurial. We don't really need that, so its ok to switch it off for backward compatibility, at least for the next year or so.
</p>
TicketjhpalmieriWed, 16 Feb 2011 21:02:10 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>hgignore</em>
</li>
</ul>
<p>
the file SAGE_ROOT/.hgignore, now including spkginstall
</p>
TicketjhpalmieriWed, 16 Feb 2011 21:06:38 GMTstatus, description changed
https://trac.sagemath.org/ticket/9433#comment:132
https://trac.sagemath.org/ticket/9433#comment:132
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=132">diff</a>)
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:127" title="Comment 127">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
The file <code>spkginstall</code> from the root repo is not under revision control. Is there a reason for this?
</p>
</blockquote>
<p>
I now realize that you're talking about the file <code>spkginstall</code> in the actual spkg file. This is just a copy of <code>rootspkginstall</code> made by <code>sagemake_devel_packages</code>, so we don't need to track it. I'm going to add it to the <code>.hgignore</code> file (by adding <code>^spkginstall$</code>, so it only matches a file with exactly that name at the top level). This change requires review.
</p>
<p>
Meanwhile, I'm giving Volker's change to the hg_script a positive review: for me, it makes any errors about dotencode go away when I use an older version of Mercurial to access the repo.
</p>
TicketjdemeyerThu, 17 Feb 2011 19:32:16 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:133
https://trac.sagemath.org/ticket/9433#comment:133
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
About the <code>dotencode</code> requirement: the change in the shell script indeed "fixes" the actual root repository. However, the created <code>sageroot</code> spkg still has a mercurial repository which requires <code>dotencode</code> (so, after sdist and make, you need a new version of <code>hg</code>). Since none of the other spkgs have this behaviour, I think this still needs work.
</p>
TicketjhpalmieriFri, 18 Feb 2011 01:16:17 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:134
https://trac.sagemath.org/ticket/9433#comment:134
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
I don't really understand this complaint. After all, Sage is distributed with a version of Mercurial, and it is completely reasonable to require that version for repositories which are part of Sage. It's easy enough to type "sage hg" instead of "hg", or make an alias, or put the Sage version of hg in your $PATH. I think this is more a problem with Mercurial  why is it not backwards compatible? Who merged this version of Mercurial into Sage? Did the spkg maintainers or release manager notice or discuss this incompatibility issue?
</p>
<p>
Also, if you're going to start inventing rules about how an spkg should be prepared, you might consider putting those rules in the developer's guide, and perhaps discussing them and getting approval for them on sagedevel before imposing them.
</p>
<p>
We can add <code>config format.dotencode=0</code> at various places in sagesdist and sagemake_devel_packages, and I'm attaching a new version of the scripts patch to do this, but this is not a viable longterm solution: the version requirements of Mercurial for anyone who wants to make a new spkg for Sage need to be made public, not imposed at anyone's whim.
</p>
TicketjhpalmieriFri, 18 Feb 2011 01:16:49 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v8.patch</em>
</li>
</ul>
<p>
patch for scripts repo
</p>
TicketjhpalmieriFri, 18 Feb 2011 01:17:18 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:135
https://trac.sagemath.org/ticket/9433#comment:135
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=135">diff</a>)
</li>
</ul>
TicketjdemeyerFri, 18 Feb 2011 08:15:03 GMT
https://trac.sagemath.org/ticket/9433#comment:136
https://trac.sagemath.org/ticket/9433#comment:136
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:134" title="Comment 134">jhpalmieri</a>:
</p>
<blockquote class="citation">
<p>
Did the spkg maintainers or release manager notice or discuss this incompatibility issue?
</p>
</blockquote>
<p>
I didn't know about this until looking at this ticket.
</p>
<p>
Why not simply avoid <code>hg clone</code> in <code>sagemake_devel_packages</code>? If the other spkgs can be made without using <code>hg clone</code>, surely the same should work for the <code>sage_root</code> spkg?
</p>
TicketjdemeyerFri, 18 Feb 2011 08:42:12 GMT
https://trac.sagemath.org/ticket/9433#comment:137
https://trac.sagemath.org/ticket/9433#comment:137
<p>
Version 1.7.3 of Mercurial was merged in sage4.6.2.alpha2, ticket <a class="closed ticket" href="https://trac.sagemath.org/ticket/10594" title="enhancement: Upgrade Mercurial to 1.8.x (closed: fixed)">#10594</a>. It can still be unmerged if you think that's a good thing to do. Then we would fall back to Mercurial 1.6.4.
</p>
TicketjhpalmieriFri, 18 Feb 2011 19:32:45 GMT
https://trac.sagemath.org/ticket/9433#comment:138
https://trac.sagemath.org/ticket/9433#comment:138
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:136" title="Comment 136">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Why not simply avoid <code>hg clone</code> in <code>sagemake_devel_packages</code>? If the other spkgs can be made without using <code>hg clone</code>, surely the same should work for the <code>sage_root</code> spkg?
</p>
</blockquote>
<p>
It certainly could be done, but it makes the repo harder to maintain. Suppose you want to add a new file to the repo. If you clone, you just run "hg add" (or apply a patch which accomplishes this) and you're done. If you manually copy everything over, as is done for the other repos, then you also have to modify sagemake_devel_packages, maybe rootspkginstall, maybe sagesdist. This is especially true for the root repo, where files need to be dealt with individually: it's not like the Sage repo where except for a handful of files, you just copy over an entire directory (devel/sage/sage/), and it's not like the scripts repo where except for a handful of files, you just copy over everything with a certain name ("sage*").
</p>
<p>
Maybe instead it could run "hg manifest" and then manually copy over the listed files. But this seems really awkward when "hg clone" does exactly what is required.
</p>
<blockquote class="citation">
<p>
Version 1.7.3 of Mercurial was merged in sage4.6.2.alpha2, ticket <a class="closed ticket" href="https://trac.sagemath.org/ticket/10594" title="enhancement: Upgrade Mercurial to 1.8.x (closed: fixed)">#10594</a>. It can still be unmerged if you think that's a good thing to do. Then we would fall back to Mercurial 1.6.4.
</p>
</blockquote>
<p>
I'm really not sure about this. Perhaps it should be discussed on <a class="closed ticket" href="https://trac.sagemath.org/ticket/10594" title="enhancement: Upgrade Mercurial to 1.8.x (closed: fixed)">#10594</a>. The lack of backwards compatibility seems problematic to me. I can try to post something there later today.
</p>
TicketvbraunFri, 18 Feb 2011 20:39:20 GMT
https://trac.sagemath.org/ticket/9433#comment:139
https://trac.sagemath.org/ticket/9433#comment:139
<p>
Anything that creates a new mercurial repository will by default require dotencode. In particular, <code>$SAGE_LOCAL/bin/sageclone</code> which is unrelated to this ticket.
</p>
<p>
It would be easy to add <code>config format.dotencode=0</code> to all <code>hg init</code>, <code>hg clone</code> commands. But current distributions already picked up the new mercurial, so I don't see it as particularly pressing issue. Moreover, we have a suitable mercurial in Sage precisely because it is sometimes finky with old versions. I would be in favor of just using dotencode. If it causes a big problem then we can always revert the repository format, you just have to clone with <code>dotencode=0</code>.
</p>
TicketjhpalmieriFri, 18 Feb 2011 21:14:13 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:140
https://trac.sagemath.org/ticket/9433#comment:140
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=140">diff</a>)
</li>
</ul>
TicketjdemeyerSun, 20 Feb 2011 15:02:26 GMT
https://trac.sagemath.org/ticket/9433#comment:141
https://trac.sagemath.org/ticket/9433#comment:141
<p>
Testing distributions:
</p>
<ol><li><a class="extlink" href="http://boxen.math.washington.edu/home/release/sage4.7.alpha0/"><span class="icon"></span>http://boxen.math.washington.edu/home/release/sage4.7.alpha0/</a>: sage4.6.2.rc0 + this ticket
</li><li><a class="extlink" href="http://boxen.math.washington.edu/home/release/sage4.7.alpha1/"><span class="icon"></span>http://boxen.math.washington.edu/home/release/sage4.7.alpha1/</a>: sage4.7.alpha0 + some random stuff (<a class="closed ticket" href="https://trac.sagemath.org/ticket/10688" title="enhancement: Remove weave from Sage (closed: fixed)">#10688</a> and <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/9433_testing.patch" title="Attachment '9433_testing.patch' in Ticket #9433">9433_testing.patch</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/9433_testing.patch" title="Download"></a>)
</li></ol>
TicketjdemeyerSun, 20 Feb 2011 15:09:24 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:142
https://trac.sagemath.org/ticket/9433#comment:142
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
Upgrading sage4.7.alpha0 > sage4.7.alpha1 fails because of uncommitted changes: during the install of <code>sage_root4.7.alpha1.spkg</code>, I get an editor with
</p>
<pre class="wiki">HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: 
HG: user: Jeroen Demeyer <jdemeyer@cage.ugent.be>
HG: branch 'default'
HG: changed spkg/install
HG: changed spkg/standard/deps
</pre><p>
<code>hg diff</code> gives:
</p>
<pre class="wiki">diff r 1c44cedc9957 spkg/install
 a/spkg/install Thu Feb 17 15:54:54 2011 +0000
+++ b/spkg/install Sun Feb 20 16:06:09 2011 +0100
@@ 1,5 +1,7 @@
#!/usr/bin/env bash
+# TESTING PATCH
+
###############################################################################
# Check if pipestatus already exists, otherwise
# create it to allow upgrade from Sage <4.5. This is a temporary fix.
@@ 422,9 +424,6 @@
TERMCAP=`$newest termcap`
export TERMCAP
WEAVE=`$newest weave`
export WEAVE

ZLIB=`$newest zlib`
export ZLIB
</pre><p>
and something similar for <code>spkg/standard/deps</code>.
</p>
<p>
It seems that <code>spkg/install</code> and <code>spkg/standard/deps</code> are changed by the upgrader before the Sage root repository is installed and that this causes trouble.
</p>
TicketjdemeyerMon, 21 Feb 2011 10:58:37 GMTreviewer changed
https://trac.sagemath.org/ticket/9433#comment:143
https://trac.sagemath.org/ticket/9433#comment:143
<ul>
<li><strong>reviewer</strong>
changed from <em>Leif Leonhardy, Volker Braun</em> to <em>Leif Leonhardy, Volker Braun, Jeroen Demeyer</em>
</li>
</ul>
<p>
John (and/or Volker), do you have time to work on this in the coming week / 2 weeks? If yes, I would like to merge this in sage4.7.alpha0. This ticket might require some more work to get it right, so it would be nice to know whether you have time. Otherwise we can postpone this to a later Sage release.
</p>
TicketvbraunMon, 21 Feb 2011 11:13:14 GMT
https://trac.sagemath.org/ticket/9433#comment:144
https://trac.sagemath.org/ticket/9433#comment:144
<p>
I'm around and am in favor of merging it asap.
</p>
TicketjhpalmieriMon, 21 Feb 2011 15:57:03 GMT
https://trac.sagemath.org/ticket/9433#comment:145
https://trac.sagemath.org/ticket/9433#comment:145
<p>
I have a fix to this problem which I am testing right now. It looks good so far, and I'll post it later if it continues to look good. (The fix is as follows: the script <code>sageupdate</code> downloads new versions of several files, like <code>install</code> and <code>deps</code>; I'm now adding a command to commit the Sage root repo after doing this download.)
</p>
TicketjhpalmieriMon, 21 Feb 2011 16:45:15 GMT
https://trac.sagemath.org/ticket/9433#comment:146
https://trac.sagemath.org/ticket/9433#comment:146
<p>
Okay, I have two options; which is better? In <code>sageupdate</code>, the files install, deps, and newest_version might get changed. We could:
</p>
<ul><li>commit the changes in <code>sageupdate</code>, and then the changes will get committed again by <code>rootspkginstall</code> when the root repo spkg is installed.
</li></ul><ul><li>or in <code>rootspkginstall</code>, we can overwrite any changes: do <code>hg pull ...</code> to pull from the new repo and then <code>hg update clean</code> to discard any other changes.
</li></ul><p>
The first of these adds a redundant commit message to the log file if the relevant files are changed. The second discards all changes, including any other ones that the user may have made. I don't know Mercurial well enough, but perhaps there is a better third choice?
</p>
TicketvbraunMon, 21 Feb 2011 18:16:01 GMT
https://trac.sagemath.org/ticket/9433#comment:147
https://trac.sagemath.org/ticket/9433#comment:147
<p>
I'm in favor of the following third option:
</p>
<ol><li>The first thing that <code>sageupdate</code> should do is to commit any outstanding changes and, in particular, abort if there is a patch queue applied. So you are safe even if you forgot to commit your changes.
</li><li>Then <code>sageupdate</code> overwrites some critical files for its operation.
</li><li>Finally, the <code>sage_root</code> spkg is installed and its <code>spkginstall</code> overwrites the remaining noncritical files in the root repository. I'd rather be guaranteed to have a clean slate after update than dying in the middle of the update because the attempted merge fails.
</li></ol><p>
So if you had any uncommitted changes before updating, you'll end up with two commit messages. The more individual commits the better. The only drawback is that if you had made modifications to the root repository that you want to keep and you are not using patch queues, then you now have to extract your changes as a patch from the repository and apply that patch. I think thats acceptable since, I think, most developers use patch queues and I don't expect too frequent edits to the root repository.
</p>
<p>
If that doesn't convince you, I'd prefer option number 1 as my second choice :)
</p>
TicketjhpalmieriMon, 21 Feb 2011 20:42:12 GMTstatus, description changed
https://trac.sagemath.org/ticket/9433#comment:148
https://trac.sagemath.org/ticket/9433#comment:148
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=148">diff</a>)
</li>
</ul>
<p>
Here is a new version of the scripts patch. The only change is in <code>sageupdate</code>. I updated the instructions, also: I'm not worrying about making the repo compatible with older versions of Mercurial right now.
</p>
TicketjhpalmieriMon, 21 Feb 2011 20:46:22 GMT
https://trac.sagemath.org/ticket/9433#comment:149
https://trac.sagemath.org/ticket/9433#comment:149
<p>
Oh, also, I created some versions for testing upgrades:
</p>
<ul><li><a class="extlink" href="http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X0/"><span class="icon"></span>http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X0/</a>
</li><li><a class="extlink" href="http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X1/"><span class="icon"></span>http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X1/</a> (spkg/install changed)
</li><li><a class="extlink" href="http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X2/"><span class="icon"></span>http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X2/</a> (Makefile changed)
</li><li><a class="extlink" href="http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X3/"><span class="icon"></span>http://sage.math.washington.edu/home/palmieri/misc/9433/sage4.6.2.X3/</a> (Makefile and spkg/standard/deps changed)
</li></ul><p>
The first of these is just 4.6.2.rc0 with the patches from this ticket applied. In each of the other ones, one or two files have been modified, as indicated. It's also worth testing by updating to the "X0" version, for example, then modifying a file tracked by the root repo (like README.txt) without committing the change, and then trying to update to "X1". The update should abort.
</p>
TicketjhpalmieriMon, 21 Feb 2011 21:12:09 GMT
https://trac.sagemath.org/ticket/9433#comment:150
https://trac.sagemath.org/ticket/9433#comment:150
<p>
I made a few minor cosmetic changes in the patch, and right now, the upgrade paths I mentioned use the old version, not the new one. I'll have them updated in a few minutes.
</p>
TicketjhpalmieriMon, 21 Feb 2011 21:23:17 GMT
https://trac.sagemath.org/ticket/9433#comment:151
https://trac.sagemath.org/ticket/9433#comment:151
<p>
Okay, everything's updated now.
</p>
TicketjdemeyerThu, 24 Feb 2011 14:01:37 GMT
https://trac.sagemath.org/ticket/9433#comment:152
https://trac.sagemath.org/ticket/9433#comment:152
<p>
I made new testing releases:
</p>
<ol><li><a class="extlink" href="http://boxen.math.washington.edu/home/release/sage4.7.alpha0/"><span class="icon"></span>http://boxen.math.washington.edu/home/release/sage4.7.alpha0/</a>: sage4.6.2.rc1 + this ticket
</li><li><a class="extlink" href="http://boxen.math.washington.edu/home/release/sage4.7.alpha1/"><span class="icon"></span>http://boxen.math.washington.edu/home/release/sage4.7.alpha1/</a>: sage4.7.alpha0 + <a class="closed ticket" href="https://trac.sagemath.org/ticket/10688" title="enhancement: Remove weave from Sage (closed: fixed)">#10688</a> + <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/9433/9433_testing.patch" title="Attachment '9433_testing.patch' in Ticket #9433">9433_testing.patch</a><a class="tracrawlink" href="https://trac.sagemath.org/rawattachment/ticket/9433/9433_testing.patch" title="Download"></a>
</li></ol><p>
Upgrading sage4.7.alpha0 > sage4.7.alpha1 succeeded this time.
</p>
TicketvbraunThu, 24 Feb 2011 20:57:54 GMT
https://trac.sagemath.org/ticket/9433#comment:153
https://trac.sagemath.org/ticket/9433#comment:153
<p>
I've tried the update and it works beautifully.
</p>
<p>
I think there is still one problem: If the user has no .hgrc (like many nondevelopers trying to upgrade), then mercurial will fail to commit with
</p>
<pre class="wiki">[vbraun@volkerdesktop sage4.7.alpha0]$ mv ~/.hgrc ~/backup.hgrc
[vbraun@volkerdesktop sage4.7.alpha0]$ hg commit m "test"
abort: no username supplied (see "hg help config")
</pre><p>
So whenever we commit to the root repo, we should specify a username explicitly. For example,
</p>
<pre class="wiki">hg commit u "committed by sage upgrade" m "test"
</pre>
TicketjhpalmieriFri, 25 Feb 2011 05:45:24 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>trac_9433scripts.v9.patch</em>
</li>
</ul>
<p>
patch for scripts repo
</p>
TicketjhpalmieriFri, 25 Feb 2011 05:45:43 GMTattachment set
https://trac.sagemath.org/ticket/9433
https://trac.sagemath.org/ticket/9433
<ul>
<li><strong>attachment</strong>
set to <em>rootspkginstall.v4</em>
</li>
</ul>
<p>
The file $SAGE_ROOT/spkg/rootspkginstall
</p>
TicketjhpalmieriFri, 25 Feb 2011 05:49:49 GMTdescription changed
https://trac.sagemath.org/ticket/9433#comment:154
https://trac.sagemath.org/ticket/9433#comment:154
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/9433?action=diff&version=154">diff</a>)
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/9433#comment:153" title="Comment 153">vbraun</a>:
</p>
<blockquote class="citation">
<p>
I think there is still one problem: If the user has no .hgrc (like many nondevelopers trying to upgrade), then mercurial will fail
</p>
</blockquote>
<p>
I wouldn't have spotted this. Good catch. I have new patches which add "u ..." to various commit commands: in sageupdate and in rootspkginstall. I haven't bothered with sagesdist or sagemake_devel_packages, since these are done by the release manager who had better have a .hgrc file. (Besides, there are already other "hg commit" commands in those scripts.)
</p>
<p>
I've also updated my versions (4.6.2.X0 etc.) for testing upgrades with these changes.
</p>
TicketvbraunFri, 25 Feb 2011 17:46:46 GMTstatus changed
https://trac.sagemath.org/ticket/9433#comment:155
https://trac.sagemath.org/ticket/9433#comment:155
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
</ul>
<p>
I'm happy with it now. I tested upgrades without <code>~/.hgrc</code> and everything worked for me.
</p>
TicketjdemeyerSat, 26 Feb 2011 10:40:19 GMT
https://trac.sagemath.org/ticket/9433#comment:156
https://trac.sagemath.org/ticket/9433#comment:156
<p>
Agreed! This will get merged in sage4.7.alpha0. I still expect breakage here and there, but that's what alpha versions are for.
</p>
TicketjdemeyerTue, 08 Mar 2011 21:44:36 GMTstatus changed; resolution, merged set
https://trac.sagemath.org/ticket/9433#comment:157
https://trac.sagemath.org/ticket/9433#comment:157
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
<li><strong>merged</strong>
set to <em>sage4.7.alpha0</em>
</li>
</ul>
Ticket