Opened 9 years ago

Closed 8 years ago

Last modified 8 years ago

#10594 closed enhancement (fixed)

Upgrade Mercurial to 1.8.x

Reported by: ryan Owned by: tbd
Priority: major Milestone: sage-4.7.2
Component: packages: standard Keywords: mercurial upgrade sd31
Cc: jason, was, leif Merged in: sage-4.7.2.alpha0
Authors: Ryan Grout, Keshav Kini Reviewers: Dan Drake, Ryan Grout, John Palmieri
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jdemeyer)

Upgrade the Mercurial spkg to 1.8.4 (probably the final 1.8.x release).

spkg: http://sage.math.washington.edu/home/keshav/files/mercurial-1.8.4.spkg

Change History (42)

comment:1 Changed 9 years ago by ryan

  • Cc jason added

comment:2 Changed 9 years ago by ryan

  • Status changed from new to needs_review

comment:3 follow-up: Changed 9 years ago by dimpase

  • Status changed from needs_review to needs_info

it seems to work for me, but how does one really test it?

comment:4 in reply to: ↑ 3 ; follow-ups: Changed 9 years ago by ddrake

Replying to dimpase:

it seems to work for me, but how does one really test it?

In a Sage shell ("sage -sh"), unpack the spkg and go into the src directory. There, you can build the package (just do make all) and then run Mercurial's built-in tests: make tests.

Interestingly, on a regular Ubuntu machine (on which Mercurial works perfectly fine) some of the tests don't pass; it seems like their tests want to have the unzip utility available.

The .spkg installs fine in Linux, and on t2.math. I'm going to take a look at OS X. Mercurial is very well tested and cross-platform, so I would be very surprised if we have any problems.

comment:5 Changed 9 years ago by ddrake

  • Status changed from needs_info to needs_review

comment:6 in reply to: ↑ 4 Changed 9 years ago by dimpase

Replying to ddrake:

Replying to dimpase:

it seems to work for me, but how does one really test it?

In a Sage shell ("sage -sh"), unpack the spkg and go into the src directory. There, you can build the package (just do make all) and then run Mercurial's built-in tests: make tests.

Interestingly, on a regular Ubuntu machine (on which Mercurial works perfectly fine) some of the tests don't pass; it seems like their tests want to have the unzip utility available.

The .spkg installs fine in Linux, and on t2.math. I'm going to take a look at OS X. Mercurial is very well tested and cross-platform, so I would be very surprised if we have any problems.

one should at least test that the combinat stuff works. This is one of the most immediate and tricky uses of hg in Sage.

comment:7 Changed 9 years ago by ddrake

  • Status changed from needs_review to positive_review

The spkg builds and installs on t2.math and bsd.math. I ran tests on those two machines and on an Ubuntu machine, and I got some failures on all three...but the failures seem to be the same as what we got with 1.6.4.

I tried the combinat patch queue, and it works the same as it does in 4.6.1.rc1 with Mercurial 1.6.4. The upgrade notes don't mention anything that affects our use of Mercurial, so positive review.

comment:8 Changed 9 years ago by jdemeyer

  • Reviewers set to Dan Drake

comment:9 Changed 9 years ago by jdemeyer

  • Merged in set to sage-4.6.2.alpha2
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:10 Changed 9 years ago by jhpalmieri

After some issues on #9433, I wonder if this should be reopened. There are some incompatibilities between Mercurial 1.7.3 and earlier versions; see this web page. I think the issues is this:

  • any new repositories created with 1.7.3 (without including a flag like --config format.dotencode=False) will not be usable by versions before 1.7.

I think a reasonable argument could be made for merging 1.7.3 anyway: when dealing with Sage, it's not a huge requirement to ask people to use the Mercurial distributed with Sage instead of some older version installed elsewhere on their machine.

But perhaps a reasonable argument could be made on the other side: why create repositories which can't be read by Mercurial 1.6.4 or earlier? Should we back out 1.7.3 altogether?

We could also keep 1.7.3 but require all repos to be created with the flag --config format.dotencode=False. I personally think that this is unwieldy.

Any opinions?

comment:11 Changed 9 years ago by vbraun

Just to reiterate my point from #9433, I think we should keep the new version. Current distributions (e.g. Fedora 14) have already mercurial 1.7+. If you have a way old mercurial install, on the other extreme, then you are probably forced to use Sage's mercurial already. This is precisely why mercurial is included in Sage.

comment:12 Changed 9 years ago by jdemeyer

Volker, I see your point but I have a small preference for delaying this new version of Mercurial until it's more widespread. I also don't see a compelling reason for including Mercurial 1.7.3 (as opposed to 1.6.4).

comment:13 Changed 8 years ago by jdemeyer

  • Merged in sage-4.6.2.alpha2 deleted
  • Milestone changed from sage-4.6.2 to sage-feature
  • Resolution fixed deleted
  • Status changed from closed to new
  • Type changed from task to enhancement

hg repositories made with 1.7.3 cannot be manipulated using versions of mercurial <1.7. Because of this backwards incompatible change and because I see no compelling reason to upgrade Mercurial, I want to be on the safe side and unmerge #10594 (i.e. revert to Mercurial version 1.6.4) in the sage-4.6.2 release.

In some later Sage release, we can still decide to upgrade Mercurial.

comment:14 Changed 8 years ago by jdemeyer

  • Status changed from new to needs_review

comment:15 Changed 8 years ago by jdemeyer

  • Status changed from needs_review to positive_review

comment:16 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-feature to sage-wait

comment:17 follow-up: Changed 8 years ago by kini

  • Description modified (diff)
  • Keywords upgrade added
  • Status changed from positive_review to needs_work
  • Summary changed from Upgrade to mercurial 1.7.3 to Upgrade Mercurial to 1.8.x

Can we upgrade yet, Jeroen? What is probably the final version of Mercurial 1.8.x, i.e. 1.8.4, is already out. Soon we will be three versions behind the latest version. Do we really need a compelling reason to upgrade packages we ship, when possible? As Volker said, anyone who is having trouble keeping up with Mercurial versions is probably using sage -hg anyway.

I'm working on an SPKG which will upgrade Mercurial to 1.8.4 and resolve #11120 and #11121 as well. Since 1.7.3 is already old, I hope you don't mind if I change around the ticket description...

comment:18 Changed 8 years ago by kini

  • Authors changed from Ryan Grout to Ryan Grout, Keshav Kini
  • Cc was added
  • Keywords sd31 added
  • Milestone changed from sage-wait to sage-4.7.1
  • Status changed from needs_work to needs_review

The spkg is available here. It also fixes #11120 and #11121, as mentioned. I'm CCing William since we talked about this in person and I don't see him on this ticket.

comment:19 in reply to: ↑ 17 ; follow-up: Changed 8 years ago by jdemeyer

Replying to kini:

Do we really need a compelling reason to upgrade packages we ship, when possible?

I think we do, I don't think we should simply upgrade packages for the sake of upgrading. I think the incompatibility with older versions of Mercurial is a negative point for upgrading Mercurial. Only if there is a particular feature of Mercurial 1.8.4 which is really useful, then I think we should upgrade.

comment:20 Changed 8 years ago by jhpalmieri

  • Description modified (diff)

On first glance, this looks okay. There are a few extraneous files in the spkg:

$ hg status
? .SPKG.txt.swp
? .SPKG.txt.un~
? .spkg-install.un~

So of course they should be cleaned up.

If someone upgrades from an earlier version of mercurial on OS X, should we check for and delete the old file SAGE/local/bin/hgmerge?

comment:21 in reply to: ↑ 19 Changed 8 years ago by kini

Replying to jdemeyer:

Replying to kini:

Do we really need a compelling reason to upgrade packages we ship, when possible?

I think we do, I don't think we should simply upgrade packages for the sake of upgrading. I think the incompatibility with older versions of Mercurial is a negative point for upgrading Mercurial. Only if there is a particular feature of Mercurial 1.8.4 which is really useful, then I think we should upgrade.

This is not a backwards-incompatible change in 1.7 -- it is a forward-incompatibility in 1.6.x. In the current state of affairs, the sage scripts cannot properly handle packages created by the user's system Mercurial, for many users whose system Mercurial is "too new" for Sage (and this proportion will only increase with time). If we upgrade Mercurial in Sage, the user's system Mercurial may not be able to handle packages created by the sage scripts, but this is not a problem because in those cases the user can do echo alias hg="sage -hg" >> ~/.bashrc and go on with their life. The proportion of users for which this will be a problem will only decrease with time. Also, rather than being an inconvenience, this will actually improve the user's life, since they have access to a newer version of Mercurial than they would otherwise. If we don't upgrade, we run into situations where you need to clone an entire repository with --config format.dotencode=False before you can run Sage's packaging tools on it, because they all use Sage's Mercurial. The proportion of users who encounter this problem will only increase with time, and I suspect that developers (i.e. the usual users of Mercurial) number in this population with more likelihood than average users.

So upgrading Mercurial would help more than it would hurt. But even if it helped exactly as much as it hurt in terms of this repository format issue, software isn't perfect. Newer versions fix bugs that existed in older versions. While newer versions may include new bugs, those too will be fixed in the future. Sitting tight on an old version of software is not safer than following the latest stable releases. I'm not advocating updating to bleeding-edge nightly builds or anything; 1.8.4 was released on June 1 and will be the final bugfix release before the feature release 1.9 (which will be released on July 1, according to Mercurial's release schedule). This makes it as stable as one could like, I think.

edit: yikes, let me clean out that junk from the spkg, hang on...

comment:22 Changed 8 years ago by kini

OK, should be fixed. Sorry about that, that was some vim detritus...

comment:23 Changed 8 years ago by jdemeyer

  • Description modified (diff)

comment:24 Changed 8 years ago by ryan

  • Reviewers changed from Dan Drake to Dan Drake, Ryan Grout

Reviewing.... If anyone else wnats to test this package, by all means help review.

comment:25 Changed 8 years ago by ryan

  • Status changed from needs_review to positive_review

The package install fine and none of the hg self-tests fail. Positive review

comment:26 Changed 8 years ago by kini

A full review of this spkg should probably include a runthrough of sage -bdist, sage -sdist, and friends. I can easily believe that hg's self-tests are successful and that Mercurial builds properly from my spkg; what is less clear is whether the newer version of Mercurial breaks any of Sage's various shell scripts that currently use it to do various mysterious things during release management and other such activities. It looks like Jeroen has the spkg in 4.7.1.alpha4 as of right now, though, so I'm guessing he's testing these things at the moment.

comment:27 Changed 8 years ago by ryan

  • Status changed from positive_review to needs_work

comment:28 follow-ups: Changed 8 years ago by ryan

  • Status changed from needs_work to needs_review

setting back to needs review. If jdemeyer is reviewing this, he can set it back to positive review when he is finished.

As I was testing this, my sage installation got corrupted (not related to this review). So I wasn't able finish testing -bdist and -sdist

comment:29 in reply to: ↑ 28 ; follow-up: Changed 8 years ago by kini

Replying to ryan:

As I was testing this, my sage installation got corrupted (not related to this review). So I wasn't able finish testing -bdist and -sdist

Why not do it on sage.math with a cleanly unpacked tarball? :)

comment:30 in reply to: ↑ 29 Changed 8 years ago by ryan

Replying to kini:

Replying to ryan:

As I was testing this, my sage installation got corrupted (not related to this review). So I wasn't able finish testing -bdist and -sdist

Why not do it on sage.math with a cleanly unpacked tarball? :)

Good idea! I will tomorrow :)

comment:31 in reply to: ↑ 28 Changed 8 years ago by jdemeyer

Replying to ryan:

setting back to needs review. If jdemeyer is reviewing this, he can set it back to positive review when he is finished.

I don't plan to formally review this.

comment:32 Changed 8 years ago by ryan

-bdist and -sdist both run without errors on my machine with the updated spkg.

Any other tests that should be run?

comment:33 Changed 8 years ago by kini

Other than those, maybe -merge and -combinat if you're familiar with them. Also, this spkg resolves #11121 and #11120 as well, so those need to be reviewed as well...

comment:34 Changed 8 years ago by jhpalmieri

This seems to work, from scratch and in an upgrade, although I've asked some questions on #11120 and #11121.

comment:35 Changed 8 years ago by jhpalmieri

  • Reviewers changed from Dan Drake, Ryan Grout to Dan Drake, Ryan Grout, John Palmieri
  • Status changed from needs_review to positive_review

I'm happy with this, so I'm setting it to "positive review".

comment:36 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-4.7.1 to sage-4.7.2

comment:37 Changed 8 years ago by leif

  • Cc leif added

comment:38 follow-up: Changed 8 years ago by jhpalmieri

By the way, I just had the opposite issue to that raised in comment 13: someone prepared an spkg for Sage using their system version of mercurial, and I couldn't check the repo with the version in Sage because the one in Sage is too old. I don't know enough about Mercurial to give instructions about how to repair their repo, so I could only check it by installing the spkg on this ticket.

So I think we should merge this in 4.7.2, not delay it further.

comment:39 in reply to: ↑ 38 Changed 8 years ago by jdemeyer

Replying to jhpalmieri:

So I think we should merge this in 4.7.2, not delay it further.

Sure!

comment:40 Changed 8 years ago by jdemeyer

  • Merged in set to sage-4.7.2.alpha0
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:41 in reply to: ↑ 4 Changed 8 years ago by kcrisman

In a Sage shell ("sage -sh"), unpack the spkg and go into the src directory. There, you can build the package (just do make all) and then run Mercurial's built-in tests: make tests.

There is currently no spkg-check here. If they have tests, we might as well add one. This is #11701.

comment:42 Changed 8 years ago by jdemeyer

Could any of the authors here please have a look at #12058: Mercurial should not enable pager by default.

Note: See TracTickets for help on using tickets.