Upgrade MPIR to a more recent upstream release

[leif]

This is a follow-up to #8664 (and a couple of other tickets).

The following new spkgs are based on the latest MPIR 2.1.3 spkg, the p9 from #12131:

New spkg: ​http://boxen.math.washington.edu/home/leif/Sage/spkgs/mpir-2.4.0.p2.spkg

md5sum: c9e6d8e2f5de4a09b4b16ee2354a9e17 mpir-2.4.0.p2.spkg

(Preliminary; changes not yet committed.)

New spkg: ​http://boxen.math.washington.edu/home/leif/Sage/spkgs/mpir-2.4.0.p1.spkg

md5sum: 0d88b7dfe358bb429a5a9aa3a3b4b00b mpir-2.4.0.p1.spkg

mpir-2.4.0.p2 (Leif Leonhardy, April 4th, 2012)

#11616 (upgrading MPIR), further fixes:

• Before enabling -march=native, minimalistically check whether the system's assembler also understands the instructions the compiler emits with that option. (Work-around for e.g. GCC 4.6.3 on MacOS X 10.x and Intel Core i7-family CPUs with AVX.)
• Do not unconditionally unset PYTHON, since Sage (>=5.0.beta10) no longer pollutes the environment with its package version variables, which previous- ly confused yasm's configure.
• Fix extraction of __GMP_CC and __GMP_CFLAGS from gmp.h, since MPIR meanwhile defines these to preprocessor variables (rather than literals). Also don't use \+ in sed patterns, as this is less portable.
• Work around GCC 4.7.0 bug (compilation error) on Linux ia64 (Itanium) by almost completely disabling optimization on that platform if GCC 4.7.x is detected. This doesn't hurt much if we later rebuild MPIR with a (non- broken) GCC from the new GCC spkg. Cf. #12765.
• Do not build the C++ interface and static libraries when bootstrapping the GCC spkg, i.e. if SAGE_BUILD_TOOLCHAIN=yes. (GMP/MPIR is a prerequisite for it, and MPIR will later get rebuilt with both enabled, with the newly built GCC.) Cf. #12782.
• Fix a potential race condition in yasm's build by patching the re2c source. Cf. #11844.
• Add "patch loop" to apply any patches (*.patch) located in patches/. Currently only the re2c patch matches that; the prepatched header to support Sun's C compiler is still copied over (and only on SunOS, although it doesn't do any harm on other platforms).
• Minor clean-up; e.g. redirect error messages and warnings to stderr, quote parameter to --libdir, add some comments and messages, also save user's setting of LDFLAGS and ABI.

mpir-2.4.0.p1 (Leif Leonhardy, March 21st, 2012)

• Upstream upgrade to MPIR 2.4.0 (#11616). The 2.4.0.p0 spkg isn't in this history, as it was based on the 2.1.3.p4 spkg, i.e., is "on another branch", and never got merged into Sage.
• Remove forcing a sequential make install again, since the potential race condition was fixed in MPIR 2.1.4.
• Fix .hgtags, which contained duplicate entries, and was missing others.

---

The following old spkgs are based on the obsolete MPIR 2.1.3.p4 spkg from #8664:

• ​http://spkg-upload.googlecode.com/files/mpir-2.3.1.p0.spkg
  md5sum: ff0104e302611e49604a1c4ead97cbf4 mpir-2.3.1.p0.spkg

• ​http://spkg-upload.googlecode.com/files/mpir-2.4.0.p0.spkg
  md5sum: 98e04aa43ceef04d83574f0ee8e152fc mpir-2.4.0.p0.spkg

You may want to give both of them a try, although the former is just intended as a "fallback" spkg in case problems introduced in the latter should arise.

Note: On 64-bit processors running 32-bit operating systems, one currently has to set ABI=32, otherwise the build will fail. (This does not apply to the newer MPIR 2.4.0 p1 and p2 spkgs.)

mpir-2.4.0.p0 (Leif Leonhardy, July 21st, 2011)

• Upstream upgrade to MPIR 2.4.0 (#11616).

mpir-2.3.1.p0 (Leif Leonhardy, July 20th, 2011)

• Upstream upgrade to MPIR 2.3.1 (#11616).
• Clean up .hgtags.

mpir-2.1.3.p4 (Leif Leonhardy, July 19th, 2011)

• #8664: Upgrade Sage's MPIR spkg to version 2.1.3
  ...

---

The list of changes between MPIR 2.1.3 (more precisely, 2.1.1) and MPIR 2.4.0 is fairly long, so I haven't put them into the description, but attached them​ in a plain text file.

For instructions on how to install / test the spkgs see also #8664. It is sufficient to just

• copy the spkg into $SAGE_ROOT/spkg/standard/, then

• run

  $ env SAGE_UPGRADING=yes make build
  

  (or omit build to also rebuild the documentation in the same make run).

To run just MPIR's test suite, you can reinstall the spkg with SAGE_CHECK=yes:

$ env SAGE_CHECK=yes ./sage -f spkg/standard/mpir-<version>.spkg

Or, if you haven't yet installed the spkg (but copied it into $SAGE_ROOT/spkg/standard/ as mentioned above), do:

$ env SAGE_CHECK=yes ./sage -i spkg/standard/mpir-<version>.spkg
$ env SAGE_UPGRADING=yes make build # rebuilds all dependent packages

Afterwards you can run make doc to (re)build the documentation, and / or make ptestlong to run Sage's full test suite in parallel.

---

Slightly related: #11844 reports a potential race condition due to yasm when building MPIR in parallel. We've never run into this [before] though. The MPIR 2.4.0.p2 spkg now includes a patch to upstream fixing that.

[leif]

Note that some changes between 2.1.1 and 2.1.4 may be missing (documented elsewhere, i.e. at least on mpir-devel I think). The segfault was fixed in 2.1.3, the race condition in 2.1.4.

[leif]

Setting this to "needs review" since the MPIR 2.1.3.p4 from #8664 got positive review again, though so far only by a single reviewer.

The current packages are mainly meant for testing the new upstream releases, hopefully on a variety of platforms supported by Sage; some improvements or changes to Sage's part will most probably follow.

It would just be nice to relatively early know whether any of them (more important, MPIR 2.4.0) causes any problems on one of our platforms. (MPIR 2.5.0 is scheduled for September 1st, which isn't that far...)

[leif]

Replying to leif:

(MPIR 2.5.0 is scheduled for September 1st, which isn't that far...)

News from ​mpir-devel:

"The release date for MPIR-2.5 was penciled in for 1st Sept, however we have decided to put it back a month to 1st Oct to allow us to get some more code in this release rather than wait for the MPIR-2.6 release on 1st Dec."

[ohanar]

I've successfully built and tested sage using the 2.4.0.p0 spkg on skynet/eno. I'm currently building/testing on some other machines as well (although, I don't have access to a ppc mac, so I can't help in that regard).

[ohanar]

Hmm, well I'm getting some random test failures on sage.math and skynet/iras with 2.4.0.p0. I'll try again with 2.3.1.p0 and report back.

[leif]

Added a reference to #11844 (potential race condition due to yasm when building MPIR in parallel).

[leif]

Added a note on the need to set ABI=32 on 64-bit machines running 32-bit operating systems. (The same is true for the already merged MPIR 2.1.3.p4 spkg.)

Also added a reference to #9858; FLINT 1.5.0's test suite won't build with any of the MPIR 2.x spkgs because it uses deprecated functions, and updated the installation instructions.

[leif]

MPIR 2.5.1 has been released on March 11th. We could also give that a try, although 2.4.0 has IMHO been tested a lot and didn't raise any problems within Sage, AFAIK, at least apparently not reproducible ones.

[leif]

It's not unlikely that I'll soon provide an MPIR 2.4.0.p2 spkg, also fixing a potential race condition in the yasm build, as well as an "experimental" MPIR 2.5.1 spkg.

Please build, test, and review!

[jdemeyer]

The following can be removed now:

# The Yasm build uses PYTHON from the environment to find python, so unset
# it since the setting from 'spkg/standard/newest_version' confuses it:
unset PYTHON
# This can be removed once #10492 has been merged, which turns the "package
# version" environment variables into solely `make` variables, such that the
# (shell) environment does no longer get polluted.

Also: do you feel like adding a work-around for #12765:

# Work around a bug in GCC 4.7.0 on ia64:
# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48496
if [ "`uname -m`" = ia64 ] && [ "`testcc.sh $CC`" = GCC ] ; then
    if $CC -dumpversion | grep >/dev/null '^4\.7\.0$' ; then
        echo >&2 "Warning: Working around bug in gcc 4.7.0"
        CFLAGS="$CFLAGS -O0"
    fi
fi

After sage-5.0.beta12 is released, I plan to test this spkg on the buildbots.

[jdemeyer]

Also: could you use patch for patching? Looking at patches/gmp-h.in.diff, there is no need for a conditional patch. The patch will only actually do something if __SUNPRO_CC is defined.

[jdemeyer]

Note that I have a mpir-2.1.3.p10 at #12782, we should coordinate somehow.

[leif]

Replying to jdemeyer:

Note that I have a mpir-2.1.3.p10 at #12782, we should coordinate somehow.

I'm currently working on a 2.4.0.p2 with a couple of fixes, including one for ia64 and GCC 4.7.0.

No way to get around this but disabling optimization completely?

[jdemeyer]

Replying to leif:

No way to get around this but disabling optimization completely?

I have investigated but didn't manage to fix the build using anything but -O0. Anyway, if #12369 gets merged, it's not really a problem.

[jdemeyer]

If you're creating a new spkg anyway, could you incorporate my changes from #12782?

[leif]

Replying to jdemeyer:

If you're creating a new spkg anyway, could you incorporate my changes from #12782?

Yes, although I kept the unset PYTHON, since it is safe and otherwise doesn't hurt.

[leif]

Replying to leif:

Replying to jdemeyer:

If you're creating a new spkg anyway, could you incorporate my changes from #12782?

Yes, although I kept the unset PYTHON, since it is safe and otherwise doesn't hurt.

We can also skip building a static library when building just for GCC in the first place I think.

[leif]

Diff between the previous spkg in Sage and my new p1 spkg. For reference / review only.

[leif]

I've uploaded a second, p2 spkg with further fixes, also incorporating [the] changes from / for

• #11844 (potential race condition in yasm build),
• #12765 (build error on Linux ia64 (Itanium) with GCC 4.7.0, which is a GCC bug),
• #12782 (build MPIR without C++ interface [and without static libraries] when bootstrapping the GCC spkg),
• configure error with the GCC spkg on MacOS X (on a Core i7 with AVX), as ​reported on sage-release.

See the changelog entries in the description and the attached diffs for details and further changes.

The p2 is still somewhat preliminary (subject to further testing), although I don't expect^TM further changes to be necessary, despite that haven't committed the changes yet.