Opened 10 years ago

Last modified 10 years ago

#11616 closed enhancement

Upgrade MPIR to a more recent upstream release — at Version 25

Reported by: leif Owned by: leif
Priority: blocker Milestone: sage-5.0
Component: packages: standard Keywords: sd32, GMP, SandyBridge, Westmere, yasm re2c race condition, Linux ia64 Itanium GCC 4.7.0 bug
Cc: justin, wbhart, jpflori Merged in:
Authors: Leif Leonhardy Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by leif)

This is a follow-up to #8664 (and a couple of other tickets).


The following new spkgs are based on the latest MPIR 2.1.3 spkg, the p9 from #12131:

New spkg: http://boxen.math.washington.edu/home/leif/Sage/spkgs/mpir-2.4.0.p2.spkg

md5sum: c9e6d8e2f5de4a09b4b16ee2354a9e17 mpir-2.4.0.p2.spkg

(Preliminary; changes not yet committed.)

New spkg: http://boxen.math.washington.edu/home/leif/Sage/spkgs/mpir-2.4.0.p1.spkg

md5sum: 0d88b7dfe358bb429a5a9aa3a3b4b00b mpir-2.4.0.p1.spkg

mpir-2.4.0.p2 (Leif Leonhardy, April 4th, 2012)

#11616 (upgrading MPIR), further fixes:

  • Before enabling -march=native, minimalistically check whether the system's assembler also understands the instructions the compiler emits with that option. (Work-around for e.g. GCC 4.6.3 on MacOS X 10.x and Intel Core i7-family CPUs with AVX.)
  • Do not unconditionally unset PYTHON, since Sage (>=5.0.beta10) no longer pollutes the environment with its package version variables, which previous- ly confused yasm's configure.
  • Fix extraction of __GMP_CC and __GMP_CFLAGS from gmp.h, since MPIR meanwhile defines these to preprocessor variables (rather than literals). Also don't use \+ in sed patterns, as this is less portable.
  • Work around GCC 4.7.0 bug (compilation error) on Linux ia64 (Itanium) by almost completely disabling optimization on that platform if GCC 4.7.x is detected. This doesn't hurt much if we later rebuild MPIR with a (non- broken) GCC from the new GCC spkg. Cf. #12765.
  • Do not build the C++ interface and static libraries when bootstrapping the GCC spkg, i.e. if SAGE_BUILD_TOOLCHAIN=yes. (GMP/MPIR is a prerequisite for it, and MPIR will later get rebuilt with both enabled, with the newly built GCC.) Cf. #12782.
  • Fix a potential race condition in yasm's build by patching the re2c source. Cf. #11844.
  • Add "patch loop" to apply any patches (*.patch) located in patches/. Currently only the re2c patch matches that; the prepatched header to support Sun's C compiler is still copied over (and only on SunOS, although it doesn't do any harm on other platforms).
  • Minor clean-up; e.g. redirect error messages and warnings to stderr, quote parameter to --libdir, add some comments and messages, also save user's setting of LDFLAGS and ABI.

mpir-2.4.0.p1 (Leif Leonhardy, March 21st, 2012)

  • Upstream upgrade to MPIR 2.4.0 (#11616). The 2.4.0.p0 spkg isn't in this history, as it was based on the 2.1.3.p4 spkg, i.e., is "on another branch", and never got merged into Sage.
  • Remove forcing a sequential make install again, since the potential race condition was fixed in MPIR 2.1.4.
  • Fix .hgtags, which contained duplicate entries, and was missing others.

The following old spkgs are based on the obsolete MPIR 2.1.3.p4 spkg from #8664:

You may want to give both of them a try, although the former is just intended as a "fallback" spkg in case problems introduced in the latter should arise.

Note: On 64-bit processors running 32-bit operating systems, one currently has to set ABI=32, otherwise the build will fail. (This does not apply to the newer MPIR 2.4.0 p1 and p2 spkgs.)

mpir-2.4.0.p0 (Leif Leonhardy, July 21st, 2011)

  • Upstream upgrade to MPIR 2.4.0 (#11616).

mpir-2.3.1.p0 (Leif Leonhardy, July 20th, 2011)

  • Upstream upgrade to MPIR 2.3.1 (#11616).
  • Clean up .hgtags.

mpir-2.1.3.p4 (Leif Leonhardy, July 19th, 2011)

  • #8664: Upgrade Sage's MPIR spkg to version 2.1.3
    ...


The list of changes between MPIR 2.1.3 (more precisely, 2.1.1) and MPIR 2.4.0 is fairly long, so I haven't put them into the description, but attached them in a plain text file.


For instructions on how to install / test the spkgs see also #8664. It is sufficient to just

  • copy the spkg into $SAGE_ROOT/spkg/standard/, then
  • run
    $ env SAGE_UPGRADING=yes make build
    
    (or omit build to also rebuild the documentation in the same make run).

To run just MPIR's test suite, you can reinstall the spkg with SAGE_CHECK=yes:

$ env SAGE_CHECK=yes ./sage -f spkg/standard/mpir-<version>.spkg

Or, if you haven't yet installed the spkg (but copied it into $SAGE_ROOT/spkg/standard/ as mentioned above), do:

$ env SAGE_CHECK=yes ./sage -i spkg/standard/mpir-<version>.spkg
$ env SAGE_UPGRADING=yes make build # rebuilds all dependent packages

Afterwards you can run make doc to (re)build the documentation, and / or make ptestlong to run Sage's full test suite in parallel.


Slightly related: #11844 reports a potential race condition due to yasm when building MPIR in parallel. We've never run into this [before] though. The MPIR 2.4.0.p2 spkg now includes a patch to upstream fixing that.

Change History (27)

comment:1 Changed 10 years ago by leif

  • Description modified (diff)

Changed 10 years ago by leif

Note that some changes between 2.1.1 and 2.1.4 may be missing (documented elsewhere, i.e. at least on mpir-devel I think). The segfault was fixed in 2.1.3, the race condition in 2.1.4.

comment:2 follow-up: Changed 10 years ago by leif

  • Description modified (diff)
  • Status changed from new to needs_review

Setting this to "needs review" since the MPIR 2.1.3.p4 from #8664 got positive review again, though so far only by a single reviewer.

The current packages are mainly meant for testing the new upstream releases, hopefully on a variety of platforms supported by Sage; some improvements or changes to Sage's part will most probably follow.

It would just be nice to relatively early know whether any of them (more important, MPIR 2.4.0) causes any problems on one of our platforms. (MPIR 2.5.0 is scheduled for September 1st, which isn't that far...)

comment:3 in reply to: ↑ 2 Changed 10 years ago by leif

Replying to leif:

(MPIR 2.5.0 is scheduled for September 1st, which isn't that far...)

News from mpir-devel:

"The release date for MPIR-2.5 was penciled in for 1st Sept, however we have decided to put it back a month to 1st Oct to allow us to get some more code in this release rather than wait for the MPIR-2.6 release on 1st Dec."

comment:4 Changed 10 years ago by ohanar

I've successfully built and tested sage using the 2.4.0.p0 spkg on skynet/eno. I'm currently building/testing on some other machines as well (although, I don't have access to a ppc mac, so I can't help in that regard).

comment:5 Changed 10 years ago by ohanar

Hmm, well I'm getting some random test failures on sage.math and skynet/iras with 2.4.0.p0. I'll try again with 2.3.1.p0 and report back.

comment:6 Changed 10 years ago by was

  • Keywords sd32 added

comment:7 Changed 10 years ago by leif

  • Description modified (diff)

comment:8 Changed 10 years ago by leif

  • Description modified (diff)

Added a reference to #11844 (potential race condition due to yasm when building MPIR in parallel).

comment:9 Changed 10 years ago by leif

  • Description modified (diff)

Added a note on the need to set ABI=32 on 64-bit machines running 32-bit operating systems. (The same is true for the already merged MPIR 2.1.3.p4 spkg.)

Also added a reference to #9858; FLINT 1.5.0's test suite won't build with any of the MPIR 2.x spkgs because it uses deprecated functions, and updated the installation instructions.

comment:10 Changed 10 years ago by leif

  • Status changed from needs_review to needs_work
  • Work issues set to Rebase the spkg(s) on the MPIR 2.1.3.p5 spkg from #11896.

comment:11 Changed 10 years ago by jdemeyer

  • Dependencies #8664 #5847 deleted
  • Description modified (diff)
  • Work issues changed from Rebase the spkg(s) on the MPIR 2.1.3.p5 spkg from #11896. to Rebase the spkg(s) on the MPIR 2.1.3.p7 spkg from #11964

comment:12 Changed 10 years ago by leif

  • Work issues changed from Rebase the spkg(s) on the MPIR 2.1.3.p7 spkg from #11964 to Rebase the spkg(s) on the MPIR 2.1.3.p9 spkg from #12131

comment:13 Changed 10 years ago by leif

MPIR 2.5.1 has been released on March 11th. We could also give that a try, although 2.4.0 has IMHO been tested a lot and didn't raise any problems within Sage, AFAIK, at least apparently not reproducible ones.

comment:14 Changed 10 years ago by leif

  • Authors set to Leif Leonhardy
  • Description modified (diff)
  • Status changed from needs_work to needs_review
  • Work issues Rebase the spkg(s) on the MPIR 2.1.3.p9 spkg from #12131 deleted

It's not unlikely that I'll soon provide an MPIR 2.4.0.p2 spkg, also fixing a potential race condition in the yasm build, as well as an "experimental" MPIR 2.5.1 spkg.

Please build, test, and review!

comment:15 Changed 10 years ago by jdemeyer

The following can be removed now:

# The Yasm build uses PYTHON from the environment to find python, so unset
# it since the setting from 'spkg/standard/newest_version' confuses it:
unset PYTHON
# This can be removed once #10492 has been merged, which turns the "package
# version" environment variables into solely `make` variables, such that the
# (shell) environment does no longer get polluted.

Also: do you feel like adding a work-around for #12765:

# Work around a bug in GCC 4.7.0 on ia64:
# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48496
if [ "`uname -m`" = ia64 ] && [ "`testcc.sh $CC`" = GCC ] ; then
    if $CC -dumpversion | grep >/dev/null '^4\.7\.0$' ; then
        echo >&2 "Warning: Working around bug in gcc 4.7.0"
        CFLAGS="$CFLAGS -O0"
    fi
fi

After sage-5.0.beta12 is released, I plan to test this spkg on the buildbots.

comment:16 Changed 10 years ago by jdemeyer

Also: could you use patch for patching? Looking at patches/gmp-h.in.diff, there is no need for a conditional patch. The patch will only actually do something if __SUNPRO_CC is defined.

comment:17 follow-up: Changed 10 years ago by jdemeyer

Note that I have a mpir-2.1.3.p10 at #12782, we should coordinate somehow.

comment:18 in reply to: ↑ 17 ; follow-up: Changed 10 years ago by leif

Replying to jdemeyer:

Note that I have a mpir-2.1.3.p10 at #12782, we should coordinate somehow.

I'm currently working on a 2.4.0.p2 with a couple of fixes, including one for ia64 and GCC 4.7.0.

No way to get around this but disabling optimization completely?

comment:19 in reply to: ↑ 18 Changed 10 years ago by jdemeyer

Replying to leif:

No way to get around this but disabling optimization completely?

I have investigated but didn't manage to fix the build using anything but -O0. Anyway, if #12369 gets merged, it's not really a problem.

comment:20 follow-up: Changed 10 years ago by jdemeyer

If you're creating a new spkg anyway, could you incorporate my changes from #12782?

comment:21 in reply to: ↑ 20 ; follow-up: Changed 10 years ago by leif

Replying to jdemeyer:

If you're creating a new spkg anyway, could you incorporate my changes from #12782?

Yes, although I kept the unset PYTHON, since it is safe and otherwise doesn't hurt.

comment:22 in reply to: ↑ 21 Changed 10 years ago by leif

Replying to leif:

Replying to jdemeyer:

If you're creating a new spkg anyway, could you incorporate my changes from #12782?

Yes, although I kept the unset PYTHON, since it is safe and otherwise doesn't hurt.

We can also skip building a static library when building just for GCC in the first place I think.

Changed 10 years ago by leif

Diff between the previous spkg in Sage and my new p1 spkg. For reference / review only.

comment:23 Changed 10 years ago by leif

  • Cc justin added
  • Description modified (diff)
  • Keywords yasm re2c race condition Linux ia64 Itanium GCC 4.7.0 bug added

I've uploaded a second, p2 spkg with further fixes, also incorporating [the] changes from / for

  • #11844 (potential race condition in yasm build),
  • #12765 (build error on Linux ia64 (Itanium) with GCC 4.7.0, which is a GCC bug),
  • #12782 (build MPIR without C++ interface [and without static libraries] when bootstrapping the GCC spkg),
  • configure error with the GCC spkg on MacOS X (on a Core i7 with AVX), as reported on sage-release.

See the changelog entries in the description and the attached diffs for details and further changes.


The p2 is still somewhat preliminary (subject to further testing), although I don't expectTM further changes to be necessary, despite that haven't committed the changes yet.

comment:24 Changed 10 years ago by leif

  • Description modified (diff)

comment:25 Changed 10 years ago by leif

  • Description modified (diff)
Note: See TracTickets for help on using tickets.