Opened 8 years ago

Last modified 8 years ago

#11666 closed enhancement

Upgrade MPFR to 3.1.0 — at Version 31

Reported by: leif Owned by: leif
Priority: major Milestone: sage-5.0
Component: packages: standard Keywords: sd36 sd32 MPFR spkg wishlist sd35.5
Cc: jpflori, zimmerma Merged in:
Authors: Mike Hansen, Jean-Pierre Flori Reviewers: Paul Zimmermann
Report Upstream: Fixed upstream, but not in a stable release. Work issues: failing doctests for 32 bits
Branch: Commit:
Dependencies: #12171 Stopgaps:

Description (last modified by jpflori)

Our current MPFR spkg is fairly old (based on MPFR 2.4.2), and upgrading it to the latest [stable] upstream release is now on the high-priority wishlist.

Use http://perso.telecom-paristech.fr/~flori/sage/mpfr-3.1.0.spkg


MPFR 3.0.1 was released on April 4th 2011; there are already some additional upstream patches for that version; see http://www.mpfr.org/mpfr-current/.


MPFR 3.1.0 has been released October 3rd 2011.

Changes from versions 3.0.* to version 3.1.0:

  • The MPFR source has been reorganized.
  • Dropped ansi2knr support.
  • TLS support is now detected automatically. If TLS is supported, MPFR is built as thread safe by default. To disable TLS explicitly, configure MPFR with --disable-thread-safe.
  • New --enable-gmp-internals configure option to use GMP's undocumented functions (not from the public API). Note that library versioning is not guaranteed to work if this option is used.
  • The mpfr_urandom() and mpfr_urandomb() functions now return identical values on processors with different word size (assuming the same random seed, and since the GMP random generator does not depend itself on the word size, cf. http://gmplib.org/list-archives/gmp-devel/2010-September/001642.html).
  • The mpfr_add_one_ulp() and mpfr_sub_one_ulp() macros (which are obsolete and no more documented) will be removed in a future release.
  • Speed improvement for the mpfr_sqr() and mpfr_div() functions using Mulders' algorithm. As a consequence, other functions using those routines are also faster.
  • Much faster formatted output (mpfr_printf(), etc.) with %Rg and similar.
  • The --with-gmp-build configure option can now be used when the GMP source directory and the GMP build directory are different (without having to copy header files manually as before).
  • New functions mpfr_buildopt_tune_case(), mpfr_frexp(), mpfr_grandom() and mpfr_z_sub().
  • New division-by-zero exception (flag) and associated functions.
  • The mpfr.h header can be included several times, while still supporting optional functions (see Section "Headers and Libraries" in the manual).
  • Updated tuning parameters.
  • Improved MPFR manual.
  • MPFR tests: libtool no longer generates wrapper scripts with make check (so that running the tests under valgrind or gdb is easier).
  • Bug fixes.
    Note: The mpfr_subnormalize() implementation up to MPFR 3.0.0 did not change the flags. In particular, it did not follow the generic rule concerning the inexact flag (and no special behavior was specified). The case of the underflow flag was more a lack of specification.

Tarballs are available in various formats (regarding the compression method); a bzipped one can be found here (upstream).

Change History (31)

comment:1 Changed 8 years ago by was

  • Keywords sd32 added

comment:2 Changed 8 years ago by jpflori

  • Cc jpflori added

comment:3 Changed 8 years ago by leif

  • Description modified (diff)
  • Owner changed from tbd to leif
  • Summary changed from Upgrade MPFR to 3.0.1 + upstream patches to Upgrade MPFR to 3.0.1 + upstream patches or later (3.1.0)

comment:4 Changed 8 years ago by leif

  • Description modified (diff)

The second release candidate is out:

[...] The main changes since this first release candidate are:

  • Fixed --enable-gmp-internals.
  • Handle the special cases in mpfr_cmp_q() and mpfr_cmp_f() (fixing the problem with the erange flag in particular).
  • Added mpfr_buildopt_gmpinternals_p() function.
  • MPFR manual update and minor corrections.

Here's a second release candidate. As there should not be new platform-specific problems, the final release is delayed by a few days only.

[...]
http://www.mpfr.org/mpfr-3.1.0/mpfr-3.1.0-rc2.tar.bz2
[...]

The MD5's:
[...]
6ba48c87687959d5e68cd695686257f4 mpfr-3.1.0-rc2.tar.bz2
[...]

[...]

Final release now scheduled for October 3rd.

comment:5 Changed 8 years ago by leif

  • Description modified (diff)

The final MPFR 3.1.0 has been released today.

See the description for links.

comment:6 Changed 8 years ago by mhansen

  • Authors set to Mike Hansen
  • Dependencies set to 12171
  • Summary changed from Upgrade MPFR to 3.0.1 + upstream patches or later (3.1.0) to Upgrade MPFR to 3.1.0

comment:7 Changed 8 years ago by mhansen

  • Status changed from new to needs_review

I have an spkg at http://sage.math.washington.edu/mpfr-3.1.0.spkg. This is needed for flint2.

comment:8 Changed 8 years ago by fbissey

Shouldn't the dependency be the other way around? That is mpfi needs mpfr rather than mpfr needs mpfi? Have you tested this on a 32bit OS?

comment:9 Changed 8 years ago by mhansen

This version of MPFI should work with both versions of MPFR. However, the old version of MPFI will not work with the new version of MPFI.

I haven't tested it on a 32bit OS. This was primarily just so we could build flint2 in Sage.

comment:10 Changed 8 years ago by zimmerma

  • Cc zimmerma added

comment:11 follow-up: Changed 8 years ago by zimmerma

I wonder why comment 8 speaks about MPFI. Also, upstream (MPFR) we have seen several problems with the --enable-thread-safe option which is now enabled by default. Unless needed for Sage, my advice would be to configure MPFR with --disable-thread-safe.

Paul

comment:12 Changed 8 years ago by zimmerma

  • Status changed from needs_review to needs_info

the url from comment 7 does not work.

Paul

comment:13 in reply to: ↑ 11 Changed 8 years ago by fbissey

Replying to zimmerma:

I wonder why comment 8 speaks about MPFI. Also, upstream (MPFR) we have seen several problems with the --enable-thread-safe option which is now enabled by default. Unless needed for Sage, my advice would be to configure MPFR with --disable-thread-safe.

I spoke about mpfi because this ticket is marked as depending on #12171 which is upgrading mpfi. My understanding of the dependency field is that it means that you need #12171 for this ticket. This is silly and should be the other way around.

I think comment 9 has not been written with a clear mind.

I asked about test on 32bit machine because of this [ https://github.com/cschwan/sage-on-gentoo/issues/13].

comment:14 Changed 8 years ago by mhansen

The spkg is actually at http://sage.math.washington.edu/home/mhansen/mpfr-3.1.0.spkg

Let me clarify the MPFI issue since I must have been too tired when I tried before. If we were to just install this version of MPFR, the current MPFI (1.3.4) will fail to build since it uses something which has been removed/moved in the new version of MPFR. If the new version of MPFI works with both the old and new versions of MPFR. So, if we want to actually include this package in Sage, we have to have the new version of MPFI in first. This is the reason I listed the MPFI ticket as a dependency of this one.

comment:15 Changed 8 years ago by fbissey

You are making sense now. I indeed remember making a patch for mpfi-1.4 to build against mpfr-3.x in gentoo, it would apply to 1.3.4 as well I think. Anyway there are a number of things to fix as far as I can see because of the inclusion of these two.

comment:16 Changed 8 years ago by jpflori

Some thoughts about the spkg-install script:

  • It sets some CFLAGS and others itself, is it needed? does not mpfr look for the gmp header and pumps from there if possible? and should we take advantage of this (as in done for mpir where spkg-install look at what mpir chooses if CFLAGS et al. are empty)?
  • Is the patch for Solaris stuff still needed? (see #6453) -> at least the code in mpn_exp.c did not change in mpfr 3.1.0, so I guess it has to be included to support outdated Solaris systems where memset is buggy?
  • It always runs the test suite, whether SAGE_CHECK is set (then it is ran twice...) or not (then just once), is there still a good reason for doing so? It seems that such a behavior was also present in the MPIR spkg but was removed (see #9522 and #8664)
  • make should be replaced by $MAKE in spkg-check

comment:17 Changed 8 years ago by zimmerma

It sets some CFLAGS and others itself, is it needed?

as a developer of MPFR I strongly recommend to let MPFR choose the same CC and CFLAGS as GMP (they are read in gmp.h). Since we do this for MPFR, this greatly improved the portability of MPFR on different platforms with different compilers, in particuler it solves the ABI=32 vs ABI=64 issues.

Is the patch for Solaris stuff still needed?

I guess yes, since this was a bug in the Solaris kernel.

Paul

comment:18 follow-up: Changed 8 years ago by zimmerma

I've tested this spkg on top of #12171 on my 64-bit laptop (Core 2 Duo): all doctests pass.

Paul

comment:19 in reply to: ↑ 18 Changed 8 years ago by fbissey

Replying to zimmerma:

I've tested this spkg on top of #12171 on my 64-bit laptop (Core 2 Duo): all doctests pass.

Yes 64bit is fine as far as I can tell. It's 32bit that's the trouble - both linux and OS X [10.5.8], don't know about other like solaris or cygwin.

comment:20 Changed 8 years ago by zimmerma

  • Keywords sd35.5 added

comment:21 follow-up: Changed 8 years ago by zimmerma

  • Dependencies changed from 12171 to #12171
  • Reviewers set to Paul Zimmermann

on a 32-bit Pentium 4, after applying #12171 and importing this spkg, I get when I start Sage:

ImportError: /localdisk/tmp/sage-4.7.2/local/lib/libmpfi.so.0: undefined symbol: mpfr_add_d
Error importing ipy_profile_sage - perhaps you should run %upgrade?
WARNING: Loading of ipy_profile_sage failed.

Maybe I should first import this spkg then that of MPFI?

Paul

comment:22 follow-up: Changed 8 years ago by jpflori

Still about the spkg itself:

  • Is there any reason to "unset RM"? it says it messes with libtool while running "make check". I do not see this restriction in the MPIR package, or the MPFI one e.g. Is this for a specific architecture? Is this really needed? I guess this is fixed by #3537
  • MAKE is set back to make before running "make install" (and then "make check" in the original package), any undocumented reason?

I'll test all of this on my system but if someone can point me to the relevant info, I'd be grateful, Google wans not that helpful.

comment:23 follow-up: Changed 8 years ago by jpflori

I've put an updated package implementing my remarks, mainly using the mpir spkg-install file, I've also updated the description and license info.

It can be found at

http://perso.telecom-paristech.fr/sage/mpfr-3.1.0.spkg

It seems to do what it should, and uccessfully built and rand MPFR test suite on a "usual" intel 64 bit system, not any idea if I broke anything for more exotic architectures. Did not ran sage test suite either.

comment:24 in reply to: ↑ 23 Changed 8 years ago by leif

comment:25 in reply to: ↑ 22 Changed 8 years ago by leif

Replying to jpflori:

Still about the spkg itself:

  • Is there any reason to "unset RM"? it says it messes with libtool while running "make check". I do not see this restriction in the MPIR package, or the MPFI one e.g. Is this for a specific architecture? Is this really needed? I guess this is fixed by #3537.

Unsetting RM is safe, but meanwhile a bit redundant. Newer autotools use $RM instead of rm -f (or $RM -f), so tend to fail if RM happens to be set to (just) rm.


  • MAKE is set back to make before running "make install" (and then "make check" in the original package), any undocumented reason?

I'm pretty sure that's a relict from when people didn't know how to properly disable parallel make. If in doubt, use $MAKE -j1 install etc.

comment:27 in reply to: ↑ 21 Changed 8 years ago by leif

Replying to zimmerma:

Maybe I should first import this spkg then that of MPFI?

Since MPFI depends on MPFR (and MPIR), you have to rebuild MPFI after upgrading one of the latter.

If you have enough time (and/or the machine is fast enough), you can copy new/updated spkgs into $SAGE_ROOT/spkg/standard/ and run

$ env SAGE_UPGRADING=yes make build

afterwards. That way all dependent packages automatically get rebuilt, which can take some time.

(Unfortunately there are also a couple of stupid transitive dependencies; e.g. ATLAS gets rebuilt if you update readline -- just because ATLAS now has an spkg-install file written in Python, and Python depends on readline... So a lot of rebuilds performed by make aren't really necessary.)

comment:28 Changed 8 years ago by zimmerma

  • Work issues set to fails on 32-bit

I tried importing the MPFI spkg+patches (#12171) and that spkg on top of 4.8.alpha6 on a 32-bit Pentium 4 and got the following failures:

sage -t  "devel/sage-main/sage/matrix/matrix_mpolynomial_dense.pyx"
sage -t  "devel/sage-main/sage/matrix/constructor.py"
sage -t  "devel/sage-main/sage/matrix/matrix2.pyx"
sage -t  "devel/sage-main/sage/misc/randstate.pyx"
sage -t  "devel/sage-main/sage/modules/free_module_element.pyx"
sage -t  "devel/sage-main/sage/rings/real_mpfi.pyx"
sage -t  "devel/sage-main/sage/rings/complex_field.py"
sage -t  "devel/sage-main/sage/rings/complex_interval_field.py"
sage -t  "devel/sage-main/sage/rings/real_mpfr.pyx" # Killed/crashed
sage -t  "devel/sage-main/sage/schemes/elliptic_curves/ell_rational_field.py"

Paul

comment:29 Changed 8 years ago by zimmerma

the failures are most likely due to the fact that the mpfr_urandom and mpfr_urandomb return identical values on processors with different word size in MPFR 3.1.0 (see http://www.mpfr.org/mpfr-3.1.0/#changes).

The bad news is that this will need changing the doctests.

The good news is that we won't need any more different doctests on 32-bit and 64-bit (assuming the MPIR random generator is independent of the word size).

Paul Zimmermann

comment:30 Changed 8 years ago by jpflori

For the record, Mike's spkg and my updated one both need to be rebased on top of #12131.

I'll take care of that later.

comment:31 Changed 8 years ago by jpflori

  • Authors changed from Mike Hansen to Mike Hansen, Jean-Pierre Flori
  • Description modified (diff)
  • Work issues changed from fails on 32-bit to failing doctests for 32 bits

You can finally find a rebased spkg where I discarded some stuff involving ABI which previously made sage print that it was building for 32 bits, whereas it actually had no effect, and which is rebased on #12131 at the usual address:

http://perso.telecom-paristech.fr/~flori/sage/mpfr-3.1.0.spkg

I guess the issue with doctests is the last step before "needs review"

Note: See TracTickets for help on using tickets.