Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#11896 closed defect (fixed)

MPIR 2.x may fail to build on 32-bit operating systems

Reported by: Nathann Cohen, Dan Drake, Steven Trogdon Owned by: leif
Priority: critical Milestone: sage-4.8
Component: packages: standard Keywords: ABI 64-bit CPU processor Linux -m32 MPN_PATH assembler error bad register lshift
Cc: ncohen, ddrake, strogdon, jdemeyer Merged in: sage-4.8.alpha0
Authors: Leif Leonhardy, Jeroen Demeyer Reviewers: Steven Trogdon, Jeroen Demeyer, Leif Leonhardy
Report Upstream: Reported upstream. Little or no feedback. Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by jdemeyer)

MPIR fails to build on 32-bit operating systems running on 64-bit processors if CFLAGS contain -m32 and ABI is not set.

The simple solution is to set ABI=32 in this case, which the new spkg now does (on Linux, where the issue occurred). It also sets ABI=64 if the operating system supports it, and ABI isn't already set, and prints according messages.


Upstream report: http://groups.google.com/group/mpir-devel/browse_thread/thread/46ccdc5dfc3485cd


New spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/mpir-2.1.3.p6.spkg

Attachments (2)

mpir-2.1.3.p4-p5.diff (2.6 KB) - added by leif 10 years ago.
Diff between the p4 (from #8664) and the p5 spkg. For reference / review only.
mpir-2.1.3.p5-p6.diff (2.3 KB) - added by jdemeyer 10 years ago.

Download all attachments as: .zip

Change History (32)

comment:1 Changed 10 years ago by leif

  • Cc ncohen ddrake strogdon added
  • Description modified (diff)
  • Reporter changed from leif to Nathann Cohen, Dan Drake, Steve Trogdon
  • Status changed from new to needs_review

Would be nice if one of you could review this.

For testing, it should be sufficient to (try to) install the spkg with an arbitrary Sage version (of course installed on some 32-bit Linux running on a 64-bit processor):

$ unset ABI # We want to test that the spkg works *without* setting it.
$ env SAGE_CHECK=yes ./sage -i http://sage.math.washington.edu/home/leif/Sage/spkgs/mpir-2.1.3.p5.spkg

After that, you should reinstall the original MPIR spkg from your Sage installation (with ./sage -f spkg/standard/mpir-<version>.spkg) in case you used a Sage version prior to Sage 4.7.2.alpha3 (into which the MPIR 2.1.3.p4 spkg from #8664, the first 2.x version in Sage, was merged).

(No need to uninstall it from Sage 4.7.2.alpha3 or if you happen to have already installed the MPIR 2.1.3.p4 spkg, the GMP-ECM 6.3.p2 spkg and the necessary patches from their tickets.)

You can of course in addition test the spkg on other (64-bit) platforms as well... :-)

Thanks!

Changed 10 years ago by leif

Diff between the p4 (from #8664) and the p5 spkg. For reference / review only.

comment:2 Changed 10 years ago by leif

P.S.:

I'll rebase the spkgs with newer upstream versions (see #11616) on this one when I find the time. I'll perhaps also provide an MPIR 2.1.4.p0 spkg based on this one which just adds a minor upstream fix (for a rare race condition which leads to an error during the installation).

comment:3 follow-up: Changed 10 years ago by leif

Oh, one more thing:

I cannot test whether the GMP-ECM and MPFR spkgs have the same problem (since they use GMP's / MPIR's CFLAGS), i.e., also fail on such systems if ABI isn't set.

If you find the time, you could also check this by unsetting ABI and reinstalling them (with ./sage -f ...).

comment:4 Changed 10 years ago by leif

  • Description modified (diff)

comment:5 in reply to: ↑ 3 ; follow-up: Changed 10 years ago by strogdon

  • Description modified (diff)
  • Reviewers set to Steven Trogdon
  • Status changed from needs_review to positive_review

Replying to leif:

Oh, one more thing: I cannot test whether the GMP-ECM and MPFR spkgs have the same problem (since they use GMP's / MPIR's CFLAGS), i.e., also fail on such systems if ABI isn't set. If you find the time, you could also check this by unsetting ABI and reinstalling them (with ./sage -f ...).

The new mpir spkg installs here on my 32-bit OS with 64-bit processor with ABI unset.

The MPIR test suite passed successfully.
Now cleaning up tmp files.
Making Sage/Python scripts relocatable...
Making script relocatable
Finished installing mpir-2.1.3.p5.spkg

It also installed on an amd64 to which I have access. The ECM and MPFR spkgs reinstalled correctly with ABI unset on both the 32-bit and amd64 machines. A positive review here.

comment:6 in reply to: ↑ 5 Changed 10 years ago by leif

  • Reporter changed from Nathann Cohen, Dan Drake, Steve Trogdon to Nathann Cohen, Dan Drake, Steven Trogdon

Replying to strogdon:

The new mpir spkg installs here on my 32-bit OS with 64-bit processor with ABI unset. It also installed on an amd64 to which I have access. The ECM and MPFR spkgs reinstalled correctly with ABI unset on both the 32-bit and amd64 machines. A positive review here.

Speed reviewing, thanks!

comment:7 Changed 10 years ago by jdemeyer

  • Cc jdemeyer added

comment:8 Changed 10 years ago by ncohen

( it compiled perfectly here ! Thank you very much leif ! :-D)

comment:9 Changed 10 years ago by jdemeyer

  • Merged in set to sage-4.7.2.alpha4
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:10 Changed 10 years ago by jdemeyer

This fails to build on Itanium:

mpir-2.1.3.p5
Machine:
Linux iras 2.6.16.46-0.12-default #1 SMP Thu May 17 14:00:09 UTC 2007 ia64 ia64 ia64 GNU/Linux
Deleting directories from past builds of previous/current versions of mpir-2.1.3.p5
Extracting package /home/buildbot/build/sage/iras-1/iras_full/build/sage-4.7.2.alpha4/spkg/standard/mpir-2.1.3.p5.spkg ...
-rw-r--r-- 1 buildbot sage 3759395 2011-10-04 22:36 /home/buildbot/build/sage/iras-1/iras_full/build/sage-4.7.2.alpha4/spkg/standard/mpir-2.1.3.p5.spkg
Finished extraction
****************************************************
Host system
uname -a:
Linux iras 2.6.16.46-0.12-default #1 SMP Thu May 17 14:00:09 UTC 2007 ia64 ia64 ia64 GNU/Linux
****************************************************
****************************************************
CC Version
gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/gcc-4.6.1/ia64-Linux-suse/libexec/gcc/ia64-unknown-linux-gnu/4.6.1/lto-wrapper
Target: ia64-unknown-linux-gnu
Configured with: /usr/local/gcc-4.6.1/src/gcc-4.6.1/configure --enable-languages=c,c++,fortran --with-gnu-as --with-as=/usr/local/binutils-2.21/ia64-Linux-suse-gcc-4.5.1/bin/as --with-gnu-ld --with-ld=/usr/local/binutils-2.21/ia64-Linux-suse-gcc-4.5.1/bin/ld --with-gmp=/usr/local/mpir-2.4.0/ia64-Linux-suse-gcc-4.6.0 --with-mpfr=/usr/local/mpfr-3.0.1/ia64-Linux-suse-mpir-2.4.0-gcc-4.6.0 --with-mpc=/usr/local/mpc-0.9/ia64-Linux-suse-mpir-2.4.0-mpfr-3.0.1-gcc-4.6.0 --prefix=/usr/local/gcc-4.6.1/ia64-Linux-suse
Thread model: posix
gcc version 4.6.1 (GCC) 
****************************************************
Warning: Building MPIR with SAGE_DEBUG=yes disables optimization.
Building a 32-bit version of MPIR
Checking what CFLAGS MPIR would use if they were empty...
Settings required to properly build MPIR, taking into account SAGE_DEBUG etc.:
  CFLAGS:   -g -O0
  LDFLAGS:  -Wl,-z,noexecstack
  ABI:     32
Settings from the "global" environment:
  CC:      gcc
  CFLAGS:  
  (CPPFLAGS, CXX and CXXFLAGS are listed below; these don't get modified.)
Using the spkg's (i.e. Sage's) default (plus mandatory) settings.
Finally using the following settings:
  CC=gcc
  CFLAGS=   -g -O0
  CPPFLAGS=
  CXX=g++
  CXXFLAGS=
  LDFLAGS= -Wl,-z,noexecstack
  ABI=32
(These settings may still get overridden by 'configure' or Makefiles.)
Configuring MPIR with the following options:
    --prefix="/home/buildbot/build/sage/iras-1/iras_full/build/sage-4.7.2.alpha4/local" --enable-shared --enable-static --enable-gmpcompat --enable-cxx=yes 
You can set MPIR_EXTRA_OPTS to pass additional parameters.
checking build system type... ia64-unknown-linux-gnu
checking host system type... ia64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make -j4 sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
configure: error: ABI=32 is not among the following valid choices: 64
Error configuring MPIR. (See above for the options passed to it.)

comment:11 Changed 10 years ago by jdemeyer

  • Resolution fixed deleted
  • Status changed from closed to new

The problem is that the -m32/-m64 switches are not supported on every system. So we should do the following:

  1. Try building and running with -m64.
  2. Try building and running with -m32.
  3. If both fail, fall back to the old behaviour (which was what?)

comment:12 Changed 10 years ago by jdemeyer

  • Description modified (diff)
  • Merged in sage-4.7.2.alpha4 deleted

The attached patch should fix the problem. New spkg for testing: http://boxen.math.washington.edu/home/jdemeyer/spkg/mpir-2.1.3.p6.spkg

comment:13 Changed 10 years ago by jdemeyer

  • Description modified (diff)

comment:14 follow-up: Changed 10 years ago by leif

A much simpler solution is to set CFLAG64="" on ia64 (Itanium), then the first attempt (building and running a 64-bit executable) will succeed.

comment:15 Changed 10 years ago by leif

By the way, I'd say that's a GCC flaw; it should support -m64 on Itanium as well since there it's just a redundant flag, as opposed to a conflicting or unsupported one.

comment:16 Changed 10 years ago by leif

Oh, it does have related options for HP-UX: :-)

       -milp32
       -mlp64
           Generate code for a 32-bit or 64-bit environment.  The 32-bit environment sets int,
           long and pointer to 32 bits.  The 64-bit environment sets int to 32 bits and long and
           pointer to 64 bits.  These are HP-UX specific flags.

comment:17 in reply to: ↑ 14 ; follow-up: Changed 10 years ago by jdemeyer

Replying to leif:

A much simpler solution is to set CFLAG64="" on ia64 (Itanium), then the first attempt (building and running a 64-bit executable) will succeed.

My patch doesn't do specific checking for Itanium, it will work on any compiler which does not support -m32 and -m64.

comment:18 in reply to: ↑ 17 Changed 10 years ago by leif

Replying to jdemeyer:

Replying to leif:

A much simpler solution is to set CFLAG64="" on ia64 (Itanium), then the first attempt (building and running a 64-bit executable) will succeed.

My patch doesn't do specific checking for Itanium, it will work on any compiler which does not support -m32 and -m64.

I'd factor out -m32 though, i.e., use $CFLAG32 in the second attempt and add

if [[ -z $CFLAG32 ]]; then
    CFLAG32=-m32
fi

or

: ${CFLAG32=-m32} # doesn't invalidate an empty setting ("")

or the like (at the top level), such that porting people can adapt it to whatever they need.

comment:19 Changed 10 years ago by leif

Alternatively, you could just set ABI=64 (without any checks) on Itanium, which is reasonable as it doesn't (or at least we don't) support anything else on that.

comment:20 Changed 10 years ago by leif

(I originally intended to restrict the test / setting the ABI to x86 / x86_64 and ppc / ppc64 anyway... ;-) )

comment:21 Changed 10 years ago by jdemeyer

  • Milestone changed from sage-4.7.2 to sage-4.7.3
  • Priority changed from blocker to critical

comment:22 follow-up: Changed 10 years ago by jdemeyer

  • Status changed from new to needs_review

New spkg which uses $CFLAG32, same location. Needs review.

comment:23 in reply to: ↑ 22 ; follow-up: Changed 10 years ago by leif

  • Authors changed from Leif Leonhardy to Leif Leonhardy, Jeroen Demeyer
  • Reviewers changed from Steven Trogdon to Steven Trogdon, Jeroen Demeyer, Leif Leonhardy
  • Status changed from needs_review to positive_review

Replying to jdemeyer:

New spkg which uses $CFLAG32, same location. Needs review.

Positive review, although the comment

            # (Usually succeeds, unless e.g. a 32-bit CPU is explicitly selected by CFLAGS.)

referred to building, not running a 64-bit executable, and the Changelog entry mentions -m32 and -m64 rather than CFLAG32 and CFLAG64.

Tested with SAGE_CHECK=yes on Linux x86, x86_64 and ia64, with and without {C,CPP,CXX}FLAGS and ABI set.

Changed 10 years ago by jdemeyer

comment:24 in reply to: ↑ 23 Changed 10 years ago by jdemeyer

Replying to leif:

Replying to jdemeyer:

New spkg which uses $CFLAG32, same location. Needs review.

Positive review, although the comment

            # (Usually succeeds, unless e.g. a 32-bit CPU is explicitly selected by CFLAGS.)

referred to building, not running a 64-bit executable

I think it is actually more correct to refer to building and running rather than just building.

the Changelog entry mentions -m32 and -m64 rather than CFLAG32 and CFLAG64.

Fixed this.

comment:25 Changed 10 years ago by jdemeyer

  • Merged in set to sage-4.7.3.alpha0
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:26 follow-up: Changed 10 years ago by jhpalmieri

For what it's worth, this fails to build on the skynet machine silius. I get the same error message with p5 and p6:

Building a 64-bit version of MPIR
Checking what CFLAGS MPIR would use if they were empty...
Settings required to properly build MPIR, taking into account SAGE_DEBUG etc.:
  CFLAGS:
  LDFLAGS:  -Wl,-z,noexecstack
  ABI:     64
Settings from the "global" environment:
  CC:      gcc
  CFLAGS:
  (CPPFLAGS, CXX and CXXFLAGS are listed below; these don't get modified.)
Using the spkg's (i.e. Sage's) default (plus mandatory) settings.
Finally using the following settings:
  CC=gcc
  CFLAGS=  -g -O3
  CPPFLAGS=
  CXX=g++
  CXXFLAGS=
  LDFLAGS= -Wl,-z,noexecstack
  ABI=64
(These settings may still get overridden by 'configure' or Makefiles.)
Configuring MPIR with the following options:
    --prefix="/home/palmieri/silius/sage-4.7.2.alpha4/local" --enable-shared --enable-static --ena\
ble-gmpcompat --enable-cxx=yes
You can set MPIR_EXTRA_OPTS to pass additional parameters.
checking build system type... power7-unknown-linux-gnu
checking host system type... power7-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make -j12 sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
configure: error: ABI=64 is not among the following valid choices: mode64 mode32 32
Error configuring MPIR. (See above for the options passed to it.)

comment:27 in reply to: ↑ 26 ; follow-up: Changed 10 years ago by leif

Replying to jhpalmieri:

For what it's worth, this fails to build on the skynet machine silius. I get the same error message with p5 and p6.

Ooops, I was pretty sure I also tested it there. The p4 builds just fine.

I'll prepare a follow-up spkg...

comment:28 in reply to: ↑ 27 Changed 10 years ago by leif

Replying to leif:

Replying to jhpalmieri:

For what it's worth, this fails to build on the skynet machine silius. I get the same error message with p5 and p6.

Ooops, I was pretty sure I also tested it there. The p4 builds just fine.

I'll prepare a follow-up spkg...

New spkg (p7) up on #11964, needs review.

comment:29 Changed 10 years ago by jdemeyer

  • Milestone sage-4.7.3 deleted

Milestone sage-4.7.3 deleted

comment:30 Changed 10 years ago by jdemeyer

  • Merged in changed from sage-4.7.3.alpha0 to sage-4.8.alpha0
  • Milestone set to sage-4.8
Note: See TracTickets for help on using tickets.