Opened 9 years ago

Closed 9 years ago

#9876 closed defect (fixed)

Building PARI/GP with SAGE_CHECK=yes fails on 32-bit big endian machines

Reported by: jdemeyer Owned by: tbd
Priority: blocker Milestone: sage-4.6
Component: packages: standard Keywords:
Cc: drkirkby, mpatel, GeorgSWeber Merged in: sage-4.6.alpha2
Authors: Jeroen Demeyer Reviewers: John Palmieri, Leif Leonhardy
Report Upstream: Completely fixed; Fix reported upstream Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jdemeyer)

This is an upstream problem with PARI/GP

Upstream report: http://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1099

The "rnfkummer" test in PARI's make test-all fails on the following systems:

  • PPC OS X 10.4
  • Linux PPC 32-bit
  • Solaris SPARC

For example, on a PPC Mac OS X 10.4:

GP/PARI CALCULATOR Version 2.4.3 (development svn-12594)
PowerPC running darwin (PPC/GMP-5.0.1 kernel) 32-bit version
compiled: Sep  8 2010, gcc-4.5.1 (GCC)
(readline v5.0 enabled, extended help enabled)

I get a failure in rnfkummer (both sta and dyn), I am attaching the dif and the output of Configure.

Attachments (5)

Configure.log (2.8 KB) - added by jdemeyer 9 years ago.
output of Configure
rnfkummer-sta.dif (4.0 KB) - added by jdemeyer 9 years ago.
diff from make test-all
9876_doctest.patch (2.2 KB) - added by jdemeyer 9 years ago.
Adds doctest to catch this error in the future
pari.p6.patch (6.7 KB) - added by jdemeyer 9 years ago.
patch for the PARI spkg with upstream fix and some cleanup
pari.p7.patch (5.5 KB) - added by jdemeyer 9 years ago.
Patch for the PARI spkg .p6 to .p7

Download all attachments as: .zip

Change History (39)

Changed 9 years ago by jdemeyer

output of Configure

Changed 9 years ago by jdemeyer

diff from make test-all

comment:1 Changed 9 years ago by jdemeyer

  • Description modified (diff)

comment:2 Changed 9 years ago by jdemeyer

  • Description modified (diff)

comment:3 Changed 9 years ago by jhpalmieri

I get the same error on t2.math.washington.edu (Solaris on sparc).

comment:4 follow-up: Changed 9 years ago by jdemeyer

So what does a Solaris sparc have in common with a PPC OS X which other systems do not have???

comment:5 follow-up: Changed 9 years ago by jhpalmieri

I've put the build directory from t2 at /home/palmieri/t2/pari-2.4.3.svn-12577.p5 in case any of the files there are helpful. (Note that the /home directories are the same on t2 as on sage.math, so you can access this directory from sage.math.)

comment:6 in reply to: ↑ 5 Changed 9 years ago by jdemeyer

Replying to jhpalmieri:

I've put the build directory from t2 at /home/palmieri/t2/pari-2.4.3.svn-12577.p5 in case any of the files there are helpful. (Note that the /home directories are the same on t2 as on sage.math, so you can access this directory from sage.math.)

Please do chmod 755 /home/palmieri/t2/pari-2.4.3.svn-12577.p5 and then I can actually access it.

comment:7 Changed 9 years ago by jdemeyer

Thanks! For reference, the PARI/GP build directory on t2 (a Solaris sparc) can be found on: http://sage.math.washington.edu/home/palmieri/t2/pari-2.4.3.svn-12577.p5/src/

comment:8 in reply to: ↑ 4 ; follow-up: Changed 9 years ago by jdemeyer

Replying to jdemeyer:

So what does a Solaris sparc have in common with a PPC OS X which other systems do not have???

32-bit big endian machines?

comment:9 Changed 9 years ago by jhpalmieri

  • Cc drkirkby added

I also get the same error on the skynet machine mark (a very slow sparc machine running solaris). I didn't the problem with fulvia (x86 running solaris).

comment:10 Changed 9 years ago by jdemeyer

  • Description modified (diff)
  • Report Upstream changed from Reported upstream. Little or no feedback. to Reported upstream. Developers acknowledge bug.
  • Summary changed from Building PARI/GP with SAGE_CHECK=yes fails on rnfkummer on a PPC Mac OS X 10.4 to Building PARI/GP with SAGE_CHECK=yes fails on 32-bit big endian machines

comment:11 in reply to: ↑ 8 ; follow-up: Changed 9 years ago by drkirkby

Replying to jdemeyer:

Replying to jdemeyer:

So what does a Solaris sparc have in common with a PPC OS X which other systems do not have???

32-bit big endian machines?

All modern SPARC systems are 64-bit. The Sun Ultra 1, which was the first 64-bit machine from Sun, was bought out in 1995 - i.e. 15 years ago, well before any Intel or AMD chip was 64-bit.

However, to maintain backward compatibility, and to increase performance, Sun chose to retain the default builds as 32-bit. So are in fact building Sage 32-bit on Solaris, though a 64-bit build is possible.

The fact John said the tests fail on 'mark' but pass on 'fulvia' does indicate it's very likely to be a big-endian issue. I have a machine running HP-UX which uses a PA-RISC processor. That is big-endian. I could try building Pari on there, though I'm not sure if it will work or not. I'd not be surprised if it failed to build on there.

Dave

comment:12 in reply to: ↑ 11 Changed 9 years ago by leif

Replying to drkirkby:

All modern SPARC systems are 64-bit. The Sun Ultra 1, which was the first 64-bit machine from Sun, was bought out in 1995 - i.e. 15 years ago, well before any Intel or AMD chip was 64-bit.

i860? That were the days of SPARC V7... ;-)

comment:13 Changed 9 years ago by jdemeyer

  • Report Upstream changed from Reported upstream. Developers acknowledge bug. to Fixed upstream, but not in a stable release.

Changed 9 years ago by jdemeyer

Adds doctest to catch this error in the future

comment:14 Changed 9 years ago by jdemeyer

  • Authors set to Jeroen Demeyer
  • Description modified (diff)
  • Report Upstream changed from Fixed upstream, but not in a stable release. to Completely fixed; Fix reported upstream
  • Status changed from new to needs_review

Fix (spkg + doctest) tested on

  • 64-bit Intel Linux
  • 32-bit Intel Linux
  • 32-bit PPC OS X

comment:15 Changed 9 years ago by leif

  • Cc mpatel added
  • Priority changed from critical to blocker

Assuming Mitesh is ok with that.

comment:16 Changed 9 years ago by leif

P.S.: Tell Mercurial some patches have been removed...

comment:17 follow-up: Changed 9 years ago by leif

Can someone test the new spkg on MacOS X 10.4 Intel? (The upstream fix only adds -fno-common on PPCs, not Tiger in general, which is afaik wrong.)

comment:18 Changed 9 years ago by leif

Last comment refers to

=== pari-2.4.3.svn-12577.p6 (Jeroen Demeyer, September 10th, 2010) ===
 * Add upstream fix for config/get_dlcflags and remove the patch

which is

  • src/config/get_dlcflags

    $ diff -Nau pari-2.4.3.svn-12577.p5/src/config/get_dlcflags pari-2.4.3.svn-12577.p6/src/config/get_dlcflags 
    old new  
    22if test -n "$__gnuc__"; then
    33  case $osname in
    44    cygwin|mingw) DLCFLAGS=;;
     5    darwin) DLCFLAGS=-fPIC
     6      case $arch in
     7        ppc|ppc64) DLCFLAGS="$DLCFLAGS -fno-common"
     8      esac;;
    59    *) DLCFLAGS=-fPIC;;
    610  esac
    711else #assume native compiler

comment:19 follow-ups: Changed 9 years ago by leif

  • Description modified (diff)

Jeroen, could you also extend the message printed when SAGE_TUNE_PARI=yes to strongly recommend running PARI's test suite after tuning? (We could in principle do that automatically from spkg-install as well, but that's IMHO not a good idea since it then might get run twice, and also doing so wouldn't fit Sage's build/test process.)

As mentioned somewhere else (#9343?), one of PARI's self-tests fails after "successful" tuning on a Pentium 4 Prescott running Ubuntu 9.04 (gcc 4.3.3). I've yet only put a comment on the wiki page; will open a ticket for that later.


I'd prefer prepending ./ to Configure, since this will give an appropriate error message in any case where src/Configure does (for whatever reason) not exist. (Otherwise bash would then attempt to find it in $PATH, which might cause weird behavoir.)


While adding -fno-common on MacOS X 10.5 and 10.6, too, is perhaps a bit odd, it doesn't hurt; otherwise one should look for Darwin 8 (10.4) with uname .... (I currently don't know the exact string; the relevant part should be the same on PPC 10.4.).

comment:20 in reply to: ↑ 19 Changed 9 years ago by leif

Replying to leif:

[...] otherwise one should look for Darwin 8 (10.4) with uname .... (I currently don't know the exact string; the relevant part should be the same on PPC 10.4.).

`uname -r` should start with 8 for MacOS X 10.4 / Tiger / Darwin 8.

comment:21 in reply to: ↑ 17 ; follow-ups: Changed 9 years ago by jdemeyer

Replying to leif:

Can someone test the new spkg on MacOS X 10.4 Intel? (The upstream fix only adds -fno-common on PPCs, not Tiger in general, which is afaik wrong.)

I agree this certainly needs to be tested.

Changed 9 years ago by jdemeyer

patch for the PARI spkg with upstream fix and some cleanup

comment:22 in reply to: ↑ 19 Changed 9 years ago by jdemeyer

Replying to leif:

Jeroen, could you also extend the message printed when SAGE_TUNE_PARI=yes to strongly recommend running PARI's test suite after tuning?


I'd prefer prepending ./ to Configure, since this will give an appropriate error message in any case where src/Configure does (for whatever reason) not exist.

Done. New spkg, same place: http://sage.math.washington.edu/home/jdemeyer/spkg/pari-2.4.3.svn-12577.p6.spkg

comment:23 in reply to: ↑ 21 ; follow-up: Changed 9 years ago by leif

Replying to jdemeyer:

Replying to leif:

Can someone test the new spkg on MacOS X 10.4 Intel? (The upstream fix only adds -fno-common on PPCs, not Tiger in general, which is afaik wrong.)

I agree this certainly needs to be tested.

Well, the incapability of 10.4's linker / loader to handle common symbols in conjunction with dynamic libraries is completely unrelated to the processor architecture; I doubt the one used on Intel processors in 10.4 will differ much from the one on PowerPCs. (PPC may imply Darwin 8, but not the other way around.)

So I'd prefer keeping our patch (which is already well-tested), or modify it to add a distinction on the Darwin release (`uname -r`, see above).

comment:24 Changed 9 years ago by jhpalmieri

The new spkg works (with SAGE_CHECK=yes) on t2.math.

comment:25 in reply to: ↑ 23 ; follow-up: Changed 9 years ago by jdemeyer

Replying to leif:

Well, the incapability of 10.4's linker / loader to handle common symbols in conjunction with dynamic libraries is completely unrelated to the processor architecture; I doubt the one used on Intel processors in 10.4 will differ much from the one on PowerPCs. (PPC may imply Darwin 8, but not the other way around.)

I sent an email to the PARI people about this, let's see what they say.

comment:26 in reply to: ↑ 21 Changed 9 years ago by leif

  • Cc GeorgSWeber added

Replying to jdemeyer:

Replying to leif:

Can someone test the new spkg on MacOS X 10.4 Intel? (The upstream fix only adds -fno-common on PPCs, not Tiger in general, which is afaik wrong.)

I agree this certainly needs to be tested.

Georg, perhaps you could confirm that -fno-common is also needed on MacOS X 10.4 Intel machines? (Or the opposite, but to me that's unlikely.)

It should be sufficient to just try

$ cd .../pari-2.4.3.svn-12577.p6/src
$ ./Configure --graphic=none
$ make gp

(with http://sage.math.washington.edu/home/jdemeyer/spkg/pari-2.4.3.svn-12577.p6.spkg)

comment:27 Changed 9 years ago by leif

The error we previously got on MacOS X 10.4 PPC looked like this:

/usr/bin/gcc  -o "/Users/jdemeyer/pari-2.4.3.svn-12577/src/Odarwin-ppc"/libpari-2.4.dylib -dynamiclib  -O3 -Wall -fno-strict-aliasing -fomit-frame-pointer    -fPIC -Wl,-flat_namespace,-undefined,suppress  mp.o mpinl.o F2x.o FF.o Flx.o FpE.o FpV.o FpX.o Qfb.o RgV.o RgX.o ZV.o ZX.o alglin1.o alglin2.o arith1.o arith2.o base1.o base2.o base3.o base4.o base5.o bb_group.o bibli1.o bibli2.o bit.o buch1.o buch2.o buch3.o buch4.o concat.o ellanal.o elliptic.o galconj.o gen1.o gen2.o gen3.o hnf_snf.o ifactor1.o lll.o perm.o polarit1.o polarit2.o polarit3.o prime.o random.o rootpol.o subcyclo.o subgroup.o trans1.o trans2.o trans3.o anal.o compat.o compile.o default.o errmsg.o es.o eval.o hash.o init.o intnum.o members.o pariinl.o parse.o sumiter.o DedekZeta.o Hensel.o QX_factor.o aprcl.o elldata.o ellsea.o galois.o galpol.o groupid.o krasner.o kummer.o mpqs.o nffactor.o part.o stark.o subfield.o thue.o darwin.o
ld: common symbols not allowed with MH_DYLIB output format with the -multi_module option
init.o definition of common _DEBUGMEM (size 4)
init.o definition of common _avma (size 4)
init.o definition of common _bot (size 4)
[...]
es.o definition of common _pariErr (size 4)
es.o definition of common _pariOut (size 4)
init.o definition of common _pari_errfile (size 4)
init.o definition of common _pari_infile (size 4)
init.o definition of common _pari_outfile (size 4)
init.o definition of common _cb_pari_err_recover (size 4)
init.o definition of common _foreignFuncFree (size 4)
init.o definition of common _cb_pari_handle_exception (size 4)
init.o definition of common _cb_pari_sigint (size 4)
init.o definition of common _cb_pari_whatnow (size 4)
init.o definition of common _foreignExprHandler (size 4)
init.o definition of common _foreignHandler (size 4)
init.o definition of common _memused (size 4)
/usr/bin/libtool: internal link edit command failed
make[1]: *** [libpari-2.4.dylib] Error 1
make: *** [gp] Error 2

comment:28 in reply to: ↑ 25 ; follow-up: Changed 9 years ago by jdemeyer

Replying to jdemeyer:

I sent an email to the PARI people about this, let's see what they say.

I haven't had much luck, see http://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1088:

Bill Allombert wrote:

Jeroen Demeyer wrote:

In that case, I strongly recommend you to apply the patch to ALL darwin architectures, not just ppc/ppc64.

Why ? Is there any documentation that recommend that ? gcc documentation certainly does not.

I do agree with leif though, I will prepare a .p7 with the fix.

comment:29 in reply to: ↑ 28 Changed 9 years ago by jdemeyer

  • Description modified (diff)

Replying to jdemeyer:

I do agree with leif though, I will prepare a .p7 with the fix.

New spkg: http://sage.math.washington.edu/home/jdemeyer/spkg/pari-2.4.3.svn-12577.p7.spkg

Changed 9 years ago by jdemeyer

Patch for the PARI spkg .p6 to .p7

comment:30 Changed 9 years ago by jdemeyer

I think this is ready for review now. The original issue seems to be fixed and the get_dlcflags change has been reverted.

comment:31 follow-up: Changed 9 years ago by leif

  • Reviewers set to John Palmieri, Leif Leonhardy
  • Status changed from needs_review to positive_review

I've again tested the latest p7 (which includes only minor* changes w.r.t. the p6) and the new Sage library patch (additional regression doctest) on Ubuntu 10.04 x86_64 (ptestlong run twice, the second time with #9898 applied and the lcalc p5 spkg from #9845). (Also, the new spkg is "sane".)

Since Jeroen has already successfully tested it on a PPC Mac (big-endian), and John on t2.math (I assume in 32-bit big-endian mode), and the patch to get_dlcflags has been reverted to what we already tested (which definitely works on MacOS X 10.4 Intel, too), I set this to "positive review".


*Although I must admit I haven't yet verified that the newly updated upstream source files claiming to fix PARI bug #1079 achieve the same as the previously applied Sage patches ("segmentation fault in rnfequation(,,1) over Q").

comment:32 in reply to: ↑ 31 Changed 9 years ago by leif

Replying to leif:

I haven't yet verified that the newly updated upstream source files claiming to fix PARI bug #1079 achieve the same as the previously applied Sage patches [...]

As far as I can tell they do. (And do not introduce other changes.)

comment:33 Changed 9 years ago by jhpalmieri

I also tested this again on t2 (32-bit) and also an a 10.6 OS X Intel Mac, just to make sure the new Darwin-only changes didn't mess that up. No problems in either case.

comment:34 Changed 9 years ago by mpatel

  • Merged in set to sage-4.6.alpha2
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.