Opened 10 years ago

Last modified 8 years ago

#5847 closed enhancement

Update GMP-ECM to 6.3 — at Version 106

Reported by: mabshoff Owned by: leif
Priority: major Milestone: sage-4.7.2
Component: packages: standard Keywords: sd32 MPIR elliptic curves libecm ecm spkg
Cc: zimmerma, dimpase Merged in:
Authors: Mike Hansen, Leif Leonhardy, Jeroen Demeyer Reviewers: Leif Leonhardy, Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by leif)

Changes between ecm-6.2.3 and ecm-6.3:
* New assembly code for 64-bit PowerPC (thanks to Philip McLaughlin)
* Allow several processes to write to the same -save file
* More routines in new P+-1 stage 2 use multi-threading in OpenMP build
* Fixed incompatibility with GMP 5.0.0
* Fixed several bugs, and now check return value from malloc() calls
* Fixed linking of GMP which prevented successful builds under Darwin 
  (and presumably other systems)
* Allow use of x86_64 asm code under MinGW

Changes between ecm-6.2.2 and ecm-6.2.3:
* Fixed incompatibility with GMP 4.3.0 when testing version in configure
* SSE2 asm code for Visual C added in stage 2 NTT code
* Small improvement to x86_64 mulredc asm code, slight speedup on Core 2
* Fixed incorrect carry propagation in subquadratic REDC code which
  could lead to incorrect arithmetic in rare cases
* Fixed memory leak with -v parameter when factor was found in ECM stage 1
* Fixed bug which caused only one ECM curve to be run in spite of -c
  parameter if input line did not end in newline
* Assembler mulredc code enabled by default on x86_64

Changes between ecm-6.2.1 and ecm-6.2.2:
* Updated build project files for Visual C by Brian Gladman, also adds
missing NTT_GFP_TWIDDLE_DI[FT]_BREAKOVER defines in VC parameter file
* Fixed uninitialised parameter to P-1 probability computation
* In tune.c : fixed generation of NTT_GFP_TWIDDLE_DI[FT]_BREAKOVER values,
avoid calling cputime() excessively often when timing short functions,
fixed access to uninitialised memory
* Fixed serious split infinitive in configure script (thanks Paul Leyland)
* Removed unnecessary carry propagation in x86_64 mulredc code, slight
speedup (thanks Philip McLaughlin)
* Fixed non-portable PIC code in x86_64/redc.asm
* Fixed problem with pattern matching host type names in configure.in
* Converted binary constants in spv.c and ntt_gfp.c to hexadecimal,
some assembler do not support binary constants

Apply: http://spkg-upload.googlecode.com/files/ecm-6.3.p2.spkg

md5sum: 8246ca74be1ee07b312a84d2a88d9142 ecm-6.3.p2.spkg


Apply this patch to the Sage library. (Can or maybe should be applied after installing the new spkg.)


Testing distribution with new mpir (#8664) and new ecm: http://sage.math.washington.edu/home/jdemeyer/release/sage-4.6.1.alpha2-mpir/sage-4.6.1.alpha2-mpir.tar

Change History (115)

comment:1 Changed 9 years ago by was

  • Report Upstream set to N/A
I think you may need the latest 6.3

 ./configure --with-gmp=/usr/local/
make -j
make -j check
and it passed
without specifing where gmp/mpir is , it got very confused , , they have their
search paths mixed up.

Jason
- Hide quoted text -

On Wednesday 02 June 2010 02:59:21 Bill Hart wrote:
> There's an open ticket by Michael Abshoff to update to 6.2.2: :-)
>
> http://trac.sagemath.org/sage_trac/ticket/5847
>
> Bill.
>
> On 2 June 2010 02:54, Jason Moxham <jason@njkfrudils.plus.com> wrote:
> > I had forgotten about this , gmp-ecm-6.2.1 is 2 years old , what is it
> > doing in sage ? :) , they fixed it in a later release , perhaps about a
> > year ago
> >
> > Jason
> >
> > On Wednesday 02 June 2010 02:46:58 Bill Hart wrote:
> >> On 2 June 2010 02:40, William Stein <wstein@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > Building Sage fails with GMP-ECM, as before.   Yes, I know this is
> >> > because of deprecation, etc...
> >>
> >> Sure. We announced a list of deprecated symbols on sage-devel and
> >> mpir-devel. Then we permanently removed mpz_random and mpz_random2
> >> *only* , the worst offenders.
> >>
> >> Bill.
> >>
> >> > ar cru .libs/libecm.a  ecm.o ecm2.o pm1.o pp1.o getprime.o listz.o
> >> > lucas.o stage2.o toomcook.o mpmod.o mul_l
> >> > o.o polyeval.o median.o schoen_strass.o ks-multiply.o rho.o bestd.o
> >> > auxlib.o random.o factor.o sp.o spv.o sp
> >> > m.o mpzspm.o mpzspv.o ntt_gfp.o ecm_ntt.o pm1fs2.o mul_fft.o
> >> > sets_long.o auxarith.otune-tune.o: In function `tune_mpres_mul':
> >> > tune.c:(.text+0xd1): undefined reference to `mpz_random'
> >> > collect2: ld returned 1 exit status
> >> > make[4]: *** [tune] Error 1
> >> > make[4]: *** Waiting for unfinished jobs....
> >> > ranlib .libs/libecm.a
> >> > creating libecm.la
> >> > (cd .libs && rm -f libecm.la && ln -s ../libecm.la libecm.la)
> >> > make[4]: Leaving directory
> >> > `/mnt/usb1/scratch/wstein/build/mpir2/sage-4.4.3.alpha1/spkg/build/ecm
> >> >-6. 2.1.p2/s rc'
> >> > make[3]: *** [all-recursive] Error 1
> >> > make[3]: Leaving directory
> >> > `/mnt/usb1/scratch/wstein/build/mpir2/sage-4.4.3.alpha1/spkg/build/ecm
> >> >-6. 2.1.p2/src' make[2]: *** [all] Error 2make[2]: Leaving directory
> >> > `/mnt/usb1/scratch/wstein/build/mpir2/sage-4.4.3.alpha1/spkg/build/ecm
> >> >-6. 2.1.p2/s rc'
> >> > There was a problem building GMP ECM.
> >> >
> >> > real    0m9.633s
> >> > user    0m8.510s
> >> > sys     0m8.950s
> >> > sage: An error occurred while installing ecm-6.2.1.p2
> >> > Please email sage-devel http://groups.google.com/group/sage-devel

comment:2 Changed 9 years ago by mhansen

  • Summary changed from Update GMP-ECM to 6.2.2 to Update GMP-ECM to 6.3

comment:3 Changed 9 years ago by mhansen

  • Authors set to Mike Hansen
  • Status changed from new to needs_review

There is a 6.3 spkg at http://sage.math.washington.edu/home/mhansen/ecm-6.3.spkg

I've checked that it works with MPIR 2.1.1 and all tests pass.

comment:4 Changed 9 years ago by leif

Since Sage with MPIR 2.1.1 (#8664) requires updating to this package, I report at that ticket.

comment:5 Changed 9 years ago by leif

Since MPIR 2.1.1 has a bug (see #9837), I've (successfully) built and tested Sage 4.6.prealpha3 (see #9343 and the NewPARI Wiki page) with GMP 5.0.1 and this new ECM 6.3 spkg on Ubuntu 10.04 x86_64 (Core2, gcc 4.4.3; parallel build from scratch with 32 jobs; native code with O3).

ptestlong passed all tests.

comment:6 Changed 9 years ago by leif

It also passed ptestlong on the same machine with Sage 4.5.3.alpha2 and MPIR 2.1.1 (because the MPIR bug apparently only shows up in combination with the new PARI package, which isn't included in that Sage version).

Same for Fedora 13 x86 (Pentium 4 Prescott, gcc 4.4.4, parallel build with 6 jobs, rest dito).

Changed 9 years ago by leif

Suggested changes - NOT (yet) a Mercurial patch. (Minor fixes, some comments added, some clean-up.)

comment:7 Changed 9 years ago by leif

I've added a reviewer patch (ordinary context diff) with some changes:

  • Remove also the manual page of previous installations.
  • Typo: rm -r -> rm -f (header file)
  • Removed setting of CXXFLAGS, since we don't have C++ code.
  • Don't overwrite CFLAGS if SAGE64=yes (instead, append). Removed -O2 -g in that case. Make use of CFLAG64 if set.
  • Quote $SAGE_LOCAL in the parameters to configure, too.
  • Use $MAKE in spkg-check, too.
  • Some messages changed (e.g. all failures now starting with "Error"), some added.
  • A few comments/notes added (SPKG.txt, spkg-install).

If you're ok with the changes, I can replace the diff with a Mercurial patch. Or simply merge them...

comment:8 Changed 9 years ago by leif

  • Authors changed from Mike Hansen to Mike Hansen, Leif Leonhardy
  • Reviewers set to Leif Leonhardy
  • Type changed from defect to enhancement

New spkg: http://spkg-upload.googlecode.com/files/ecm-6.3.p0.spkg

md5sum: b9b1fcd5ebc2e3689fd379c1dba3a372 ecm-6.3.p0.spkg

New spkg based on Mike's with some more changes (than mentioned above).

Should be installed with the MPIR 2.1.3 spkg from #8664. See instructions there.

(Tested with Sage 4.6.1.alpha0 on Ubuntu 9.04 x86 and Ubuntu 10.04 x86_64.)

Changed 9 years ago by leif

SPKG "reviewer" patch, based on Mike's, i.e. ecm-6.3 vs. ecm-6.3.p0. For reference/review.

comment:9 Changed 9 years ago by jdemeyer

Built and tested on sage.math.washington.edu without problems.

comment:10 Changed 9 years ago by leif

  • Owner changed from mabshoff to leif

comment:11 Changed 9 years ago by jdemeyer

This fails to compile on my OS X 10.4 powerpc G5 machine, full log attached but here is the interesting part:

****************************************************
Host system
uname -a:
Darwin moufang.ugent.be 8.11.0 Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power         Macintosh powerpc
****************************************************
****************************************************
CC Version
gcc -v
Using built-in specs.
Target: powerpc-apple-darwin8
Configured with: /private/var/tmp/gcc/gcc-5367.obj~1/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man -- enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --with-    slibdir=/usr/lib --build=powerpc-apple-darwin8 --host=powerpc-apple-darwin8 --target=powerpc-apple-darwin8
Thread model: posix
gcc version 4.0.1 (Apple Computer, Inc. build 5367)
****************************************************
...
checking build system type... powerpc-apple-darwin8.11.0
checking host system type... powerpc-apple-darwin8.11.0
...
configure: Configuration:
configure: Build for host type powerpc-apple-darwin8.11.0
configure: CC=gcc, CFLAGS=-g -O3  -fPIC
configure: Linking GMP with -lgmp
configure: Using asm redc code from directory powerpc64
configure: Not using SSE2 instructions in NTT code
configure: Assertions disabled
configure: Shell command execution disabled
configure: OpenMP disabled
configure: Memory debugging disabled
make  all-recursive
Making all in powerpc64
m4 -I../ -DOPERATION_mulredc1 `test -f mulredc1.asm || echo './'`mulredc1.asm >mulredc1.s
/bin/sh ../libtool   --mode=compile gcc  -g -O3  -fPIC -c -o mulredc1.lo mulredc1.s
libtool: compile:  gcc -g -O3 -fPIC -c mulredc1.s -o mulredc1.o
mulredc1.s:40:mulld instruction is only for 64-bit implementations (not allowed without -force_cpusubtype_ALL option)
mulredc1.s:41:mulhdu instruction is only for 64-bit implementations (not allowed without -force_cpusubtype_ALL option)
mulredc1.s:42:mulld instruction is only for 64-bit implementations (not allowed without -force_cpusubtype_ALL option)
mulredc1.s:43:mulld instruction is only for 64-bit implementations (not allowed without -force_cpusubtype_ALL option)
mulredc1.s:44:mulhdu instruction is only for 64-bit implementations (not allowed without -force_cpusubtype_ALL option)
mulredc1.s:47:std instruction is only for 64-bit implementations (not allowed without -force_cpusubtype_ALL option)
make[4]: *** [mulredc1.lo] Error 1
rm mulredc1.s
make[3]: *** [all-recursive] Error 1
make[2]: *** [all] Error 2
Error building GMP-ECM.

Changed 9 years ago by jdemeyer

Log file for failed build on OS X 10.4 powerpc G5

comment:12 Changed 9 years ago by jdemeyer

  • Description modified (diff)

comment:13 follow-up: Changed 9 years ago by leif

Well, I think these are the relevant parts:

...
checking whether we can link against GMP... yes
checking if gmp.h version and libgmp version are the same... (5.0.1/5.0.1) yes
checking for __gmpn_add_nc... yes
checking for __gmpn_mod_34lsub1... yes
checking for __gmpn_redc_1... no
...
configure: Using asm redc code from directory powerpc64
...

So it's probably an upstream problem, either MPIR or ECM.

Or should we pass -force_cpusubtype_ALL? (To the assembler?) I think rather not.

What happens on other PPCs?

Can you try installing it with GMP 5.0.1?

comment:14 follow-up: Changed 9 years ago by leif

  • Cc zimmerma added

Paul, perhaps you have an idea what's going wrong there (ECM trying to use "64-bit" instructions on MacOS X 10.4 PPC [G5], with MPIR 2.1.3).

comment:15 follow-ups: Changed 9 years ago by leif

Jeroen, can you try configuring with --disable-asm-redc?

(The 64-bit PPC asm code is new in 6.3 btw.)

comment:16 in reply to: ↑ 13 ; follow-up: Changed 9 years ago by fbissey

Replying to leif:

Well, I think these are the relevant parts:

...
checking whether we can link against GMP... yes
checking if gmp.h version and libgmp version are the same... (5.0.1/5.0.1) yes
checking for __gmpn_add_nc... yes
checking for __gmpn_mod_34lsub1... yes
checking for __gmpn_redc_1... no
...
configure: Using asm redc code from directory powerpc64
...

So it's probably an upstream problem, either MPIR or ECM.

Or should we pass -force_cpusubtype_ALL? (To the assembler?) I think rather not.

What happens on other PPCs?

Can you try installing it with GMP 5.0.1?

A remark about installing with GMP 5. On Gentoo I had to apply the following for it to compile

	# fixes for gmp-5
	sed -i "s:__GMP_BITS_PER_MP_LIMB:GMP_LIMB_BITS:g" bestd.c mpmod.c \
		schoen_strass.c sp.h || die "failed to patch files for gmp-5"

This is backward compatible with GMP 4.3. I am a bit surprised that with the stuff that mpir has deprecated, they didn't deprecate that as well.

comment:17 in reply to: ↑ 14 Changed 9 years ago by zimmerma

Replying to leif:

Paul, perhaps you have an idea what's going wrong there (ECM trying to use "64-bit" instructions on MacOS X 10.4 PPC [G5], with MPIR 2.1.3).

is this a 32-bit machine? What does ECM config.guess return? Can you try the svn version of GMP-ECM using svn checkout svn://scm.gforge.inria.fr/svn/ecm?

Paul

comment:18 in reply to: ↑ 15 ; follow-up: Changed 9 years ago by zimmerma

Replying to leif:

Jeroen, can you try configuring with --disable-asm-redc?

(The 64-bit PPC asm code is new in 6.3 btw.)

it should work with --disable-asm-redc.

Paul

comment:19 Changed 9 years ago by jdemeyer

Thanks for all the suggestions, I will try them but probably not today.

comment:20 in reply to: ↑ 16 Changed 9 years ago by leif

Replying to fbissey:

Replying to leif:

Can you try installing it with GMP 5.0.1?

A remark about installing with GMP 5. On Gentoo I had to apply the following for it to compile

	# fixes for gmp-5
	sed -i "s:__GMP_BITS_PER_MP_LIMB:GMP_LIMB_BITS:g" bestd.c mpmod.c \
		schoen_strass.c sp.h || die "failed to patch files for gmp-5"

This is backward compatible with GMP 4.3. I am a bit surprised that with the stuff that mpir has deprecated, they didn't deprecate that as well.

Worked for me without patching with both MPIR 2.1.1 and [vanilla] GMP 5.0.1 on Ubuntu (see above).

comment:21 in reply to: ↑ 18 ; follow-up: Changed 9 years ago by leif

  • Cc dimpase added

Replying to zimmerma:

Replying to leif:

Jeroen, can you try configuring with --disable-asm-redc?

(The 64-bit PPC asm code is new in 6.3 btw.)

it should work with --disable-asm-redc.

Thanks!

Replying to jdemeyer:

Thanks for all the suggestions, I will try them but probably not today.

Should I update the spkg / spkg patch to enable --disable-asm-redc (on MacOS X PPC 10.4 and perhaps 10.5, or simply PPC)?

IIRC we don't support 64-bit builds (SAGE64=yes) on PowerPC (and MacOS X < 10.6) anyway, and Apple doesn't support MacOS X 10.6 on PPCs.

Setting ABI=32 would perhaps also work.

comment:22 Changed 9 years ago by leif

Any objections to enable (also) building a shared library?

(And also enabling the build of a static MPIR library, which is better for ECM?)

comment:23 follow-up: Changed 9 years ago by leif

I wonder how the GNU assembler (Linux PPC) behaves...

François, would you like to test this?

comment:24 follow-up: Changed 9 years ago by zimmerma

in fact this bug is already fixed upstream (in revision 1516), see https://gforge.inria.fr/tracker/index.php?func=detail&aid=10646&group_id=135&atid=623

The patch is the following:

--- configure.in        (revision 1515)
+++ configure.in        (revision 1516)
@@ -195,7 +195,7 @@
 # asm_redc enabled by default for x86_64 and 64 bit PowerPC
 if test "x$enable_asm_redc" = x; then
   case $host in
-    x86_64* | powerpc-apple-darwin* | powerpc64-*-linux*) enable_asm_redc=yes;;
+    x86_64*-*-* | powerpc-apple-darwin* | powerpc64-*-linux*) enable_asm_redc=yes;;
     *) enable_asm_redc=no;;
   esac
 fi
@@ -203,8 +203,18 @@
 if test "x$enable_asm_redc" = xyes; then
   case $host in
     pentium4-*-* | pentium3-*-* | viac7-*-* | i786-*-*) ASMPATH=pentium4;;
-    x86_64-*-*)  ASMPATH=x86_64;; 
-    powerpc-apple-darwin*) ASMPATH=powerpc64;;
+    x86_64*-*-*)  ASMPATH=x86_64;; 
+# warning: with powerpc-apple-darwin* we can have ABI=32
+# see bug #10646 on the bug tracker, where config.guess says
+# powerpc-apple-darwin8.11.0 (this a 64-bit machine, but most applications
+# are compiled in 32 bits). It works with --disable-asm-redc.
+    powerpc-apple-darwin*)
+AC_PREPROC_IFELSE([AC_LANG_PROGRAM([
+#if defined(__ppc__)
+#error
+#endif])], [], [AC_MSG_NOTICE([32-bit PowerPC, disabling asm-redc])
+                enable_asm_redc=no])
+                          ASMPATH=powerpc64;;
     powerpc64-*-linux*)
                          ECM_INCLUDE([<"$srcdir"/powerpc64/powerpc-defs.m4>])
                          ASMPATH=powerpc64;;
@@ -213,7 +223,9 @@
                          ASMPATH=athlon;;
     *) AC_MSG_ERROR([[asm redc not available on this machine $host]]);;
   esac
+fi
 
+if test "x$enable_asm_redc" = xyes; then
 # do the necessary definitions and includes
   AC_DEFINE([NATIVE_REDC],1,[Define to 1 to use asm redc])
   test "x$CCAS" != x || CCAS="$CC -c"

Please can you check it works correctly with this patch?

Paul

comment:25 in reply to: ↑ 21 Changed 9 years ago by dimpase

Replying to leif:

the required MPIR spkg does not build on PPC (see my comment on #8664)

comment:26 in reply to: ↑ 15 ; follow-up: Changed 9 years ago by jdemeyer

  • Status changed from needs_review to needs_work
  • Work issues set to include patch for 32-bit ppc

Replying to leif:

Jeroen, can you try configuring with --disable-asm-redc?

The build (outside of Sage) works.

Replying to zimmerma:

is this a 32-bit machine?

Well, technically the processor is 64-bit capable (I believe) but it runs a 32-bit system and gcc produces 32-bit code by default.

What does ECM config.guess return?

powerpc-apple-darwin8.11.0

Can you try the svn version of

GMP-ECM using svn checkout svn://scm.gforge.inria.fr/svn/ecm?

This works.

Replying to zimmerma:

in fact this bug is already fixed upstream (in revision 1516)

Please can you check it works correctly with this patch?

This works.

Replying to leif:

Setting ABI=32 would perhaps also work.

No, it does not.

comment:27 in reply to: ↑ 26 ; follow-up: Changed 9 years ago by leif

Replying to jdemeyer:

Replying to leif:

Jeroen, can you try configuring with --disable-asm-redc?

The build (outside of Sage) works.

Well, I'm a bit unsure what to do now. I can of course include the patch, but that does not what we actually want (it's just a work-around).

I think passing -Wa,-force_cpusubtype_ALL on MacOS PPCs should also work (which does not disable the assembly code, but avoids the odd assembler error, such that we get better performance).

Note that this worked on Dima's G4 with MPIR, so I expect it to work with a G5, too.

Don't know if G3s run into the same problem, or support these instructions. We then might have to really disable the code on PPCs < G4.

We could of course just test this in spkg-install, i.e. feed some extended instruction set assembly code to the assembler with -force_cpusubtype_ALL and see if we get an error before we disable the redc asm code. (I'd have to look what option gas takes on Linux PPC.)

Objections against adding yet another environment variable (ECM_EXTRA_OPTS, to add to ECM's configure, analoguous to PARI_EXTRA_OPTS), to ease working around potential problems showing up later?

comment:28 Changed 9 years ago by leif

  • Status changed from needs_work to needs_info

From my gas manpage (relevant for Linux PPC only):

...
       Target PowerPC options:
          [-mpwrx|-mpwr2|-mpwr|-m601|-mppc|-mppc32|-m603|-m604|
           -m403|-m405|-mppc64|-m620|-mppc64bridge|-mbooke]
          [-mcom|-many|-maltivec|-mvsx] [-memb]
          [-mregnames|-mno-regnames]
          [-mrelocatable|-mrelocatable-lib]
          [-mlittle|-mlittle-endian|-mbig|-mbig-endian]
          [-msolaris|-mno-solaris]
...

The options for other architectures in addition have more detailed descriptions, but unfortunately not the ones for PPC as the target. But it seems gas enables instruction set extensions by default.


P.S.: That the build works with -force_cpusubtype_ALL (without SAGE_CHECK=yes) doesn't necessarily mean we couldn't get illegal instruction exceptions when actually running the code.

comment:29 in reply to: ↑ 23 ; follow-up: Changed 9 years ago by fbissey

Replying to leif:

I wonder how the GNU assembler (Linux PPC) behaves...

François, would you like to test this?

I know that gmp-ecm-6.3 builds on my linux ppc system but I don't believe I have the assembler enabled. I will try that as soon as I have access to the machine next week. I will also have a look at mpir. Note that my test machine is a G4.

I am a bit surprised that no changes where needed to build it against GMP 5.0.1 I add a bug report specifically on that in Gentoo and I could reproduce it on one of my machine which has GMP 5.0.1.

comment:30 in reply to: ↑ 29 ; follow-up: Changed 9 years ago by leif

Replying to fbissey:

Replying to leif:

I wonder how the GNU assembler (Linux PPC) behaves...

François, would you like to test this?

I know that gmp-ecm-6.3 builds on my linux ppc system but I don't believe I have the assembler enabled. I will try that as soon as I have access to the machine next week. I will also have a look at mpir. Note that my test machine is a G4.

Ok.

I am a bit surprised that no changes where needed to build it against GMP 5.0.1

Hmmm, we have to upgrade ECM because of the upgrade of MPIR / GMP:

Changes between ecm-6.2.3 and ecm-6.3:

  • New assembly code for 64-bit PowerPC (thanks to Philip McLaughlin)
  • Allow several processes to write to the same -save file
  • More routines in new P+-1 stage 2 use multi-threading in OpenMP build
  • Fixed incompatibility with GMP 5.0.0
  • Fixed several bugs, and now check return value from malloc() calls
  • Fixed linking of GMP which prevented successful builds under Darwin (and presumably other systems)
  • Allow use of x86_64 asm code under MinGW


I add a bug report specifically on that in Gentoo and I could reproduce it on one of my machine which has GMP 5.0.1.

comment:31 in reply to: ↑ 30 Changed 9 years ago by fbissey

Replying to leif:

Replying to fbissey:

Replying to leif:

I wonder how the GNU assembler (Linux PPC) behaves...

François, would you like to test this?

I know that gmp-ecm-6.3 builds on my linux ppc system but I don't believe I have the assembler enabled. I will try that as soon as I have access to the machine next week. I will also have a look at mpir. Note that my test machine is a G4.

Ok.

I am a bit surprised that no changes where needed to build it against GMP 5.0.1

Hmmm, we have to upgrade ECM because of the upgrade of MPIR / GMP:

Changes between ecm-6.2.3 and ecm-6.3:

  • New assembly code for 64-bit PowerPC (thanks to Philip McLaughlin)
  • Allow several processes to write to the same -save file
  • More routines in new P+-1 stage 2 use multi-threading in OpenMP build
  • Fixed incompatibility with GMP 5.0.0
  • Fixed several bugs, and now check return value from malloc() calls
  • Fixed linking of GMP which prevented successful builds under Darwin (and presumably other systems)
  • Allow use of x86_64 asm code under MinGW

I remember adding 6.3 to the sage-on-gentoo tree for GMP-5 {{{# ChangeLog? for sci-mathematics/ecm # Copyright 1999-2010 Gentoo Foundation; Distributed under the GPL v2 # $Header: $

08 Sep 2010; François Bissey <f.r.bissey@…> metadata.xml: fix metadata

03 Aug 2010; Christopher Schwan <cschwan@…> -ecm-6.2.3.ebuild, -ecm-6.3.ebuild: Removed old versions

*ecm-6.3-r1 (23 Jul 2010)

23 Jul 2010; Christopher Schwan <cschwan@…> +ecm-6.3-r1.ebuild: Migrated to autotools-utils.eclass

08 Jul 2010; François Bissey <f.r.bissey@…> ecm-6.3.ebuild: Hopefully fixed for gmp-5

*ecm-6.3 (07 Jul 2010)

07 Jul 2010; François Bissey <f.r.bissey@…> +ecm-6.3.ebuild: Version bump. Hopefully helpfull with gmp-5.

}}} I'll have to check my inbox for the details. I imported it on the 7th of July because it said it was compatible with GMP-5 and one of our user was using that. The user then reported that he still couldn't get ECM to build and I introduced the fix above the next day which solved the problem. A possibility is interference with by a patch to GMP Gentoo side.

On a positive note ECM 6.3 is the default in sage-on-gentoo since the 3rd of August so it has been extensively tested on x86, amd64 and ppc. However assembler code is at the user's discretion.

comment:32 follow-up: Changed 9 years ago by dimpase

  • Status changed from needs_info to needs_work

on MacOSX 10.5 PPC (G4) I get

checking what assembly label suffix to use... :
checking if globals are prefixed by underscore... configure: error: Test program links neither with nor without underscore.
Error configuring GMP-ECM.

$ gcc -v

Using built-in specs. Target: powerpc-apple-darwin9 Configured with: /var/tmp/gcc_42/gcc_42-5577~1/src/configure --disable-checking --prefix=/usr --mandir=/usr/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/[cg][.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin9 --with-gxx-include-dir=/usr/include/c++/4.0.0 --program-prefix= --host=powerpc-apple-darwin9 --target=powerpc-apple-darwin9 Thread model: posix gcc version 4.2.1 (Apple Inc. build 5577)

comment:33 in reply to: ↑ 32 ; follow-up: Changed 9 years ago by leif

Replying to dimpase:

on MacOSX 10.5 PPC (G4) I get

checking what assembly label suffix to use... :
checking if globals are prefixed by underscore... configure: error: Test program links neither with nor without underscore.
Error configuring GMP-ECM.

Can you attach the config.log?

(Maybe weird CFLAGS etc. settings in your environment?)

comment:34 Changed 9 years ago by leif

  • Status changed from needs_work to needs_info

Changed 9 years ago by dimpase

macosx 10.5 ppc G4 failure of ./configure

comment:35 in reply to: ↑ 33 ; follow-ups: Changed 9 years ago by dimpase

Replying to leif:

Replying to dimpase:

on MacOSX 10.5 PPC (G4) I get

 checking what assembly label suffix to use... :
 checking if globals are prefixed by underscore... configure: error: Test program links neither with nor without underscore.
 Error configuring GMP-ECM.

Can you attach the config.log?

done

(Maybe weird CFLAGS etc. settings in your environment?)

no, it's rather prosaic

$ gcc -g conftes1.o conftes2.o 
collect2: ld terminated with signal 10 [Bus error]
ld warning: can't find atom for stabs FUN at 00000000 in conftes2.o

comment:36 in reply to: ↑ 35 ; follow-up: Changed 9 years ago by dimpase

Replying to dimpase:

$ gcc -g conftes1.o conftes2.o 
collect2: ld terminated with signal 10 [Bus error]
ld warning: can't find atom for stabs FUN at 00000000 in conftes2.o

more details - it looks like "-g" flag does not work here:

$ cat conftes1.c
#ifdef __cplusplus
extern "C" { void underscore_test(); }
#endif
main () { underscore_test(); }
$
$ cat conftes2.s
      	.text
	.globl _underscore_test
_underscore_test:
$
$ gcc -g -o x conftes1.c conftes2.s
collect2: ld terminated with signal 10 [Bus error]
ld warning: can't find atom for stabs FUN at 00000000 in /var/folders/wg/wghOV5z8H7Ox9E0VSGnzm++++TM/-Tmp-//cc5M42pV.o
$ gcc -o x conftes1.c conftes2.s
$ ./x
$ 

could it be due to "empty" conftes2.s ?

comment:37 in reply to: ↑ 36 ; follow-up: Changed 9 years ago by dimpase

  • Status changed from needs_info to needs_work

Replying to dimpase:

more details - it looks like "-g" flag does not work here:

 
 $ cat conftes1.c
 #ifdef __cplusplus
 extern "C" { void underscore_test(); }
 #endif
 main () { underscore_test(); }
 $
 $ cat conftes2.s
       	.text
 	.globl _underscore_test
 _underscore_test:
 $
 $ gcc -g -o x conftes1.c conftes2.s
 collect2: ld terminated with signal 10 [Bus error]
ld warning: can't find atom for stabs FUN at 00000000 in /var/folders/wg/wghOV5z8H7Ox9E0VSGnzm++++TM/-Tmp-//cc5M42pV.o
$ gcc -o x conftes1.c conftes2.s
$ ./x
$ 

could it be due to "empty" conftes2.s ?

Indeed:

$ cat c2.s
.text
.globl _underscore_test
_underscore_test:
	mflr r0
$ gcc -g -o x conftes1.c c2.s
$ ./x
$ gcc -g -O2 -o x conftes1.c c2.s
$ ./x
$ gcc -o x conftes1.c c2.s
$ ./x
$  

So, one (not me :-( )needs to explain this to the autoconf, I suppose

comment:38 in reply to: ↑ 37 Changed 9 years ago by dimpase

Replying to dimpase:

So, one (not me :-( )needs to explain this to the autoconf, I suppose

I went ahead and tweaked configure with the extra asm command in conftes2.s above. It passes, but then I get it misconfigured to ppc64, and then make does not work, as it wants to build ppc64-specific asm code. The following does it better:

$ ./configure --disable-asm-redc

with this, both make and make check work OK. This is all so far out of Sage tree. I'll attach config.log, just in case.

Changed 9 years ago by dimpase

macosx 10.5 ppc G4 with tweaked ./configure --disable-asm-redc (adding extra command in conftes2.s)

comment:39 in reply to: ↑ 35 Changed 9 years ago by jdemeyer

Replying to dimpase:

no, it's rather prosaic

$ gcc -g conftes1.o conftes2.o 
collect2: ld terminated with signal 10 [Bus error]
ld warning: can't find atom for stabs FUN at 00000000 in conftes2.o

Please do $ /usr/libexec/gcc/powerpc-apple-darwin9/4.2.1/ld --version $ which ld $ ld --version

comment:40 follow-up: Changed 9 years ago by jdemeyer

Again, with proper formatting:

$ /usr/libexec/gcc/powerpc-apple-darwin9/4.2.1/ld --version
$ which ld
$ ld --version 

comment:41 follow-up: Changed 9 years ago by zimmerma

[I write here as a developer of GMP-ECM]

the configure error reported in comment 32 seems to be a ld problem. User "dimpase", do you manage to configure GMP successfully?

Also, which version of GMP-ECM did you use in comment 32? The svn version (see comment 24) should fix the fact that 64-bit assembly is used on 32-bit machines. If not, please report the problem upstream on http://ecm.gforge.inria.fr/.

Paul Zimmermann

comment:42 in reply to: ↑ 40 Changed 9 years ago by dimpase

Replying to jdemeyer:

$ /usr/libexec/gcc/powerpc-apple-darwin9/4.2.1/ld -v
@(#)PROGRAM:ld  PROJECT:ld64-85.2.1
$ ld -v
@(#)PROGRAM:ld  PROJECT:ld64-85.2.1
$ which ld
/usr/bin/ld

I think it's the latest ld available under Xcode for ppc. (Same for gcc-4.0.1)

comment:43 in reply to: ↑ 41 Changed 9 years ago by dimpase

Replying to zimmerma:

[I write here as a developer of GMP-ECM]

the configure error reported in comment 32 seems to be a ld problem. User "dimpase", do you manage to configure GMP successfully?

Yes, I do. MPIR, to be more precise. The version on #8664.

Also, which version of GMP-ECM did you use in comment 32?

the one packed in the spkg on this ticket.

Subsequent experiments were made on the same version (without whatever patches in the spkg, however).

I then packaged the configure fix and the extra config flag into Sage spkg and am building Sage using it and the MPIR spkg on #8664. (it's not done yet, this machine is slooooow...)

The svn version (see comment 24) should fix the fact that 64-bit assembly is used on 32-bit machines. If not, please report the problem upstream on http://ecm.gforge.inria.fr/.

do you want me to test the svn version on this machine?

Paul Zimmermann

Dima Pasechnik

comment:44 follow-up: Changed 9 years ago by zimmerma

do you want me to test the svn version on this machine?

yes, as upstream developer I cannot help unless you report a problem with the (vanilla) upstream version.

Of course you are also free to solve the problem within Sage, but if it is not due to the Sage packaging, it would be useful to report it upstream.

Paul Zimmermann

comment:45 in reply to: ↑ 44 ; follow-ups: Changed 9 years ago by dimpase

Replying to zimmerma:

do you want me to test the svn version on this machine?

yes, as upstream developer I cannot help unless you report a problem with the (vanilla) upstream version.

OK, I got the svn version, cd to trunk, and am stuck:

$ autoconf
configure.in:8: error: possibly undefined macro: AM_INIT_AUTOMAKE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.in:81: error: possibly undefined macro: AM_CONDITIONAL
configure.in:155: error: possibly undefined macro: AM_PROG_AS
configure.in:156: error: possibly undefined macro: AM_PROG_CC_C_O
configure.in:185: error: possibly undefined macro: AC_OPENMP

Am I doing something wrong, or my autoconf is too old?

Dima

Of course you are also free to solve the problem within Sage, but if it is not due to the Sage packaging, it would be useful to report it upstream.

Paul Zimmermann

comment:46 in reply to: ↑ 45 ; follow-up: Changed 9 years ago by jdemeyer

Replying to dimpase:

$ autoconf
configure.in:8: error: possibly undefined macro: AM_INIT_AUTOMAKE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.in:81: error: possibly undefined macro: AM_CONDITIONAL
configure.in:155: error: possibly undefined macro: AM_PROG_AS
configure.in:156: error: possibly undefined macro: AM_PROG_CC_C_O
configure.in:185: error: possibly undefined macro: AC_OPENMP

Try the following instead:

$ autoreconf -i

If this doesn't work, please post the output of

$ autoconf --version
$ automake --version

comment:47 follow-up: Changed 9 years ago by zimmerma

you can also try the snapshot from http://www.loria.fr/~zimmerma/ecm-6.3.1.tar.gz (note this is NOT a release).

Paul Zimmermann

comment:48 in reply to: ↑ 45 Changed 9 years ago by dimpase

Replying to dimpase:

Replying to zimmerma:

do you want me to test the svn version on this machine?

yes, as upstream developer I cannot help unless you report a problem with the (vanilla) upstream version.

OK, I got the svn version, cd to trunk, and am stuck:

oops, please ignore this; I should have read README.dev...

Dima

comment:49 in reply to: ↑ 47 ; follow-up: Changed 9 years ago by dimpase

Replying to zimmerma:

you can also try the snapshot from http://www.loria.fr/~zimmerma/ecm-6.3.1.tar.gz

this one builds (with GMP provided by MPIR) and passes all tests on this computer (macosx 10.5 ppc G4), without any specific configure flags...

(note this is NOT a release).

Paul Zimmermann

Dima

comment:50 in reply to: ↑ 46 Changed 9 years ago by dimpase

Replying to jdemeyer:

Try the following instead:

with current autotools from gnu.org:

$ autoreconf -i
configure.in:106: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:106: the top level
configure.in:111: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:111: the top level
configure.in:139: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:139: the top level
configure.in:106: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:106: the top level
configure.in:111: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:111: the top level
configure.in:139: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:139: the top level
configure.in:106: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:106: the top level
configure.in:111: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:111: the top level
configure.in:139: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:139: the top level
configure.in:106: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:106: the top level
configure.in:111: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:111: the top level
configure.in:139: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:139: the top level
configure.in:106: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:106: the top level
configure.in:111: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:111: the top level
configure.in:139: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
configure.in:139: the top level
configure.in:156: installing `./compile'
configure.in:10: installing `./config.guess'
configure.in:10: installing `./config.sub'
configure.in:8: installing `./install-sh'
configure.in:8: installing `./missing'
athlon/Makefile.am:9: Libtool library used but `LIBTOOL' is undefined
athlon/Makefile.am:9:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
athlon/Makefile.am:9:   to `configure.in' and run `aclocal' and `autoconf' again.
athlon/Makefile.am:9:   If `AC_PROG_LIBTOOL' is in `configure.in', make sure
athlon/Makefile.am:9:   its definition is in aclocal's search path.
pentium4/Makefile.am:9: Libtool library used but `LIBTOOL' is undefined
pentium4/Makefile.am:9:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
pentium4/Makefile.am:9:   to `configure.in' and run `aclocal' and `autoconf' again.
pentium4/Makefile.am:9:   If `AC_PROG_LIBTOOL' is in `configure.in', make sure
pentium4/Makefile.am:9:   its definition is in aclocal's search path.
powerpc64/Makefile.am:10: Libtool library used but `LIBTOOL' is undefined
powerpc64/Makefile.am:10:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
powerpc64/Makefile.am:10:   to `configure.in' and run `aclocal' and `autoconf' again.
powerpc64/Makefile.am:10:   If `AC_PROG_LIBTOOL' is in `configure.in', make sure
powerpc64/Makefile.am:10:   its definition is in aclocal's search path.
x86_64/Makefile.am:12: Libtool library used but `LIBTOOL' is undefined
x86_64/Makefile.am:12:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
x86_64/Makefile.am:12:   to `configure.in' and run `aclocal' and `autoconf' again.
x86_64/Makefile.am:12:   If `AC_PROG_LIBTOOL' is in `configure.in', make sure
x86_64/Makefile.am:12:   its definition is in aclocal's search path.
Makefile.am:8: Libtool library used but `LIBTOOL' is undefined
Makefile.am:8:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
Makefile.am:8:   to `configure.in' and run `aclocal' and `autoconf' again.
Makefile.am:8:   If `AC_PROG_LIBTOOL' is in `configure.in', make sure
Makefile.am:8:   its definition is in aclocal's search path.
Makefile.am: installing `./depcomp'
autoreconf: automake failed with exit status: 1
$ 

If this doesn't work, please post the output of

$ autoconf --version
autoconf (GNU Autoconf) 2.68
[...]
Written by David J. MacKenzie and Akim Demaille.
$ automake --version
automake (GNU automake) 1.11
[...]

As well:

$ aclocal 
$ automake -c -a
$ autoconf
$ ./configure
[...]
checking whether gcc and cc understand -c and -o together... yes
./configure: line 4661: syntax error near unexpected token `2.2.6'
./configure: line 4661: `LT_PREREQ(2.2.6)'

I see exactly the same error as the latter on Debian squeeze x86_64.

comment:51 in reply to: ↑ 49 Changed 9 years ago by zimmerma

Replying to dimpase:

Replying to zimmerma:

you can also try the snapshot from http://www.loria.fr/~zimmerma/ecm-6.3.1.tar.gz

this one builds (with GMP provided by MPIR) and passes all tests on this computer (macosx 10.5 ppc G4), without any specific configure flags...

this confirms the problem is fixed upstream, and the patch at comment 24 should be enough.

Paul

comment:52 in reply to: ↑ 27 Changed 9 years ago by jdemeyer

Replying to leif:

Replying to jdemeyer:

Replying to leif:

Jeroen, can you try configuring with --disable-asm-redc?

The build (outside of Sage) works.

Well, I'm a bit unsure what to do now. I can of course include the patch, but that does not what we actually want (it's just a work-around).

Well, since the patch comes from upstream and fixes the problem is all known cases, I think it makes sense to make a new ecm spkg with this patch included. Leif, what do you think (and could you make the spkg? YES/NO)

comment:53 in reply to: ↑ 24 Changed 9 years ago by dimpase

Replying to zimmerma:

in fact this bug is already fixed upstream (in revision 1516), see https://gforge.inria.fr/tracker/index.php?func=detail&aid=10646&group_id=135&atid=623

The patch is the following: [...]

Please can you check it works correctly with this patch?

I wish I could. The patch is against configure.in. But I cannot make autotools work on the package, neither on any local machine, nor on boxen (where autoreconf is version 2.61):

dima@boxen:/tmp/ecm-6.3.p0/src$ autoreconf -i
Remember to add `AC_PROG_LIBTOOL' to `configure.in'.
libtoolize: `config.guess' exists: use `--force' to overwrite
libtoolize: `config.sub' exists: use `--force' to overwrite
libtoolize: `ltmain.sh' exists: use `--force' to overwrite
configure.in:185: error: possibly undefined macro: AC_OPENMP
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1
dima@boxen:/tmp/ecm-6.3.p0/src$ 

sorry...

Dima

Paul

comment:54 Changed 9 years ago by jdemeyer

  • Authors changed from Mike Hansen, Leif Leonhardy to Mike Hansen, Leif Leonhardy, Jeroen Demeyer
  • Description modified (diff)
  • Status changed from needs_work to needs_review
  • Work issues include patch for 32-bit ppc deleted

New spkg with upstream patch from comment:24: http://sage.math.washington.edu/home/jdemeyer/spkg/ecm-6.3.p1.spkg

I still have to test it properly though.

Changed 9 years ago by jdemeyer

patch from p0 to p1, for review

comment:55 Changed 9 years ago by jdemeyer

  • Merged in set to sage-4.6.1.alpha1

comment:56 follow-up: Changed 9 years ago by jdemeyer

Tested new spkg succesfully on the previously-failing OS X 10.4 PPC machine.

comment:57 in reply to: ↑ 56 Changed 9 years ago by dimpase

  • Reviewers changed from Leif Leonhardy to Leif Leonhardy, Dima Pasechnik
  • Status changed from needs_review to positive_review

Replying to jdemeyer:

Tested new spkg succesfully on the previously-failing OS X 10.4 PPC machine.

works on MacOSX 10.5 PPC. Positive review!

comment:58 Changed 9 years ago by jdemeyer

  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:59 Changed 9 years ago by jdemeyer

There is some serious trouble with this spkg: #10252

comment:60 Changed 9 years ago by jdemeyer

  • Merged in sage-4.6.1.alpha1 deleted
  • Resolution fixed deleted
  • Status changed from closed to new

comment:61 Changed 9 years ago by jdemeyer

  • Status changed from new to needs_work

comment:62 Changed 9 years ago by leif

Just for the record:

$ hg status 
M .hgignore
M SPKG.txt
M spkg-install
A patches/configure.patch

I would still prefer to use the extended instruction set on capable PowerPCs (cf. this comment above), regardless if the operating system's ABI is 32 bit or not. (And the configure file in patches/ is huge, as usual, though not under revision control.)

comment:63 Changed 9 years ago by leif

  • Description modified (diff)
  • Status changed from needs_work to needs_review

New spkg: http://spkg-upload.googlecode.com/files/ecm-6.3.p2.spkg

md5sum: 85eabecaa8974a270d5587ef8de06da1 ecm-6.3.p2.spkg


ecm-6.3.p2 (Leif Leonhardy, November 23rd, 2010)

  • Apply another patch from upstream to 'configure.in' to fix compilation on 32-bit x86 processors supporting SSE2. (#10252) (There's only a single, cumulative patch file since both patches are to 'configure.in'.)
  • Work around linker bug on MacOS X 10.5 PPC (see Special Update/Build? Instructions above, and #5847 comment 35 ff.).
  • Allow passing extra arguments to 'configure' through ECM_EXTRA_OPTS.
  • Add "-march=native" to CFLAGS on platforms that support it if CFLAGS do not already contain similar.
  • Print settings of CC, CFLAGS etc. and how we configure.
  • Print settings of CC and CFLAGS found in 'SAGE_LOCAL/include/gmp.h' and eventually a system-wide 'gmp.h'; these aren't yet used though.
  • Further clean-up.

Special Update/Build? Instructions

  • src/ contains "stable" upstream code, to which we currently apply two upstream patches (both to 'configure.in'). These (i.e. the files 'patches/configure.in' and 'patches/configure') should be removed on the next upgrade to a stable release.
  • GMP-ECM comes with a self-tuning feature; we could support that as an option ($SAGE_TUNE_*=yes) in the future.
  • We currently work around a linker bug on MacOS X 10.5 PPC (with GCC 4.2.1) which breaks 'configure' if debug symbols are enabled. This *might* get fixed in later upstream releases.
  • ECM currently does not use the CC and CFLAGS settings from 'gmp.h' since we pass (other) options in CFLAGS (though MPIR currently doesn't use its own CFLAGS for the same reason); we could fix that somehow s.t. "optimized" code generation options ('-mcpu=...', '-mtune=...') are used by gcc. Of course a user can also manually enable them by setting the "global" CFLAGS to e.g. "-march=native" on x86 systems. Note that this doesn't affect the packages' selection of processor- specific optimized [assembly] code. 'spkg-install' already reads those settings now, but doesn't yet use them. If SAGE_FAT_BINARY="yes", we should avoid too specific settings of "-mcpu=...", and perhaps pass a more generic "--host=..." to 'configure'.

Please build, test and report!

(As you know, it still requires the new MPIR spkg from #8664. Note that I didn't have the same versions of autotools, so the patched configure looks quite different and might give warnings, but works, at least for me.)

comment:64 Changed 9 years ago by leif

FWIW, you can also test this package with the old MPIR (1.2.2), or some older GMP. Should work as well.

comment:65 follow-up: Changed 9 years ago by leif

Ooops, I just noticed I've cleaned up a little bit too much (and did not enable building also a shared library by default).

As is, the Sage library won't build with the new package, unless you either install it with

  • CFLAGS containing (also) -fPIC, or
  • ECM_EXTRA_OPTS=--enable-shared.

(Both together also works. So yet another reason we have to pass our custom CFLAGS.)

I'll update the spkg later, after some feedback I think.

comment:66 in reply to: ↑ 65 Changed 9 years ago by leif

Replying to leif:

As is, the Sage library won't build with the new package, unless you either install it with

  • CFLAGS containing (also) -fPIC, or
  • ECM_EXTRA_OPTS=--enable-shared.

Another option is to pass ECM_EXTRA_OPTS=--with-pic, not sure if this works on all platforms.

comment:67 Changed 9 years ago by leif

Just for the record: I'll also update the MPIR spkg s.t. we can get "tuned" CFLAGS from it (essentially some potentially better / more specific -march=... for non-x86 systems).

The current GMP-ECM spkg uses -march=native on x86 / x86_64 systems.

comment:68 follow-up: Changed 9 years ago by zimmerma

just a comment: the ticket title mentions ECM 6.3, but the changes in the description refer to changes between 6.2.1 and 6.2.2.

Paul

comment:69 Changed 9 years ago by jdemeyer

Leif: why not let ECM use MPIR's CFLAGS? It seems to me like you're making this package more complicated that it should be.

comment:70 Changed 9 years ago by jdemeyer

  • Status changed from needs_review to needs_work

Sorry, but this will totally break C compilers which do not support -march=native:

    case "`uname -m`" in
        i[3456]86|i86pc|x86_64|amd64)
            # Only add it if CFLAGS do not already contain similar:
            if ! (echo "$CFLAGS" | egrep -- '-march=|-mcpu=|-mtune=' >/dev/null);
            then
                CFLAGS="-march=native $CFLAGS"
            fi;;
    esac

comment:71 follow-up: Changed 9 years ago by jdemeyer

On sage.math.washington.edu, I get a failure while installing the Sage library (sage-4.6.1.alpha2 + #8664 + #5847):

building 'sage.ext.interpreters.wrapper_el' extension
gcc -pthread -shared build/temp.linux-x86_64-2.6/sage/libs/libecm.o -L/mnt/usb1/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local//lib
-L/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local/lib -lcsage -lecm -lgmp -lstdc++ -lntl -lpython2.6 -o build/lib.linux-x86_64-2.6/sage/libs/libecm.so
/usr/bin/ld: /mnt/usb1/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local//lib/libecm.a(libecm_la-factor.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/mnt/usb1/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local//lib/libecm.a: could not read symbols: Bad value
collect2: ld returned 1 exit status

comment:72 Changed 9 years ago by jdemeyer

  • Description modified (diff)

comment:73 follow-up: Changed 9 years ago by leif

Replying to jdemeyer:

Sorry, but this will totally break C compilers which do not support -march=native:

    case "`uname -m`" in
        i[3456]86|i86pc|x86_64|amd64)
            # Only add it if CFLAGS do not already contain similar:
            if ! (echo "$CFLAGS" | egrep -- '-march=|-mcpu=|-mtune=' >/dev/null);
            then
                CFLAGS="-march=native $CFLAGS"
            fi;;
    esac

Well, cite properly:

...
if [ "$SAGE_FAT_BINARY" = yes ]; then
    echo "Warning: SAGE_FAT_BINARY is currently not really supported by this package."
    # XXX Disable SSE2 on x86? (by passing '--enable-sse2=no' to 'configure')
    # XXX Disable asm-redc? Or pass some "generic" '--host=...' to 'configure'?
else
    # XXX We don't yet test if CC is really gcc here.
    # gcc's "-march=native" only works on some platforms:
    case "`uname -m`" in
        i[3456]86|i86pc|x86_64|amd64)
            # Only add it if CFLAGS do not already contain similar:
            if ! (echo "$CFLAGS" | egrep -- '-march=|-mcpu=|-mtune=' >/dev/null);
            then
                CFLAGS="-march=native $CFLAGS"
            fi;;
    esac
fi
...

(There are also notes on this in both the changelog and the Special Update/Build? Instructions, see above.)

What C compilers does Sage currently support (and will Sage support in the near future)?

Cf. this comment:

There's very little support for other compilers in Sage, and it's easy to add a distinction when the day it gets necessary comes, though I could add it now.

Btw, if a user decides or has to set CC for some reason (which might even be just to use a different gcc), GMP-ECM won't use MPIR's / GMP's CFLAGS either. And MPIR isn't omniscient...

comment:74 in reply to: ↑ 71 ; follow-up: Changed 9 years ago by leif

Replying to jdemeyer:

On sage.math.washington.edu, I get a failure while installing the Sage library (sage-4.6.1.alpha2 + #8664 + #5847):

building 'sage.ext.interpreters.wrapper_el' extension
gcc -pthread -shared build/temp.linux-x86_64-2.6/sage/libs/libecm.o -L/mnt/usb1/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local//lib -L/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local/lib -lcsage -lecm -lgmp -lstdc++ -lntl -lpython2.6 -o build/lib.linux-x86_64-2.6/sage/libs/libecm.so
/usr/bin/ld: /mnt/usb1/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/loca//lib/libecm.a(libecm_la-factor.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/mnt/usb1/scratch/jdemeyer/merger/sage-4.6.1.alpha2-mpir/local//lib/libecm.a: could not read symbols: Bad value
collect2: ld returned 1 exit status

Do you read the comments?

comment:75 follow-up: Changed 9 years ago by leif

P.S.: As I said, I first want to get some test results (e.g. on PowerPCs etc.) before I upload further changes.

comment:76 in reply to: ↑ 73 ; follow-up: Changed 9 years ago by jdemeyer

Replying to leif:

What C compilers does Sage currently support (and will Sage support in the near future)?

Okay, I was referring to older versions of gcc which do not support -march=native (it is a relatively new command line option, so we cannot assume that it will work).

Anyway, the proper way to do this is to actually run gcc -march=native on some stupid .c file and check whether it compiles (there is no need to actually run the compiled program, so we can compile to /dev/null).

comment:77 in reply to: ↑ 74 Changed 9 years ago by jdemeyer

Replying to leif:

Do you read the comments?

Clearly, I did not...

comment:78 in reply to: ↑ 76 ; follow-ups: Changed 9 years ago by leif

Replying to jdemeyer:

Replying to leif:

What C compilers does Sage currently support (and will Sage support in the near future)?

Okay, I was referring to older versions of gcc which do not support -march=native (it is a relatively new command line option, so we cannot assume that it will work).

At least since four years ago, don't know the exact version...

Anyway, the proper way to do this is to actually run gcc -march=native on some stupid .c file and check whether it compiles (there is no need to actually run the compiled program, so we can compile to /dev/null).

gcc -v -march=native should be sufficient, since the preprocessor also needs to know about it.

comment:79 in reply to: ↑ 78 ; follow-up: Changed 9 years ago by leif

Replying to leif:

Replying to jdemeyer:

Replying to leif:

What C compilers does Sage currently support (and will Sage support in the near future)?

Okay, I was referring to older versions of gcc which do not support -march=native (it is a relatively new command line option, so we cannot assume that it will work).

At least since four years ago, don't know the exact version...

GCC 4.0.1 might not support it (could you test that on an x86 machine?); GCC 4.2.1 certainly does.

comment:80 in reply to: ↑ 79 Changed 9 years ago by leif

Replying to leif:

Replying to leif:

Replying to jdemeyer:

Replying to leif:

What C compilers does Sage currently support (and will Sage support in the near future)?

Okay, I was referring to older versions of gcc which do not support -march=native (it is a relatively new command line option, so we cannot assume that it will work).

At least since four years ago, don't know the exact version...

GCC 4.0.1 might not support it (could you test that on an x86 machine?); GCC 4.2.1 certainly does.

... and 4.1.2 also does.

comment:81 in reply to: ↑ 78 ; follow-up: Changed 9 years ago by jdemeyer

Replying to leif:

gcc -v -march=native should be sufficient, since the preprocessor also needs to know about it.

No, that's not sufficient to test it, it does not run the preprocessor.

If you want to run to preprocessor, the following works:

gcc -march=native -E -x c /dev/null -o /dev/null

Using this test, -march=native works for me on gcc 4.2.3, NOT on gcc 4.0.3

comment:82 in reply to: ↑ 81 ; follow-up: Changed 9 years ago by leif

Replying to jdemeyer:

Replying to leif:

gcc -v -march=native should be sufficient, since the preprocessor also needs to know about it.

No, that's not sufficient to test it, it does not run the preprocessor.

If you want to run to preprocessor, the following works:

gcc -march=native -E -x c /dev/null -o /dev/null

gcc -march=... -dM -E -x c /dev/null e.g. outputs the definitions of predefined macros like __i386__, __core2__, __SSE2__ etc.


Using this test, -march=native works for me on gcc 4.2.3, NOT on gcc 4.0.3

Thanks, I use:

    if touch foo.c && $CC -march=native -c foo.c &>/dev/null; then
       ...
    fi
    rm -f foo.*

Funny: GCC 4.3.3 and 4.4.3 don't build GCC 4.0.1 on Ubuntu 9.04 or 10.04... ;-)

comment:83 in reply to: ↑ 82 Changed 9 years ago by jdemeyer

Replying to leif:

Thanks, I use:

    if touch foo.c && $CC -march=native -c foo.c &>/dev/null; then
       ...
    fi
    rm -f foo.*

That's the best way. I would add -o /dev/null so you don't need rm -f foo.* but that's a minor point.

comment:84 Changed 9 years ago by jdemeyer

I'm still thinking that these changes should go to the mpir package and that ecm should use the default CFLAGS provided by mpir.

comment:85 in reply to: ↑ 68 Changed 9 years ago by leif

  • Description modified (diff)

Replying to zimmerma:

just a comment: the ticket title mentions ECM 6.3, but the changes in the description refer to changes between 6.2.1 and 6.2.2.

That's a historical relict; I've now also added the other changes between Sage's current version and 6.3.

comment:86 in reply to: ↑ 75 ; follow-up: Changed 9 years ago by leif

  • Description modified (diff)
  • Keywords MPIR elliptic curves libecm added
  • Status changed from needs_work to needs_review

Replying to leif:

P.S.: As I said, I first want to get some test results (e.g. on PowerPCs etc.) before I upload further changes.

Although (or because) there's not much feedback yet, I've uploaded a corrected spkg (still p2) with also some other changes. This obsoletes setting ECM_EXTRA_OPTS=--with-pic or -fPIC in CFLAGS etc., works with all GCC 4.0.x on x86 as well and also uses processor-specific settings from MPIR if available.

Updated spkg: http://spkg-upload.googlecode.com/files/ecm-6.3.p2.spkg (same place)

md5sum: 8246ca74be1ee07b312a84d2a88d9142 ecm-6.3.p2.spkg (fortunately differs ;-) )


ecm-6.3.p2 (Leif Leonhardy, November 25th, 2010)

  • #5847: Apply another patch from upstream to 'configure.in' to fix com- pilation on 32-bit x86 processors supporting SSE2. (Also #10252.) (There's only a single, cumulative patch file since both patches are to 'configure.in'.)
  • Work around linker bug on MacOS X 10.5 PPC (see Special Update/Build? Instructions above, and #5847 comment 35 ff.).
  • Allow passing extra arguments to 'configure' through ECM_EXTRA_OPTS.
  • Add "-march=native" to CFLAGS on platforms that support it, or use processor-specific CFLAGS from GMP's / MPIR's 'gmp.h' if available, but only if CFLAGS do not already contain similar (i.e., don't over- ride a user's choice). [Subject to further improvement.]
  • Print settings of CC, CFLAGS etc. and how we configure.
  • Print settings of CC and CFLAGS found in 'SAGE_LOCAL/include/gmp.h' and eventually a system-wide 'gmp.h', although the latter aren't (yet) used at all, and only processor-specific parts of the former.
  • Add '-fPIC' conditionally, i.e. not if we're also building a shared library (or '--with-pic' was given).
  • Don't delete previous installations unless the build succeeded.
  • Further clean-up.

Special Update/Build? Instructions

[...]

  • ECM currently does not (by itself) use the CC and CFLAGS settings from 'gmp.h' since we pass (other) options in CFLAGS, and CC is set by Sage and might got set by the user (though MPIR currently doesn't use its own CFLAGS for the same reason, which is fixed in an MPIR 2.1.3.p2 spkg). We now at least partially fix that s.t. "optimized" code generation options ('-mcpu=...', '-mtune=...') are used by gcc. Of course a user can also manually enable them by setting the "global" CFLAGS to e.g. '-march=native' on x86[_64] systems, or '-mcpu=...' and '-mtune=...' on other architectures where "native" isn't supported. Note that this doesn't affect the packages' selection of processor- specific optimized [assembly] code. 'spkg-install' already reads the settings from Sage's and also a system-wide GMP / MPIR now, but doesn't (yet) use all of them. If SAGE_FAT_BINARY="yes", we should avoid too specific settings of "-mcpu=...", and perhaps pass a more generic "--host=..." to 'configure'. (MPIR honors '--enable-fat' to some extent, but this option isn't used on anything other than x86 / x86_64.)

A new MPIR (2.1.3.p2) spkg which handles CFLAGS better (allowing us to use processor-specific settings chosen by MPIR) is on the way, but will live on another ticket (not #8664). Just haven't committed the changes yet.

Changed 9 years ago by leif

Diff between the p1 and the p2 spkg. For reference / review.

Changed 9 years ago by leif

Mercurial patch to get the p2 from the p1 spkg. Except for the commit messages perhaps less readable. (6 changesets.)

comment:87 in reply to: ↑ 86 ; follow-up: Changed 9 years ago by dimpase

Replying to leif:

Replying to leif:

P.S.: As I said, I first want to get some test results (e.g. on PowerPCs etc.) before I upload further changes.

builts OK on OSX 10.5 PPC, with SAGE_CHECK=yes

comment:88 in reply to: ↑ 87 Changed 9 years ago by leif

Replying to dimpase:

Replying to leif:

Replying to leif:

P.S.: As I said, I first want to get some test results (e.g. on PowerPCs etc.) before I upload further changes.

builts OK on OSX 10.5 PPC, with SAGE_CHECK=yes

Happy to hear that. I wonder what happens on other MacOS X versions with GCC 4.2.1 (regarding the linker bus error in configure when -g was given to gcc).

comment:89 follow-up: Changed 9 years ago by jdemeyer

  • Description modified (diff)
  • Keywords ecm spkg added

comment:90 in reply to: ↑ 89 Changed 9 years ago by leif

comment:91 Changed 9 years ago by jdemeyer

Tested successfully on

  • sage.math.washington.edu
  • OS X 10.4 PPC
  • 32-bit Gentoo Linux Pentium 4

comment:92 Changed 9 years ago by jdemeyer

  • Milestone changed from sage-4.6.1 to sage-4.6.2

comment:93 follow-up: Changed 8 years ago by drkirkby

SAGE_FAT_BINARY is unsupported in this package, but to compond the problem, the compiler option

-march=native

is added. That's just asking for trouble for distributing binaries.

comment:94 follow-up: Changed 8 years ago by mariah

  • Status changed from needs_review to needs_work

SAGE_FAT_BINARY needs to be supported.

comment:95 in reply to: ↑ 93 Changed 8 years ago by leif

Replying to drkirkby:

SAGE_FAT_BINARY is unsupported in this package, but to compond the problem, the compiler option

-march=native

is added.


Well, apparently SAGE_FAT_BINARY has never been supported by the ECM spkg.

And no, -march=native isn't added by spkg-install in case SAGE_FAT_BINARY=yes:

if [ "$SAGE_FAT_BINARY" = yes ]; then
    # XXX Disable SSE2 on x86? (by passing '--enable-sse2=no' to 'configure')
    # XXX Disable asm-redc? Or pass some "generic" '--host=...' to 'configure'?
    echo "Warning: SAGE_FAT_BINARY is currently not really supported by this package."
    echo "         Add e.g. '--disable-asm-redc' and/or '--enable-sse2=no'"
    echo "         to ECM_EXTRA_OPTS if you run into problems."
else
    # Tune the code generation to the machine we build on:

    ...
fi

(Note that some 'optimized' parameters may come from gmp.h anyway, but these aren't used either if SAGE_FAT_BINARY=yes.)

We had some discussion on that before, and Paul said they're considering adding a --enable-fat switch to ECM's configure IIRC. There are also a few comments in ECM's SPKG.txt.

comment:96 in reply to: ↑ 94 ; follow-up: Changed 8 years ago by leif

  • Status changed from needs_work to needs_info

Replying to mariah:

SAGE_FAT_BINARY needs to be supported.


In which way / to what extent?

(See my previous post.)

comment:97 in reply to: ↑ 96 Changed 8 years ago by mariah

  • Status changed from needs_info to needs_review

Replying to leif:

Replying to mariah:

SAGE_FAT_BINARY needs to be supported.


In which way / to what extent?

(See my previous post.)

If I build a SAGE_FAT_BINARY version of sage on one of the skynet x86_64 machines, then move that build to another skynet x86_64 machine, then all the tests should pass. That is my understanding of what SAGE_FAT_BINARY should accomplish.

In spite of ecm not explicitly recognizing SAGE_FAT_BINARY, when I have built SAGE_FAT_BINARY versions of previous versions of sage (for example 4.6.2), I have not run into any problems. (Perhaps I have just been lucky.)

Does this spkg maintain this SAGE_FAT_BINARY functionality?

comment:98 follow-up: Changed 8 years ago by mariah

  • Status changed from needs_review to needs_work

sage-4.7.1-alpha1 built with http://spkg-upload.googlecode.com/files/ecm-6.3.p2.spkg on skynet/flavius (x86_64-Linux-k8-fc) fails test:

flavius% ./sage -t -long -force_lib "devel/sage/sage/libs/libecm.pyx"
sage -t -long -force_lib "devel/sage/sage/libs/libecm.pyx"  
**********************************************************************
File "/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-k8-fc-review-5847/devel/sage/sage/libs/libecm.pyx", line 14:
    sage: import sage.libs.libecm
Exception raised:
    Traceback (most recent call last):
      File "/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-k8-fc-review-5847/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-k8-fc-review-5847/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-k8-fc-review-5847/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_0[2]>", line 1, in <module>
        import sage.libs.libecm###line 14:
    sage: import sage.libs.libecm
    ImportError: /home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-k8-fc-review-5847/local/lib/python/site-packages/sage/libs/libecm.so: cannot enable executable stack as shared object requires: Permission denied
**********************************************************************

Looking at the build I see

Copying patched files...
Adding '-fPIC' to CFLAGS since we don't (also) build a shared library.
Using additional host-specific CFLAGS: -march=native

Settings from SAGE_LOCAL/include/gmp.h:
    CC=gcc 
    CFLAGS=-O2 -m64 -march=k8 -mtune=k8 
Using CC=gcc
Using CFLAGS=-march=native -g -O3  -fPIC
Using CPPFLAGS=
Using LDFLAGS=
Using ABI=
(These settings may get overridden by 'configure' or Makefiles.)

Strongly suggest ecm be built with same CFLAGS values as gmp/mpir as recommended by ecm INSTALL file.

comment:99 in reply to: ↑ 98 Changed 8 years ago by leif

  • Status changed from needs_work to needs_info

Replying to mariah:

sage-4.7.1-alpha1 built with http://spkg-upload.googlecode.com/files/ecm-6.3.p2.spkg on skynet/flavius (x86_64-Linux-k8-fc) fails test:

...
    ImportError: /home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-k8-fc-review-5847/local/lib/python/site-packages/sage/libs/libecm.so: cannot enable executable stack as shared object requires: Permission denied

Looking at the build I see

...
Adding '-fPIC' to CFLAGS since we don't (also) build a shared library.
...
Settings from SAGE_LOCAL/include/gmp.h:
    CC=gcc 
    CFLAGS=-O2 -m64 -march=k8 -mtune=k8 
Using CC=gcc
Using CFLAGS=-march=native -g -O3  -fPIC
Using CPPFLAGS=
Using LDFLAGS=
...

Strongly suggest ecm be built with same CFLAGS values as gmp/mpir as recommended by ecm INSTALL file.


Hi Mariah,

this failure is rather unrelated to the GMP-ECM package itself (since we only build a static library, see above), but a "common" Fedora (>=14 / SELinux) / GCC problem; cf. e.g. #10188.

The libecm.so extension module -- which erroneously carries the "executable stack" flag -- is built by Cython from libecm.pyx.

Could you please first try

$ execstack -c local/lib/python/site-packages/sage/libs/libecm.so

and rerun the test?

(Taking the flags from MPIR / gmp.h wouldn't help anyway in this case.)


If the above works, you could then patch module_list.py:

  • module_list.py

    diff --git a/module_list.py b/module_list.py
    a b  
    561561    Extension('sage.libs.libecm',
    562562              sources = ['sage/libs/libecm.pyx'],
    563563              libraries = ['ecm', 'gmp'],
     564              extra_compile_args=["-Wl,-z,noexecstack"],
    564565              depends = [SAGE_ROOT + "/local/include/ecm.h"]),
    565566     
    566567    Extension('sage.libs.mwrank.mwrank',

Then you probably have to touch libecm.pyx in order to convince Cython to rebuild the module before issuing ./sage -b. Afterwards the tests should pass without the need to rerun execstack. (You can check the flag's setting by running execstack -q ....)

Let me know if the above works and/or if you need a Mercurial patch for that... ;-)

(Just curious: Which GCC version are you using now? The problem may or should have arised with earlier version of Sage on Fedora, too.)

comment:100 Changed 8 years ago by leif

Disclaimer for Dave:

The patch above is a bit hackish. We should either pass the additional compiler flag conditionally (on Linux systems only, where we can rely on a GNU linker), or perhaps instead use GCC's --noexecstack parameter, of which I currently don't know whether it is supported by ancient versions of GCC.

comment:101 Changed 8 years ago by leif

Hmmm, apparently the patch should read:

  • module_list.py

    diff --git a/module_list.py b/module_list.py
    a b  
    561561    Extension('sage.libs.libecm',
    562562              sources = ['sage/libs/libecm.pyx'],
    563563              libraries = ['ecm', 'gmp'],
     564              extra_link_args=["-Wl,-z,noexecstack"],
    564565              depends = [SAGE_ROOT + "/local/include/ecm.h"]),
    565566     
    566567    Extension('sage.libs.mwrank.mwrank',

(Still hackish though.)

comment:102 follow-up: Changed 8 years ago by mariah

On skynet/flavius I get

flavius% execstack -c local/lib/python/site-packages/sage/libs/libecm.so
execstack: Could not set security context for local/lib/python/site-packages/sage/libs/libecm.so: Operation not supported
flavius%

I built with gcc-4.6.0:

flavius% gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/gcc-4.6.0/x86_64-Linux-k8-fc/libexec/gcc/x86_64-unknown-linux-gnu/4.6.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: /usr/local/gcc-4.6.0/src/gcc-4.6.0/configure --enable-languages=c,c++,fortran --with-gnu-as --with-as=/usr/local/binutils-2.21/x86_64-Linux-k8-fc-gcc-4.5.1-rh/bin/as --with-gnu-ld --with-ld=/usr/local/binutils-2.21/x86_64-Linux-k8-fc-gcc-4.5.1-rh/bin/ld --with-gmp=/usr/local/mpir-2.3.0/x86_64-Linux-k8-fc-gcc-4.5.1-rh --with-mpfr=/usr/local/mpfr-3.0.0/x86_64-Linux-k8-fc-mpir-2.3.0-gcc-4.5.1-rh --with-mpc=/usr/local/mpc-0.9/x86_64-Linux-k8-fc-mpir-2.3.0-mpfr-3.0.0-gcc-4.5.1-rh --prefix=/usr/local/gcc-4.6.0/x86_64-Linux-k8-fc
Thread model: posix
gcc version 4.6.0 (GCC) 
flavius%

comment:103 in reply to: ↑ 102 ; follow-up: Changed 8 years ago by leif

Replying to mariah:

On skynet/flavius I get

flavius% execstack -c local/lib/python/site-packages/sage/libs/libecm.so
execstack: Could not set security context for local/lib/python/site-packages/sage/libs/libecm.so: Operation not supported
flavius%


Not supported? WTF...

No idea what's causing that. Though I don't need it, I can toggle the flag here on Ubuntu (10.4) as I like.

Maybe an obsolete version of execstack? Or maybe just a wrong, misleading error message? The path is correct (relative to SAGE_ROOT)?

If all doesn't help, simply try applying the patch (to devel/sage/module_list.py), touch devel/sage/sage/libs/libecm.pyx and run ./sage -b.

comment:104 in reply to: ↑ 103 ; follow-up: Changed 8 years ago by mariah

  • Description modified (diff)

Replying to leif:

If all doesn't help, simply try applying the patch (to devel/sage/module_list.py), touch devel/sage/sage/libs/libecm.pyx and run ./sage -b.

This works. Now the offending test passes.

(I thought I did the execstack command in SAGE_ROOT.)

If you will add the patch to this ticket, I will have another go at building on all the skynet machines.

comment:105 in reply to: ↑ 104 Changed 8 years ago by leif

Replying to mariah:

(I thought I did the execstack command in SAGE_ROOT.)

Well, my execstack (version 1.0) gives an appropriate error message on non-existing.so. Perhaps you have an old version installed that only supports 32-bit ELF.


If you will add the patch to this ticket, I will have another go at building on all the skynet machines.

Ok, I'll make the extra linker flag depend on the OS (i.e., Linux).

May take 1-2 hours until I upload a Mercurial patch.

Changed 8 years ago by leif

Sage library patch. Fixes 'execstack' issue on Fedora with GCC 4.6.0. Based on Sage 4.7.rc2.

comment:106 Changed 8 years ago by leif

  • Description modified (diff)
  • Status changed from needs_info to needs_review

Patch (to the Sage library) is up.

Note: See TracTickets for help on using tickets.