Upgrade to NTL 6.2.1

[jpflori]

Changelog at ​http://www.shoup.net/ntl/doc/tour-changes.html Upstream tarball at:

• ​http://www.shoup.net/ntl/ntl-6.2.1.tar.gz

Sage repackaged tarball (see spkg-src) at:

• ​http://www.lmona.de/files/sage/ntl-6.2.1.tar.bz2

[jpflori]

Our patches need to be updated, working on it.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3fc9def

Update NTL to version 6.2.0.

[jpflori]

At least it builds. Full testsuite not run yet (nor strange install like cygwin tested).

If you wan to try it out, please use SAGE_UPGRADING=yes, as it seems to me the dynamic lib version number got updated once again.

---

New commits:

​3fc9def

Update NTL to version 6.2.0.

[jpflori]

It seems FLINT<->NTL interface (shipped by FLINT) is broken.

[cremona]

Replying to jpflori:

It seems FLINT<->NTL interface (shipped by FLINT) is broken.

How annoying. Will you inform flint-devel?

[jpflori]

I guess it's trivial. Some private stuff must have been made public by Victor (or so I hope).

And yes I made a minimal report at:

• ​https://groups.google.com/forum/#!topic/flint-devel/YEsOO_lnQNs

[jpflori]

Quite unrelated, but we should update flint anyway, I think we still ship the version with the broken is_prime stuff.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​98e68c7	Let FLINT build against NTL 6.2.
​a6b95ac	Update FLINT patch revision.

[jpflori]

Tentative fix pushed. (Nothing really tested yet though.)

[jpflori]

And Singular broke as well...

[jpflori]

And that might be an NTL issue in fact.

[jpflori]

Forwarded the problem to Shoup.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8d60304

Fix some template issue with NTL 6.2.0.

[jpflori]

Fix from Victor included.

[jpflori]

(And tested by he and I, and there should be an NTL bugfix release soon.)

[jpflori]

Victor just released a bugfix release, that leverages the need for the fix I took from his mails.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​311a52c

Update NTL to version 6.2.1.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b67d968

Fix checksum for NTL.

[fbissey]

Does flint still needs to be fixed or did you forget to revert it?

[jpflori]

Nope it still needs to be fixed.

[fbissey]

OK I got two questions then:

• is the fix backward compatible with ntl-6.1/6.0
• is flint planning a new release to include the fix? In the past Bill has been very fast to include stuff like that in a release.

[jpflori]

Replying to fbissey:

OK I got two questions then:

• is the fix backward compatible with ntl-6.1/6.0

No idea.

• is flint planning a new release to include the fix? In the past Bill has been very fast to include stuff like that in a release.

I guess so. I've already posted on flint-devel but Bill did not announce anything. And I fear hes quite busy right now.

[fbissey]

Replying to jpflori:

Replying to fbissey:

OK I got two questions then:

• is the fix backward compatible with ntl-6.1/6.0

No idea.

It looks like the answer is yes, which means that flint doesn't have to check the version of ntl which will make the inclusion of a fix in flint so much easier.

[fbissey]

I will create a pull request for flint (hopefully today) tell me if I am duplicating work that you are ready to push. Once that's done I will test things a bit further but if the patchbot is happy this ticket is done.

[fbissey]

Pull request for flint done: ​https://github.com/wbhart/flint2/pull/95

[jpflori]

Thanks for the pull request. I'll be away from the internet for a few days, so i's very welcome.

[jdemeyer]

This causes a build error in Singular for me:

g++ -O2 -g  -fPIC -I.. -I/usr/local/src/sage-git/local -pipe -I. -I.. -I/usr/local/src/sage-git/local -I/usr/local/src/sage-git/local/include -I/usr/local/src/sage-git/local/include   -fno-implicit-templates -I.. -I/usr/local/src/sage-git/local -DNDEBUG -DOM_NDEBUG -Dx86_64_Linux -DHAVE_CONFIG_H \
  -o Singular \
  tesths.cc iparith.o mpsr_Tok.o claptmpl.o\
  grammar.o scanner.o attrib.o blackbox.o eigenval_ip.o extra.o fehelp.o feOpt.o ipassign.o ipconv.o ipid.o iplib.o ipprint.o ipshell.o newstruct.o lists.o sdb.o fglm.o interpolation.o silink.o ssiLink.o s_buff.o subexpr.o janet.o wrapper.o libparse.o sing_win.o gms.o pcv.o maps_ip.o walk.o walk_ip.o cntrlc.o misc_ip.o calcSVD.o pipeLink.o Minor.o MinorProcessor.o MinorInterface.o bigintm.o pyobject_setup.o denom_list.o minpoly.o countedref.o semaphore.o slInit_Dynamic.o -rdynamic -L/usr/local/src/sage-git/local/kernel -L../kernel -lkernel -L/usr/local/src/sage-git/local/lib  -L/usr/local/src/sage-git/local/lib -lflint -lmpfr -lmpir  -ldl -lm -lsingfac -lsingcf -lflint -lmpfr -lntl -lgmp -lreadline -lrt -lnsl -lm  -lnsl  -lomalloc -lpthread  ../kernel/mmalloc.o 
/usr/local/src/sage-git/local/lib/libsingcf.a(cfNewtonPolygon.o): In function `NTL::Vec<NTL::ZZ>::SetLength(long)':
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::ZZ>::DoSetLength(long)'
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::ZZ>::DoSetLength(long)'
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::ZZ>::DoSetLength(long)'
/usr/local/src/sage-git/local/lib/libsingcf.a(facHensel.o): In function `NTL::Vec<NTL::zz_pEX>::SetLength(long)':
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::zz_pEX>::DoSetLength(long)'
/usr/local/src/sage-git/local/lib/libsingcf.a(facNTLzzpEXGCD.o): In function `NTL::Vec<NTL::zz_pX>::SetLength(long)':
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::zz_pX>::DoSetLength(long)'
collect2: ld returned 1 exit status
make[4]: *** [Singular] Error 1

[jpflori]

Wasn't that the issue fixed between 6.2.0 and 6.2.1?

[fbissey]

Replying to jpflori:

Wasn't that the issue fixed between 6.2.0 and 6.2.1?

You would be the only one to know, you never posted your singular problem with 6.2.0.

[jpflori]

Euh, yes indeed. This looks different as my problem was with template instantiation. Here there is an undefined reference. What tricked me earlier is that the problematic function is the same.

For the record, here is my report to Victor Shoup: I've been playing for a few moments with NTL 6.2.0 and it looks really nice. Nonetheless I don't seem able to build singular 3-6-0 on top of it.

Indeed g++ spits: {{{../../../../../../../../include/NTL/vector.h:427:4: error: no matching function for call to 'NTL::Vec<NTL::Pair<NTL::GF2X, long int>

::Init(long int&, const NTL::Pair<NTL::GF2X, long int>&, const

NTL::Pair<NTL::GF2X, long int>&)' ../../../../../../../../include/NTL/vector.h:427:4: note: candidates are: ../../../../../../../../include/NTL/vector.h:379:6: note: void NTL::Vec<T>::Init(long int) [with T = NTL::Pair<NTL::GF2X, long int>] ../../../../../../../../include/NTL/vector.h:379:6: note: candidate expects 1 argument, 3 provided ../../../../../../../../include/NTL/vector.h:389:6: note: void NTL::Vec<T>::Init(long int, const T*) [with T = NTL::Pair<NTL::GF2X, long int>] ../../../../../../../../include/NTL/vector.h:389:6: note: candidate expects 2 arguments, 3 provided ../../../../../../../../include/NTL/vector.h:400:6: note: void NTL::Vec<T>::Init(long int, const T&) [with T = NTL::Pair<NTL::GF2X, long int>] ../../../../../../../../include/NTL/vector.h:400:6: note: candidate expects 2 arguments, 3 provided }}}

And looking at the new vector.h from NTL, on line 427, there is indeed a call to: Init(n, a, *src); within the DoSetLength? definition.

And no corresponding function in vector.h (whereas there are indeed three definitions for Init with different signatures.)

Am I missing something here?

Best regards,

[fbissey]

Not sure you are missing something. C++ always look like spaghetti to me (in a different way from classic FORTRAN - FORTRAN was abstract spaghetti, C++ is litteral as far as I am concerned). I could be that "DoSetLength?" is actually defined in another header and it is not properly included before vector.h or in vector.h itself.

[fbissey]

Well a local build here shows

objdump -T -C /scratch/portage/dev-libs/ntl-6.2.1/image/usr/lib64/libntl.so.5.0.0 | grep DoSetLength
00000000000b8c26  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
00000000000973fc g    DF .text  00000000000001b9  Base        NTL::WordVector::DoSetLength(long)
000000000005aac6  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::GF2EX, long> >::DoSetLength(long)
00000000001015c0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
00000000000b7af2  w   DF .text  000000000000005a  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000eb136  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pX, long> >::DoSetLength(long)
000000000009d610  w   DF .text  0000000000000038  Base        NTL::Vec<char>::DoSetLength(long)
0000000000070aae  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::GF2X, long> >::DoSetLength(long)
000000000013246a  w   DF .text  0000000000000089  Base        NTL::Vec<NTL::ZZVec>::DoSetLength(long)
00000000000845ca  w   DF .text  0000000000000080  Base        NTL::Vec<NTL::quad_float>::DoSetLength(long)
00000000000b8c98  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Vec<long> >::DoSetLength(long)
0000000000090854  w   DF .text  0000000000000083  Base        NTL::Vec<NTL::xdouble>::DoSetLength(long)
0000000000122424  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::zz_pX, long> >::DoSetLength(long)
000000000004495e  w   DF .text  0000000000000038  Base        NTL::Vec<unsigned long>::DoSetLength(long)
0000000000052128  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
00000000000c78f0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)
0000000000101988  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
0000000000045350  w   DF .text  0000000000000038  Base        NTL::Vec<NTL::IntFactor>::DoSetLength(long)
000000000007534e  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)
00000000000b8a4c  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
000000000007bb40  w   DF .text  0000000000000038  Base        NTL::Vec<double>::DoSetLength(long)
000000000010814c  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::zz_pEX, long> >::DoSetLength(long)
000000000007b72a  w   DF .text  0000000000000082  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
00000000000b886c  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::ZZX, long> >::DoSetLength(long)
0000000000101e08  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
0000000000051b32  w   DF .text  000000000000005a  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
00000000000cdda6  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pEX, long> >::DoSetLength(long)
00000000000447be  w   DF .text  0000000000000038  Base        NTL::Vec<long>::DoSetLength(long)
000000000006e66a  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::GF2X>::DoSetLength(long)
00000000000753c0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)
00000000000b7ff6  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000c7470  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)
0000000000127b74  w   DF .text  0000000000000089  Base        NTL::Vec<NTL::GF2XVec>::DoSetLength(long)

Jeroen's failure was on three symbols:

NTL::Vec<NTL::ZZ>::DoSetLength(long)
NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
NTL::Vec<NTL::zz_pX>::DoSetLength(long)

And they are all present. So two possibilities I see:

• Jeroen's build of ntl was incomplete
• there is an ordering problem in the singular's linking

Jeroen can easily check the first one. The second would involve a little bit more experimentation and cut and paste.

I will try to reproduce this locally.

[fbissey]

/usr/local/src/sage-git/local/lib/libsingcf.a(cfNewtonPolygon.o): In function 
`NTL::Vec<NTL::ZZ>::SetLength(long)':

We have interference from a previously installed singular. It gets the wrong libsingcf.a. Obvious once you look at it closely.

[fbissey]

After examination of what I am doing in sage-on-gentoo is there a need in sage for libsingcf.a and the other static archives installed? Is Macauley2 using them?

[fbissey]

Replying to fbissey:

After examination of what I am doing in sage-on-gentoo is there a need in sage for libsingcf.a and the other static archives installed? Is Macauley2 using them?

Skip that. I forgot that singular install directly under SAGE_LOCAL while compiling. That and the fact the symbol are referenced in the library. Going back to testing.

[fbissey]

Couldn't reproduce it so far.

[jpflori]

I would also say this comes from an old NTL install, but only Jeroen can confirm...

[fbissey]

Replying to jpflori:

I would also say this comes from an old NTL install, but only Jeroen can confirm...

Nice theory but I just checked and these symbols are also in ntl 6.1.0. Unless of course it comes from a much older ntl (I could look into that if needed). So there is probably something more subtle at play. I would very much like Jeroen to check the symbols are in libntl and if they are we should look carefully at the full log from the singular build.

[jdemeyer]

OK, I'm trying the build again...

[jpflori]

Replying to fbissey:

Replying to jpflori:

I would also say this comes from an old NTL install, but only Jeroen can confirm...

Nice theory but I just checked and these symbols are also in ntl 6.1.0. Unless of course it comes from a much older ntl (I could look into that if needed). So there is probably something more subtle at play. I would very much like Jeroen to check the symbols are in libntl and if they are we should look carefully at the full log from the singular build.

Yeah, but this part of the templating stuff changed between 6.1.0 and 6.2.0, so it might also be the case that Jeroen built on top of the 6.2.0 version with broken lazy templates initialization...

[jdemeyer]

I rebuilt and it failed again. The output of the analogous objdump command is:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000a4d30 g    DF .text  00000000000001e9  Base        NTL::WordVector::DoSetLength(long)

FYI:

(sage-sh)$ find local -name libntl* |xargs ls -l
-rw-r--r-- 1 jdemeyer jdemeyer 37380910 Oct  8 10:41 local/lib/libntl.a
-rw-r--r-- 1 jdemeyer jdemeyer     1058 Oct  8 10:41 local/lib/libntl.la
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so -> libntl.so.5.0.0
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so.5 -> libntl.so.5.0.0
-rwxr-xr-x 1 jdemeyer jdemeyer 14786659 Oct  8 10:41 local/lib/libntl.so.5.0.0

[jdemeyer]

Also FYI:

(sage-sh)$ uname -a
Linux tamiyo 3.7.10-gentoo #4 SMP PREEMPT Thu Oct 31 21:23:56 CET 2013 x86_64 Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz GenuineIntel GNU/Linux

(sage-sh)$ g++ --version
g++ (Gentoo 4.6.4 p1.2, pie-0.5.2) 4.6.4
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(sage-sh)$ ld --version
GNU ld (GNU Binutils) 2.22
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

[jdemeyer]

Full log of NTL build

[jdemeyer]

Full log of failed Singular build

[fbissey]

Replying to jdemeyer:

I rebuilt and it failed again. The output of the analogous objdump command is:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000a4d30 g    DF .text  00000000000001e9  Base        NTL::WordVector::DoSetLength(long)

FYI:

(sage-sh)$ find local -name libntl* |xargs ls -l
-rw-r--r-- 1 jdemeyer jdemeyer 37380910 Oct  8 10:41 local/lib/libntl.a
-rw-r--r-- 1 jdemeyer jdemeyer     1058 Oct  8 10:41 local/lib/libntl.la
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so -> libntl.so.5.0.0
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so.5 -> libntl.so.5.0.0
-rwxr-xr-x 1 jdemeyer jdemeyer 14786659 Oct  8 10:41 local/lib/libntl.so.5.0.0

OK so it looks like libntl is not built properly, although I have no idea where there is room for variation. At this stage I think we need the build log for ntl.

[jdemeyer]

Replying to fbissey:

At this stage I think we need the build log for ntl.

It's attached on this ticket.

[fbissey]

By the look of it you posted the logs a few seconds before me. Just as an aside I just noticed that libntl.so is currently linked with gcc rather than g++. Not fatal but it leaves a lot of libstdc++ symbols undefined. I'll see if I can come up with something for that tomorrow.

[jdemeyer]

With NTL-6.1.0, I get

(sage-sh)$ objdump -T -C local/lib/libntl.so.3.0.0 | grep DoSetLength
00000000000d3f60  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::ZZ_p> >::DoSetLength(long)
00000000000c2e80  w   DF .text  00000000000001f3  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000d3780  w   DF .text  0000000000000243  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)                                                                   
00000000000772b0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)                                                           
000000000005fcf0  w   DF .text  0000000000000247  Base        NTL::Vec<NTL::GF2XVec>::DoSetLength(long)                                                                 
00000000000738e0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)                                                                      
00000000000c3180  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
0000000000073b00  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::GF2X>::DoSetLength(long)
00000000000fae10  w   DF .text  0000000000000247  Base        NTL::Vec<NTL::ZZVec>::DoSetLength(long)
0000000000047820  w   DF .text  00000000000001d3  Base        NTL::Vec<double>::DoSetLength(long)
0000000000075fe0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::GF2X, long> >::DoSetLength(long)
0000000000113a60  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::zz_p> >::DoSetLength(long)
0000000000154ba0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::zz_pE> >::DoSetLength(long)
00000000000c2c60  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
0000000000047460  w   DF .text  00000000000001d3  Base        NTL::Vec<unsigned long>::DoSetLength(long)
00000000000c37b0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
0000000000047640  w   DF .text  00000000000001d3  Base        NTL::Vec<NTL::FFTPrimeInfo*>::DoSetLength(long)
0000000000047c20  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<unsigned long> >::DoSetLength(long)
000000000011a1c0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::zz_pEX, long> >::DoSetLength(long)
000000000009e460 g    DF .text  00000000000001e9  Base        NTL::WordVector::DoSetLength(long)
000000000005f8e0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::GF2EX, long> >::DoSetLength(long)
0000000000081680  w   DF .text  0000000000000214  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
0000000000047a00  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<long> >::DoSetLength(long)
000000000013d110  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::GF2E> >::DoSetLength(long)
00000000000faa00  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pX, long> >::DoSetLength(long)
0000000000055fb0  w   DF .text  00000000000001f3  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
00000000000566b0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
0000000000113840  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
00000000000c33a0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::ZZX, long> >::DoSetLength(long)
00000000000965c0  w   DF .text  0000000000000214  Base        NTL::Vec<NTL::xdouble>::DoSetLength(long)
0000000000113280  w   DF .text  0000000000000243  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
0000000000048100  w   DF .text  00000000000001dc  Base        NTL::Vec<NTL::IntFactor>::DoSetLength(long)
000000000014d500  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::ZZ_pE> >::DoSetLength(long)
00000000000da930  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pEX, long> >::DoSetLength(long)
00000000000568d0  w   DF .text  0000000000000247  Base        NTL::Vec<NTL::Vec<NTL::GF2> >::DoSetLength(long)
0000000000047280  w   DF .text  00000000000001d3  Base        NTL::Vec<long>::DoSetLength(long)
000000000008a4f0  w   DF .text  0000000000000214  Base        NTL::Vec<NTL::quad_float>::DoSetLength(long)
00000000000c3eb0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::zz_pX, long> >::DoSetLength(long)
00000000000818d0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::RR> >::DoSetLength(long)
00000000000c3a00  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000d3d40  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)

[jdemeyer]

@fbissey and @jpflori: does this ticket actually work for the both of you? If so, I'm surprised that it doesn't work for me...

[jdemeyer]

Something else: patches should apply without fuzz:

Applying patches to NTL.
patching file src/tools.c
Hunk #1 succeeded at 18 with fuzz 1.
patching file include/NTL/tools.h
Hunk #2 succeeded at 311 (offset 58 lines).
patching file src/DoConfig
patching file src/mfile
patching file src/mfile
Hunk #1 succeeded at 325 (offset -6 lines).
Hunk #2 succeeded at 338 (offset -6 lines).
Hunk #3 succeeded at 358 (offset -9 lines).
Hunk #4 succeeded at 397 (offset -9 lines).
patching file src/WizardAux
Hunk #3 succeeded at 254 (offset 40 lines).
patching file include/NTL/new.h

[jdemeyer]

With g++-4.8.3, things change:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000c6290  w   DF .text  0000000000000064  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000d82b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)
000000000007b780  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)
0000000000144980  w   DF .text  0000000000000080  Base        NTL::Vec<NTL::GF2XVec>::DoSetLength(long)
000000000007b710  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)
00000000000c7760  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
0000000000074050  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::GF2X>::DoSetLength(long)
0000000000082a90  w   DF .text  0000000000000036  Base        NTL::Vec<double>::DoSetLength(long)
00000000001199d0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
00000000000c7840  w   DF .text  0000000000000036  Base        NTL::Vec<unsigned long>::DoSetLength(long)
00000000000c7540  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
00000000000a0670 g    DF .text  0000000000000200  Base        NTL::WordVector::DoSetLength(long)
00000000000825d0  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
00000000000c77d0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::Vec<long> >::DoSetLength(long)
0000000000056770  w   DF .text  0000000000000064  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
0000000000056ea0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
000000000011a400  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
0000000000119f10  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
00000000000a75c0  w   DF .text  0000000000000036  Base        NTL::Vec<char>::DoSetLength(long)
000000000007b7f0  w   DF .text  0000000000000036  Base        NTL::Vec<long>::DoSetLength(long)
000000000008bdd0  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::quad_float>::DoSetLength(long)
00000000000c6a20  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000d87a0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)

[fbissey]

OK, now that's weird, and yes it works for me and I re-built singular and flint after ntl. Haven't done sage however.

[jpflori]

Yup, I've been using it on different systems (one redhat ppc64 and a ubuntu x86_64) for a while now... In fact the release of 6.2.1 over 6.2.0 was made because I tried to use 6.2.0 and it failed and I contacted Shoup who was very reactive...

[jdemeyer]

With g++-4.7.3 from Sage:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000c6ea0  w   DF .text  0000000000000074  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000d95b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)
00000000000c82b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)
000000000007e150  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)
0000000000101a20  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
00000000000852c0  w   DF .text  0000000000000046  Base        NTL::Vec<double>::DoSetLength(long)
0000000000079090  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::GF2X, long> >::DoSetLength(long)
000000000011afa0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
0000000000049c40  w   DF .text  0000000000000046  Base        NTL::Vec<unsigned long>::DoSetLength(long)
000000000013f470  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
0000000000122370  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::zz_pEX, long> >::DoSetLength(long)
00000000000a1f40 g    DF .text  0000000000000208  Base        NTL::WordVector::DoSetLength(long)
00000000000627b0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::GF2EX, long> >::DoSetLength(long)
0000000000084e30  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
0000000000058b70  w   DF .text  0000000000000074  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
00000000001019a0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pX, long> >::DoSetLength(long)
0000000000062830  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
00000000001223f0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
00000000000c80b0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::ZZX, long> >::DoSetLength(long)
000000000009a180  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::xdouble>::DoSetLength(long)
000000000011b3b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
000000000004a6e0  w   DF .text  0000000000000046  Base        NTL::Vec<NTL::IntFactor>::DoSetLength(long)
00000000000a9060  w   DF .text  0000000000000046  Base        NTL::Vec<char>::DoSetLength(long)
00000000000e06c0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pEX, long> >::DoSetLength(long)
0000000000049a70  w   DF .text  0000000000000046  Base        NTL::Vec<long>::DoSetLength(long)
000000000013f3f0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::zz_pX, long> >::DoSetLength(long)
00000000000c7730  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000e0740  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)

[jdemeyer]

With vanilla g++-4.6.4, same problem as with Gentoo's g++-4.6.4

I think that GCC 4.6.4 is too recent to not support it, so the DoSetLength problem should be fixed...

[fbissey]

I am guessing the latest release of ntl relies on templating features from recent g++, probably 4.7.x. Basically we would almost require someone with gcc<4.7 to use something that is like ntl 6.1. I'll have a look at the diff between 6.1 and 6.2.1.

[jdemeyer]

Replying to fbissey:

I am guessing the latest release of ntl relies on templating features from recent g++, probably 4.7.x.

I'm not entirely against requiring g++-4.7 (we do have the GCC spkg after all), but is upstream aware of this? Anyway, if an easy fix is possible for this issue, that would be the best.

[jdemeyer]

Replying to fbissey:

probably 4.7.x.

Yes, as I posted before, g++-4.7.3 does work.

[fbissey]

Jean-Pierre has contact upstream I guess he could ask if that is a known requirement. Regarding this ticket I would like to understand why the .so library is linked with gcc rather than g++ - like upstream intended. Upstream source indicate g++, my sage-on-gentoo ebuild uses g++. This spkg: gcc. What the...?

[fbissey]

OK the configure script we currently ship is insufficient to generate a libtool that can link with g++. I am going to clean the spkg a bit later (yes I'll rebase the patches) but basically I will put this as the new configure.ac

AC_INIT(NTL, 6.2.1, victor@shoup.net)
AM_INIT_AUTOMAKE([1.9 foreign])
AC_CONFIG_FILES([Makefile])
LT_INIT
AC_PROG_CXX
AC_PROG_CC
AC_PROG_LIBTOOL
AC_OUTPUT

Looking for the C compiler is possibly over the top but it doesn't hurt. I grossly tested things and with that I now have a clean linking with g++. Before

/home/work/fbissey/sandbox/sage-6.4.beta4/local/var/tmp/sage/build/ntl-6.2.1.p0/src/libtool/libtool --mode=link g++ -I../include -I.  -O2 -g     -o libntl.la FFT.lo FacVec.lo GF2.lo GF2E.lo GF2EX.lo GF2EXFactoring.lo GF2X.lo GF2X1.lo GF2XFactoring.lo GF2XVec.lo GetTime.lo HNF.lo ctools.lo LLL.lo LLL_FP.lo LLL_QP.lo LLL_RR.lo LLL_XD.lo RR.lo WordVector.lo ZZ.lo ZZVec.lo ZZX.lo ZZX1.lo ZZXCharPoly.lo ZZXFactoring.lo ZZ_p.lo ZZ_pE.lo ZZ_pEX.lo ZZ_pEXFactoring.lo ZZ_pX.lo ZZ_pX1.lo ZZ_pXCharPoly.lo ZZ_pXFactoring.lo fileio.lo lip.lo lzz_p.lo lzz_pE.lo lzz_pEX.lo lzz_pEXFactoring.lo lzz_pX.lo lzz_pX1.lo lzz_pXCharPoly.lo lzz_pXFactoring.lo mat_GF2.lo mat_GF2E.lo mat_RR.lo mat_ZZ.lo mat_ZZ_p.lo mat_ZZ_pE.lo mat_lzz_p.lo mat_lzz_pE.lo mat_poly_ZZ.lo mat_poly_ZZ_p.lo mat_poly_lzz_p.lo quad_float.lo tools.lo vec_GF2.lo vec_GF2E.lo vec_RR.lo vec_ZZ.lo vec_ZZ_p.lo vec_ZZ_pE.lo vec_lzz_p.lo vec_lzz_pE.lo xdouble.lo G_LLL_FP.lo G_LLL_QP.lo G_LLL_XD.lo G_LLL_RR.lo -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -
lgmp  -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -lgf2x  -lm -rpath /home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -version-info `cat VERSION_INFO` #LSHAR
libtool: link: gcc -shared  -fPIC -DPIC  .libs/FFT.o .libs/FacVec.o .libs/GF2.o .libs/GF2E.o .libs/GF2EX.o .libs/GF2EXFactoring.o .libs/GF2X.o .libs/GF2X1.o .libs/GF2XFactoring.o .libs/GF2XVec.o .libs/GetTime.o .libs/HNF.o .libs/ctools.o .libs/LLL.o .libs/LLL_FP.o .libs/LLL_QP.o .libs/LLL_RR.o .libs/LLL_XD.o .libs/RR.o .libs/WordVector.o .libs/ZZ.o .libs/ZZVec.o .libs/ZZX.o .libs/ZZX1.o .libs/ZZXCharPoly.o .libs/ZZXFactoring.o .libs/ZZ_p.o .libs/ZZ_pE.o .libs/ZZ_pEX.o .libs/ZZ_pEXFactoring.o .libs/ZZ_pX.o .libs/ZZ_pX1.o .libs/ZZ_pXCharPoly.o .libs/ZZ_pXFactoring.o .libs/fileio.o .libs/lip.o .libs/lzz_p.o .libs/lzz_pE.o .libs/lzz_pEX.o .libs/lzz_pEXFactoring.o .libs/lzz_pX.o .libs/lzz_pX1.o .libs/lzz_pXCharPoly.o .libs/lzz_pXFactoring.o .libs/mat_GF2.o .libs/mat_GF2E.o .libs/mat_RR.o .libs/mat_ZZ.o .libs/mat_ZZ_p.o .libs/mat_ZZ_pE.o .libs/mat_lzz_p.o .libs/mat_lzz_pE.o .libs/mat_poly_ZZ.o .libs/mat_poly_ZZ_p.o .libs/mat_poly_lzz_p.o .libs/quad_float.o .libs/tools.o .libs/vec_GF2.o .libs/vec_GF2E.o .libs/vec_
RR.o .libs/vec_ZZ.o .libs/vec_ZZ_p.o .libs/vec_ZZ_pE.o .libs/vec_lzz_p.o .libs/vec_lzz_pE.o .libs/xdouble.o .libs/G_LLL_FP.o .libs/G_LLL_QP.o .libs/G_LLL_XD.o .libs/G_LLL_RR.o   -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -lgmp -lgf2x -lm  -O2   -Wl,-soname -Wl,libntl.so.5 -o .libs/libntl.so.5.0.0

After

/home/work/fbissey/sandbox/sage-6.4.beta4/local/var/tmp/sage/build/ntl-6.2.1.p0/src/libtool/libtool --mode=link g++ -I../include -I.  -O2   -o libntl.la FFT.lo FacVec.lo GF2.lo GF2E.lo GF2EX.lo GF2EXFactoring.lo GF2X.lo GF2X1.lo GF2XFactoring.lo GF2XVec.lo GetTime.lo HNF.lo ctools.lo LLL.lo LLL_FP.lo LLL_QP.lo LLL_RR.lo LLL_XD.lo RR.lo WordVector.lo ZZ.lo ZZVec.lo ZZX.lo ZZX1.lo ZZXCharPoly.lo ZZXFactoring.lo ZZ_p.lo ZZ_pE.lo ZZ_pEX.lo ZZ_pEXFactoring.lo ZZ_pX.lo ZZ_pX1.lo ZZ_pXCharPoly.lo ZZ_pXFactoring.lo fileio.lo lip.lo lzz_p.lo lzz_pE.lo lzz_pEX.lo lzz_pEXFactoring.lo lzz_pX.lo lzz_pX1.lo lzz_pXCharPoly.lo lzz_pXFactoring.lo mat_GF2.lo mat_GF2E.lo mat_RR.lo mat_ZZ.lo mat_ZZ_p.lo mat_ZZ_pE.lo mat_lzz_p.lo mat_lzz_pE.lo mat_poly_ZZ.lo mat_poly_ZZ_p.lo mat_poly_lzz_p.lo quad_float.lo tools.lo vec_GF2.lo vec_GF2E.lo vec_RR.lo vec_ZZ.lo vec_ZZ_p.lo vec_ZZ_pE.lo vec_lzz_p.lo vec_lzz_pE.lo xdouble.lo G_LLL_FP.lo G_LLL_QP.lo G_LLL_XD.lo G_LLL_RR.lo -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -lgmp  -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -lgf2x  -lm -rpath /home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -version-info `cat VERSION_INFO` #LSHAR
libtool: link: g++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/crtbeginS.o  .libs/FFT.o .libs/FacVec.o .libs/GF2.o .libs/GF2E.o .libs/GF2EX.o .libs/GF2EXFactoring.o .libs/GF2X.o .libs/GF2X1.o .libs/GF2XFactoring.o .libs/GF2XVec.o .libs/GetTime.o .libs/HNF.o .libs/ctools.o .libs/LLL.o .libs/LLL_FP.o .libs/LLL_QP.o .libs/LLL_RR.o .libs/LLL_XD.o .libs/RR.o .libs/WordVector.o .libs/ZZ.o .libs/ZZVec.o .libs/ZZX.o .libs/ZZX1.o .libs/ZZXCharPoly.o .libs/ZZXFactoring.o .libs/ZZ_p.o .libs/ZZ_pE.o .libs/ZZ_pEX.o .libs/ZZ_pEXFactoring.o .libs/ZZ_pX.o .libs/ZZ_pX1.o .libs/ZZ_pXCharPoly.o .libs/ZZ_pXFactoring.o .libs/fileio.o .libs/lip.o .libs/lzz_p.o .libs/lzz_pE.o .libs/lzz_pEX.o .libs/lzz_pEXFactoring.o .libs/lzz_pX.o .libs/lzz_pX1.o .libs/lzz_pXCharPoly.o .libs/lzz_pXFactoring.o .libs/mat_GF2.o .libs/mat_GF2E.o .libs/mat_RR.o .libs/mat_ZZ.o .libs/mat_ZZ_p.o .libs/mat_ZZ_pE.o .libs/mat_lzz_p.o .libs/mat_lzz_pE.o .libs/mat_poly_ZZ.o .libs/mat_poly_ZZ_p.o .libs/mat_poly_lzz_p.o .libs/quad_float.o .libs/tools.o .libs/vec_GF2.o .libs/vec_GF2E.o .libs/vec_RR.o .libs/vec_ZZ.o .libs/vec_ZZ_p.o .libs/vec_ZZ_pE.o .libs/vec_lzz_p.o .libs/vec_lzz_pE.o .libs/xdouble.o .libs/G_LLL_FP.o .libs/G_LLL_QP.o .libs/G_LLL_XD.o .libs/G_LLL_RR.o   -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -lgmp -lgf2x -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../lib64/crtn.o  -O2   -Wl,-soname -Wl,libntl.so.5 -o .libs/libntl.so.5.0.0

[fbissey]

OK pushed and updated tarball let's see if I did this right.

---

New commits:

​5fdce94

Refreshing patches and new configure.ac to generate a libtool that supports c++ properly.

[jdemeyer]

Changing only configure.ac obviously does nothing, you also need to add patches for the generated files.

[fbissey]

As far as your problem with gcc 4.6.4 goes, it indeed does nothing. That was not quite the point. I don't need to add the generated files because this is not a pristine source, it includes the generated files in a separate directory as did its predecessors.

[fbissey]

I also looked at the diff between 6.1.0 and 6.2.1 and I don't think I would try to touch anything to make it work with an earlier g++ there is too much all over the place.

[jpflori]

If I understand correctly NTL 6.2.x needs GCC 4.7.x at least? If so I can report to Shoup.

[fbissey]

That's our conclusion indeed.

[jpflori]

Report transmitted to Shoup.

[jdemeyer]

Replying to jpflori:

Report transmitted to Shoup.

Good, I'm curious what the reply will be...

[jdemeyer]

Also: if it's decided that GCC 4.7.x is needed, that will require a discussion on sage-devel and a new ticket to implement this.

[jpflori]

Shoup replied he uses an old GCC 4.2.1 without problems and would like to have more details... So we need to conduct further investigation. But maybe he's not trying to use the problematic symbol. Can you provide a one liner to check his lib is correctly built with the old GCC?

[jdemeyer]

Replying to jpflori:

Shoup replied he uses an old GCC 4.2.1 without problems

Did he try to compile Singular on top of NTL? Because NTL itself always compiles.

[jpflori]

Would asking Shoup to run

objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength

on his libntl be a sensible question?

[jpflori]

Here is the feedboack from Victor:

This new issue seems to be related to template instantiation and linking,
which admittedly is a potentially murky area.  I don't think there is
an NTL bug…but it's hard to say.  I haven't tested NTL recently
as a dynamic library -- I almost always use static libraries, just because
it is simpler.  I could imagine that there is some subtle interaction between
template instantiation and dynamic library loading…since it seems
the problem goes away in some versions of gcc, it could be a gcc problem.
If you can send me a simple test that illustrates the problem that would be useful.
If there is anything in the code or the NTL build scrips that would help with this,
I'd be glad to help.

To clarify one point: NTL makes no use of any features not present in the C++98.
standard, and it is only within the last couple of years that I have much 
confidence that C++98 is widely and correctly implemented.

and just now:

I looked for closely at the ticket.
I see that the code is compiled with the -fno-implicit-templates
flag.  I'm not sure, but that could be a source of the problem...

[fbissey]

That's singular that is using -fno-implicit-templates. It would be interesting to check if libntl.a is usable. I found the argument about .a compared to .so a bit specious. Symbols can be checked in .a library with "nm" there is just no option to get the non hashed names in the output. But it looks like piping through c++filt works

nm libntl.a | c++filt

nm also works with .so files if you use the -D switch.

[jdemeyer]

I highly doubt that static vs. dynamic linking is the issue here. If the symbol isn't in the .so file, there is no reason why it should be in the .a file.

[jdemeyer]

The latest version actually fails:

Applying patches to NTL.
patching file src/tools.c
patching file include/NTL/tools.h
patching file src/DoConfig
patching file src/mfile
patching file src/tools.c
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file src/tools.c.rej
patching file include/NTL/tools.h
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/NTL/tools.h.rej
Error applying '/usr/local/src/sage-config/local/var/tmp/sage/build/ntl-6.2.1.p0/patches/make.patch'.

Also: applying patches after running ./configure makes no sense, can you change that?

[fbissey]

Looks like I have a duplicate patch, I will fix this and patch first as you ask.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2d3584f

Correct patch and change ordering between libtool generation and patching in spkg-install

[fbissey]

I must have done a wrong cp somewhere, make.patch was identical to errorcallback.patch. All fixed as per your request.

[jdemeyer]

Still doesn't work:

Applying patches to NTL.   
patching file src/tools.c  
patching file include/NTL/tools.h
patching file src/DoConfig 
patching file src/mfile
patching file src/mfile
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file src/mfile.rej
patching file src/WizardAux
Error applying '/home/jdemeyer/sage-git/local/var/tmp/sage/build/ntl-6.2.1.p0/patches/make.patch'.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e2623ec

Reseparate make.patch properly from the libtool patch

[fbissey]

OK I tested the patches apply properly this time.

[fbissey]

Any news on this? Also we may want to check if the problem is gcc-4.6.x only. I may spend a bit of time investigating this week end.

[jpflori]

Not on my side. I just came back to work this week and had a bunch of other stuff to do, some of them were even Sage related.

I'm sorry, a lot has been said here, but the ticket is getting hairy and I would like that we properly define what the problem is and how to detect it (other than, try to build Singular on top and NTL and see if it fails). What's not completely clear to me is what the problem actually is when using GCC 4.6.3 (and potentially prior versions:

• is a symbol not getting defined in NTL with some GCC version and templating magic? as Singular needs that symbol it fails to build, and we can just detect the problematic GCC versions by using nm/objdump on the produced NTL .so/.a files.
• is this a problem with template instantiation magic when Singular gets built, so nothing is actually missing from the NTL .so file, but there is some issue with let's say the header of NTL or the way Singular uses them, or the flags it passes to the compiler/linker? (that does not make a lot of sense, but as the situation is not clear to me, I'm just putting here all that goes through my head).

So in the first hypothesis which looks the more reasonable to me, what is the missing symbol? Can we produce a three lines C file which would fail to link against NTL as Singular does?

I might personally be able to test GCC series between 4.6 and 4.9 on Linux x86_64, maybe ppc64, vanilla FSF and Debian versions. Not sure I'll have the time to twiddle with pre 4.5 versions. What about clang? :)

[fbissey]

There are several symbols missing but the ones that cause singular to fail are the following

NTL::Vec<NTL::ZZ>::DoSetLength(long)
NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
NTL::Vec<NTL::zz_pX>::DoSetLength(long)

c++ and me don't mix well so I wouldn't try to write a small test program for these but it should be easy for someone who has a clue about c++. Which also means that I have no idea on how it fails to build. I should be able to test gcc 4.3 easily on ppc64 since sles11sp1 comes with that as default.

[jpflori]

Ok, so I just built NTL 6.2.1 on a x86_64 Ubuntu 14.04.1 LTS using the available Linaro GCC (4.4, 4.6, 4.7, 4.8) and can confirm that:

• 4.7 and 4.8 yield a whole bunch of DoSetLength? symbols in static and shared libs,
• 4.4 and 4.6 don't.

I'm now trying other versions of NTL to see what happens.

[jpflori]

Random pointers to investigate:

• ​https://gcc.gnu.org/ml/gcc-help/2012-08/msg00280.html
• ​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48883
• ​https://gcc.gnu.org/onlinedocs/gcc/Template-Instantiation.html

[jpflori]

As far as NTL versions (with g++-4.6) are concerned, and as expected:

• 6.1.0 gets the bunch of symbols;
• 6.2.0 and 6.2.1 don't.

[jpflori]

A little more info:

• with Debian's GCC 4.4 and 4.6, only a few symbols get defined;
• with Debian's GCC 4.7, 4.8 and 4.9, a lof of symbols get defined;
• with FSF's GCC 4.3, only a few symbols get defined;
• with FSF's GCC 4.2 and 4.5, a lot of symbols get defined.

So it's kind of unfortunate that GCC behavior varies in that way.

Whatsoever, removing the -fno-implicit-templates flag from Singular 3-1-6 build system, let it link to a libntl.so without all the explicit DoSetLength symbols.

So I guess the proper fix would be to explicitly list the needed templates in Singular sources (the NTL6 compatibility patch does already this kind of stuff), I'm now trying this solution. Or we could just get rid of the -fno-implicit-templates flag from Singular 3-1-6 build system. Or do you think, we should make sure NTL always define the whole bunch of symbols.

Not sure how singular-4.x behaves.

[jpflori]

Let Singular instantiate NTL templates

[jpflori]

(Note this is a 3-1-7 based patch.)

[jpflori]

Last 10 new commits:

​311a52c	Update NTL to version 6.2.1.
​b67d968	Fix checksum for NTL.
​071b254	Update Singular to 3-1-7.
​13760d8	Let Singular instantiate NTL templates.
​5fdce94	Refreshing patches and new configure.ac to generate a libtool that supports c++ properly.
​2d3584f	Correct patch and change ordering between libtool generation and patching in spkg-install
​e2623ec	Reseparate make.patch properly from the libtool patch
​86b0663	Merge remote-tracking branch 'trac/u/fbissey/ntl-6.2.1' into ticket/16882
​8b5a659	Merge remote-tracking branch 'trac/develop' into ticket/16882
​5a11868	Rebase Singular patches.

[jpflori]

FYI, grepping through the Singular 4.0.1 sources for no-implicit yields nothing.

So the use of that flag in the 3-1-x branch is surely an "historical artifact".

[jpflori]

And I have no problem building Singular 4-0-1 on top of a lightweight NTL in a Sage shell (without any patches).

[jpflori]

I definitely think that not passing -fno-implicit-templates is the way to go. It's useless to fight with a soon to be obsolete Singular version.

[jdemeyer]

Does Singular-3-1-7 require NTL-6.2.1? If not, I strongly suggest to split off the Singular upgrade to a different ticket (2 spkg upgrades on one ticket sounds scary).

[jpflori]

As far as I saw, Singular 3-1-7 is mainly bug fixes and in particular inclusion of Sage patches... And I wanted to see if it changed anything as far as the templates issue was concerned. It did not, but the update was done. Feel free to split it apart, but on my side I've fought enough for this ticket. It should not be that hard. Just keep the templates.patch here (note that you could also get a similar efect playing with env vars.

[jdemeyer]

What is the "submit upstream" status of the ntl62.patch for FLINT?

---

New commits:

​7c3f691	Upgrade to Singular-3-1-7
​b21d42c	Upgrade to NTL 6.2.1

[fbissey]

Replying to jdemeyer:

What is the "submit upstream" status of the ntl62.patch for FLINT?

---

New commits:

​7c3f691	Upgrade to Singular-3-1-7
​b21d42c	Upgrade to NTL 6.2.1

It has been merged into trunk but there are no release containing it at this time.

[jpflori]

I see you added doc fir the lazy template instantiation patch, which I forgot to do, but put it in NTL's SPKG.txt instead of Singular's one, and with a wrong name (template_fix.patch instead of templates.patch).

Thanks for splitting the updates into two tickets.

[jdemeyer]

Replying to jpflori:

I see you added doc fir the lazy template instantiation patch, which I forgot to do, but put it in NTL's SPKG.txt instead of Singular's one, and with a wrong name (template_fix.patch instead of templates.patch).

I didn't add anything, I literally just split up the ticket in two.

[jpflori]

Oh right, the template_fix.patch thing mentioned in NTL's SPKG.txt is a leftover of the fix needed for 6.2.0 which was included into 6.2.1.

[jpflori]

Doc updated.

---

Last 10 new commits:

​071b254	Update Singular to 3-1-7.
​13760d8	Let Singular instantiate NTL templates.
​5fdce94	Refreshing patches and new configure.ac to generate a libtool that supports c++ properly.
​2d3584f	Correct patch and change ordering between libtool generation and patching in spkg-install
​e2623ec	Reseparate make.patch properly from the libtool patch
​86b0663	Merge remote-tracking branch 'trac/u/fbissey/ntl-6.2.1' into ticket/16882
​8b5a659	Merge remote-tracking branch 'trac/develop' into ticket/16882
​5a11868	Rebase Singular patches.
​7157fa9	Update patch documentation for NTL and Singular.
​840aebf	Merge remote-tracking branch 'trac/u/jdemeyer/ticket/16882' into ticket/16882

[jdemeyer]

NTL + Singular builds fine now on the machine where it failed before.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1027b8f	Document templates.patch
​3968376	Update patches for Singular 3-1-7.
​f70af8c	Add new alias for Singular debug memory allocation.
​5f76994	Merge branch 'ticket/17184' into ticekt/16882
​cdedfc6	Correct fuzz for Singular patches.
​ba3aec7	Merge branch 'ticket/17184' into ticket/16882

[jpflori]

I can also confirm it builds in normal and debug modes with nice and mean versions of GCC.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​30b9594	Avoid redefining Singular mock ring.
​32f3be6	Avoid redefining Singular mock ring.
​7c1fdea	Merge branch 'ticket/17184' into ticket/16882

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3d1a298

Remove debug stuff.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​47ea250

Update doctest for Singular new behavior.

[vbraun]

lgtm