Opened 5 years ago

Closed 5 years ago

#16882 closed enhancement (fixed)

Upgrade to NTL 6.2.1

Reported by: jpflori Owned by:
Priority: major Milestone: sage-6.4
Component: packages: standard Keywords: spkg ntl
Cc: snark, cremona Merged in:
Authors: Jean-Pierre Flori, François Bissey Reviewers: Volker Braun
Report Upstream: N/A Work issues:
Branch: 47ea250 (Commits) Commit: 47ea2504aff21da6e7f40ecb0265a8b81ecd249d
Dependencies: #17184 Stopgaps:

Attachments (3)

ntl-6.2.1.p0.log (75.5 KB) - added by jdemeyer 5 years ago.
Full log of NTL build
singular-3.1.6.p3.log (629.9 KB) - added by jdemeyer 5 years ago.
Full log of failed Singular build
templates.patch (808 bytes) - added by jpflori 5 years ago.
Let Singular instantiate NTL templates

Download all attachments as: .zip

Change History (117)

comment:1 Changed 5 years ago by jpflori

  • Branch set to u/jpflori/ticket/16882
  • Description modified (diff)

comment:2 Changed 5 years ago by jpflori

  • Description modified (diff)

Our patches need to be updated, working on it.

comment:3 Changed 5 years ago by git

  • Commit set to 3fc9def85f68b23a62614c0c0981590685db137c

Branch pushed to git repo; I updated commit sha1. New commits:

3fc9defUpdate NTL to version 6.2.0.

comment:4 Changed 5 years ago by jpflori

  • Authors set to Jean-Pierre Flori
  • Description modified (diff)
  • Keywords spkg ntl added
  • Status changed from new to needs_review

At least it builds. Full testsuite not run yet (nor strange install like cygwin tested).

If you wan to try it out, please use SAGE_UPGRADING=yes, as it seems to me the dynamic lib version number got updated once again.


New commits:

3fc9defUpdate NTL to version 6.2.0.

comment:5 follow-up: Changed 5 years ago by jpflori

  • Status changed from needs_review to needs_work

It seems FLINT<->NTL interface (shipped by FLINT) is broken.

comment:6 in reply to: ↑ 5 Changed 5 years ago by cremona

Replying to jpflori:

It seems FLINT<->NTL interface (shipped by FLINT) is broken.

How annoying. Will you inform flint-devel?

comment:7 Changed 5 years ago by jpflori

I guess it's trivial. Some private stuff must have been made public by Victor (or so I hope).

And yes I made a minimal report at:

comment:8 Changed 5 years ago by jpflori

Quite unrelated, but we should update flint anyway, I think we still ship the version with the broken is_prime stuff.

comment:9 Changed 5 years ago by git

  • Commit changed from 3fc9def85f68b23a62614c0c0981590685db137c to a6b95acccbd0202d02bd235dfbe2de4496e91334

Branch pushed to git repo; I updated commit sha1. New commits:

98e68c7Let FLINT build against NTL 6.2.
a6b95acUpdate FLINT patch revision.

comment:10 Changed 5 years ago by jpflori

  • Status changed from needs_work to needs_review

Tentative fix pushed. (Nothing really tested yet though.)

comment:11 Changed 5 years ago by jpflori

  • Status changed from needs_review to needs_work

And Singular broke as well...

comment:12 Changed 5 years ago by jpflori

And that might be an NTL issue in fact.

comment:13 Changed 5 years ago by jpflori

Forwarded the problem to Shoup.

comment:14 Changed 5 years ago by git

  • Commit changed from a6b95acccbd0202d02bd235dfbe2de4496e91334 to 8d603048705ffd1c2671739a535b8ab713cb5eed

Branch pushed to git repo; I updated commit sha1. New commits:

8d60304Fix some template issue with NTL 6.2.0.

comment:15 Changed 5 years ago by jpflori

  • Status changed from needs_work to needs_review

Fix from Victor included.

comment:16 Changed 5 years ago by jpflori

(And tested by he and I, and there should be an NTL bugfix release soon.)

comment:17 Changed 5 years ago by jpflori

  • Description modified (diff)
  • Status changed from needs_review to needs_work
  • Summary changed from Update to NTL 6.2 to Update to NTL 6.2.1

Victor just released a bugfix release, that leverages the need for the fix I took from his mails.

comment:18 Changed 5 years ago by git

  • Commit changed from 8d603048705ffd1c2671739a535b8ab713cb5eed to 311a52ce5bc9d9b74af21aeb93ae87e69567bc1b

Branch pushed to git repo; I updated commit sha1. New commits:

311a52cUpdate NTL to version 6.2.1.

comment:19 Changed 5 years ago by git

  • Commit changed from 311a52ce5bc9d9b74af21aeb93ae87e69567bc1b to b67d968eeb957854a0e1f1e5906bb61ec48e8a5e

Branch pushed to git repo; I updated commit sha1. New commits:

b67d968Fix checksum for NTL.

comment:20 Changed 5 years ago by jpflori

  • Description modified (diff)
  • Status changed from needs_work to needs_review

comment:21 Changed 5 years ago by fbissey

Does flint still needs to be fixed or did you forget to revert it?

comment:22 Changed 5 years ago by jpflori

Nope it still needs to be fixed.

comment:23 follow-up: Changed 5 years ago by fbissey

OK I got two questions then:

  • is the fix backward compatible with ntl-6.1/6.0
  • is flint planning a new release to include the fix? In the past Bill has been very fast to include stuff like that in a release.

comment:24 in reply to: ↑ 23 ; follow-up: Changed 5 years ago by jpflori

Replying to fbissey:

OK I got two questions then:

  • is the fix backward compatible with ntl-6.1/6.0

No idea.

  • is flint planning a new release to include the fix? In the past Bill has been very fast to include stuff like that in a release.

I guess so. I've already posted on flint-devel but Bill did not announce anything. And I fear hes quite busy right now.

comment:25 in reply to: ↑ 24 Changed 5 years ago by fbissey

Replying to jpflori:

Replying to fbissey:

OK I got two questions then:

  • is the fix backward compatible with ntl-6.1/6.0

No idea.

It looks like the answer is yes, which means that flint doesn't have to check the version of ntl which will make the inclusion of a fix in flint so much easier.

comment:26 Changed 5 years ago by fbissey

I will create a pull request for flint (hopefully today) tell me if I am duplicating work that you are ready to push. Once that's done I will test things a bit further but if the patchbot is happy this ticket is done.

comment:27 Changed 5 years ago by fbissey

Pull request for flint done: https://github.com/wbhart/flint2/pull/95

comment:28 Changed 5 years ago by jpflori

Thanks for the pull request. I'll be away from the internet for a few days, so i's very welcome.

comment:29 Changed 5 years ago by jdemeyer

  • Status changed from needs_review to needs_work

This causes a build error in Singular for me:

g++ -O2 -g  -fPIC -I.. -I/usr/local/src/sage-git/local -pipe -I. -I.. -I/usr/local/src/sage-git/local -I/usr/local/src/sage-git/local/include -I/usr/local/src/sage-git/local/include   -fno-implicit-templates -I.. -I/usr/local/src/sage-git/local -DNDEBUG -DOM_NDEBUG -Dx86_64_Linux -DHAVE_CONFIG_H \
  -o Singular \
  tesths.cc iparith.o mpsr_Tok.o claptmpl.o\
  grammar.o scanner.o attrib.o blackbox.o eigenval_ip.o extra.o fehelp.o feOpt.o ipassign.o ipconv.o ipid.o iplib.o ipprint.o ipshell.o newstruct.o lists.o sdb.o fglm.o interpolation.o silink.o ssiLink.o s_buff.o subexpr.o janet.o wrapper.o libparse.o sing_win.o gms.o pcv.o maps_ip.o walk.o walk_ip.o cntrlc.o misc_ip.o calcSVD.o pipeLink.o Minor.o MinorProcessor.o MinorInterface.o bigintm.o pyobject_setup.o denom_list.o minpoly.o countedref.o semaphore.o slInit_Dynamic.o -rdynamic -L/usr/local/src/sage-git/local/kernel -L../kernel -lkernel -L/usr/local/src/sage-git/local/lib  -L/usr/local/src/sage-git/local/lib -lflint -lmpfr -lmpir  -ldl -lm -lsingfac -lsingcf -lflint -lmpfr -lntl -lgmp -lreadline -lrt -lnsl -lm  -lnsl  -lomalloc -lpthread  ../kernel/mmalloc.o 
/usr/local/src/sage-git/local/lib/libsingcf.a(cfNewtonPolygon.o): In function `NTL::Vec<NTL::ZZ>::SetLength(long)':
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::ZZ>::DoSetLength(long)'
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::ZZ>::DoSetLength(long)'
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::ZZ>::DoSetLength(long)'
/usr/local/src/sage-git/local/lib/libsingcf.a(facHensel.o): In function `NTL::Vec<NTL::zz_pEX>::SetLength(long)':
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::zz_pEX>::DoSetLength(long)'
/usr/local/src/sage-git/local/lib/libsingcf.a(facNTLzzpEXGCD.o): In function `NTL::Vec<NTL::zz_pX>::SetLength(long)':
/usr/local/src/sage-git/local/var/tmp/sage/build/singular-3.1.6.p3/src/latest/factory/../../../../../../../../include/NTL/vector.h:129: undefined reference to `NTL::Vec<NTL::zz_pX>::DoSetLength(long)'
collect2: ld returned 1 exit status
make[4]: *** [Singular] Error 1

comment:30 follow-up: Changed 5 years ago by jpflori

Wasn't that the issue fixed between 6.2.0 and 6.2.1?

comment:31 in reply to: ↑ 30 Changed 5 years ago by fbissey

Replying to jpflori:

Wasn't that the issue fixed between 6.2.0 and 6.2.1?

You would be the only one to know, you never posted your singular problem with 6.2.0.

comment:32 Changed 5 years ago by jpflori

Euh, yes indeed. This looks different as my problem was with template instantiation. Here there is an undefined reference. What tricked me earlier is that the problematic function is the same.

For the record, here is my report to Victor Shoup: I've been playing for a few moments with NTL 6.2.0 and it looks really nice. Nonetheless I don't seem able to build singular 3-6-0 on top of it.

Indeed g++ spits: {{{../../../../../../../../include/NTL/vector.h:427:4: error: no matching function for call to 'NTL::Vec<NTL::Pair<NTL::GF2X, long int>

::Init(long int&, const NTL::Pair<NTL::GF2X, long int>&, const

NTL::Pair<NTL::GF2X, long int>&)' ../../../../../../../../include/NTL/vector.h:427:4: note: candidates are: ../../../../../../../../include/NTL/vector.h:379:6: note: void NTL::Vec<T>::Init(long int) [with T = NTL::Pair<NTL::GF2X, long int>] ../../../../../../../../include/NTL/vector.h:379:6: note: candidate expects 1 argument, 3 provided ../../../../../../../../include/NTL/vector.h:389:6: note: void NTL::Vec<T>::Init(long int, const T*) [with T = NTL::Pair<NTL::GF2X, long int>] ../../../../../../../../include/NTL/vector.h:389:6: note: candidate expects 2 arguments, 3 provided ../../../../../../../../include/NTL/vector.h:400:6: note: void NTL::Vec<T>::Init(long int, const T&) [with T = NTL::Pair<NTL::GF2X, long int>] ../../../../../../../../include/NTL/vector.h:400:6: note: candidate expects 2 arguments, 3 provided }}}

And looking at the new vector.h from NTL, on line 427, there is indeed a call to: Init(n, a, *src); within the DoSetLength? definition.

And no corresponding function in vector.h (whereas there are indeed three definitions for Init with different signatures.)

Am I missing something here?

Best regards,

comment:33 Changed 5 years ago by fbissey

Not sure you are missing something. C++ always look like spaghetti to me (in a different way from classic FORTRAN - FORTRAN was abstract spaghetti, C++ is litteral as far as I am concerned). I could be that "DoSetLength?" is actually defined in another header and it is not properly included before vector.h or in vector.h itself.

comment:34 Changed 5 years ago by fbissey

Well a local build here shows

objdump -T -C /scratch/portage/dev-libs/ntl-6.2.1/image/usr/lib64/libntl.so.5.0.0 | grep DoSetLength
00000000000b8c26  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
00000000000973fc g    DF .text  00000000000001b9  Base        NTL::WordVector::DoSetLength(long)
000000000005aac6  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::GF2EX, long> >::DoSetLength(long)
00000000001015c0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
00000000000b7af2  w   DF .text  000000000000005a  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000eb136  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pX, long> >::DoSetLength(long)
000000000009d610  w   DF .text  0000000000000038  Base        NTL::Vec<char>::DoSetLength(long)
0000000000070aae  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::GF2X, long> >::DoSetLength(long)
000000000013246a  w   DF .text  0000000000000089  Base        NTL::Vec<NTL::ZZVec>::DoSetLength(long)
00000000000845ca  w   DF .text  0000000000000080  Base        NTL::Vec<NTL::quad_float>::DoSetLength(long)
00000000000b8c98  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Vec<long> >::DoSetLength(long)
0000000000090854  w   DF .text  0000000000000083  Base        NTL::Vec<NTL::xdouble>::DoSetLength(long)
0000000000122424  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::zz_pX, long> >::DoSetLength(long)
000000000004495e  w   DF .text  0000000000000038  Base        NTL::Vec<unsigned long>::DoSetLength(long)
0000000000052128  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
00000000000c78f0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)
0000000000101988  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
0000000000045350  w   DF .text  0000000000000038  Base        NTL::Vec<NTL::IntFactor>::DoSetLength(long)
000000000007534e  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)
00000000000b8a4c  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
000000000007bb40  w   DF .text  0000000000000038  Base        NTL::Vec<double>::DoSetLength(long)
000000000010814c  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::zz_pEX, long> >::DoSetLength(long)
000000000007b72a  w   DF .text  0000000000000082  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
00000000000b886c  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::ZZX, long> >::DoSetLength(long)
0000000000101e08  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
0000000000051b32  w   DF .text  000000000000005a  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
00000000000cdda6  w   DF .text  0000000000000077  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pEX, long> >::DoSetLength(long)
00000000000447be  w   DF .text  0000000000000038  Base        NTL::Vec<long>::DoSetLength(long)
000000000006e66a  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::GF2X>::DoSetLength(long)
00000000000753c0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)
00000000000b7ff6  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000c7470  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)
0000000000127b74  w   DF .text  0000000000000089  Base        NTL::Vec<NTL::GF2XVec>::DoSetLength(long)

Jeroen's failure was on three symbols:

NTL::Vec<NTL::ZZ>::DoSetLength(long)
NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
NTL::Vec<NTL::zz_pX>::DoSetLength(long)

And they are all present. So two possibilities I see:

  • Jeroen's build of ntl was incomplete
  • there is an ordering problem in the singular's linking

Jeroen can easily check the first one. The second would involve a little bit more experimentation and cut and paste.

I will try to reproduce this locally.

comment:35 Changed 5 years ago by fbissey

/usr/local/src/sage-git/local/lib/libsingcf.a(cfNewtonPolygon.o): In function 
`NTL::Vec<NTL::ZZ>::SetLength(long)':

We have interference from a previously installed singular. It gets the wrong libsingcf.a. Obvious once you look at it closely.

comment:36 follow-up: Changed 5 years ago by fbissey

After examination of what I am doing in sage-on-gentoo is there a need in sage for libsingcf.a and the other static archives installed? Is Macauley2 using them?

comment:37 in reply to: ↑ 36 Changed 5 years ago by fbissey

Replying to fbissey:

After examination of what I am doing in sage-on-gentoo is there a need in sage for libsingcf.a and the other static archives installed? Is Macauley2 using them?

Skip that. I forgot that singular install directly under SAGE_LOCAL while compiling. That and the fact the symbol are referenced in the library. Going back to testing.

comment:38 Changed 5 years ago by fbissey

Couldn't reproduce it so far.

comment:39 follow-up: Changed 5 years ago by jpflori

I would also say this comes from an old NTL install, but only Jeroen can confirm...

comment:40 Changed 5 years ago by jpflori

  • Status changed from needs_work to needs_info

comment:41 in reply to: ↑ 39 ; follow-up: Changed 5 years ago by fbissey

Replying to jpflori:

I would also say this comes from an old NTL install, but only Jeroen can confirm...

Nice theory but I just checked and these symbols are also in ntl 6.1.0. Unless of course it comes from a much older ntl (I could look into that if needed). So there is probably something more subtle at play. I would very much like Jeroen to check the symbols are in libntl and if they are we should look carefully at the full log from the singular build.

comment:42 Changed 5 years ago by jdemeyer

OK, I'm trying the build again...

comment:43 in reply to: ↑ 41 Changed 5 years ago by jpflori

Replying to fbissey:

Replying to jpflori:

I would also say this comes from an old NTL install, but only Jeroen can confirm...

Nice theory but I just checked and these symbols are also in ntl 6.1.0. Unless of course it comes from a much older ntl (I could look into that if needed). So there is probably something more subtle at play. I would very much like Jeroen to check the symbols are in libntl and if they are we should look carefully at the full log from the singular build.

Yeah, but this part of the templating stuff changed between 6.1.0 and 6.2.0, so it might also be the case that Jeroen built on top of the 6.2.0 version with broken lazy templates initialization...

comment:44 follow-up: Changed 5 years ago by jdemeyer

I rebuilt and it failed again. The output of the analogous objdump command is:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000a4d30 g    DF .text  00000000000001e9  Base        NTL::WordVector::DoSetLength(long)

FYI:

(sage-sh)$ find local -name libntl* |xargs ls -l
-rw-r--r-- 1 jdemeyer jdemeyer 37380910 Oct  8 10:41 local/lib/libntl.a
-rw-r--r-- 1 jdemeyer jdemeyer     1058 Oct  8 10:41 local/lib/libntl.la
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so -> libntl.so.5.0.0
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so.5 -> libntl.so.5.0.0
-rwxr-xr-x 1 jdemeyer jdemeyer 14786659 Oct  8 10:41 local/lib/libntl.so.5.0.0

comment:45 Changed 5 years ago by jdemeyer

Also FYI:

(sage-sh)$ uname -a
Linux tamiyo 3.7.10-gentoo #4 SMP PREEMPT Thu Oct 31 21:23:56 CET 2013 x86_64 Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz GenuineIntel GNU/Linux
(sage-sh)$ g++ --version
g++ (Gentoo 4.6.4 p1.2, pie-0.5.2) 4.6.4
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
(sage-sh)$ ld --version
GNU ld (GNU Binutils) 2.22
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
Last edited 5 years ago by jdemeyer (previous) (diff)

Changed 5 years ago by jdemeyer

Full log of NTL build

Changed 5 years ago by jdemeyer

Full log of failed Singular build

comment:46 in reply to: ↑ 44 ; follow-up: Changed 5 years ago by fbissey

Replying to jdemeyer:

I rebuilt and it failed again. The output of the analogous objdump command is:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000a4d30 g    DF .text  00000000000001e9  Base        NTL::WordVector::DoSetLength(long)

FYI:

(sage-sh)$ find local -name libntl* |xargs ls -l
-rw-r--r-- 1 jdemeyer jdemeyer 37380910 Oct  8 10:41 local/lib/libntl.a
-rw-r--r-- 1 jdemeyer jdemeyer     1058 Oct  8 10:41 local/lib/libntl.la
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so -> libntl.so.5.0.0
lrwxrwxrwx 1 jdemeyer jdemeyer       15 Oct  8 10:41 local/lib/libntl.so.5 -> libntl.so.5.0.0
-rwxr-xr-x 1 jdemeyer jdemeyer 14786659 Oct  8 10:41 local/lib/libntl.so.5.0.0

OK so it looks like libntl is not built properly, although I have no idea where there is room for variation. At this stage I think we need the build log for ntl.

comment:47 in reply to: ↑ 46 Changed 5 years ago by jdemeyer

Replying to fbissey:

At this stage I think we need the build log for ntl.

It's attached on this ticket.

comment:48 Changed 5 years ago by fbissey

By the look of it you posted the logs a few seconds before me. Just as an aside I just noticed that libntl.so is currently linked with gcc rather than g++. Not fatal but it leaves a lot of libstdc++ symbols undefined. I'll see if I can come up with something for that tomorrow.

comment:49 Changed 5 years ago by jdemeyer

With NTL-6.1.0, I get

(sage-sh)$ objdump -T -C local/lib/libntl.so.3.0.0 | grep DoSetLength
00000000000d3f60  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::ZZ_p> >::DoSetLength(long)
00000000000c2e80  w   DF .text  00000000000001f3  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000d3780  w   DF .text  0000000000000243  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)                                                                   
00000000000772b0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)                                                           
000000000005fcf0  w   DF .text  0000000000000247  Base        NTL::Vec<NTL::GF2XVec>::DoSetLength(long)                                                                 
00000000000738e0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)                                                                      
00000000000c3180  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
0000000000073b00  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::GF2X>::DoSetLength(long)
00000000000fae10  w   DF .text  0000000000000247  Base        NTL::Vec<NTL::ZZVec>::DoSetLength(long)
0000000000047820  w   DF .text  00000000000001d3  Base        NTL::Vec<double>::DoSetLength(long)
0000000000075fe0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::GF2X, long> >::DoSetLength(long)
0000000000113a60  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::zz_p> >::DoSetLength(long)
0000000000154ba0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::zz_pE> >::DoSetLength(long)
00000000000c2c60  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
0000000000047460  w   DF .text  00000000000001d3  Base        NTL::Vec<unsigned long>::DoSetLength(long)
00000000000c37b0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
0000000000047640  w   DF .text  00000000000001d3  Base        NTL::Vec<NTL::FFTPrimeInfo*>::DoSetLength(long)
0000000000047c20  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<unsigned long> >::DoSetLength(long)
000000000011a1c0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::zz_pEX, long> >::DoSetLength(long)
000000000009e460 g    DF .text  00000000000001e9  Base        NTL::WordVector::DoSetLength(long)
000000000005f8e0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::GF2EX, long> >::DoSetLength(long)
0000000000081680  w   DF .text  0000000000000214  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
0000000000047a00  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<long> >::DoSetLength(long)
000000000013d110  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::GF2E> >::DoSetLength(long)
00000000000faa00  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pX, long> >::DoSetLength(long)
0000000000055fb0  w   DF .text  00000000000001f3  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
00000000000566b0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
0000000000113840  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
00000000000c33a0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::ZZX, long> >::DoSetLength(long)
00000000000965c0  w   DF .text  0000000000000214  Base        NTL::Vec<NTL::xdouble>::DoSetLength(long)
0000000000113280  w   DF .text  0000000000000243  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
0000000000048100  w   DF .text  00000000000001dc  Base        NTL::Vec<NTL::IntFactor>::DoSetLength(long)
000000000014d500  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::ZZ_pE> >::DoSetLength(long)
00000000000da930  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pEX, long> >::DoSetLength(long)
00000000000568d0  w   DF .text  0000000000000247  Base        NTL::Vec<NTL::Vec<NTL::GF2> >::DoSetLength(long)
0000000000047280  w   DF .text  00000000000001d3  Base        NTL::Vec<long>::DoSetLength(long)
000000000008a4f0  w   DF .text  0000000000000214  Base        NTL::Vec<NTL::quad_float>::DoSetLength(long)
00000000000c3eb0  w   DF .text  0000000000000224  Base        NTL::Vec<NTL::Pair<NTL::zz_pX, long> >::DoSetLength(long)
00000000000818d0  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::Vec<NTL::RR> >::DoSetLength(long)
00000000000c3a00  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000d3d40  w   DF .text  000000000000021b  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)

comment:50 Changed 5 years ago by jdemeyer

@fbissey and @jpflori: does this ticket actually work for the both of you? If so, I'm surprised that it doesn't work for me...

comment:51 Changed 5 years ago by jdemeyer

Something else: patches should apply without fuzz:

Applying patches to NTL.
patching file src/tools.c
Hunk #1 succeeded at 18 with fuzz 1.
patching file include/NTL/tools.h
Hunk #2 succeeded at 311 (offset 58 lines).
patching file src/DoConfig
patching file src/mfile
patching file src/mfile
Hunk #1 succeeded at 325 (offset -6 lines).
Hunk #2 succeeded at 338 (offset -6 lines).
Hunk #3 succeeded at 358 (offset -9 lines).
Hunk #4 succeeded at 397 (offset -9 lines).
patching file src/WizardAux
Hunk #3 succeeded at 254 (offset 40 lines).
patching file include/NTL/new.h

comment:52 Changed 5 years ago by jdemeyer

With g++-4.8.3, things change:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000c6290  w   DF .text  0000000000000064  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000d82b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)
000000000007b780  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)
0000000000144980  w   DF .text  0000000000000080  Base        NTL::Vec<NTL::GF2XVec>::DoSetLength(long)
000000000007b710  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)
00000000000c7760  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
0000000000074050  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::GF2X>::DoSetLength(long)
0000000000082a90  w   DF .text  0000000000000036  Base        NTL::Vec<double>::DoSetLength(long)
00000000001199d0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
00000000000c7840  w   DF .text  0000000000000036  Base        NTL::Vec<unsigned long>::DoSetLength(long)
00000000000c7540  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
00000000000a0670 g    DF .text  0000000000000200  Base        NTL::WordVector::DoSetLength(long)
00000000000825d0  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
00000000000c77d0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::Vec<long> >::DoSetLength(long)
0000000000056770  w   DF .text  0000000000000064  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
0000000000056ea0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
000000000011a400  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
0000000000119f10  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
00000000000a75c0  w   DF .text  0000000000000036  Base        NTL::Vec<char>::DoSetLength(long)
000000000007b7f0  w   DF .text  0000000000000036  Base        NTL::Vec<long>::DoSetLength(long)
000000000008bdd0  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::quad_float>::DoSetLength(long)
00000000000c6a20  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000d87a0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)

comment:53 Changed 5 years ago by fbissey

OK, now that's weird, and yes it works for me and I re-built singular and flint after ntl. Haven't done sage however.

comment:54 Changed 5 years ago by jpflori

Yup, I've been using it on different systems (one redhat ppc64 and a ubuntu x86_64) for a while now... In fact the release of 6.2.1 over 6.2.0 was made because I tried to use 6.2.0 and it failed and I contacted Shoup who was very reactive...

comment:55 Changed 5 years ago by jdemeyer

With g++-4.7.3 from Sage:

(sage-sh)$ objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength
00000000000c6ea0  w   DF .text  0000000000000074  Base        NTL::Vec<NTL::ZZ_p>::DoSetLength(long)
00000000000d95b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pE>::DoSetLength(long)
00000000000c82b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::Vec<NTL::ZZ> >::DoSetLength(long)
000000000007e150  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ>::DoSetLength(long)
0000000000101a20  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pX>::DoSetLength(long)
00000000000852c0  w   DF .text  0000000000000046  Base        NTL::Vec<double>::DoSetLength(long)
0000000000079090  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::GF2X, long> >::DoSetLength(long)
000000000011afa0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_p>::DoSetLength(long)
0000000000049c40  w   DF .text  0000000000000046  Base        NTL::Vec<unsigned long>::DoSetLength(long)
000000000013f470  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pX>::DoSetLength(long)
0000000000122370  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::zz_pEX, long> >::DoSetLength(long)
00000000000a1f40 g    DF .text  0000000000000208  Base        NTL::WordVector::DoSetLength(long)
00000000000627b0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::GF2EX, long> >::DoSetLength(long)
0000000000084e30  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::RR>::DoSetLength(long)
0000000000058b70  w   DF .text  0000000000000074  Base        NTL::Vec<NTL::GF2E>::DoSetLength(long)
00000000001019a0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pX, long> >::DoSetLength(long)
0000000000062830  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::GF2EX>::DoSetLength(long)
00000000001223f0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
00000000000c80b0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::ZZX, long> >::DoSetLength(long)
000000000009a180  w   DF .text  0000000000000078  Base        NTL::Vec<NTL::xdouble>::DoSetLength(long)
000000000011b3b0  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::zz_pE>::DoSetLength(long)
000000000004a6e0  w   DF .text  0000000000000046  Base        NTL::Vec<NTL::IntFactor>::DoSetLength(long)
00000000000a9060  w   DF .text  0000000000000046  Base        NTL::Vec<char>::DoSetLength(long)
00000000000e06c0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::ZZ_pEX, long> >::DoSetLength(long)
0000000000049a70  w   DF .text  0000000000000046  Base        NTL::Vec<long>::DoSetLength(long)
000000000013f3f0  w   DF .text  0000000000000071  Base        NTL::Vec<NTL::Pair<NTL::zz_pX, long> >::DoSetLength(long)
00000000000c7730  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZX>::DoSetLength(long)
00000000000e0740  w   DF .text  0000000000000070  Base        NTL::Vec<NTL::ZZ_pEX>::DoSetLength(long)

comment:56 Changed 5 years ago by jdemeyer

With vanilla g++-4.6.4, same problem as with Gentoo's g++-4.6.4

I think that GCC 4.6.4 is too recent to not support it, so the DoSetLength problem should be fixed...

comment:57 follow-ups: Changed 5 years ago by fbissey

I am guessing the latest release of ntl relies on templating features from recent g++, probably 4.7.x. Basically we would almost require someone with gcc<4.7 to use something that is like ntl 6.1. I'll have a look at the diff between 6.1 and 6.2.1.

comment:58 in reply to: ↑ 57 Changed 5 years ago by jdemeyer

Replying to fbissey:

I am guessing the latest release of ntl relies on templating features from recent g++, probably 4.7.x.

I'm not entirely against requiring g++-4.7 (we do have the GCC spkg after all), but is upstream aware of this? Anyway, if an easy fix is possible for this issue, that would be the best.

comment:59 in reply to: ↑ 57 Changed 5 years ago by jdemeyer

Replying to fbissey:

probably 4.7.x.

Yes, as I posted before, g++-4.7.3 does work.

comment:60 Changed 5 years ago by fbissey

Jean-Pierre has contact upstream I guess he could ask if that is a known requirement. Regarding this ticket I would like to understand why the .so library is linked with gcc rather than g++ - like upstream intended. Upstream source indicate g++, my sage-on-gentoo ebuild uses g++. This spkg: gcc. What the...?

comment:61 Changed 5 years ago by fbissey

OK the configure script we currently ship is insufficient to generate a libtool that can link with g++. I am going to clean the spkg a bit later (yes I'll rebase the patches) but basically I will put this as the new configure.ac

AC_INIT(NTL, 6.2.1, victor@shoup.net)
AM_INIT_AUTOMAKE([1.9 foreign])
AC_CONFIG_FILES([Makefile])
LT_INIT
AC_PROG_CXX
AC_PROG_CC
AC_PROG_LIBTOOL
AC_OUTPUT

Looking for the C compiler is possibly over the top but it doesn't hurt. I grossly tested things and with that I now have a clean linking with g++. Before

/home/work/fbissey/sandbox/sage-6.4.beta4/local/var/tmp/sage/build/ntl-6.2.1.p0/src/libtool/libtool --mode=link g++ -I../include -I.  -O2 -g     -o libntl.la FFT.lo FacVec.lo GF2.lo GF2E.lo GF2EX.lo GF2EXFactoring.lo GF2X.lo GF2X1.lo GF2XFactoring.lo GF2XVec.lo GetTime.lo HNF.lo ctools.lo LLL.lo LLL_FP.lo LLL_QP.lo LLL_RR.lo LLL_XD.lo RR.lo WordVector.lo ZZ.lo ZZVec.lo ZZX.lo ZZX1.lo ZZXCharPoly.lo ZZXFactoring.lo ZZ_p.lo ZZ_pE.lo ZZ_pEX.lo ZZ_pEXFactoring.lo ZZ_pX.lo ZZ_pX1.lo ZZ_pXCharPoly.lo ZZ_pXFactoring.lo fileio.lo lip.lo lzz_p.lo lzz_pE.lo lzz_pEX.lo lzz_pEXFactoring.lo lzz_pX.lo lzz_pX1.lo lzz_pXCharPoly.lo lzz_pXFactoring.lo mat_GF2.lo mat_GF2E.lo mat_RR.lo mat_ZZ.lo mat_ZZ_p.lo mat_ZZ_pE.lo mat_lzz_p.lo mat_lzz_pE.lo mat_poly_ZZ.lo mat_poly_ZZ_p.lo mat_poly_lzz_p.lo quad_float.lo tools.lo vec_GF2.lo vec_GF2E.lo vec_RR.lo vec_ZZ.lo vec_ZZ_p.lo vec_ZZ_pE.lo vec_lzz_p.lo vec_lzz_pE.lo xdouble.lo G_LLL_FP.lo G_LLL_QP.lo G_LLL_XD.lo G_LLL_RR.lo -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -
lgmp  -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -lgf2x  -lm -rpath /home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -version-info `cat VERSION_INFO` #LSHAR
libtool: link: gcc -shared  -fPIC -DPIC  .libs/FFT.o .libs/FacVec.o .libs/GF2.o .libs/GF2E.o .libs/GF2EX.o .libs/GF2EXFactoring.o .libs/GF2X.o .libs/GF2X1.o .libs/GF2XFactoring.o .libs/GF2XVec.o .libs/GetTime.o .libs/HNF.o .libs/ctools.o .libs/LLL.o .libs/LLL_FP.o .libs/LLL_QP.o .libs/LLL_RR.o .libs/LLL_XD.o .libs/RR.o .libs/WordVector.o .libs/ZZ.o .libs/ZZVec.o .libs/ZZX.o .libs/ZZX1.o .libs/ZZXCharPoly.o .libs/ZZXFactoring.o .libs/ZZ_p.o .libs/ZZ_pE.o .libs/ZZ_pEX.o .libs/ZZ_pEXFactoring.o .libs/ZZ_pX.o .libs/ZZ_pX1.o .libs/ZZ_pXCharPoly.o .libs/ZZ_pXFactoring.o .libs/fileio.o .libs/lip.o .libs/lzz_p.o .libs/lzz_pE.o .libs/lzz_pEX.o .libs/lzz_pEXFactoring.o .libs/lzz_pX.o .libs/lzz_pX1.o .libs/lzz_pXCharPoly.o .libs/lzz_pXFactoring.o .libs/mat_GF2.o .libs/mat_GF2E.o .libs/mat_RR.o .libs/mat_ZZ.o .libs/mat_ZZ_p.o .libs/mat_ZZ_pE.o .libs/mat_lzz_p.o .libs/mat_lzz_pE.o .libs/mat_poly_ZZ.o .libs/mat_poly_ZZ_p.o .libs/mat_poly_lzz_p.o .libs/quad_float.o .libs/tools.o .libs/vec_GF2.o .libs/vec_GF2E.o .libs/vec_
RR.o .libs/vec_ZZ.o .libs/vec_ZZ_p.o .libs/vec_ZZ_pE.o .libs/vec_lzz_p.o .libs/vec_lzz_pE.o .libs/xdouble.o .libs/G_LLL_FP.o .libs/G_LLL_QP.o .libs/G_LLL_XD.o .libs/G_LLL_RR.o   -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -lgmp -lgf2x -lm  -O2   -Wl,-soname -Wl,libntl.so.5 -o .libs/libntl.so.5.0.0

After

/home/work/fbissey/sandbox/sage-6.4.beta4/local/var/tmp/sage/build/ntl-6.2.1.p0/src/libtool/libtool --mode=link g++ -I../include -I.  -O2   -o libntl.la FFT.lo FacVec.lo GF2.lo GF2E.lo GF2EX.lo GF2EXFactoring.lo GF2X.lo GF2X1.lo GF2XFactoring.lo GF2XVec.lo GetTime.lo HNF.lo ctools.lo LLL.lo LLL_FP.lo LLL_QP.lo LLL_RR.lo LLL_XD.lo RR.lo WordVector.lo ZZ.lo ZZVec.lo ZZX.lo ZZX1.lo ZZXCharPoly.lo ZZXFactoring.lo ZZ_p.lo ZZ_pE.lo ZZ_pEX.lo ZZ_pEXFactoring.lo ZZ_pX.lo ZZ_pX1.lo ZZ_pXCharPoly.lo ZZ_pXFactoring.lo fileio.lo lip.lo lzz_p.lo lzz_pE.lo lzz_pEX.lo lzz_pEXFactoring.lo lzz_pX.lo lzz_pX1.lo lzz_pXCharPoly.lo lzz_pXFactoring.lo mat_GF2.lo mat_GF2E.lo mat_RR.lo mat_ZZ.lo mat_ZZ_p.lo mat_ZZ_pE.lo mat_lzz_p.lo mat_lzz_pE.lo mat_poly_ZZ.lo mat_poly_ZZ_p.lo mat_poly_lzz_p.lo quad_float.lo tools.lo vec_GF2.lo vec_GF2E.lo vec_RR.lo vec_ZZ.lo vec_ZZ_p.lo vec_ZZ_pE.lo vec_lzz_p.lo vec_lzz_pE.lo xdouble.lo G_LLL_FP.lo G_LLL_QP.lo G_LLL_XD.lo G_LLL_RR.lo -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -lgmp  -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib  -lgf2x  -lm -rpath /home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -version-info `cat VERSION_INFO` #LSHAR
libtool: link: g++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/crtbeginS.o  .libs/FFT.o .libs/FacVec.o .libs/GF2.o .libs/GF2E.o .libs/GF2EX.o .libs/GF2EXFactoring.o .libs/GF2X.o .libs/GF2X1.o .libs/GF2XFactoring.o .libs/GF2XVec.o .libs/GetTime.o .libs/HNF.o .libs/ctools.o .libs/LLL.o .libs/LLL_FP.o .libs/LLL_QP.o .libs/LLL_RR.o .libs/LLL_XD.o .libs/RR.o .libs/WordVector.o .libs/ZZ.o .libs/ZZVec.o .libs/ZZX.o .libs/ZZX1.o .libs/ZZXCharPoly.o .libs/ZZXFactoring.o .libs/ZZ_p.o .libs/ZZ_pE.o .libs/ZZ_pEX.o .libs/ZZ_pEXFactoring.o .libs/ZZ_pX.o .libs/ZZ_pX1.o .libs/ZZ_pXCharPoly.o .libs/ZZ_pXFactoring.o .libs/fileio.o .libs/lip.o .libs/lzz_p.o .libs/lzz_pE.o .libs/lzz_pEX.o .libs/lzz_pEXFactoring.o .libs/lzz_pX.o .libs/lzz_pX1.o .libs/lzz_pXCharPoly.o .libs/lzz_pXFactoring.o .libs/mat_GF2.o .libs/mat_GF2E.o .libs/mat_RR.o .libs/mat_ZZ.o .libs/mat_ZZ_p.o .libs/mat_ZZ_pE.o .libs/mat_lzz_p.o .libs/mat_lzz_pE.o .libs/mat_poly_ZZ.o .libs/mat_poly_ZZ_p.o .libs/mat_poly_lzz_p.o .libs/quad_float.o .libs/tools.o .libs/vec_GF2.o .libs/vec_GF2E.o .libs/vec_RR.o .libs/vec_ZZ.o .libs/vec_ZZ_p.o .libs/vec_ZZ_pE.o .libs/vec_lzz_p.o .libs/vec_lzz_pE.o .libs/xdouble.o .libs/G_LLL_FP.o .libs/G_LLL_QP.o .libs/G_LLL_XD.o .libs/G_LLL_RR.o   -L/home/work/fbissey/sandbox/sage-6.4.beta4/local/lib -lgmp -lgf2x -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../lib64/crtn.o  -O2   -Wl,-soname -Wl,libntl.so.5 -o .libs/libntl.so.5.0.0

comment:62 Changed 5 years ago by fbissey

  • Branch changed from u/jpflori/ticket/16882 to u/fbissey/ntl-6.2.1
  • Commit changed from b67d968eeb957854a0e1f1e5906bb61ec48e8a5e to 5fdce9403048ed832b12e61982e816778fbdb7c2
  • Description modified (diff)

OK pushed and updated tarball let's see if I did this right.


New commits:

5fdce94Refreshing patches and new configure.ac to generate a libtool that supports c++ properly.

comment:63 Changed 5 years ago by jdemeyer

  • Status changed from needs_info to needs_work

Changing only configure.ac obviously does nothing, you also need to add patches for the generated files.

comment:64 Changed 5 years ago by fbissey

As far as your problem with gcc 4.6.4 goes, it indeed does nothing. That was not quite the point. I don't need to add the generated files because this is not a pristine source, it includes the generated files in a separate directory as did its predecessors.

comment:65 Changed 5 years ago by fbissey

I also looked at the diff between 6.1.0 and 6.2.1 and I don't think I would try to touch anything to make it work with an earlier g++ there is too much all over the place.

comment:66 Changed 5 years ago by jpflori

If I understand correctly NTL 6.2.x needs GCC 4.7.x at least? If so I can report to Shoup.

comment:67 Changed 5 years ago by fbissey

That's our conclusion indeed.

comment:68 follow-up: Changed 5 years ago by jpflori

Report transmitted to Shoup.

comment:69 in reply to: ↑ 68 Changed 5 years ago by jdemeyer

Replying to jpflori:

Report transmitted to Shoup.

Good, I'm curious what the reply will be...

comment:70 Changed 5 years ago by jdemeyer

Also: if it's decided that GCC 4.7.x is needed, that will require a discussion on sage-devel and a new ticket to implement this.

comment:71 follow-up: Changed 5 years ago by jpflori

Shoup replied he uses an old GCC 4.2.1 without problems and would like to have more details... So we need to conduct further investigation. But maybe he's not trying to use the problematic symbol. Can you provide a one liner to check his lib is correctly built with the old GCC?

comment:72 in reply to: ↑ 71 Changed 5 years ago by jdemeyer

Replying to jpflori:

Shoup replied he uses an old GCC 4.2.1 without problems

Did he try to compile Singular on top of NTL? Because NTL itself always compiles.

comment:73 Changed 5 years ago by jpflori

Would asking Shoup to run

objdump -T -C local/lib/libntl.so.5.0.0 | grep DoSetLength

on his libntl be a sensible question?

comment:74 Changed 5 years ago by jpflori

Here is the feedboack from Victor:

This new issue seems to be related to template instantiation and linking,
which admittedly is a potentially murky area.  I don't think there is
an NTL bug…but it's hard to say.  I haven't tested NTL recently
as a dynamic library -- I almost always use static libraries, just because
it is simpler.  I could imagine that there is some subtle interaction between
template instantiation and dynamic library loading…since it seems
the problem goes away in some versions of gcc, it could be a gcc problem.
If you can send me a simple test that illustrates the problem that would be useful.
If there is anything in the code or the NTL build scrips that would help with this,
I'd be glad to help.

To clarify one point: NTL makes no use of any features not present in the C++98.
standard, and it is only within the last couple of years that I have much 
confidence that C++98 is widely and correctly implemented.

and just now:

I looked for closely at the ticket.
I see that the code is compiled with the -fno-implicit-templates
flag.  I'm not sure, but that could be a source of the problem...

comment:75 Changed 5 years ago by fbissey

That's singular that is using -fno-implicit-templates. It would be interesting to check if libntl.a is usable. I found the argument about .a compared to .so a bit specious. Symbols can be checked in .a library with "nm" there is just no option to get the non hashed names in the output. But it looks like piping through c++filt works

nm libntl.a | c++filt

nm also works with .so files if you use the -D switch.

comment:76 Changed 5 years ago by jdemeyer

I highly doubt that static vs. dynamic linking is the issue here. If the symbol isn't in the .so file, there is no reason why it should be in the .a file.

comment:77 Changed 5 years ago by jdemeyer

The latest version actually fails:

Applying patches to NTL.
patching file src/tools.c
patching file include/NTL/tools.h
patching file src/DoConfig
patching file src/mfile
patching file src/tools.c
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file src/tools.c.rej
patching file include/NTL/tools.h
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/NTL/tools.h.rej
Error applying '/usr/local/src/sage-config/local/var/tmp/sage/build/ntl-6.2.1.p0/patches/make.patch'.

Also: applying patches after running ./configure makes no sense, can you change that?

comment:78 Changed 5 years ago by fbissey

Looks like I have a duplicate patch, I will fix this and patch first as you ask.

comment:79 Changed 5 years ago by git

  • Commit changed from 5fdce9403048ed832b12e61982e816778fbdb7c2 to 2d3584feed014ce09007933a972b9886ee566327

Branch pushed to git repo; I updated commit sha1. New commits:

2d3584fCorrect patch and change ordering between libtool generation and patching in spkg-install

comment:80 Changed 5 years ago by fbissey

I must have done a wrong cp somewhere, make.patch was identical to errorcallback.patch. All fixed as per your request.

comment:81 Changed 5 years ago by jdemeyer

Still doesn't work:

Applying patches to NTL.   
patching file src/tools.c  
patching file include/NTL/tools.h
patching file src/DoConfig 
patching file src/mfile
patching file src/mfile
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file src/mfile.rej
patching file src/WizardAux
Error applying '/home/jdemeyer/sage-git/local/var/tmp/sage/build/ntl-6.2.1.p0/patches/make.patch'.

comment:82 Changed 5 years ago by git

  • Commit changed from 2d3584feed014ce09007933a972b9886ee566327 to e2623ecfa6ec929f542765f2d0ca80a87dc762d0

Branch pushed to git repo; I updated commit sha1. New commits:

e2623ecReseparate make.patch properly from the libtool patch

comment:83 Changed 5 years ago by fbissey

OK I tested the patches apply properly this time.

comment:84 Changed 5 years ago by fbissey

Any news on this? Also we may want to check if the problem is gcc-4.6.x only. I may spend a bit of time investigating this week end.

comment:85 Changed 5 years ago by jpflori

Not on my side. I just came back to work this week and had a bunch of other stuff to do, some of them were even Sage related.

I'm sorry, a lot has been said here, but the ticket is getting hairy and I would like that we properly define what the problem is and how to detect it (other than, try to build Singular on top and NTL and see if it fails). What's not completely clear to me is what the problem actually is when using GCC 4.6.3 (and potentially prior versions:

  • is a symbol not getting defined in NTL with some GCC version and templating magic? as Singular needs that symbol it fails to build, and we can just detect the problematic GCC versions by using nm/objdump on the produced NTL .so/.a files.
  • is this a problem with template instantiation magic when Singular gets built, so nothing is actually missing from the NTL .so file, but there is some issue with let's say the header of NTL or the way Singular uses them, or the flags it passes to the compiler/linker? (that does not make a lot of sense, but as the situation is not clear to me, I'm just putting here all that goes through my head).

So in the first hypothesis which looks the more reasonable to me, what is the missing symbol? Can we produce a three lines C file which would fail to link against NTL as Singular does?

I might personally be able to test GCC series between 4.6 and 4.9 on Linux x86_64, maybe ppc64, vanilla FSF and Debian versions. Not sure I'll have the time to twiddle with pre 4.5 versions. What about clang? :)

comment:86 Changed 5 years ago by fbissey

There are several symbols missing but the ones that cause singular to fail are the following

NTL::Vec<NTL::ZZ>::DoSetLength(long)
NTL::Vec<NTL::zz_pEX>::DoSetLength(long)
NTL::Vec<NTL::zz_pX>::DoSetLength(long)

c++ and me don't mix well so I wouldn't try to write a small test program for these but it should be easy for someone who has a clue about c++. Which also means that I have no idea on how it fails to build. I should be able to test gcc 4.3 easily on ppc64 since sles11sp1 comes with that as default.

comment:87 Changed 5 years ago by jpflori

Ok, so I just built NTL 6.2.1 on a x86_64 Ubuntu 14.04.1 LTS using the available Linaro GCC (4.4, 4.6, 4.7, 4.8) and can confirm that:

  • 4.7 and 4.8 yield a whole bunch of DoSetLength? symbols in static and shared libs,
  • 4.4 and 4.6 don't.

I'm now trying other versions of NTL to see what happens.

comment:89 Changed 5 years ago by jpflori

As far as NTL versions (with g++-4.6) are concerned, and as expected:

  • 6.1.0 gets the bunch of symbols;
  • 6.2.0 and 6.2.1 don't.

comment:90 Changed 5 years ago by jpflori

A little more info:

  • with Debian's GCC 4.4 and 4.6, only a few symbols get defined;
  • with Debian's GCC 4.7, 4.8 and 4.9, a lof of symbols get defined;
  • with FSF's GCC 4.3, only a few symbols get defined;
  • with FSF's GCC 4.2 and 4.5, a lot of symbols get defined.

So it's kind of unfortunate that GCC behavior varies in that way.

Whatsoever, removing the -fno-implicit-templates flag from Singular 3-1-6 build system, let it link to a libntl.so without all the explicit DoSetLength symbols.

So I guess the proper fix would be to explicitly list the needed templates in Singular sources (the NTL6 compatibility patch does already this kind of stuff), I'm now trying this solution. Or we could just get rid of the -fno-implicit-templates flag from Singular 3-1-6 build system. Or do you think, we should make sure NTL always define the whole bunch of symbols.

Not sure how singular-4.x behaves.

Changed 5 years ago by jpflori

Let Singular instantiate NTL templates

comment:91 Changed 5 years ago by jpflori

(Note this is a 3-1-7 based patch.)

comment:92 Changed 5 years ago by jpflori

  • Branch changed from u/fbissey/ntl-6.2.1 to u/jpflori/trac/16882
  • Commit e2623ecfa6ec929f542765f2d0ca80a87dc762d0 deleted
  • Description modified (diff)

comment:93 Changed 5 years ago by jpflori

  • Branch changed from u/jpflori/trac/16882 to u/jpflori/ticket/16882
  • Commit set to 5a118683670cd343cd63b7fe7eab1882e4c4b839

Last 10 new commits:

311a52cUpdate NTL to version 6.2.1.
b67d968Fix checksum for NTL.
071b254Update Singular to 3-1-7.
13760d8Let Singular instantiate NTL templates.
5fdce94Refreshing patches and new configure.ac to generate a libtool that supports c++ properly.
2d3584fCorrect patch and change ordering between libtool generation and patching in spkg-install
e2623ecReseparate make.patch properly from the libtool patch
86b0663Merge remote-tracking branch 'trac/u/fbissey/ntl-6.2.1' into ticket/16882
8b5a659Merge remote-tracking branch 'trac/develop' into ticket/16882
5a11868Rebase Singular patches.

comment:94 Changed 5 years ago by jpflori

FYI, grepping through the Singular 4.0.1 sources for no-implicit yields nothing.

So the use of that flag in the 3-1-x branch is surely an "historical artifact".

comment:95 Changed 5 years ago by jpflori

And I have no problem building Singular 4-0-1 on top of a lightweight NTL in a Sage shell (without any patches).

comment:96 follow-up: Changed 5 years ago by jpflori

  • Authors changed from Jean-Pierre Flori to Jean-Pierre Flori, François Bissey
  • Status changed from needs_work to needs_review
  • Summary changed from Update to NTL 6.2.1 to Update to NTL 6.2.1 and to Singular 3-1-7

I definitely think that not passing -fno-implicit-templates is the way to go. It's useless to fight with a soon to be obsolete Singular version.

comment:97 in reply to: ↑ 96 Changed 5 years ago by jdemeyer

Does Singular-3-1-7 require NTL-6.2.1? If not, I strongly suggest to split off the Singular upgrade to a different ticket (2 spkg upgrades on one ticket sounds scary).

comment:98 Changed 5 years ago by jpflori

As far as I saw, Singular 3-1-7 is mainly bug fixes and in particular inclusion of Sage patches... And I wanted to see if it changed anything as far as the templates issue was concerned. It did not, but the update was done. Feel free to split it apart, but on my side I've fought enough for this ticket. It should not be that hard. Just keep the templates.patch here (note that you could also get a similar efect playing with env vars.

comment:99 Changed 5 years ago by jdemeyer

  • Dependencies set to #17184
  • Description modified (diff)
  • Summary changed from Update to NTL 6.2.1 and to Singular 3-1-7 to Upgrade to NTL 6.2.1

comment:100 Changed 5 years ago by jdemeyer

  • Branch changed from u/jpflori/ticket/16882 to u/jdemeyer/ticket/16882
  • Created changed from 08/26/14 14:16:19 to 08/26/14 14:16:19
  • Modified changed from 10/20/14 20:05:01 to 10/20/14 20:05:01

comment:101 follow-up: Changed 5 years ago by jdemeyer

  • Commit changed from 5a118683670cd343cd63b7fe7eab1882e4c4b839 to b21d42c649a38ec97bd62b3cc48970205677ce16

What is the "submit upstream" status of the ntl62.patch for FLINT?


New commits:

7c3f691Upgrade to Singular-3-1-7
b21d42cUpgrade to NTL 6.2.1

comment:102 in reply to: ↑ 101 Changed 5 years ago by fbissey

Replying to jdemeyer:

What is the "submit upstream" status of the ntl62.patch for FLINT?


New commits:

7c3f691Upgrade to Singular-3-1-7
b21d42cUpgrade to NTL 6.2.1

It has been merged into trunk but there are no release containing it at this time.

comment:103 follow-up: Changed 5 years ago by jpflori

I see you added doc fir the lazy template instantiation patch, which I forgot to do, but put it in NTL's SPKG.txt instead of Singular's one, and with a wrong name (template_fix.patch instead of templates.patch).

Thanks for splitting the updates into two tickets.

comment:104 in reply to: ↑ 103 Changed 5 years ago by jdemeyer

Replying to jpflori:

I see you added doc fir the lazy template instantiation patch, which I forgot to do, but put it in NTL's SPKG.txt instead of Singular's one, and with a wrong name (template_fix.patch instead of templates.patch).

I didn't add anything, I literally just split up the ticket in two.

comment:105 Changed 5 years ago by jpflori

Oh right, the template_fix.patch thing mentioned in NTL's SPKG.txt is a leftover of the fix needed for 6.2.0 which was included into 6.2.1.

comment:106 Changed 5 years ago by jpflori

  • Branch changed from u/jdemeyer/ticket/16882 to u/jpflori/ticket/16882
  • Commit changed from b21d42c649a38ec97bd62b3cc48970205677ce16 to 840aebf3e4f4e617d65bbc0a0b27d05df1c41004

Doc updated.


Last 10 new commits:

071b254Update Singular to 3-1-7.
13760d8Let Singular instantiate NTL templates.
5fdce94Refreshing patches and new configure.ac to generate a libtool that supports c++ properly.
2d3584fCorrect patch and change ordering between libtool generation and patching in spkg-install
e2623ecReseparate make.patch properly from the libtool patch
86b0663Merge remote-tracking branch 'trac/u/fbissey/ntl-6.2.1' into ticket/16882
8b5a659Merge remote-tracking branch 'trac/develop' into ticket/16882
5a11868Rebase Singular patches.
7157fa9Update patch documentation for NTL and Singular.
840aebfMerge remote-tracking branch 'trac/u/jdemeyer/ticket/16882' into ticket/16882

comment:107 Changed 5 years ago by jdemeyer

NTL + Singular builds fine now on the machine where it failed before.

comment:108 Changed 5 years ago by git

  • Commit changed from 840aebf3e4f4e617d65bbc0a0b27d05df1c41004 to ba3aec74109d7f92fa25e6b044842d84fee0c996

Branch pushed to git repo; I updated commit sha1. New commits:

1027b8fDocument templates.patch
3968376Update patches for Singular 3-1-7.
f70af8cAdd new alias for Singular debug memory allocation.
5f76994Merge branch 'ticket/17184' into ticekt/16882
cdedfc6Correct fuzz for Singular patches.
ba3aec7Merge branch 'ticket/17184' into ticket/16882

comment:109 Changed 5 years ago by jpflori

I can also confirm it builds in normal and debug modes with nice and mean versions of GCC.

comment:110 Changed 5 years ago by git

  • Commit changed from ba3aec74109d7f92fa25e6b044842d84fee0c996 to 7c1fdea5a81c77498119e65491fa47b820c6df47

Branch pushed to git repo; I updated commit sha1. New commits:

30b9594Avoid redefining Singular mock ring.
32f3be6Avoid redefining Singular mock ring.
7c1fdeaMerge branch 'ticket/17184' into ticket/16882

comment:111 Changed 5 years ago by git

  • Commit changed from 7c1fdea5a81c77498119e65491fa47b820c6df47 to 3d1a298aea6e2ee92d6cbbbdff6fb8d8c5f9a096

Branch pushed to git repo; I updated commit sha1. New commits:

3d1a298Remove debug stuff.

comment:112 Changed 5 years ago by git

  • Commit changed from 3d1a298aea6e2ee92d6cbbbdff6fb8d8c5f9a096 to 47ea2504aff21da6e7f40ecb0265a8b81ecd249d

Branch pushed to git repo; I updated commit sha1. New commits:

47ea250Update doctest for Singular new behavior.

comment:113 Changed 5 years ago by vbraun

  • Reviewers set to Volker Braun
  • Status changed from needs_review to positive_review

lgtm

comment:114 Changed 5 years ago by vbraun

  • Branch changed from u/jpflori/ticket/16882 to 47ea2504aff21da6e7f40ecb0265a8b81ecd249d
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.