Move ntl_wrap to Sage library

[jdemeyer]

New commits:

​bf7497a

Move ntl_wrap to the Sage library

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​2ed346b	Remove unneeded inclusions of cdefs.pxi
​57e9278	More removals of cdefs.pxi
​370106d	Even more removals of cdefs
​1accad6	Remove cdefs.pxi from .pxd/.pxi files
​ec497b4	Merge tag '6.8.beta0' into HEAD
​b5a143f	Move ntl_wrap to Sage library

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6cf2f45

Merge tag '6.8.beta1' into t/18367/move_ntl_wrap_to_sage_library

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​97215ae

Rename ntl_wrap -> ntlwrap

[fbissey]

Looks ready to be sent to the bots to me.

[vbraun]

On arando, snapperkob, and arm I get this:

Deleting empty directory /home/buildslave-sage/slave/sage_git/build/src/doc/hu/a_tour_of_sage/static
Traceback (most recent call last):
  File "/home/buildslave-sage/slave/sage_git/build/src/doc/common/builder.py", line 1619, in <module>
    import sage.all
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/all.py", line 99, in <module>
    from sage.rings.all      import *
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/all.py", line 68, in <module>
    from number_field.all import *
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/number_field/all.py", line 7, in <module>
    from totallyreal import enumerate_totallyreal_fields_prim
  File "sage/rings/number_field/totallyreal_data.pxd", line 12, in init sage.rings.number_field.totallyreal (build/cythonized/sage/rings/number_field/totallyreal.c:10354)
  File "sage/rings/number_field/totallyreal_data.pyx", line 40, in init sage.rings.number_field.totallyreal_data (build/cythonized/sage/rings/number_field/totallyreal_data.c:11148)
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 465, in PolynomialRing
    R = _single_variate(base_ring, name, sparse, implementation)
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 539, in _single_variate
    R = m.PolynomialRing_integral_domain(base_ring, name, sparse, implementation)
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1544, in __init__
    sparse=sparse, element_class=element_class)
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1451, in __init__
    sparse=sparse, element_class=element_class, category=category)
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 305, in __init__
    from sage.matrix.matrix_space import is_MatrixSpace
  File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 46, in <module>
    import matrix_modn_sparse
  File "sage/matrix/matrix_integer_dense.pxd", line 11, in init sage.matrix.matrix_modn_sparse (build/cythonized/sage/matrix/matrix_modn_sparse.c:18762)
  File "sage/matrix/matrix_modn_dense_template_header.pxi", line 9, in init sage.matrix.matrix_integer_dense (build/cythonized/sage/matrix/matrix_integer_dense.c:50329)
ImportError: /home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/matrix/matrix_modn_dense_float.so: undefined symbol: _ZN3NTL8ZZ_pInfoE

[fbissey]

What gcc versions are those? It probably needs a compiler flag, the symbol comes from ZZ_p.h.

[vbraun]

As one data point, arando is gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1)

[fbissey]

Hum, the symbol should be in libntl which suggests that it didn't link appropriately to it.

[jdemeyer]

Volker, did you also include #18450 in that test run since that also affects libraries?

[vbraun]

No, all tickets that were included are closed by now

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0bd4c02

linbox needs ntl

[fbissey]

Actually info from sage-on-gentoo which is all compiled with -as-needed suggest a slightly different approach:

fbissey@QCD-nzi3 ~ $ ldd -r /usr/lib64/python2.7/site-packages/sage/matrix/matrix_modn_dense_float.so
	linux-vdso.so.1 (0x00007fff24fff000)
	libgivaro.so.0 => /usr/lib64/libgivaro.so.0 (0x00007f318a8ce000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f318a65d000)
	libopenblas_openmp.so.0 => /usr/lib64/libopenblas_openmp.so.0 (0x00007f318a0d5000)
	libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libstdc++.so.6 (0x00007f3189dc6000)
	libpython2.7.so.1.0 => /usr/lib64/libpython2.7.so.1.0 (0x00007f3189a08000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f3189703000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1 (0x00007f31894ec000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f318914f000)
	libgmpxx.so.4 => /usr/lib64/libgmpxx.so.4 (0x00007f3188f48000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f3188d2d000)
	libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgomp.so.1 (0x00007f3188b16000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f318ade4000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f3188911000)
	libutil.so.1 => /lib64/libutil.so.1 (0x00007f318870e000)
fbissey@QCD-nzi3 ~ $ ldd -r /usr/lib64/python2.7/site-packages/sage/matrix/matrix_modn_dense_double.so
	linux-vdso.so.1 (0x00007fff91dff000)
	libgivaro.so.0 => /usr/lib64/libgivaro.so.0 (0x00007f9413287000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f9413016000)
	libopenblas_openmp.so.0 => /usr/lib64/libopenblas_openmp.so.0 (0x00007f9412a8e000)
	libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libstdc++.so.6 (0x00007f941277f000)
	libpython2.7.so.1.0 => /usr/lib64/libpython2.7.so.1.0 (0x00007f94123c1000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f94120bc000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1 (0x00007f9411ea5000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f9411b08000)
	libgmpxx.so.4 => /usr/lib64/libgmpxx.so.4 (0x00007f9411901000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f94116e6000)
	libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgomp.so.1 (0x00007f94114cf000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f941379b000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f94112ca000)
	libutil.so.1 => /lib64/libutil.so.1 (0x00007f94110c7000)

Neither of those get libntl and they don't get liblinbox either. All the libraries are resolved so they are definitely not needed. Furthermore liblinbox doesn't need libntl. Only liblinboxsage does and curiously I don't think we ever link to it.

fbissey@QCD-nzi3 ~ $ ldd -r /usr/lib64/liblinbox.so
	linux-vdso.so.1 (0x00007fff2d334000)
	libgivaro.so.0 => /usr/lib64/libgivaro.so.0 (0x00007fbb7557f000)
	libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libstdc++.so.6 (0x00007fbb7526f000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fbb74ed2000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1 (0x00007fbb74cbb000)
	libgmpxx.so.4 => /usr/lib64/libgmpxx.so.4 (0x00007fbb74ab5000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007fbb74843000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fbb7453f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fbb75a18000)
fbissey@QCD-nzi3 ~ $ ldd -r /usr/lib64/liblinboxsage.so
	linux-vdso.so.1 (0x00007ffffbbcf000)
	libgmpxx.so.4 => /usr/lib64/libgmpxx.so.4 (0x00007fed8461a000)
	libntl.so.5 => /usr/lib64/libntl.so.5 (0x00007fed84260000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007fed83fee000)
	libgf2x.so.1 => /usr/lib64/libgf2x.so.1 (0x00007fed83dd9000)
	libopenblas_openmp.so.0 => /usr/lib64/libopenblas_openmp.so.0 (0x00007fed83852000)
	liblinbox.so.0 => /usr/lib64/liblinbox.so.0 (0x00007fed8364a000)
	libgivaro.so.0 => /usr/lib64/libgivaro.so.0 (0x00007fed833ee000)
	libiml.so.0 => /usr/lib64/libiml.so.0 (0x00007fed831cd000)
	libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libstdc++.so.6 (0x00007fed82ebd000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fed82bb9000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fed8281c000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1 (0x00007fed82604000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fed823e9000)
	libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgomp.so.1 (0x00007fed821d2000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fed84ca3000)

In the first instance Jeroen's latest commit would solve the present problem though.

[fbissey]

However, I will go with caution. Because #18450 could have brought something in, that I cannot see yet, in this ticket. The commit looks good to me.

[jdemeyer]

OK, maybe the commit message was wrong, but at least the fix works...

[fbissey]

Interestingly you have that stuff in the commit for #17854, which now may need some rebasing?

[jdemeyer]

Replying to fbissey:

Interestingly you have that stuff in the commit for #17854, which now may need some rebasing?

Cool git tip: if you make an identical change in two different commits, git recognizes that and it doesn't cause a conflict.