Opened 13 months ago

Closed 6 months ago

Last modified 6 months ago

#26932 closed enhancement (fixed)

Upgrade to givaro-4.1.1 fflas-ffpack-2.4.3 linbox-1.6.3

Reported by: cpernet Owned by:
Priority: major Milestone: sage-8.9
Component: packages: standard Keywords:
Cc: fbissey, gh-timokau Merged in:
Authors: Clément Pernet Reviewers: Timo Kaufmann
Report Upstream: N/A Work issues:
Branch: ead33ba (Commits) Commit:
Dependencies: Stopgaps:

Description (last modified by cpernet)

Tarballs of released packages (with appropriate names) can be found at:

Example of timing improvements (in seconds) on my laptop running an i7-8650U

  • for random square matrices of order n=4000 over GF(4194301) (22 bits)
develop #26932
A * B 3.61 3.57
A.det() 2.96 1.52
A.echelonize() 3.59 1.86
A\b 8.9 5.13
  • for random square matrices of order n=4000 over GF(251) (8 bits)
develop #26932
A * B 1.59 1.5
A.det() 1.54 0.731
A.echelonize() 1.82 0.692
A\b 3.7 1.79

Change History (80)

comment:1 Changed 13 months ago by fbissey

  • Cc fbissey added

comment:2 Changed 13 months ago by embray

  • Milestone changed from sage-8.6 to sage-8.7

Retarging tickets optimistically to the next milestone. If you are responsible for this ticket (either its reporter or owner) and don't believe you are likely to complete this ticket before the next release (8.7) please retarget this ticket's milestone to sage-pending or sage-wishlist.

comment:3 Changed 11 months ago by cpernet

  • Branch set to u/cpernet/upgrade_to_givaro_4_1_0_fflas_ffpack_2_4_0_linbox_1_6_0

comment:4 Changed 11 months ago by git

  • Commit set to f7aa5c04a155e8d690a674bd489f43153bd0d467

Branch pushed to git repo; I updated commit sha1. New commits:

f7aa5c0fix and clean fetch_int

comment:5 Changed 11 months ago by cpernet

Tarballs of released packages (with appropriate names) can be found at:

I'll release the 3 libraries soon, but I meant to check that they could be smoothly updating into sage before hand, therefore this ticket proposes to apply the rc0 versions temporarily. Once everything is in order, I'll cut the releases, and checksums accordingly in this branch.

comment:6 Changed 11 months ago by cpernet

  • Description modified (diff)

comment:7 Changed 11 months ago by git

  • Commit changed from f7aa5c04a155e8d690a674bd489f43153bd0d467 to ff8e05871c467be7803fd0457a870f54456c58e6

Branch pushed to git repo; I updated commit sha1. New commits:

ff8e058update checksums

comment:8 Changed 11 months ago by cpernet

For the record, the branch currently breaks several doctests.

----------------------------------------------------------------------
sage -t src/sage/algebras/free_algebra.py  # 2 doctests failed
sage -t src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # 9 doctests failed
sage -t src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # 3 doctests failed
sage -t src/sage/coding/bch.py  # 2 doctests failed
sage -t src/sage/coding/extended_code.py  # 1 doctest failed
sage -t src/sage/dynamics/arithmetic_dynamics/generic_ds.py  # 4 doctests failed
sage -t src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # 1 doctest failed
sage -t src/sage/schemes/elliptic_curves/ell_curve_isogeny.py  # 1 doctest failed
sage -t src/sage/schemes/generic/scheme.py  # 2 doctests failed
sage -t src/sage/schemes/curves/projective_curve.py  # 9 doctests failed
sage -t src/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py  # 1 doctest failed
sage -t src/sage/interfaces/sage0.py  # 1 doctest failed
sage -t src/sage/rings/fraction_field.py  # 1 doctest failed
sage -t src/sage/rings/polynomial/multi_polynomial.pyx  # 1 doctest failed
sage -t src/sage/rings/polynomial/flatten.py  # 1 doctest failed
sage -t src/sage/rings/polynomial/multi_polynomial_ideal.py  # 5 doctests failed
sage -t src/sage/rings/polynomial/multi_polynomial_element.py  # 1 doctest failed
sage -t src/sage/crypto/mq/rijndael_gf.py  # 73 doctests failed
----------------------------------------------------------------------

Givaro::GFqDom seems to be the culprit. I'm working on it.

comment:9 Changed 10 months ago by embray

  • Milestone changed from sage-8.7 to sage-8.8

Ticket retargeted after milestone closed (if you don't believe this ticket is appropriate for the Sage 8.8 release please retarget manually)

comment:10 Changed 10 months ago by git

  • Commit changed from ff8e05871c467be7803fd0457a870f54456c58e6 to e879bfecb479d5290a006de25047495d7887d59b

Branch pushed to git repo; I updated commit sha1. New commits:

e879bfeMerge branch 'develop' into t/26932/upgrade_to_givaro_4_1_0_fflas_ffpack_2_4_0_linbox_1_6_0

comment:11 Changed 9 months ago by cpernet

For the record, the source of the above-mentionned bugs has been identified to a change in givaro GFqDom interface. It has been fixed now, and we can move forward on this ticket.

comment:12 Changed 9 months ago by embray

I don't suppose any of these are tested on Cygwin? I'm happy to do so if not, I just wondered.

comment:13 Changed 9 months ago by cpernet

The 3 libraries are tested on cygwin. We currently have an unsolved bug of fflas-ffpack which seem to be cygwin specific: https://github.com/linbox-team/fflas-ffpack/issues/248

However, I personnaly don't have a cygwin box on which I could test this ticket, so I appreciate your proposition to test the ticket once I put it in "needs review", or shortly before doing so.

comment:14 Changed 9 months ago by embray

Thanks Clement, will do. I'll also have a look at that bug.

comment:15 Changed 9 months ago by cpernet

  • Description modified (diff)

comment:16 Changed 9 months ago by git

  • Commit changed from e879bfecb479d5290a006de25047495d7887d59b to c11d9cfa23ff9f77681a8f12742f68143eed4504

Branch pushed to git repo; I updated commit sha1. New commits:

4a60894new api for FFPACK::Det
2a99383fix gfqDom interface
c4f97e1fix new echelon API in LinBox
eb00ddeupdate to new echelon API
fbbd37aupdate to new Echelon API
c11d9cfupdate checksum to the freshly cut releases

comment:17 Changed 9 months ago by cpernet

  • Authors set to Clément Pernet
  • Status changed from new to needs_review

I have just released the 3 libraries and updated the sage bindings accordingly. Sage test suite passes on my box. The ticket is ready for review. I may post later on some benchmarks advertising the performance gain from this update.

comment:18 Changed 9 months ago by gh-timokau

  • Cc gh-timokau added

comment:19 Changed 9 months ago by git

  • Commit changed from c11d9cfa23ff9f77681a8f12742f68143eed4504 to 058bfb34c4ba2cc11ae3dcaca9f0bd2abaa1feb6

Branch pushed to git repo; I updated commit sha1. New commits:

058bfb3update to minor version of LinBox

comment:20 Changed 9 months ago by cpernet

  • Description modified (diff)
  • Summary changed from Upgrade to givaro-4.1.0 fflas-ffpack-2.4.0 linbox-1.6.0 to Upgrade to givaro-4.1.0 fflas-ffpack-2.4.0 linbox-1.6.1

Update: I realized that there was a mismatch between the tar.gz on github and the published md5, sha1 and cksum checks. I updated the tar.gz of the three libraries to their actual values on github.

In addition, we fixed some minor issues in the linbox-auto-install and the dependency version checking system and therefore release 1.6.1 consequently. I have thus updated this ticket to linbox-1.6.1 accordingly.

comment:21 Changed 9 months ago by gh-timokau

For what it's wroth the linbox 1.6.0 update broke the build *on nixos* for darwin and aarch64. That may be an issue for sage as well.

comment:22 Changed 9 months ago by cpernet

Thanks for the feedback.

  • for the darwin bug, this is a bug in the test-suite file tests/test-qlup.C which I just fixed upstream. It does not have any impact on the main library itself.
  • for the aarch64 bug, it seems that some SIMD has been detected at configure time by reading hte CPUID (SSE2 enabled), but the definition of the __m128i type is absent. The header file with these definition should be included in fflas-ffpack/utils/fflas_intrinsic.h where I read that arm_neon.h gets included. However its seems that the intrinsics there are totally different from the intel/amd ones (I have no experience with ARM). I guess you will hit the same bugs building fflas-ffpack and givaro don't you? And they probably already existed with linbox-1.5.2 fflas-ffpack-2.3.2, givaro-4.0.4. As an easy fix I suggest to force disable these SIMD at configure time (--disable-sse --disable-sse2). If no-one with experience on ARM and SIMD can help enabling it, I will then auto-disable these simd in the configure scripts when ARM target is detected. Bug reported upstream https://github.com/linbox-team/linbox/issues/215
Last edited 9 months ago by cpernet (previous) (diff)

comment:23 Changed 9 months ago by cpernet

Ok scratch what I said about the aarch64 problem. I got confused between the 2 logs and did not realized that no SIMD was detected with the aarch64 build. The bug is much simpler, and I think I fixed it upstream. As I can not test it on an aarch64 system, could you confirm that it is now fixed? I built a prerelease https://ljk.imag.fr/membres/Clement.Pernet/soft/linbox-1.6.2alpha0.tar.gz for you to test it. It should fix the two above issues.

comment:24 Changed 9 months ago by gh-timokau

Thanks for looking into this!

The tarball you linked wasn't able to run tests (apparently some files are missing: test-smith-form.h: No such file or directory). Instead I'm testing the latest commit on github.

This is running on our CI right now (I don't have direct access to aarch64/darwin myself). You can see the status in the "checks" on this GitHub PR: https://github.com/NixOS/nixpkgs/pull/61535

comment:25 Changed 9 months ago by cpernet

I can't believe this one did not get caught earlier. I fixed it upstream and prepared https://ljk.imag.fr/membres/Clement.Pernet/soft/linbox-1.6.2alpha1.tar.gz accordingly in case it helps.

comment:26 Changed 9 months ago by gh-timokau

We usually build from the git repository anyways, so it doesn't make much difference.

The aarch64 build succeeded, but darwin is still failing: https://logs.nix.ci/?key=nixos/nixpkgs.61535&attempt_id=da868da3-0fdc-45df-98b0-7cb18ed6f867

comment:27 Changed 8 months ago by git

  • Commit changed from 058bfb34c4ba2cc11ae3dcaca9f0bd2abaa1feb6 to 4287e026ac4c8d1655b364e5be7a4309224dc7b9

Branch pushed to git repo; I updated commit sha1. New commits:

4287e02upgrade to linbox-1.6.2

comment:28 Changed 8 months ago by cpernet

  • Description modified (diff)
  • Summary changed from Upgrade to givaro-4.1.0 fflas-ffpack-2.4.0 linbox-1.6.1 to Upgrade to givaro-4.1.0 fflas-ffpack-2.4.0 linbox-1.6.2

I released linbox-1.6.2 fixing the bugs raised by gh-timokau and updated the branch accordingly.

comment:29 Changed 8 months ago by gh-timokau

I can confirm that those are fixed, thank you!

comment:30 Changed 8 months ago by cpernet

  • Description modified (diff)

comment:31 Changed 8 months ago by cpernet

  • Description modified (diff)

comment:32 Changed 8 months ago by tscrim

Is someone who has already looked at it willing to set this to a positive review? Otherwise I will doublecheck things on my (ubuntu) machine and then flip it positive.

comment:33 Changed 8 months ago by gh-timokau

  • Reviewers set to Timo Kaufmann
  • Status changed from needs_review to positive_review

Thanks!

comment:34 Changed 8 months ago by vbraun

  • Branch changed from u/cpernet/upgrade_to_givaro_4_1_0_fflas_ffpack_2_4_0_linbox_1_6_0 to 4287e026ac4c8d1655b364e5be7a4309224dc7b9
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:35 Changed 8 months ago by fbissey

  • Commit 4287e026ac4c8d1655b364e5be7a4309224dc7b9 deleted

I was going to report that I had this issue testing linbox in gentoo

x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I..  -DLinBoxTestOnly -O2 -Wall -g -DNDEBUG -U_LB_DEBUG -g -DNDEBUG -U_LB_DEBUG -I.. -DARCH_X86_64=1 -DOPENBLAS___64BIT__=1 -DARCH_X86_64=1 -DOPENBLAS___64BIT__=1 -fopenmp -DFFLAS_COMPILED -DFFPACK_COMPILED -fabi-version=6 -I/usr/include/openblas        -march=native -O2 -pipe  -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx -mavx2 -mfma -c -o test-serialization.o test-serialization.C
test-serialization.C:32:10: fatal error: linbox/util/serialization.h: No such file or directory
   32 | #include "linbox/util/serialization.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I see that a fix has been committed in linbox https://github.com/linbox-team/linbox/commit/5a0aac2662969289c50fb9dec2fd2e597a276815 unfortunately it requires a new tarball to be completely fixed since the problem is that some files have not been included in the 1.6.2 tarball [and earlier probably]. What is the ETA for that?

comment:36 follow-up: Changed 8 months ago by cpernet

These missing files do not impact the part of LinBox? used in Sage. The compilation failure is in the test-suite. I can release v1.6.3 of LinBox? very shortly if you think it should be fixed for the next sage release. I guess, it will happen in another ticket as this one is now closed.

comment:37 in reply to: ↑ 36 Changed 8 months ago by fbissey

Replying to cpernet:

These missing files do not impact the part of LinBox? used in Sage. The compilation failure is in the test-suite. I can release v1.6.3 of LinBox? very shortly if you think it should be fixed for the next sage release. I guess, it will happen in another ticket as this one is now closed.

Yes, I should have said it is in the test suite. I don't know, Volker may choose to re-open if it causes troubles with bots running spkg-check on packages.

comment:38 Changed 8 months ago by vbraun

  • Resolution fixed deleted
  • Status changed from closed to new

comment:39 Changed 8 months ago by vbraun

On 32-bit the sage library doesn't build:

[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp: In function ‘long unsigned int __pyx_f_4sage_4libs_6linbox_22linbox_flint_interface_linbox_fmpz_mat_rank(fmpz_mat_struct*)’:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: error: no matching function for call to ‘rank(long unsigned int&, LinBox::DenseMatrix<Givaro::ZRing<Givaro::Integer> >&)’
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:75:17: note: candidate: template<class Blackbox, class Method, class DomainCategory> size_t& LinBox::rank(size_t&, const Blackbox&, const DomainCategory&, const Method&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t &r, const Blackbox &A,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:75:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:92:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t &r, const Blackbox &A)
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:92:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   cannot convert ‘__pyx_v_r’ (type ‘long unsigned int’) to type ‘size_t& {aka unsigned int&}’
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:109:17: note: candidate: template<class Blackbox, class Method> size_t& LinBox::rank(size_t&, const Blackbox&, const Method&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t &r, const Blackbox &A,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:109:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 3 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:44:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::Auto&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                    &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:44:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:60:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::Elimination&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                     &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:60:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:75:17: note: candidate: template<class Field, class Vector> size_t& LinBox::rank(size_t&, const LinBox::SparseMatrix<_Field, _Storage>&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::Elimination&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                      &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:75:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   ‘LinBox::DenseMatrix<Givaro::ZRing<Givaro::Integer> > {aka LinBox::BlasMatrix<Givaro::ZRing<Givaro::Integer>, std::vector<Givaro::Integer> >}’ is not derived from ‘const LinBox::SparseMatrix<_Field, _Storage>’
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:503:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::Blackbox&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                     &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:503:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:93:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::Wiedemann&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                     &res,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:93:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:388:17: note: candidate: template<class Field> size_t& LinBox::rank(size_t&, const LinBox::SparseMatrix<_Field, LinBox::SparseMatrixFormat::SparseSeq>&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::SparseElimination&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                       &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:388:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   ‘LinBox::DenseMatrix<Givaro::ZRing<Givaro::Integer> > {aka LinBox::BlasMatrix<Givaro::ZRing<Givaro::Integer>, std::vector<Givaro::Integer> >}’ is not derived from ‘const LinBox::SparseMatrix<_Field, LinBox::SparseMatrixFormat::SparseSeq>’
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:400:17: note: candidate: template<class Blackbox, class DomainCategory> size_t& LinBox::rank(size_t&, const Blackbox&, const DomainCategory&, const LinBox::Method::SparseElimination&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                       &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:400:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:413:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::ModularTag&, const LinBox::Method::DenseElimination&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                      &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:413:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:458:17: note: candidate: template<class Blackbox, class MyMethod> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::IntegerTag&, const MyMethod&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                     &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:458:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:468:17: note: candidate: template<class Blackbox, class Method> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::RationalTag&, const Method&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t                           &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:468:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));
[sagelib-8.8.beta7]                                                   ^
[sagelib-8.8.beta7] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:131:0,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47,
[sagelib-8.8.beta7]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.8.beta7]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1233:
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:483:17: note: candidate: template<class Blackbox> size_t& LinBox::rank(size_t&, const Blackbox&, const LinBox::RingCategories::RationalTag&, const LinBox::Method::SparseElimination&)
[sagelib-8.8.beta7]   inline size_t &rank (size_t           &r,
[sagelib-8.8.beta7]                  ^~~~
[sagelib-8.8.beta7] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.inl:483:17: note:   template argument deduction/substitution failed:
[sagelib-8.8.beta7] build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:1792:50: note:   candidate expects 4 arguments, 2 provided
[sagelib-8.8.beta7]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_LBA[0])));

comment:40 Changed 8 months ago by fbissey

You may as well get a linbox 1.6.3 in at the same time, me and the other distros that look at testsuite results would be very grateful to have a working testsuite.

comment:41 Changed 8 months ago by cpernet

  • Branch changed from 4287e026ac4c8d1655b364e5be7a4309224dc7b9 to u/cpernet/4287e026ac4c8d1655b364e5be7a4309224dc7b9

comment:42 follow-up: Changed 8 months ago by gh-timokau

  • Commit set to bd8b2a0e3c84eaf05db55687b43503457ad9deca

I am a little confused, the testsuite runs fine on nixos (64 bit). Is it maybe because we fetch directly from the github tag?


Last 10 new commits:

e879bfeMerge branch 'develop' into t/26932/upgrade_to_givaro_4_1_0_fflas_ffpack_2_4_0_linbox_1_6_0
4a60894new api for FFPACK::Det
2a99383fix gfqDom interface
c4f97e1fix new echelon API in LinBox
eb00ddeupdate to new echelon API
fbbd37aupdate to new Echelon API
c11d9cfupdate checksum to the freshly cut releases
058bfb3update to minor version of LinBox
4287e02upgrade to linbox-1.6.2
bd8b2a0fix 32 compilation failure for linbox-rank

comment:43 in reply to: ↑ 42 Changed 8 months ago by fbissey

Replying to gh-timokau:

I am a little confused, the testsuite runs fine on nixos (64 bit). Is it maybe because we fetch directly from the github tag?

Yes. I am using the tarball generated by make sdist and which is the one linked in the ticket description. A git checkout will test fine but not the generated tarball.

comment:44 Changed 8 months ago by git

  • Commit changed from bd8b2a0e3c84eaf05db55687b43503457ad9deca to ddacbf01a64e7830f2ad03f13200c3edff4f0dc5

Branch pushed to git repo; I updated commit sha1. New commits:

ddacbf0upgrade to linbox-1.6.3 fflas-ffpack-2.4.1 and givaro-4.1.1

comment:45 Changed 8 months ago by cpernet

  • Description modified (diff)
  • Summary changed from Upgrade to givaro-4.1.0 fflas-ffpack-2.4.0 linbox-1.6.2 to Upgrade to givaro-4.1.1 fflas-ffpack-2.4.1 linbox-1.6.3

I just released new minor revisions of linbox (1.6.3) fflas-ffpack (2.4.1) and givaro (4.1.1),

  • taking into account the above problems and fixing other minor issues such as
  • including the README.md in the distro,
  • fixing fflas-ffpack API for parallel calls from SageMath (in preparation of #27444).

I updated the current branch and ticket accordingly, and it is now again ready for review.

comment:46 Changed 8 months ago by cpernet

  • Status changed from new to needs_review

comment:47 Changed 8 months ago by git

  • Commit changed from ddacbf01a64e7830f2ad03f13200c3edff4f0dc5 to f2d1f29dcf5860053d8861a516058cc48e3f5bbb

Branch pushed to git repo; I updated commit sha1. New commits:

f2d1f29upgrade to fflas-ffpack-2.4.2 fixing testsuite errors with echlon forms

comment:48 Changed 8 months ago by git

  • Commit changed from f2d1f29dcf5860053d8861a516058cc48e3f5bbb to 784815648e9e8bac9c81e6336014f0021d91f853

Branch pushed to git repo; I updated commit sha1. New commits:

7848156upgrade do fflas-ffpack 2.4.3 fixing an error in charpoly autotune

comment:49 Changed 8 months ago by cpernet

  • Description modified (diff)

comment:50 Changed 8 months ago by cpernet

  • Description modified (diff)

comment:51 Changed 8 months ago by cpernet

I just did 2 last fixes of some bugs which shamelessly went through the prerelease testing. No more fixes are expected, and the ticket is still open for review, with fflas-ffpack-2.4.3 version (I updated the branch and ticket description accordingly).

comment:52 Changed 8 months ago by cpernet

  • Summary changed from Upgrade to givaro-4.1.1 fflas-ffpack-2.4.1 linbox-1.6.3 to Upgrade to givaro-4.1.1 fflas-ffpack-2.4.3 linbox-1.6.3

comment:53 Changed 7 months ago by isuruf

Testsuite of givaro, fflas-ffpack, linbox passes for me on Linux-64, osx-64 on conda. Any chance we could get this ticket tested and reviewed soon?

comment:54 Changed 7 months ago by vbraun

  • Status changed from needs_review to positive_review

comment:55 Changed 7 months ago by chapoton

  • Milestone changed from sage-8.8 to sage-8.9

comment:56 Changed 7 months ago by vbraun

  • Status changed from positive_review to needs_work

Doesn't build on 32-bit:

Debian 10 32-bit:

[sagelib-8.9.beta0] [ 5/15] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I./sage/libs/flint -I./sage/cpython -I./sage/ext -I./sage/libs/ntl -I/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals -I./sage/rings -I/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cypari2 -I./sage/rings/finite_rings -Isage/cpython -I/var/lib/buildbot/slave/sage_git/build/local/include -I/var/lib/buildbot/slave/sage_git/build/src -I/var/lib/buildbot/slave/sage_git/build/src/sage/ext -I/var/lib/buildbot/slave/sage_git/build/local/include/python2.7 -I/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/core/include -Ibuild/cythonized -I/var/lib/buildbot/slave/sage_git/build/local/include/python2.7 -c build/cythonized/sage/matrix/matrix_integer_dense.c -o build/temp.linux-i686-2.7/build/cythonized/sage/matrix/matrix_integer_dense.o -fno-strict-aliasing -DCYTHON_CLINE_IN_TRACEBACK=1 -std=c99 -g -fPIC -Wall -O2 -mmmx -msse -msse2 -msse3 -I/var/lib/buildbot/slave/sage_git/build/local/include/m4ri -I/var/lib/buildbot/slave/sage_git/build/local/include
[sagelib-8.9.beta0] In file included from /var/lib/buildbot/slave/sage_git/build/local/include/recint/ruint.h:53,
[sagelib-8.9.beta0]                  from /var/lib/buildbot/slave/sage_git/build/local/include/givaro/modular-ruint.h:14,
[sagelib-8.9.beta0]                  from /var/lib/buildbot/slave/sage_git/build/local/include/givaro/modular.h:26,
[sagelib-8.9.beta0]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:722:
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h: In member function ‘Givaro::Modular<_Storage_t, _Compute_t, typename std::enable_if<(is_same_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value || is_smaller_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value)>::type>::Element& Givaro::Modular<_Storage_t, _Compute_t, typename std::enable_if<(is_same_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value || is_smaller_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value)>::type>::init(Givaro::Modular<_Storage_t, _Compute_t, typename std::enable_if<(is_same_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value || is_smaller_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value)>::type>::Element&, const Givaro::Integer&) const [with _Storage_t = RecInt::ruint<7>; _Compute_t = RecInt::ruint<8>]’:
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:214:65: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta0]              udiv_qrnnd(q.Value, c, a2.Value, a1.Value, b1.Value);
[sagelib-8.9.beta0]                                                                  ^
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:214:65: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta0]              udiv_qrnnd(q.Value, c, a2.Value, a1.Value, b1.Value);
[sagelib-8.9.beta0]                                                                  ^
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta0]          umul_ppmm(d1, d0, q.Value, b0.Value);
[sagelib-8.9.beta0]                                              ^
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta0]          umul_ppmm(d1, d0, q.Value, b0.Value);
[sagelib-8.9.beta0]                                              ^
[sagelib-8.9.beta0] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta0] [ 6/15] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I./sage/cpython -I./sage/ext -I./sage/libs/flint -I./sage/libs/ntl -I/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals -Isage/libs/flint -I/var/lib/buildbot/slave/sage_git/build/local/include -I/var/lib/buildbot/slave/sage_git/build/src -I/var/lib/buildbot/slave/sage_git/build/src/sage/ext -I/var/lib/buildbot/slave/sage_git/build/local/include/python2.7 -I/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/core/include -Ibuild/cythonized -I/var/lib/buildbot/slave/sage_git/build/local/include/python2.7 -c build/cythonized/sage/matrix/matrix_integer_sparse.cpp -o build/temp.linux-i686-2.7/build/cythonized/sage/matrix/matrix_integer_sparse.o -fno-strict-aliasing -DCYTHON_CLINE_IN_TRACEBACK=1 -O2 -Wall -g -DNDEBUG -U_LB_DEBUG -DDISABLE_COMMENTATOR -DFFLAS_COMPILED -DFFPACK_COMPILED -std=gnu++11 -mfpmath=sse -fabi-version=6 -I/var/lib/buildbot/slave/sage_git/build/local/include
[sagelib-8.9.beta0] cc1plus: warning: SSE instruction set disabled, using 387 arithmetics
[sagelib-8.9.beta0] error: command 'gcc' failed with exit status 1
[sagelib-8.9.beta0] make[4]: *** [Makefile:33: sage] Error 1

Debian 8, 9 32-bit:

[sagelib-8.9.beta0] [ 6/13] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I./sage/cpython -I./sage/ext -I./sage/libs/flint -I./sage/libs/ntl -I/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals -Isage/libs/flint -I/home/buildbot/slave/sage_git/build/local/include -I/home/buildbot/slave/sage_git/build/src -I/home/buildbot/slave/sage_git/build/src/sage/ext -I/home/buildbot/slave/sage_git/build/local/include/python2.7 -I/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/core/include -Ibuild/cythonized -I/home/buildbot/slave/sage_git/build/local/include/python2.7 -c build/cythonized/sage/matrix/matrix_integer_sparse.cpp -o build/temp.linux-i686-2.7/build/cythonized/sage/matrix/matrix_integer_sparse.o -fno-strict-aliasing -DCYTHON_CLINE_IN_TRACEBACK=1 -O2 -Wall -g -DNDEBUG -U_LB_DEBUG -DDISABLE_COMMENTATOR -DFFLAS_COMPILED -DFFPACK_COMPILED -fpermissive -std=gnu++11 -mfpmath=sse -fabi-version=6 -I/home/buildbot/slave/sage_git/build/local/include
[sagelib-8.9.beta0] build/cythonized/sage/matrix/matrix_integer_sparse.cpp:1:0: warning: SSE instruction set disabled, using 387 arithmetics
[sagelib-8.9.beta0]  /* Generated by Cython 0.29.10 */
[sagelib-8.9.beta0]  ^
[sagelib-8.9.beta0] build/cythonized/sage/matrix/matrix_integer_sparse.cpp: In function ‘PyObject* __pyx_pf_4sage_6matrix_21matrix_integer_sparse_21Matrix_integer_sparse_30_rank_linbox(__pyx_obj_4sage_6matrix_21matrix_integer_sparse_Matrix_integer_sparse*)’:
[sagelib-8.9.beta0] build/cythonized/sage/matrix/matrix_integer_sparse.cpp:7765:48: error: no matching function for call to ‘rank(long unsigned int&, LinBox::SparseMatrix<Givaro::ZRing<Givaro::Integer> >&)’
[sagelib-8.9.beta0]    (void)(LinBox::rank(__pyx_v_r, (__pyx_v_M[0])));
[sagelib-8.9.beta0]                                                 ^
[sagelib-8.9.beta0] build/cythonized/sage/matrix/matrix_integer_sparse.cpp:7765:48: note: candidates are:
[sagelib-8.9.beta0] In file included from /home/buildbot/slave/sage_git/build/local/include/linbox/algorithms/bbcharpoly.h:47:0,
[sagelib-8.9.beta0]                  from /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/charpoly.h:34,
[sagelib-8.9.beta0]                  from build/cythonized/sage/matrix/matrix_integer_sparse.cpp:4384:
[sagelib-8.9.beta0] /home/buildbot/slave/sage_git/build/local/include/linbox/solutions/rank.h:75:17: note: template<class Blackbox, class Method, class DomainCategory> size_t& LinBox::rank(size_t&, const Blackbox&, const DomainCategory&, const Method&)
[sagelib-8.9.beta0]   inline size_t &rank (size_t &r, const Blackbox &A,
[sagelib-8.9.beta0]                  ^

comment:57 Changed 7 months ago by git

  • Commit changed from 784815648e9e8bac9c81e6336014f0021d91f853 to 43e1b7f236db20e098450953c90e368ebdcc8dcc

Branch pushed to git repo; I updated commit sha1. New commits:

43e1b7ffix ulong->size_t bug in 32 bits debian 8,9

comment:58 Changed 7 months ago by cpernet

I pushed a fix to the second bug (size_t vs unsigned long in calling LinBox::rank). I'm still waiting for a build of sage on a 32 bit ubuntu to finish, to confirm that the bug is no longer there.

As for the first bug, do you confirm that it only shows up in Debian 10 ? I have no clue what is going on there, and can not reproduce it as I don't have a 32 bit Debian at hand. Looking at other instances of the bug, it seems related to asm code explicitly using some registers (like %ebx) that gcc reserves for PIC offsets. Altough the umul_ppmm indeed uses some asm code (in reclonglong.h) in does not use %ebx explicitly, and more surprisingly, the other error, for udiv_qrnd does not involve any asm code.

This part of givaro's code is not used in Sage, so an easy fix is to comment out line 26 of givaro/src/kernel/ring/modular.h.

comment:59 Changed 7 months ago by git

  • Commit changed from 43e1b7f236db20e098450953c90e368ebdcc8dcc to b0221421819f534f72a7282fe842737de9fa9702

Branch pushed to git repo; I updated commit sha1. New commits:

b022142fix ulong -> size_t

comment:60 Changed 7 months ago by cpernet

I added a fix to another size_t issue. I confirm that the current branch compiles and passes the test-suite on a 32 bit ubuntu xenial I could have access to.

comment:61 Changed 7 months ago by vbraun

Yes, I get the error only on Debian 10 32-bit. Thats gcc 8.3.0. My guess would be that all sufficiently new compiler versions fail. The next oldest 32-bit machine is Ubuntu 18 / gcc 7.3.0 and that passes.

I don't think the error is about EBX, you are free to use it in inline asm as long as you restore it (at least on gcc >=5 and as long as you don't make function calls / use the PLT).

I haven't tried to figure out which of the many inline asm implementations is selected but various do specify constraints "=&r" (sl) which maybe be inconsistent (presumably meaning unsatisfyable).

comment:62 Changed 6 months ago by isuruf

  • Branch changed from u/cpernet/4287e026ac4c8d1655b364e5be7a4309224dc7b9 to u/isuruf/26932
  • Commit changed from b0221421819f534f72a7282fe842737de9fa9702 to 8dbef6857366344bcff0b8f789e59fd4b183d23e
  • Status changed from needs_work to needs_review

New commits:

0616252Merge remote-tracking branch 'trac/develop' into u/cpernet/4287e026ac4c8d1655b364e5be7a4309224dc7b9
8dbef68Avoid modular-ruint.h

comment:63 Changed 6 months ago by fbissey

  • Status changed from needs_review to positive_review

Let's try it again.

comment:64 Changed 6 months ago by cpernet

Ok, I finally got some time to get back to this problem. I doubt the fix above will work (I tried on a 32bit box and it still fails, as recint is also included in fflas-ffpack).

The problem is that two copies of the file longlong.h from GNU GMP are shipped: one in flint (longlong.h) and one in givaro (reclonglong.h). Flint has stripped down the file to a minimalistic set of definitions while givaro's version is the complete original one.

The problem is that these file define macros in the global namespace and conflict one with the other. I therefore added a #ifdef add_ssaaaa in givaro's reclonglong.h to yield whenever flint's longlong.h was already included.

However, givaro's forces the NO_ASM macro which skips all ASM dependent defines and makes the calls to umul_ppmm and other also work on i386 even when passed 64 bits ints. However, flint longlong.h removed the NO_ASM section, and therefore compiles the ASM variants of these routines, thus triggering the compilation failure. I will fix givaro's reclonglong.h so that it undefs the symbols conflicting with flint and redefines them in its own way.

comment:65 Changed 6 months ago by vbraun

  • Status changed from positive_review to needs_work

Fix doesn't work, for the record. Still fails with the

[sagelib-8.9.beta2] In file included from /var/lib/buildbot/slave/sage_git/build/local/include/recint/ruint.h:53:0,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/givaro/modular-ruint.h:14,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/givaro/modular.h:26,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/linbox/field/field-traits.h:30,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/linbox/field/gf2.h:40,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/linbox/vector/stream-gf2.h:77,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/linbox/vector/stream.h:872,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/linbox/matrix/densematrix/blas-matrix.h:42,
[sagelib-8.9.beta2]                  from /var/lib/buildbot/slave/sage_git/build/local/include/linbox/matrix/dense-matrix.h:79,
[sagelib-8.9.beta2]                  from build/cythonized/sage/libs/linbox/linbox_flint_interface.cpp:727:
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h: In member function ‘Givaro::Modular<_Storage_t, _Compute_t, typename std::enable_if<(is_same_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value || is_smaller_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value)>::type>::Element& Givaro::Modular<_Storage_t, _Compute_t, typename std::enable_if<(is_same_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value || is_smaller_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value)>::type>::init(Givaro::Modular<_Storage_t, _Compute_t, typename std::enable_if<(is_same_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value || is_smaller_ruint< <template-parameter-1-1>, <template-parameter-1-2> >::value)>::type>::Element&, const Givaro::Integer&) const [with _Storage_t = RecInt::ruint<7>; _Compute_t = RecInt::ruint<8>]’:
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:214:65: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta2]              udiv_qrnnd(q.Value, c, a2.Value, a1.Value, b1.Value);
[sagelib-8.9.beta2]                                                                  ^
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:214:65: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta2]              udiv_qrnnd(q.Value, c, a2.Value, a1.Value, b1.Value);
[sagelib-8.9.beta2]                                                                  ^
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta2]          umul_ppmm(d1, d0, q.Value, b0.Value);
[sagelib-8.9.beta2]                                              ^
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’
[sagelib-8.9.beta2]          umul_ppmm(d1, d0, q.Value, b0.Value);
[sagelib-8.9.beta2]                                              ^
[sagelib-8.9.beta2] /var/lib/buildbot/slave/sage_git/build/local/include/recint/rudiv.h:222:45: error: inconsistent operand constraints in an ‘asm’

comment:66 Changed 6 months ago by cpernet

  • Branch changed from u/isuruf/26932 to u/cpernet/26932

comment:67 Changed 6 months ago by cpernet

  • Commit changed from 8dbef6857366344bcff0b8f789e59fd4b183d23e to ead33baa4ac1b72158dee587c72ac3b4300d96ef
  • Status changed from needs_work to needs_review

I pushed a patch to givaro, which basically changes the name on givaro's side of all macros which definition where in conflict between flint (longlong.h) and givaro (reclonglong.h) by prefixing them with recint_.

This change solves the problem on the 32bit ubuntu machine where I could reproduce the bug.


New commits:

ead33bafixing double definition of macros between flint and givaro by isolating givaro's with recint_ prefix

comment:68 Changed 6 months ago by isuruf

Thanks. @vbraun, can you review this?

comment:69 Changed 6 months ago by arojas

With this branch and using sagelib built against system libraries, I'm getting

File "/usr/lib/python2.7/site-packages/sage/matrix/matrix_double_dense.pyx", line 2517, in sage.matrix.matrix_double_dense.Matrix_double_dense.is_unitary
Failed example:
    P.is_unitary(algorithm='orthonormal')
Expected:
    Traceback (most recent call last):
    ...
    error: ((lwork==-1)||(lwork >= MAX(1,2*n))) failed for 3rd keyword lwork: zgees:lwork=0
Got:
    <BLANKLINE>
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python2.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.matrix.matrix_double_dense.Matrix_double_dense.is_unitary[23]>", line 1, in <module>
        P.is_unitary(algorithm='orthonormal')
      File "sage/matrix/matrix_double_dense.pyx", line 2568, in sage.matrix.matrix_double_dense.Matrix_double_dense.is_unitary (build/cythonized/sage/matrix/matrix_double_dense.c:16152)
        _, T = self.schur(base_ring=sage.rings.complex_double.CDF)
      File "sage/matrix/matrix_double_dense.pyx", line 3268, in sage.matrix.matrix_double_dense.Matrix_double_dense.schur (build/cythonized/sage/matrix/matrix_double_dense.c:19274)
        T._matrix_numpy, Q._matrix_numpy = scipy.linalg.schur(self._matrix_numpy, output=format)
      File "/usr/lib/python2.7/site-packages/scipy/linalg/decomp_schur.py", line 139, in schur
        result = gees(lambda x: None, a1, lwork=-1)
    ValueError: On entry to ZGEES parameter number 6 had an illegal value

Any ideas?

comment:70 follow-up: Changed 6 months ago by fbissey

Yes. It is not related to this ticket but to the way we sometimes package openblas in distro compared to sage. I have had this doctest failure for several months now. https://github.com/cschwan/sage-on-gentoo/commit/1a5fefec74c222ee5d0673bb439c6d5a3b0c6e1e#commitcomment-30999803 has some gory details if you get down in it. The culprit is not so much sage than it is scipy in a way. It is all down to the xerbla routine effectively used https://github.com/cschwan/sage-on-gentoo/commit/1a5fefec74c222ee5d0673bb439c6d5a3b0c6e1e#commitcomment-31035447 and after.

In any case this is not related to this ticket.

comment:71 in reply to: ↑ 70 Changed 6 months ago by arojas

Replying to fbissey:

In any case this is not related to this ticket.

After some investigation, in my case this turned out to be caused by linking fflas-ffpack with --as-needed - apparently this strips a bit too many linked libs. I started linking with --as-needed with the 2.4.0 update so that made me think it was related to the upgrade.

comment:72 Changed 6 months ago by fbissey

That's very strange because the failure above occurs during a call to a function in scipy. It is supposed to fail but the error message and the point it break is different depending on the version of xerbla (C or Fortran) used.

comment:73 Changed 6 months ago by fbissey

I guess the linking may influence the order things are loaded in memory and privilege a call over another.

comment:74 follow-up: Changed 6 months ago by isuruf

Numpy has a python_xerbla.c file and a xerbla_ inside which is supposed to override the xerbla from BLAS and give a nice exception. We should create a new ticket for the xerbla issue

comment:75 in reply to: ↑ 74 Changed 6 months ago by fbissey

Replying to isuruf:

Numpy has a python_xerbla.c file and a xerbla_ inside which is supposed to override the xerbla from BLAS and give a nice exception. We should create a new ticket for the xerbla issue

I didn't know that. In that case the over-ridding is obviously not working under some conditions. Is it a bug in sage, scipy or numpy? Digging this a bit more.

comment:76 Changed 6 months ago by fbissey

Actually, the numpy source file you are pointing to gives the error message we see in some sage-on-distros as opposed to the one seen in vanilla sage. The source file is in two places in numpy. Once in linalg/lapack_lite which seems to indicate that this is used when numpy is configured without an external lapack. The other copy is in core/src/common, it is unclear if it is always compiled or not at this stage.

comment:77 Changed 6 months ago by fbissey

I'd say python_xerbla.c is always compiled. This is from core/src/common/setupy.py

    blas_info = get_info('blas_opt', 0)
    if blas_info and ('HAVE_CBLAS', None) in blas_info.get('define_macros', []):
        extra_info = blas_info
        # These files are also in MANIFEST.in so that they are always in
        # the source distribution independently of HAVE_CBLAS.
        common_src.extend([join('src', 'common', 'cblasfuncs.c'),
                           join('src', 'common', 'python_xerbla.c'),
                          ])
        if uses_accelerate_framework(blas_info):
            common_src.extend(get_sgemv_fix())
    else:
        extra_info = {}

For those who don't know, numpy's build system sets -DHAVE_CBLAS=NONE when blas/lapack is found :) When no blas/lapack is provided found, HAVE_CBLAS is undefined.

The xerbla_ symbol is inserted in numpy/core/_multiarray_umath.so. But from experience the scipy call that can give the differing messages never use that and must load a version from blas/lapack otherwise we would never see this difference.

comment:78 Changed 6 months ago by isuruf

  • Status changed from needs_review to positive_review

Putting this into positive review, so that it's tested again.

comment:79 Changed 6 months ago by vbraun

  • Branch changed from u/cpernet/26932 to ead33baa4ac1b72158dee587c72ac3b4300d96ef
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:80 Changed 6 months ago by vdelecroix

  • Commit ead33baa4ac1b72158dee587c72ac3b4300d96ef deleted

This released fixed

sage: m = diagonal_matrix(ZZ, [2] * 46)
sage: m._det_linbox()
70368744177664
sage: m = diagonal_matrix(ZZ, [3] * 100)
sage: m._det_linbox()
515377520732011331036461129765621272702107522001

see #28318

Note: See TracTickets for help on using tickets.