Opened 7 years ago

Last modified 5 years ago

#17635 closed enhancement

Update Givaro, FFLAS-FFPACK and LinBox — at Version 18

Reported by: jpflori Owned by:
Priority: major Milestone: sage-7.4
Component: packages: standard Keywords: linbox, givaro, fflas-ffpack, sd75
Cc: fbissey, cpernet Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: u/cpernet/givaro_fflasffpack_linbox (Commits, GitHub, GitLab) Commit: dd81c26b4ece4e206e5c3eb8c42e5a22337bc458
Dependencies: #19298 Stopgaps:

Status badges

Change History (18)

comment:1 Changed 7 years ago by jpflori

Requires to update:

  • fflas-ffpack to 2.0.0, see #17636
  • linbox, no release yet

comment:2 follow-up: Changed 7 years ago by fbissey

  • Cc fbissey added

So we have to wait for linbox? Do we have an idea of when it will happen?

comment:3 in reply to: ↑ 2 ; follow-up: Changed 7 years ago by jpflori

Replying to fbissey:

So we have to wait for linbox? Do we have an idea of when it will happen?

Yes we have to wait for a new Linbox, expected about the end of february, Clément Pernet wrote me, together with updated Givaro and fflas-ffpack.

By the way, I started this for #17630 which might interest you as well, I'll push code there tomorrow.

comment:4 in reply to: ↑ 3 Changed 6 years ago by jdemeyer

  • Description modified (diff)
  • Milestone changed from sage-6.5 to sage-6.7
  • Summary changed from Update Givaro to 3.8.0 to Update Givaro, FFLAS-FFPACK and LinBox

Replying to jpflori:

Replying to fbissey:

So we have to wait for linbox? Do we have an idea of when it will happen?

Yes we have to wait for a new Linbox, expected about the end of february, Clément Pernet wrote me, together with updated Givaro and fflas-ffpack.

It's June already...

Since givaro, LinBox and FFLAS-FFPACK all depend on each other, we should make one ticket.

Last edited 6 years ago by jdemeyer (previous) (diff)

comment:5 Changed 6 years ago by jdemeyer

  • Description modified (diff)
  • Milestone changed from sage-6.7 to sage-6.8

comment:6 follow-up: Changed 6 years ago by cpernet

  • Cc cpernet added

Sorry for the delay in releasing the fflas-ffpack/Givaro/LinBox environment. We're working on it now, and I'll keep you posted about the status. It should happen before the end of the year and hopefully much earlier.

I also meant to propose to replace ATLAS by OpenBLAS in sage. Looking at your clean-up work of the BLAS in #17630, I understand that we should proceed in the following order:

  1. release linbox
  2. upgrade linbox fflas-ffpack Givaro in Sage (this ticket #17635)
  3. clean-up the BLAS usage #17630
  4. replace ATLAS by OpenBLAS (ticket to be created)

I'll be happy to help in these tasks.

Last edited 6 years ago by cpernet (previous) (diff)

comment:7 Changed 6 years ago by jdemeyer

Let me remind you also about #15535, which pops up regularly. I don't know if it's a bug in Linbox or in the way that Sage uses Linbox, but it would be good to have a Linbox developer look at it.

comment:8 in reply to: ↑ 6 Changed 6 years ago by fbissey

Replying to cpernet:

Sorry for the delay in releasing the fflas-ffpack/Givaro/LinBox environment. We're working on it now, and I'll keep you posted about the status. It should happen before the end of the year and hopefully much earlier.

I also meant to propose to replace ATLAS by OpenBLAS in sage. Looking at your clean-up work of the BLAS in #17630, I understand that we should proceed in the following order:

  1. release linbox
  2. upgrade linbox fflas-ffpack Givaro in Sage (this ticket #17635)
  3. clean-up the BLAS usage #17630
  4. replace ATLAS by OpenBLAS (ticket to be created)

I'll be happy to help in these tasks.

I'll admit to have been a bit demotivated on the blas cleaning front. The tickets needs re-organizing and have good descriptions so we know what is supposed to go in them. Getting ATLAS to produce .pc file and using them is probably a priority task. Once that's done someone could create an optional openblas spkg or something.

comment:9 follow-ups: Changed 6 years ago by cpernet

  • Dependencies set to 18323

I'm working on it, and will release a new version of the 3 libs for the occasion, so as to ensure a smoother integration upstream. This will very likely be givaro-4.0.1 fflas-ffpack-2.2.0 linbox-1.4.1

These libraries now require c++11 support and this tickets therefore depends on #18323.

Inspired by what's propose in there, I suggest to add lines of the form

+# distutils: extra_compile_args = -std=c++11

to libs/linbox/linbox.pxd lib/linbox/fflas.pxd lib/linbox/echelonform.pxd but I understand that this is subject to a debate on sage-devel https://groups.google.com/forum/#!topic/sage-devel/aGwotQA9to8

If this option happens to be accepted, we would then need to add more cflags in the list: namely those produced by fflas-ffpack configure related to the target architecture: -mavx -msse4.1 and others like -fabi-version=6 -fopenmp.

I could not figure a way to append the string returned by bin/fflas-ffpack-config --cflags to these pxd files. Any hint?

Last edited 6 years ago by cpernet (previous) (diff)

comment:10 Changed 6 years ago by cpernet

  • Dependencies changed from 18323 to #18323

comment:11 in reply to: ↑ 9 Changed 6 years ago by fbissey

Replying to cpernet:

I could not figure a way to append the string returned by bin/fflas-ffpack-config --cflags to these pxd files. Any hint?

Add a definition in module_list.py, you define a LINBOX_FLAGS. It would be easier if you had a .pc file as we have a python interface to pkgconfig that we can use easily. Here you have to do your own parsing from python. Anyway you can then use LINBOX_FLAGS in the .pxd file.

comment:12 in reply to: ↑ 9 ; follow-up: Changed 6 years ago by jdemeyer

Replying to cpernet:

-mavx -msse4.1

I understand that the libraries themselves want these, but why the Cython files?

and others like -fabi-version=6

This looks like it should be used always for all C++ files then. Mixing ABI's doesn't look like a good idea.

-fopenmp

Same question as before, is this really needed for the Cython interface?

comment:13 in reply to: ↑ 12 Changed 6 years ago by cpernet

Replying to jdemeyer:

Replying to cpernet:

-mavx -msse4.1

I understand that the libraries themselves want these, but why the Cython files?

fflas-ffpack and linbox are essentially source libraries, so when you include the headers, you include the whole code. This is by design, due to the ubiquity of template programming in these libraries.

Now the old linbox-sage interface compiles some specializations and offer them through a C api. These extra flags would not be required if we would stick to using this interface.

But, since then, Cython started to offer the capability to hook C++ code directly and more functionnalities have been added in sage where fflas-ffpack code is called directly, like in libs/linbox/fflas.pxd where the C++ routine FFLAS::fgemm is called.

and others like -fabi-version=6

This looks like it should be used always for all C++ files then. Mixing ABI's doesn't look like a good idea.

Sure. Maybe one should change the ABI higher up then.

-fopenmp

Same question as before, is this really needed for the Cython interface?

Same answer as above.

comment:14 Changed 6 years ago by jdemeyer

  • Dependencies changed from #18323 to #19298

comment:15 Changed 6 years ago by cpernet

  • Branch set to u/cpernet/givaro_fflasffpack_linbox

comment:16 follow-up: Changed 6 years ago by cpernet

  • Commit set to dd81c26b4ece4e206e5c3eb8c42e5a22337bc458

I added my working branch to the ticket. I could not find if and where I should store the tarballs of the new spkg?

As of today (sept 28 2015); they correspond to the upstream version of the 3 libs: givaro, fflas-ffpack and linbox.

I added .pc files for givaro and fflas-ffpack (coming soon for linbox), and defined the extra_compile_flages in modules_list.py as suggested by fbissey.

This is still work in progress:

  • LinBox sparse solver make the test-suite fail: I'm investigating the problem, which I could not reproduce in LinBox for the moment.
  • the extra_compile_flags should be inferred from the .pc files (harcoded for the moment). I did not find how to use pkg-config through python to do it.

New commits:

dd81c26Update givaro to v4.0.1 fflas-ffpack to v2.2.0 and linbox to v1.4.1rc0.

comment:17 in reply to: ↑ 16 Changed 6 years ago by jdemeyer

Replying to cpernet:

I could not find if and where I should store the tarballs of the new spkg?

Just add a link to the upstream tarballs on this ticket. If any renaming is needed, you should rename the tarball and add a link to the renamed tarball.

comment:18 Changed 6 years ago by cpernet

  • Description modified (diff)
Note: See TracTickets for help on using tickets.