Meta-ticket: allow more Fortran compilers

[dimpase]

There are Fortran compilers which give better than gfortran performance, e.g. Intel's ifort. It is currently not possible (without a hack) to build Sage using any of these.

This ought to be fixed.

See also:

• #32176 Support building Sage with the Flang Fortran compiler
• #32177 Support building Sage with the LFortran Fortran compiler

Discussions:

• ​https://groups.google.com/g/sage-devel/c/NfUKjAaTcUg/m/9-wV7jigAwAJ (François Bissey, Sep 2021)

[dimpase]

​flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

I am able to build Sage using the system-provided (pkg install flang) flang on FreeBSD, by removing need_to_install_gfortran=yes from configure.ac, running configure with ./configure CC=cc CXX=c++ FC=flang (that is, the whole toolchain is LLVM-based).

scipy needed a minor fix, as it hardcodes not needed libgfortran, as well as apparently gfortran too, so I tricked it by making symbolic links libgfortan.so to point to libz.so (something innocent) and gfortran to point to flang. (The latter could be replaced by export FC=flang).

Also, r needed a fix, it was unable to determine what should be in FPICFLAGS, and so I did FPICFLAGS=-fPIC ./sage -f r.

[isuruf]

Replying to dimpase:

​flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

We package flang for only Linux and Windows on conda. OSX is still not supprted, but I don't think it would take much time given that flang can be used on FreeBSD. (Windows support is also by a fork we maintain until upstream decides on a CLA which seems to be taking a lot of time.)

I am able to build Sage using the system-provided (pkg install flang) flang on FreeBSD, by removing need_to_install_gfortran=yes from configure.ac, running configure with ./configure CC=cc CXX=c++ FC=flang (that is, the whole toolchain is LLVM-based).

scipy needed a minor fix, as it hardcodes not needed libgfortran, as well as apparently gfortran too, so I tricked it by making symbolic links libgfortan.so to point to libz.so (something innocent) and gfortran to point to flang. (The latter could be replaced by export FC=flang).

numpy distutils PR is at ​https://github.com/numpy/numpy/pull/9942

[dimpase]

Replying to isuruf:

Replying to dimpase:

​flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

We package flang for only Linux and Windows on conda. OSX is still not supprted, but I don't think it would take much time given that flang can be used on FreeBSD. (Windows support is also by a fork we maintain until upstream decides on a CLA which seems to be taking a lot of time.)

By the way, is there any chance that FreeBSD (or perhaps DragonflyBSD, which is also clang-based w.r.t. the toolchain) gets properly supported by Conda (i.e. in more or less the same way as Linux and OSX are)?

I am able to build Sage using the system-provided (pkg install flang) flang on FreeBSD, by removing need_to_install_gfortran=yes from configure.ac, running configure with ./configure CC=cc CXX=c++ FC=flang (that is, the whole toolchain is LLVM-based).

scipy needed a minor fix, as it hardcodes not needed libgfortran, as well as apparently gfortran too, so I tricked it by making symbolic links libgfortan.so to point to libz.so (something innocent) and gfortran to point to flang. (The latter could be replaced by export FC=flang).

numpy distutils PR is at ​https://github.com/numpy/numpy/pull/9942

Apart from some linking issues in src/sage/misc/inline_fortran.py (somewhere -lgfortran hardcoded, etc), I see little to no difference with Sage tests using gfortran.

[fbissey]

Out of interest, does autoconf think that flang is gfortran? In other words, does it put GFC to yes?

[dimpase]

Replying to fbissey:

Out of interest, does autoconf think that flang is gfortran? In other words, does it put GFC to yes?

If I omit

--- a/configure.ac
+++ b/configure.ac
@@ -466,8 +466,6 @@ elif test $need_to_install_gcc = yes; then
     true
 elif test x$GCC != xyes; then
     SAGE_SHOULD_INSTALL_GCC([your C compiler isn't GCC (GNU C)])
-elif test x$GFC != xyes; then
-    need_to_install_gfortran=yes
 else
     # Since need_to_install_gcc is "no", we know that
     # at least C, C++ and Fortran compilers are available.

then an attempt to install gfortran package is made.

[isuruf]

Replying to dimpase:

By the way, is there any chance that FreeBSD (or perhaps DragonflyBSD, which is also clang-based w.r.t. the toolchain) gets properly supported by Conda (i.e. in more or less the same way as Linux and OSX are)?

I don't think Anaconda, Inc ever will unless there is a paying customer.

That said there are some people maintaining conda packages for other non-supported platforms like ​https://github.com/jjhelmus/berryconda

If you want to build conda packages in FreeBSD, conda-forge recipes are a good place. Another option would be to create a conda package from the sage package like I used to do. (I abandoned it because it wouldn't be compatible with the rest of conda packages on other channels, but in the FreeBSD case, that shouldn't matter)

[dimpase]

Replying to isuruf:

Replying to dimpase:

​flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

We package flang for only Linux and Windows on conda. OSX is still not supprted, but I don't think it would take much time given that flang can be used on FreeBSD. (Windows support is also by a fork we maintain until upstream decides on a CLA which seems to be taking a lot of time.)

Yeah, there seem to be no flang on OSX available as yet. Flang on FreeBSD is an out-of-tree branch ATM, too. So this is all in a flux now.

I guess it might be wise to wait until flang can be built with unmodified clang...

[dimpase]

Just for giggles and lols, I tried building Sage 8.7.beta0 using flang on FreeBSD 12.0, and it just works!

Well, it needs various FFLAGS and LDFLAGS to be passed, which is probably more of a FreeBSD bug than anything else, and I didn't trace down why scipy tries to link with libgfortran.so (I ended up just faking libgfortran.so, making it a link to libm.so :-))

But the fact that the configuration just works otherwise, means that this ticket is almost done, all of a sudden!

---

The quality of flang leaves much to be desired especially in complex domain arithmetic, at least openblas tests and Sage tests show a number of errors, but, OK, it is probably fixable on the flang side...

[embray]

Ticket retargeted after milestone closed

[mkoeppe]

Batch modifying tickets that will likely not be ready for 9.1, based on a review of the ticket title, branch/review status, and last modification date.

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[dimpase]

I cannot figure out how to pass the fortran compiler to pip while building SciPy and NumPy. Yes, I know one can do python setup.py build --fcompiler=$FC, assuming FC holds the compiler, but how to combine this with installation by pip3?

[mkoeppe]

numpy's spkg-install.in knows how

[dimpase]

Replying to mkoeppe:

numpy's spkg-install.in knows how

Does it? How does one tell config_fc which Fortran compiler one wants?

Another "interesting" story there is how does one tell config_fc what to do with Fortran files which need a pre-processor? There are Fortran compilers that don't do preprocessing itself, e.g. nagfor from NAG doesn't understand -D and -U options one might need to run on .F files (with C-style #ifdefs); it comes with an fpp utility that is supposed to be run on such files. See ​https://github.com/scipy/scipy/issues/13222#issuecomment-927962803