#29056 closed defect (fixed)

numpy: make numpy find openblas from Homebrew

Reported by: gh-mwageringel Owned by:
Priority: minor Milestone: sage-9.1
Component: packages: standard Keywords: numpy, scipy
Cc: dimpase, mkoeppe Merged in:
Authors: Markus Wageringel Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: eed1d19 (Commits, GitHub, GitLab) Commit: eed1d192ff0e7832bcfbd2e6f6aaa03e1a71878c
Dependencies: #29051, #29071, #29084 Stopgaps:

Status badges

Description

For numpy to find Homebrew's openblas on macOS, this ticket adds an [openblas] section to numpy's site.cfg if Sage is using openblas. This is also what Homebrew's numpy formula does. See also the upstream site.cfg.example.

Before this ticket, numpy would use the macOS Accelerate framework instead of openblas, as can be seen from the logs:

  FOUND:
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]

After:

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/Cellar/openblas/0.3.7/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

This problem was previously discussed on sage-devel.

Change History (15)

comment:1 Changed 21 months ago by gh-mwageringel

  • Authors set to Markus Wageringel
  • Branch set to u/gh-mwageringel/29056
  • Cc dimpase mkoeppe added
  • Commit set to b840c8c550a5287b9a327c14a1e000f276e03960
  • Priority changed from major to minor
  • Status changed from new to needs_review

In order to test this, one currently needs to force-link Homebrew's readline to /usr/local (#29000) and configure Sage with

./configure LDFLAGS="-L/usr/local/opt/openblas/lib" CPPFLAGS="-I/usr/local/opt/openblas/include" PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

as openblas is keg-only.


New commits:

b840c8c29056: make numpy find openblas from Homebrew

comment:2 Changed 20 months ago by git

  • Commit changed from b840c8c550a5287b9a327c14a1e000f276e03960 to 4d549f2897acce6defc06acd9b5ea1c2be1eb3fe

Branch pushed to git repo; I updated commit sha1. New commits:

4d549f229056: fix block syntax

comment:3 follow-up: Changed 20 months ago by dimpase

won't you be getting KeyError in the case of pkg-config not available (and so pkgconfig creating empty dicts)?

comment:4 Changed 20 months ago by git

  • Commit changed from 4d549f2897acce6defc06acd9b5ea1c2be1eb3fe to 052c06c6e8b3a950699034b8b178fd51ac90ac8e

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

052c06c29056: make numpy find openblas from Homebrew

comment:5 in reply to: ↑ 3 Changed 20 months ago by gh-mwageringel

Replying to dimpase:

won't you be getting KeyError in the case of pkg-config not available (and so pkgconfig creating empty dicts)?

Is pkg-config not always present, possibly by installing the pkgconf spkg? In any case, I changed the relevant line to account for the possibility of an empty dictionary:

-    if 'openblas' in pc_blas['libraries']:
+    if 'openblas' in pc_blas.get('libraries', []):

I have tested this on 9.1.beta1 both using openblas from Homebrew and from Sage and in each case both numpy and scipy find the correct openblas.

comment:6 Changed 20 months ago by dimpase

could this be made dependent on #29051 ?

comment:7 Changed 20 months ago by gh-mwageringel

  • Dependencies set to #29051

Of course. Let us postpone this then.

comment:8 follow-up: Changed 20 months ago by dimpase

perhaps MacOS with pkg-config and openblas may be handled the same way as the "generic" systems by this script.

comment:9 in reply to: ↑ 8 Changed 20 months ago by gh-mwageringel

Replying to dimpase:

perhaps MacOS with pkg-config and openblas may be handled the same way as the "generic" systems by this script.

You mean by removing the Darwin if-case? I tried that first, but for some reason it did not work, possibly because Numpy has the preference: OpenBLAS > Accelerate > BLAS/LAPACK.

This preference can be changed using the variables NPY_BLAS_ORDER/NPY_LAPACK_ORDER, but setting these in Numpy's spkg-install did not seem to help either.

comment:10 Changed 20 months ago by mkoeppe

I would suggest to also add #29071, #29084 as prereqs

comment:11 Changed 20 months ago by mkoeppe

  • Branch changed from u/gh-mwageringel/29056 to u/mkoeppe/29056

comment:12 Changed 20 months ago by mkoeppe

  • Commit changed from 052c06c6e8b3a950699034b8b178fd51ac90ac8e to eed1d192ff0e7832bcfbd2e6f6aaa03e1a71878c
  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

I've merged in the tickets mentioned above and tested using tox -e local-homebrew-macos-standard from #29104.


New commits:

06f46ebAdd blas to fflas_ffpack linked libraries so that openblas is picked up on Arch
eed1d19Merge branch 'u/arojas/make_fflas_ffpack_detect_and_use_system_openblas_on_arch' of git://trac.sagemath.org/sage into t/29056/29056

comment:13 Changed 20 months ago by mkoeppe

  • Dependencies changed from #29051 to #29051, #29071, #29084

comment:14 Changed 20 months ago by gh-mwageringel

Thank you.

comment:15 Changed 20 months ago by vbraun

  • Branch changed from u/mkoeppe/29056 to eed1d192ff0e7832bcfbd2e6f6aaa03e1a71878c
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.