Opened 21 months ago
Closed 20 months ago
#29056 closed defect (fixed)
numpy: make numpy find openblas from Homebrew
Reported by:  ghmwageringel  Owned by:  

Priority:  minor  Milestone:  sage9.1 
Component:  packages: standard  Keywords:  numpy, scipy 
Cc:  dimpase, mkoeppe  Merged in:  
Authors:  Markus Wageringel  Reviewers:  Matthias Koeppe 
Report Upstream:  N/A  Work issues:  
Branch:  eed1d19 (Commits, GitHub, GitLab)  Commit:  eed1d192ff0e7832bcfbd2e6f6aaa03e1a71878c 
Dependencies:  #29051, #29071, #29084  Stopgaps: 
Description
For numpy to find Homebrew's openblas on macOS, this ticket adds an [openblas]
section to numpy's site.cfg if Sage is using openblas. This is also what Homebrew's numpy formula does. See also the upstream site.cfg.example.
Before this ticket, numpy would use the macOS Accelerate framework instead of openblas, as can be seen from the logs:
FOUND: extra_compile_args = ['msse3', 'I/System/Library/Frameworks/vecLib.framework/Headers'] extra_link_args = ['Wl,framework', 'Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
After:
FOUND: libraries = ['openblas', 'openblas'] library_dirs = ['/usr/local/Cellar/openblas/0.3.7/lib'] language = c define_macros = [('HAVE_CBLAS', None)]
This problem was previously discussed on sagedevel.
Change History (15)
comment:1 Changed 21 months ago by
 Branch set to u/ghmwageringel/29056
 Cc dimpase mkoeppe added
 Commit set to b840c8c550a5287b9a327c14a1e000f276e03960
 Priority changed from major to minor
 Status changed from new to needs_review
comment:2 Changed 20 months ago by
 Commit changed from b840c8c550a5287b9a327c14a1e000f276e03960 to 4d549f2897acce6defc06acd9b5ea1c2be1eb3fe
Branch pushed to git repo; I updated commit sha1. New commits:
4d549f2  29056: fix block syntax

comment:3 followup: ↓ 5 Changed 20 months ago by
won't you be getting KeyError
in the case of pkgconfig not available (and so pkgconfig creating empty dicts)?
comment:4 Changed 20 months ago by
 Commit changed from 4d549f2897acce6defc06acd9b5ea1c2be1eb3fe to 052c06c6e8b3a950699034b8b178fd51ac90ac8e
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
052c06c  29056: make numpy find openblas from Homebrew

comment:5 in reply to: ↑ 3 Changed 20 months ago by
Replying to dimpase:
won't you be getting
KeyError
in the case of pkgconfig not available (and so pkgconfig creating empty dicts)?
Is pkgconfig not always present, possibly by installing the pkgconf spkg? In any case, I changed the relevant line to account for the possibility of an empty dictionary:
 if 'openblas' in pc_blas['libraries']: + if 'openblas' in pc_blas.get('libraries', []):
I have tested this on 9.1.beta1 both using openblas from Homebrew and from Sage and in each case both numpy and scipy find the correct openblas.
comment:6 Changed 20 months ago by
could this be made dependent on #29051 ?
comment:8 followup: ↓ 9 Changed 20 months ago by
perhaps MacOS with pkgconfig and openblas may be handled the same way as the "generic" systems by this script.
comment:9 in reply to: ↑ 8 Changed 20 months ago by
Replying to dimpase:
perhaps MacOS with pkgconfig and openblas may be handled the same way as the "generic" systems by this script.
You mean by removing the Darwin ifcase? I tried that first, but for some reason it did not work, possibly because Numpy has the preference: OpenBLAS > Accelerate > BLAS/LAPACK.
This preference can be changed using the variables NPY_BLAS_ORDER
/NPY_LAPACK_ORDER
, but setting these in Numpy's spkginstall
did not seem to help either.
comment:10 Changed 20 months ago by
comment:11 Changed 20 months ago by
 Branch changed from u/ghmwageringel/29056 to u/mkoeppe/29056
comment:12 Changed 20 months ago by
 Commit changed from 052c06c6e8b3a950699034b8b178fd51ac90ac8e to eed1d192ff0e7832bcfbd2e6f6aaa03e1a71878c
 Reviewers set to Matthias Koeppe
 Status changed from needs_review to positive_review
I've merged in the tickets mentioned above and tested using tox e localhomebrewmacosstandard
from #29104.
New commits:
06f46eb  Add blas to fflas_ffpack linked libraries so that openblas is picked up on Arch

eed1d19  Merge branch 'u/arojas/make_fflas_ffpack_detect_and_use_system_openblas_on_arch' of git://trac.sagemath.org/sage into t/29056/29056

comment:13 Changed 20 months ago by
 Dependencies changed from #29051 to #29051, #29071, #29084
comment:14 Changed 20 months ago by
Thank you.
comment:15 Changed 20 months ago by
 Branch changed from u/mkoeppe/29056 to eed1d192ff0e7832bcfbd2e6f6aaa03e1a71878c
 Resolution set to fixed
 Status changed from positive_review to closed
In order to test this, one currently needs to forcelink Homebrew's readline to
/usr/local
(#29000) and configure Sage withas openblas is kegonly.
New commits:
29056: make numpy find openblas from Homebrew