Opened 9 years ago
Closed 9 years ago
#12519 closed defect (fixed)
cvxopt should not add -lcblas and -latlas on Darwin
Reported by: | jdemeyer | Owned by: | tbd |
---|---|---|---|
Priority: | major | Milestone: | sage-5.0 |
Component: | packages: standard | Keywords: | |
Cc: | Merged in: | sage-5.0.beta8 | |
Authors: | Jeroen Demeyer | Reviewers: | Dmitrii Pasechnik |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
The following is Sage-specific code in cvxopt's setup.py
:
if os.popen('sage_fortran --version').read()[:3]=='G95': libraries = ['m','lapack','gsl','gslcblas','blas','f95'] GCC_LIB_DIR=SAGE_LIB if os.path.exists(GCC_LIB_DIR + "/gcc-lib"): GCC_LIB_DIR += "/gcc-lib/" GCC_LIB_DIR += os.listdir(GCC_LIB_DIR)[0] + "/" GCC_LIB_DIR += os.listdir(GCC_LIB_DIR)[0] + "/" libdirs = [ATLAS_LIB_DIR,GCC_LIB_DIR] elif os.environ['UNAME'] == 'CYGWIN': libraries = ['lapack','gsl','gslcblas','blas', 'gfortran'] else: libraries = ['m','lapack','gsl','blas','gslcblas','cblas','gfortran','atlas']
At the end, cblas
and atlas
are added, even though they aren't installed on Darwin. This causes a build failure for cvxopt on OS X 10.4 with #12369 (without #12369, the first "if" evaluates to True).
In this ticket, I also fixed the "spkg-check" script, which will now actually fail if there is a failure.
spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/cvxopt-1.1.3.p1.spkg
Attachments (4)
Change History (30)
comment:1 Changed 9 years ago by
- Description modified (diff)
- Summary changed from cvxopt should only add -lcblas when that library exists to cvxopt should not add -lcblas on Darwin
comment:2 Changed 9 years ago by
- Description modified (diff)
- Summary changed from cvxopt should not add -lcblas on Darwin to cvxopt should not add -lcblas and -latlas on Darwin
comment:3 Changed 9 years ago by
- Description modified (diff)
comment:4 follow-up: ↓ 6 Changed 9 years ago by
- Status changed from new to needs_review
comment:5 Changed 9 years ago by
- Status changed from needs_review to needs_work
comment:6 in reply to: ↑ 4 ; follow-up: ↓ 7 Changed 9 years ago by
Replying to jdemeyer:
Am I correct in saying that this was caused by Darwin being determined by the presence of G95, which is no longer the right way to do this (or perhaps it never was --- mea culpa then...) ?
Regarding the failures of tests on OSX 10.7 (see #12011): it would be great if the exact Sage version (gcc/clang?) /Xcode version/hardware state (32 vs 64 bits) on these are reported. As it looks at the moment, that there is a bug in Apple-supplied lapack/blas, and this might force us to build Atlas on OSX 10.7 and use it.
comment:7 in reply to: ↑ 6 ; follow-ups: ↓ 9 ↓ 10 ↓ 11 Changed 9 years ago by
Replying to dimpase:
Regarding the failures of tests on OSX 10.7 (see #12011): it would be great if the exact Sage version (gcc/clang?) /Xcode version/hardware state (32 vs 64 bits) on these are reported.
In my experience, the tests fail on every OS X 10.7 machine I've tried, with Xcode 4.1, 4.2, 4.3, with both gcc and clang.
As it looks at the moment, that there is a bug in Apple-supplied lapack/blas, and this might force us to build Atlas on OSX 10.7 and use it.
That would be unfortunate. I also wonder why we don't see doctest failures elsewhere in Sage if there are bugs in these libraries.
comment:8 Changed 9 years ago by
- Status changed from needs_work to needs_review
Fixed, needs review.
comment:9 in reply to: ↑ 7 Changed 9 years ago by
Replying to jhpalmieri:
That would be unfortunate. I also wonder why we don't see doctest failures elsewhere in Sage if there are bugs in these libraries.
Because the cvxopt test suite itself is more complete than Sage? Maybe it just fails on some corner case which isn't in the doctests. I agree it would be a good idea to add a doctest to catch the OS X 10.7 failures.
comment:10 in reply to: ↑ 7 Changed 9 years ago by
Replying to jhpalmieri:
As it looks at the moment, that there is a bug in Apple-supplied lapack/blas, and this might force us to build Atlas on OSX 10.7 and use it.
That would be unfortunate. I also wonder why we don't see doctest failures elsewhere in Sage if there are bugs in these libraries.
I asked upstream for examples demonstrating lapack/blas bug(s) on OSX 10.7. They are looking into it.
comment:11 in reply to: ↑ 7 ; follow-up: ↓ 12 Changed 9 years ago by
Replying to jhpalmieri:
In my experience, the tests fail on every OS X 10.7 machine I've tried, with Xcode 4.1, 4.2, 4.3, with both gcc and clang.
here is a beautiful OSX 10.7-specific bug (got it on sqrt5 with Sage 5.0.beta6): (the syntax const-M, for M a matrix, is supposed to replace each M_ij with const-M_ij)
from cvxopt import matrix b=matrix([ 1.00e+00, 1.63e-01, 2.83e-01, 9.47e-01, 2.32e-01, 4.85e-01, 9.57e-01, 7.44e-01]) c=1.-b print "b :\n", b print "1-b:\n", c b : [ 1.00e+00] [ 1.63e-01] [ 2.83e-01] [ 9.47e-01] [ 2.32e-01] [ 4.85e-01] [ 9.57e-01] [ 7.44e-01] 1-b: [-1.00e+00] [-1.63e-01] [-2.83e-01] [-9.47e-01] [-2.32e-01] [-4.85e-01] [-9.57e-01] [-7.44e-01]
Interestingly, a shorter vector b works OK...
Do you have any idea what goes wrong here? (this is actually the bug responsible for failure of "acent" test in here)
comment:12 in reply to: ↑ 11 ; follow-up: ↓ 13 Changed 9 years ago by
comment:13 in reply to: ↑ 12 ; follow-up: ↓ 14 Changed 9 years ago by
Replying to jdemeyer:
Replying to dimpase:
Do you have any idea what goes wrong here?
Probably Apple shipping a broken BLAS. See #12011.
that's what one of CVXOPT people says (as I reported :)), but it would be good to know for sure. I've reported this example upstream (to CVXOPT), too.
I forgot to mention that this is a plain Python example. To run it in Sage proper, one needs to do first
sage: preparser(on=False)
comment:14 in reply to: ↑ 13 ; follow-up: ↓ 16 Changed 9 years ago by
comment:15 Changed 9 years ago by
- Status changed from needs_review to needs_work
This needs to be coordinated with #12011.
comment:16 in reply to: ↑ 14 ; follow-up: ↓ 17 Changed 9 years ago by
comment:17 in reply to: ↑ 16 Changed 9 years ago by
- Status changed from needs_work to needs_info
Replying to dimpase:
Replying to dimpase:
Replying to dimpase:
Probably Apple shipping a broken BLAS. See #12011.
confirmed! see the following sage-devel thread
In fact, it turns out to be a CVXOPT bug. Please see this message and this discussion)
So we do not need Atlas on OSX 10.7, after all (or at least not yet).
Hopefully CVXOPT people can fix this quickly: the affected BLAS calls are used quite a bit in their code, so I'd prefer to wait for them to have it fixed rather than going at it myself (it looks straightforward, but quite a bit of code...)
comment:18 Changed 9 years ago by
- Status changed from needs_info to needs_review
comment:19 follow-up: ↓ 20 Changed 9 years ago by
- Status changed from needs_review to needs_work
Please revert to using Apple's blas, not GSL's ones.
comment:20 in reply to: ↑ 19 ; follow-up: ↓ 21 Changed 9 years ago by
- Status changed from needs_work to needs_review
comment:21 in reply to: ↑ 20 ; follow-up: ↓ 22 Changed 9 years ago by
- Status changed from needs_review to needs_info
comment:22 in reply to: ↑ 21 ; follow-ups: ↓ 23 ↓ 24 Changed 9 years ago by
- Status changed from needs_info to needs_review
Replying to dimpase:
I suppose inclusion of gslcblas in the list of libaries means we use this very slow beast...
Are you sure? If we're not using Apple's BLAS, how do you explain that cvxopt fails its self-tests only on OS X 10.7?
Anyway: let me also clarify that I'm simply restoring the old behaviour, such that cvxopt
builds with GCC 4.6.2. So if there really is something to be done, I prefer to get this ticket reviewed as-is and continue on a different ticket.
comment:23 in reply to: ↑ 22 Changed 9 years ago by
Replying to jdemeyer:
Replying to dimpase:
I suppose inclusion of gslcblas in the list of libaries means we use this very slow beast...
Are you sure? If we're not using Apple's BLAS, how do you explain that cvxopt fails its self-tests only on OS X 10.7?
-lblas
links against Apple's "classical" BLAS (which you call with Fortran conventions), and use names ending with '_'. And
that's how CVXOPT makes these fateful calls to daxpy().
Actually, it occurs to me now that it's GSL extension of CVXOPT that needs its GSLCBLAS, nothing else. Sorry for noise. Please disregard my request on this.
Anyway: let me also clarify that I'm simply restoring the old behaviour, such that
cvxopt
builds with GCC 4.6.2. So if there really is something to be done, I prefer to get this ticket reviewed as-is and continue on a different ticket.
OK, I'll look into it ASAP.
comment:24 in reply to: ↑ 22 Changed 9 years ago by
- Status changed from needs_review to positive_review
Replying to jdemeyer:
Replying to dimpase:
I suppose inclusion of gslcblas in the list of libaries means we use this very slow beast...
Are you sure? If we're not using Apple's BLAS, how do you explain that cvxopt fails its self-tests only on OS X 10.7?
Anyway: let me also clarify that I'm simply restoring the old behaviour, such that
cvxopt
builds with GCC 4.6.2. So if there really is something to be done, I prefer to get this ticket reviewed as-is and continue on a different ticket.
looks OK on OSX 10.6.8, and on 10.5.8 PPC (installs with SAGE_CHECK=yes). I presume you have tested on OSX 10.4. Positive review. I'd like to coordinate the upgrade to (now available) cvxopt 1.1.4 with them fixing the OSX 10.7-specific bug.
comment:25 Changed 9 years ago by
- Reviewers set to Dmitrii Pasechnik
comment:26 Changed 9 years ago by
- Merged in set to sage-5.0.beta8
- Resolution set to fixed
- Status changed from positive_review to closed
In spkg-install, these lines should be removed:
since they are taken care of by patch.