Opened 9 years ago

Closed 9 years ago

#12011 closed defect (fixed)

cvxopt: fix illegal BLAS call and fix Solaris build

Reported by: jhpalmieri Owned by: tbd
Priority: blocker Milestone: sage-5.0
Component: packages: standard Keywords: Lion Darwin cvxopt atlas blas Solaris
Cc: dimpase, vbraun Merged in: sage-5.0.beta13
Authors: Jeroen Demeyer, John Palmieri Reviewers: Jeroen Demeyer, John Palmieri
Report Upstream: Fixed upstream, in a later stable release. Work issues:
Branch: Commit:
Dependencies: #12519 Stopgaps:

Description (last modified by jdemeyer)

On OS X Lion, if you set SAGE_CHECK=yes and build cvxopt, some tests fail and then it hangs before completing the tests. See http://sage.math.washington.edu/home/palmieri/misc/cvxopt-1.1.3.log for the install log.

CVXOPT devs claimed that Apple shipped a broken BLAS. But it turned out that they cannot provide valid examples showing this; they did not adhere to a de facto BLAS standard, and they agreed to fix the corresponding CVXOPT bug quickly.

This spkg also fixes an issue building with GCC on Solaris 10. There already was a fix for this in the spkg, but it was only enabled for GCC versions < 4.5 (without any mention of why). Enable this fix for all GCC versions.


New spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/cvxopt-1.1.4.p1.spkg

Attachments (2)

trac_12011-cvxopt.patch (8.7 KB) - added by jhpalmieri 9 years ago.
patch for cvxopt spkg; for review only
cvxopt-1.1.4.p1.diff (2.9 KB) - added by jdemeyer 9 years ago.
Diff for the cvxopt spkg (p0->p1). For review only

Download all attachments as: .zip

Change History (65)

comment:1 Changed 9 years ago by jdemeyer

  • Milestone changed from sage-4.8 to sage-5.0

comment:2 Changed 9 years ago by jhpalmieri

In a related matter, we should patch spkg-check to exit with a nonzero return value if tests fail. We can either exit immediately once there's a failure, or we can run all tests and then exit. I'm not sure which of these is better. In general I would suggest the latter, but since it's actually hanging on OS X Lion (after at least one failed test), I might lean toward the former...

Also, should we upgrade to version 1.1.4?

comment:3 Changed 9 years ago by jdemeyer

See #12519 for a new cvxopt spkg. I fixed spkg-check such that it now actually exists when there is a failure. However, it still fails on OS X Lion.

comment:4 follow-up: Changed 9 years ago by jhpalmieri

  • Cc dimpase added

Dima: any ideas about the self-test failures?

comment:5 in reply to: ↑ 4 ; follow-up: Changed 9 years ago by dimpase

  • Report Upstream changed from N/A to Reported upstream. Little or no feedback.

Replying to jhpalmieri:

Dima: any ideas about the self-test failures?

I just asked upstream here. Is there a way to get access to 10.7 for me?

Dima

comment:6 in reply to: ↑ 5 ; follow-up: Changed 9 years ago by dimpase

  • Status changed from new to needs_info

Replying to dimpase:

Replying to jhpalmieri:

Dima: any ideas about the self-test failures?

I just asked upstream here. Is there a way to get access to 10.7 for me?

Dima

here it is written that it's probably a bug in Apple's blas/lapack. A workaround is to use Atlas. I have no way to check this at the moment, but it looks very reasonable to me.

It would be great if you tested this with the newest Xcode...

comment:7 in reply to: ↑ 6 ; follow-up: Changed 9 years ago by jhpalmieri

Replying to dimpase:

here it is written that it's probably a bug in Apple's blas/lapack. A workaround is to use Atlas. I have no way to check this at the moment, but it looks very reasonable to me.

Do you mean Apple's ATLAS? We don't build ATLAS on Mac OS X, we just use the system's libraries. What should I do to try this out? Can I just make some changes in the spkg-install script?

It would be great if you tested this with the newest Xcode...

I've tried this with various versions of Xcode, including the newly released 4.3, and I get failures in all of them.

comment:8 Changed 9 years ago by jhpalmieri

By the way, it might make sense to base further discussion on the new cvxopt package at #12519.

comment:9 in reply to: ↑ 7 Changed 9 years ago by dimpase

Replying to jhpalmieri:

Replying to dimpase:

here it is written that it's probably a bug in Apple's blas/lapack. A workaround is to use Atlas. I have no way to check this at the moment, but it looks very reasonable to me.

Do you mean Apple's ATLAS? We don't build ATLAS on Mac OS X, we just use the system's libraries. What should I do to try this out? Can I just make some changes in the spkg-install script?

Well, no. We need to build ATLAS spkg (or get some other non-Apple lapack/blas somehow) and use it. Actually, if I recall right, this (native lapack/blas) is an artefact of the PPC support. We can perfectly well use Atlas on x86, right?

It would be great if you tested this with the newest Xcode...

I've tried this with various versions of Xcode, including the newly released 4.3, and I get failures in all of them.

comment:10 Changed 9 years ago by jdemeyer

  • Cc dimpasLion Darwin cvxopt e added; dimpase removed
  • Description modified (diff)
  • Keywords atlas blas added
  • Summary changed from OS X Lion: cvxopt fails self tests to Build ATLAS on OS X Lion

comment:11 Changed 9 years ago by jdemeyer

  • Cc dimpase added; dimpasLion Darwin cvxopt e removed
  • Description modified (diff)

comment:12 Changed 9 years ago by dimpase

doesnt work on OSX10.6:

Calling sage-spkg on 'http://boxen.math.washington.edu/home/jdemeyer/spkg/atlas-3.8.4.p2.spkg'
atlas-3.8.4.p2
Machine:
Darwin nash 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386
Deleting directories from past builds of previous/current versions of atlas-3.8.4.p2
/usr/local/src/sage/sage-4.8.alpha4/local/bin/sage-spkg: file atlas-3.8.4.p2 does not exist
Attempting to download it.
http://boxen.math.washington.edu/home/jdemeyer/spkg/atlas-3.8.4.p2.spkg --> atlas-3.8.4.p2.spkg
[..................................................]
Extracting package /usr/local/src/sage/sage-4.8.alpha4/spkg/optional/atlas-3.8.4.p2.spkg ...
-rw-r--r--  1 dima  staff  2866339 Mar  6 19:27 /usr/local/src/sage/sage-4.8.alpha4/spkg/optional/atlas-3.8.4.p2.spkg
Finished extraction
****************************************************
Host system
uname -a:
Darwin nash 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386
****************************************************
****************************************************
CC Version
gcc -v
Using built-in specs.
Target: i686-apple-darwin10
Configured with: /var/tmp/gcc/gcc-5666.3~6/src/configure --disable-checking --enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin10 --program-prefix=i686-apple-darwin10- --host=x86_64-apple-darwin10 --target=i686-apple-darwin10 --with-gxx-include-dir=/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)
****************************************************
Traceback (most recent call last):
  File "./spkg-install", line 13, in <module>
    from configuration import conf, cp, try_run, edit_in_place
  File "/usr/local/src/sage/sage-4.8.alpha4/spkg/build/atlas-3.8.4.p2/configuration.py", line 108, in <module>
    conf['release'] = subprocess.check_output(['uname', '-r'])
AttributeError: 'module' object has no attribute 'check_output'

real	0m0.092s
user	0m0.038s
sys	0m0.044s
************************************************************************
Error installing package atlas-3.8.4.p2
************************************************************************
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and including the relevant part of the log file
  /usr/local/src/sage/sage-4.8.alpha4/spkg/logs/atlas-3.8.4.p2.log
Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/usr/local/src/sage/sage-4.8.alpha4/spkg/build/atlas-3.8.4.p2 and type 'make' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
  (cd '/usr/local/src/sage/sage-4.8.alpha4/spkg/build/atlas-3.8.4.p2' && '/usr/local/src/sage/sage-4.8.alpha4/sage' -sh)
When you are done debugging, you can type "exit" to leave the subshell.
************************************************************************
Error: Failed to install package 'atlas-3.8.4.p2'.
nash:sage-5.0.beta6 dima$ 

comment:13 follow-up: Changed 9 years ago by jdemeyer

You're using a too old version of Sage.

comment:14 in reply to: ↑ 13 Changed 9 years ago by dimpase

Replying to jdemeyer:

You're using a too old version of Sage.

/me bangs head on the kbd...

comment:15 follow-up: Changed 9 years ago by kcrisman

Does this only build ATLAS in Lion, or all Mac? Seems silly to do it from scratch on others, maybe we can check Dima's program mentioned at #12519 on older versions...

comment:16 in reply to: ↑ 15 Changed 9 years ago by jdemeyer

Replying to kcrisman:

Does this only build ATLAS in Lion, or all Mac?

Only on Lion.

comment:17 Changed 9 years ago by jdemeyer

  • Cc vbraun added

After more than 3 hours, it finally managed to fail to build on sqrt5:

ATLAS install complete.  Examine
ATLAS/bin/<arch>/INSTALL_LOG/SUMMARY.LOG for details.
make -j1 clean
rm -f *.o x* config?.out *core*
rm: xarchinfo_freebsd.dSYM: is a directory
rm: xarchinfo_x86.dSYM: is a directory
rm: xconfig.dSYM: is a directory
rm: xprobe_gas_x8632.dSYM: is a directory
rm: xprobe_gas_x8664.dSYM: is a directory
rm: xspew.dSYM: is a directory
make[1]: *** [clean] Error 1
make: *** [build] Error 2
ATLAS failed to build, possibly because of a loaded system.
Traceback (most recent call last):
  File "./spkg-install", line 473, in <module>
    assert rc==0, 'Failed to build ATLAS.'
AssertionError: Failed to build ATLAS.

real    202m1.081s
user    182m33.861s
sys     10m48.195s

Weird that it reported that the build succeeded, but still it failed somehow.

comment:18 Changed 9 years ago by jdemeyer

Replace a "rm -f" by "rm -rf" probably will solve this problem.

This explain the dSYM directories: http://stackoverflow.com/questions/584825/dsym-directories-while-compiling-c-code-in-macos

comment:19 Changed 9 years ago by kcrisman

Aah, you beat me by five seconds! I concur.

comment:20 Changed 9 years ago by jdemeyer

I put up a new version hopefully fixing this, currently building on sqrt5...

comment:21 Changed 9 years ago by jdemeyer

:-(

ATLAS install complete.  Examine
ATLAS/bin/<arch>/INSTALL_LOG/SUMMARY.LOG for details.
make -j1 clean
# Xcode (OS X) may create directories like xspew.dSYM.
# Add -r option to delete also these.
rm -rf *.o x* config?.out *core*
Finished building ATLAS core.
Finished building ATLAS core.
Running make -j1 shared cshared
rm -f libatlas.so liblapack.so
make -j1 libatlas.so liblapack.so libf77blas.so libcblas.so liblapack.so
ld -melf_x86_64 -shared -soname libatlas.so -o libatlas.so \
        --whole-archive libatlas.a --no-whole-archive -lc -lpthread -lm
ld: unknown option: -melf_x86_64
make[1]: *** [libatlas.so] Error 1
make: *** [shared] Error 2
Traceback (most recent call last):
  File "./spkg-install", line 480, in <module>
    assert rc==0, 'Building shared ATLAS library failed.'
AssertionError: Building shared ATLAS library failed.

comment:22 Changed 9 years ago by jdemeyer

  • Authors set to Jeroen Demeyer
  • Status changed from needs_info to needs_review

We need to totally rewrite the command line

ld -melf_x86_64 -shared -soname libatlas.so -o libatlas.so \
        --whole-archive libatlas.a --no-whole-archive -lc -lpthread -lm

I unfortunately don't know enough about Apple's linker to know what to do here...

comment:23 Changed 9 years ago by jdemeyer

  • Status changed from needs_review to needs_work

comment:24 Changed 9 years ago by vbraun

According to Clint Whaley, only ATLAS 3.9+ supports building a shared library on OSX:

http://sourceforge.net/tracker/?func=detail&aid=3178751&group_id=23725&atid=379483

comment:25 follow-up: Changed 9 years ago by vbraun

I've made a new atlas spkg here:

http://www.stp.dias.ie/~vbraun/Sage/spkg/atlas-3.9.68.spkg

Testing will take some time...

comment:26 in reply to: ↑ 25 ; follow-up: Changed 9 years ago by dimpase

Replying to vbraun:

I've made a new atlas spkg here:

http://www.stp.dias.ie/~vbraun/Sage/spkg/atlas-3.9.68.spkg

Testing will take some time...

on sqrt5 with Sage 5.0.beta7:

...
    Intel?: True
    processor: i386
Installing ATLAS on OS X Lion, as the BLAS shipped with the system
is broken, see http://trac.sagemath.org/sage_trac/ticket/12011
Copying patches/Makefile to src/CONFIG/src/Makefile
Copying patches/archinfo_linux.c to src/CONFIG/src/backend/archinfo_linux.c
Copying patches/archinfo_x86.c to src/CONFIG/src/backend/archinfo_x86.c
Copying patches/probe_comp.c to src/CONFIG/src/probe_comp.c
First attempt: automatic tuning.
Running configure with arch = None and isa extensions None
Traceback (most recent call last):
  File "./spkg-install", line 408, in <module>
    rc = build()
  File "./spkg-install", line 396, in build
    rc = configure(arch, isa_ext)    
  File "./spkg-install", line 278, in configure
    cmd += ' -O'+atlas_system
TypeError: cannot concatenate 'str' and 'int' objects

comment:27 in reply to: ↑ 26 Changed 9 years ago by dimpase

Replying to dimpase:

on sqrt5 with Sage 5.0.beta7:

  File "./spkg-install", line 278, in configure
    cmd += ' -O'+atlas_system

Replace this line with

cmd += ' -O '+str(atlas_system)

(note the space after O!)

The result can be obtained here: http://boxen.math.washington.edu/home/dima/packages/atlas-3.9.68.spkg

Then is starts building. I'll report when it's done.

comment:28 follow-up: Changed 9 years ago by vbraun

I've also fixed the spkg in the meantime, for the record

comment:29 in reply to: ↑ 28 Changed 9 years ago by dimpase

Replying to vbraun:

I've also fixed the spkg in the meantime, for the record

sqrt5:standard dima$ mailx vbraun
Subject: let's not try to build 2 atlaces at the same time 
OK? :)
EOT

comment:30 Changed 9 years ago by jdemeyer

Unrelated to this ticket, but why include LAPACK inside the ATLAS spkg?

comment:31 Changed 9 years ago by dimpase

The following code (compile with Sage's gfortran, with -lblas library option) crashes not only on OSX 10.7, but also on OSX 10.6.

Should we also provide Atlas on 10.6 ?

        PROGRAM TEST
        COMPLEX*16 A(1), B(1),C
        COMPLEX*16 ZDOTC
        EXTERNAL ZDOTC
        A(1)=1;
        B(1)=2;
        C=ZDOTC( 1, A, 1, B, 1 )
        WRITE (*,*) C
        END

comment:32 Changed 9 years ago by dimpase

well, no, no luck yet. atlas-3.9.68 fails on sqrt5 with plenty of ld: unknown option: -melf_x86_64 messages at the end.

...
make[2]: *** [LDTRY] Error 1
ld -melf_x86_64 -shared -soname /Users/dima/Sage/sage-5.0.beta7/local/lib/libtatlas.so -o libtatlas.so \
           -rpath-link /Users/dima/Sage/sage-5.0.beta7/local/lib \
           --whole-archive libptlapack.a libptf77blas.a libptcblas.a libatlas.a --no-whole-archive -lpthread -lm
ld: unknown option: -melf_x86_64
make[2]: *** [LDTRY] Error 1
gcc -fPIC -m64 -shared -o libtatlas.so -Wl,"rpath-link /Users/dima/Sage/sage-5.0.beta7/local/lib" \
           -Wl,--whole-archive libptlapack.a libptf77blas.a libptcblas.a libatlas.a -Wl,--no-whole-archive -lpthread -lm
ld: file not found: rpath-link /Users/dima/Sage/sage-5.0.beta7/local/lib
collect2: ld returned 1 exit status
make[2]: *** [GCCTRY] Error 1
make[1]: *** [TRYALL] Error 2
make: *** [fat_ptshared] Error 2
Traceback (most recent call last):
  File "./spkg-install", line 431, in <module>
    assert rc==0, 'Building shared ATLAS library failed.'
AssertionError: Building shared ATLAS library failed.

real    683m42.661s
user    200m29.372s
sys     36m57.615s
************************************************************************
Error installing package atlas-3.9.68
************************************************************************

comment:33 follow-up: Changed 9 years ago by vbraun

atlas-3.9 is designed to build lapack as part of its build process. This is why lapack is now included and can be removed as separate spkg (see discussion at #10508).

Fortunately, atlas-3.9.68 has actually make targets for OSX dylibs so fixing the remaining issue should be easy. Updated spkg at http://www.stp.dias.ie/~vbraun/Sage/spkg/atlas-3.9.68.spkg, though its still compiling of course...

comment:34 Changed 9 years ago by fbissey

I am not thrilled by atlas-3.9.xx. Do we know if they have have finally solved some of the problems they have in dgemm? At least 3.9.68 seems to have all the symbols unlike 3.9.67 which somehow left a pile of lapack symbols behind.

comment:35 Changed 9 years ago by vbraun

For the record, I've tentatively run some tests on Linux haven't found any breakage so far with atlas-3.9.68.

comment:36 Changed 9 years ago by fbissey

Didn't test 3.9.68 (last one I checked was 3.9.49). The following tests were historically failing with 3.9.xx:

sage -t -force_lib "devel/sage/sage/modular/modsym/space.py"
sage -t -force_lib "devel/sage/sage/calculus/interpolators.pyx"

comment:37 Changed 9 years ago by vbraun

  • Description modified (diff)

comment:38 in reply to: ↑ 33 ; follow-up: Changed 9 years ago by jdemeyer

Replying to vbraun:

atlas-3.9 is designed to build lapack as part of its build process. This is why lapack is now included and can be removed as separate spkg (see discussion at #10508).

Does this imply we have to build ATLAS+LAPACK on every system (we used to build LAPACK on every system)? Not saying this is a bad thing, just asking...

Putting lapack in patches is just weird, you should probably distribute it extracted (either inside src or in a lapack top-level directory in the spkg).

comment:39 Changed 9 years ago by jdemeyer

If you patch spkg/install, please keep in mind #10492 (which has positive_review).

comment:40 in reply to: ↑ 38 Changed 9 years ago by dimpase

Replying to jdemeyer:

Replying to vbraun:

atlas-3.9 is designed to build lapack as part of its build process. This is why lapack is now included and can be removed as separate spkg (see discussion at #10508).

Does this imply we have to build ATLAS+LAPACK on every system (we used to build LAPACK on every system)? Not saying this is a bad thing, just asking...

hmm, all? Are we dropping OSX PPC support? IMHO Atlas doesn't run on OSX PPC. Also, I don't think we build Atlas on ARM. (And Linux ARM support sort of works, it's matter of just a little bit more work)

Putting lapack in patches is just weird, you should probably distribute it extracted (either inside src or in a lapack top-level directory in the spkg).

comment:41 Changed 9 years ago by jhpalmieri

The new ATLAS spkg builds for me on several OS X Lion machines. The self-tests are disabled on Darwin (I realized after installing it), so they didn't run; I'm trying again after modifying spkg-check. In an existing Sage build (one in which lapack and blas were already installed), after building ATLAS, the cvxopt spkg still fails self-tests. Do we need to modify that spkg to tell it where to find the ATLAS libraries? Or could there be a problem with the old lapack or blas?

comment:42 Changed 9 years ago by jhpalmieri

As far as installing this on OS X: I think we can test the existing ATLAS and/or BLAS libraries with Python, and Python is a prerequisite for this spkg. So why not test to see if the system libraries seem to be functioning; if so, don't install this. Then we shouldn't install this on pre-Lion machines, and if Apple ever fixes their system libraries, we won't install it then either.

comment:43 follow-up: Changed 9 years ago by vbraun

I think we have some more work to do until we can ship the new atlas on any system, you are a bit premature in discussing how to support old systems.

The only way to be sure that you don't have statically linked to another blas is to build Sage from source with the new atlas spkg. This doesn't add much overhead over building atlas.

Also, the atlas shared libraries are now named lib{t,s}atlas.{so,dylib} for threaded/singlethreaded and unix/osx respectively. We need to change modules_list.py and sage/misc/cython.py to actually use the new libraries. How about one of you with an actual OSX machine makes these config changes.

Also, on Linux I'm getting

/home/vbraun/opt/sage-atlas/sage-5.0.beta7/local/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: dpotrf_

when installing scipy. I'll try to track this down.

comment:44 in reply to: ↑ 43 Changed 9 years ago by fbissey

Replying to vbraun:

I think we have some more work to do until we can ship the new atlas on any system, you are a bit premature in discussing how to support old systems.

The only way to be sure that you don't have statically linked to another blas is to build Sage from source with the new atlas spkg. This doesn't add much overhead over building atlas.

Also, the atlas shared libraries are now named lib{t,s}atlas.{so,dylib} for threaded/singlethreaded and unix/osx respectively. We need to change modules_list.py and sage/misc/cython.py to actually use the new libraries. How about one of you with an actual OSX machine makes these config changes.

Also, on Linux I'm getting

/home/vbraun/opt/sage-atlas/sage-5.0.beta7/local/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: dpotrf_

when installing scipy. I'll try to track this down.

Which lapack version are you using in the package. We had quite a few problem of that kind in Gentoo and Sebastien Fabbro (the package maintainer of atlas) did go from lapack-3.4.0_p20120215 back to lapack-3.4.0 to solve the problem.

comment:45 follow-up: Changed 9 years ago by vbraun

I'm using lapack-3.4.0 in atlas-3.9.68 (did you mean lapack-3.3.0?)

comment:46 in reply to: ↑ 45 Changed 9 years ago by fbissey

Replying to vbraun:

I'm using lapack-3.4.0 in atlas-3.9.68 (did you mean lapack-3.3.0?)

No I meant 3.4.0, the reports that we had were for a different symbols but there could have been several missing for all I know. dpotrf_ is definitely present in my system install of atlas-lapack which is atlas 3.9.68 + lapack-3.4.0. But it may be worth a try with 3.3.1. I am looking at the spkg (I am on irc right now).

comment:47 Changed 9 years ago by jhpalmieri

Now on OS X Lion, with Sage built from scratch (using the root repo patch from this ticket), this spkg passes its self-tests, but cvxopt doesn't:

Running the test suite for cvxopt-1.1.3.p1...
Testing in /Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/spkg/build/cvxopt-1.1.3.p1/src/examples/doc/chap10
Testing l1svc.py ...
Traceback (most recent call last):
  File "l1svc.py", line 12, in <module>
    op(sum(abs(x)) + sum(u), [A*x >= 1-u, u >= 0]).solve()
  File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/modeling.py", line 2575, in solve
    sol = solvers.lp(c[:], G, h, A, b, solver=solver)
  File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/coneprog.py", line 2794, in lp
    from cvxopt import base, blas, misc
  File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/misc.py", line 20, in <module>
    from cvxopt import base, blas, lapack, cholmod, misc_solvers
ImportError: dlopen(/Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/lapack.so, 2): Symbol not found: _ATL_cGetNB
  Referenced from: /Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/lapack.so
  Expected in: flat namespace
 in /Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/lapack.so
Error: test /Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/spkg/build/cvxopt-1.1.3.p1/src/examples/doc/chap10/l1svc.py failed

comment:48 Changed 9 years ago by fbissey

We talked on irc with Volker it is not impossible that you didn't link against the right library because there is a slightly difference in naming scheme in 3.9.68. What does

otool -L /Users/palmieri/Desktop/Sage_stuff/sage_builds/new-atlas/sage-5.0.beta7-gcc/local/lib/python2.7/site-packages/cvxopt/lapack.so

says?

comment:49 follow-up: Changed 9 years ago by dimpase

  • Status changed from needs_work to needs_info

In fact, there is no urgent need to build Atlas on OSX 10.7, as I just learned. it turns out to be a CVXOPT bug. Please see this message and this discussion)

comment:50 Changed 9 years ago by dimpase

  • Description modified (diff)

comment:51 in reply to: ↑ 49 Changed 9 years ago by dimpase

Replying to dimpase:

In fact, there is no urgent need to build Atlas on OSX 10.7, as I just learned. it turns out to be a CVXOPT bug. Please see this message and this discussion)

what is true is that in OSX 10.7 there is a change in behavour of Apple's BLAS on a technically illegal input. What used to work as (wrongly!) expected does not work any more.

comment:52 Changed 9 years ago by dimpase

  • Description modified (diff)
  • Report Upstream changed from Reported upstream. Little or no feedback. to Reported upstream. Developers acknowledge bug.

comment:53 Changed 9 years ago by jdemeyer

  • Summary changed from Build ATLAS on OS X Lion to Fix illegal BLAS call in cvxopt

comment:54 Changed 9 years ago by jdemeyer

  • Dependencies set to #12519
  • Description modified (diff)

comment:55 follow-up: Changed 9 years ago by jdemeyer

Any news from upstream CVXOPT?

comment:56 in reply to: ↑ 55 ; follow-up: Changed 9 years ago by dimpase

  • Report Upstream changed from Reported upstream. Developers acknowledge bug. to Fixed upstream, in a later stable release.

Replying to jdemeyer:

Any news from upstream CVXOPT?

They have released 1.1.5 fixing this issue. Any takers to update the cvxopt spkg ? I, right now, have too many other things to do.

comment:57 in reply to: ↑ 56 Changed 9 years ago by jhpalmieri

Replying to dimpase:

They have released 1.1.5 fixing this issue. Any takers to update the cvxopt spkg ? I, right now, have too many other things to do.

I'm working on it. More details in a few minutes.

comment:58 Changed 9 years ago by jhpalmieri

  • Authors changed from Jeroen Demeyer to Jeroen Demeyer, John Palmieri
  • Description modified (diff)
  • Status changed from needs_info to needs_review

I'm posting a new spkg, but not a perfect one. The flaw is that it is version 1.1.4 rather than 1.1.5. I looked at 1.1.4 and 1.1.5, and (using diff -r ...) I found two changes in the code: the file src/C/dense.c was modified, and I imported that change into the 1.1.4 spkg. With this change, it builds and self-tests pass on OS X Lion, and also on sage.math and OpenSolaris. So that's the version I'm posting here.

The other change was to setup.py, and I don't understand enough of the changes in our patch to modify it. That is, setup.py in version 1.1.4 was almost identical to the version in 1.1.3, so it was easy to rebase our patch, but I couldn't figure out how to rebase to 1.1.5. C'est la vie.

Changed 9 years ago by jhpalmieri

patch for cvxopt spkg; for review only

comment:59 Changed 9 years ago by jdemeyer

Unfortunately, I don't have access anymore to a OS X 10.7 machine.

Changed 9 years ago by jdemeyer

Diff for the cvxopt spkg (p0->p1). For review only

comment:61 Changed 9 years ago by jdemeyer

  • Description modified (diff)
  • Keywords Solaris added
  • Reviewers set to Jeroen Demeyer
  • Summary changed from Fix illegal BLAS call in cvxopt to cvxopt: fix illegal BLAS call and fix Solaris build

Reviewer spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/cvxopt-1.1.4.p1.spkg

Changes: cvxopt-1.1.4.p1.diff

John, if you confirm that my spkg builds with SAGE_CHECK=yes on OS X Lion and you agree with my changes, you may set this to positive review.

comment:62 Changed 9 years ago by jhpalmieri

  • Reviewers changed from Jeroen Demeyer to Jeroen Demeyer, John Palmieri
  • Status changed from needs_review to positive_review

Changes look good and this passes tests on OS X Lion (and OpenSolaris, which I tried just for fun).

comment:63 Changed 9 years ago by jdemeyer

  • Merged in set to sage-5.0.beta13
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.