cvxopt: undefined symbol: ilaenv_

[vbraun]

After updating ATLAS (#10508), cvxopt fails with

sage: from cvxopt import lapack
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/home/vbraun/opt/sage-4.6.1.alpha3-atlas/<ipython console> in <module>()

ImportError: /home/vbraun/opt/sage-4.6.1.alpha3-atlas/local/lib/python2.6/site-packages/cvxopt/lapack.so: undefined symbol: ilaenv_

I've looked into the cvxopt spkg and its lapack.so uses the Fortran blas function ilaenv without linking to f77blas. Adding f77blas to the libraries in setup.py fixes the undefined symbol error.

Attached is a patch against patch/setup.py to add the extra library.

[dimpase]

Volker, what platform do you get it on? I cannot reproduce it on Linux x86-64 Debian... Could it be an Altas hack done wrongly?

[vbraun]

Hi Dima, I'm talking about my new atlas-3.9.32.spkg from #10508. This version now builds shared libraries for everything, whereas the old atlas-3.8.3 did not provide a shared libf77blas.so. I think that cvxopt so far dragged in the static f77blas, so you did not trigger this issue.

[dimpase]

Replying to vbraun:

Hi Dima, I'm talking about my new atlas-3.9.32.spkg from #10508. This version now builds shared libraries for everything, whereas the old atlas-3.8.3 did not provide a shared libf77blas.so. I think that cvxopt so far dragged in the static f77blas, so you did not trigger this issue.

Hmm, how did cvxopt manage this? I'd rather think that something got changed in Atlas - IMHO Atlas 3.8 used to provide a complete wrapper to La@ack...

Actually, on some platforms, e.g. OSX PPC, f77blas does not even exist. On this platform, Atlas is not installed. So your patch will break this, for sure.

[vbraun]

Atlas-3.8.3 does only provide a static lapack library, so cvxopt's lapack.so statically linked it all:

(sage subshell) volker-desktop:sage-4.6.1.alpha0 vbraun$ ls -al local/lib/liblapack*
-rw-r--r--. 1 vbraun vbraun 10254224 Nov 15 16:06 local/lib/liblapack.a
SAGE_ROOT=/home/vbraun/opt/sage-4.6.1.alpha0
(sage subshell) volker-desktop:sage-4.6.1.alpha0 vbraun$ readelf -a local/lib/python/site-packages/cvxopt/lapack.so | grep ilaenv
000000340600  019000000007 R_X86_64_JUMP_SLO 00000000000d6610 ilaenv_ + 0
   400: 00000000000d6610  5800 FUNC    GLOBAL DEFAULT   11 ilaenv_
   361: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ilaenv.f
   924: 00000000000d6610  5800 FUNC    GLOBAL DEFAULT   11 ilaenv_
SAGE_ROOT=/home/vbraun/opt/sage-4.6.1.alpha0

Atlas-3.9.32 builds shared libraries and cvxopt now links against the shared library.

(sage subshell) volker-desktop:sage-4.6.1.alpha3-atlas vbraun$ ls -al local/lib/liblapack* local/lib/libatlas* local/lib/lib*blas*
-rw-r--r--. 1 vbraun vbraun 10964328 Dec 20 22:36 local/lib/libatlas.a
-rw-r--r--. 1 vbraun vbraun  6402481 Dec 20 20:39 local/lib/libatlas.so
-rw-rw-r--. 1 vbraun vbraun   786962 Dec 20 22:36 local/lib/libblas.a
-rw-r--r--. 1 vbraun vbraun   499528 Dec 20 22:36 local/lib/libcblas.a
-rw-r--r--. 1 vbraun vbraun   157226 Dec 20 20:39 local/lib/libcblas.so
-rw-r--r--. 1 vbraun vbraun   599690 Dec 20 22:36 local/lib/libf77blas.a
-rw-r--r--. 1 vbraun vbraun   154978 Dec 20 20:39 local/lib/libf77blas.so
-rw-r--r--. 1 vbraun vbraun  1640938 Dec 20 22:36 local/lib/libgslcblas.a
-rwxr-xr-x. 1 vbraun vbraun      988 Dec 20 22:36 local/lib/libgslcblas.la
lrwxrwxrwx. 1 vbraun vbraun       20 Dec 20 20:43 local/lib/libgslcblas.so -> libgslcblas.so.0.0.0
lrwxrwxrwx. 1 vbraun vbraun       20 Dec 20 20:43 local/lib/libgslcblas.so.0 -> libgslcblas.so.0.0.0
-rwxr-xr-x. 1 vbraun vbraun   896520 Dec 20 20:43 local/lib/libgslcblas.so.0.0.0
-rw-r--r--. 1 vbraun vbraun 10311776 Dec 20 22:36 local/lib/liblapack.a
-rw-r--r--. 1 vbraun vbraun  5455501 Dec 20 20:39 local/lib/liblapack.so
-rw-r--r--. 1 vbraun vbraun   500036 Dec 20 22:36 local/lib/libptcblas.a
-rw-r--r--. 1 vbraun vbraun   157257 Dec 20 20:39 local/lib/libptcblas.so
-rw-r--r--. 1 vbraun vbraun   600086 Dec 20 22:36 local/lib/libptf77blas.a
-rw-r--r--. 1 vbraun vbraun   155009 Dec 20 20:39 local/lib/libptf77blas.so
SAGE_ROOT=/home/vbraun/opt/sage-4.6.1.alpha3-atlas
(sage subshell) volker-desktop:sage-4.6.1.alpha3-atlas vbraun$ readelf -a local/lib/python/site-packages/cvxopt/lapack.so | grep ilaenv
0000002997f8  004600000007 R_X86_64_JUMP_SLO 0000000000000000 ilaenv_ + 0
    70: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND ilaenv_
   471: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND ilaenv_
SAGE_ROOT=/home/vbraun/opt/sage-4.6.1.alpha3-atlas
(sage subshell) volker-desktop:sage-4.6.1.alpha3-atlas vbraun$ readelf -a local/lib/liblapack.so | grep ilaenv
00000070ff08  04e200000007 R_X86_64_JUMP_SLO 0000000000000000 clapack_ilaenv + 0
000000710ba8  070d00000007 R_X86_64_JUMP_SLO 0000000000000000 ilaenv_ + 0
  1250: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND clapack_ilaenv
  1805: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ilaenv_
  3237: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND clapack_ilaenv
  3792: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ilaenv_
SAGE_ROOT=/home/vbraun/opt/sage-4.6.1.alpha3-atlas
(sage subshell) volker-desktop:sage-4.6.1.alpha3-atlas vbraun$ readelf -a local/lib/libf77blas.so | grep ilaenv
00000021e288  004700000007 R_X86_64_JUMP_SLO 00000000000081b0 atl_f77wrap_ilaenv_ + 0
00000021e2b0  005100000007 R_X86_64_JUMP_SLO 0000000000000000 ATL_ilaenv + 0
    71: 00000000000081b0    45 FUNC    GLOBAL DEFAULT    6 atl_f77wrap_ilaenv_
    81: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ATL_ilaenv
   409: 00000000000089bc  3147 FUNC    GLOBAL DEFAULT    6 ilaenv_
    12: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ATL_f77wrap_ilaenv.c
    44: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ilaenv.f
   400: 00000000000081b0    45 FUNC    GLOBAL DEFAULT    6 atl_f77wrap_ilaenv_
   410: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ATL_ilaenv
   738: 00000000000089bc  3147 FUNC    GLOBAL DEFAULT    6 ilaenv_
SAGE_ROOT=/home/vbraun/opt/sage-4.6.1.alpha3-atlas

In particular, ilaenv remains UNDefined in cvxopt lapack.so and atlas liblapack.so and needs to be dynamically linked in from libf77blas.so.

I guess we should keep statically linking the static system-provided lapack on Windows and OSX, or change cvxopt to use the C version clapack_ilaenv (since it already links to libcblas.so)

[vbraun]

I meant: keep statically linking the static system-provided lapack on Windows and OSX, but add libf77blas.so on real unices. Or use clapack_ilaenv, then we are fine since we already link against the C blas.

[vbraun]

In order to also remove the standalone blas in #10508, we need to remove -lblas and use -lcblas -lf77blas instead.

[vbraun]

Patch against patches/setup.py to add cblas, f77blas and remove standalone blas

[leif]

Ping.

[dimpase]

Replying to leif:

Ping.

will look into it today.

[leif]

If #10508 is not an option yet, we should at least upgrade our dead old LAPACK, since people are having problems installing external packages into Sage which use it, but need a more recent version. (I.e., they get linker errors due to missing symbols.)

Also, someone^TM should provide a cvxopt spkg with the patch already applied.

[dimpase]

Replying to leif:

If #10508 is not an option yet, we should at least upgrade our dead old LAPACK, since people are having problems installing external packages into Sage which use it, but need a more recent version. (I.e., they get linker errors due to missing symbols.)

Also, someone^TM should provide a cvxopt spkg with the patch already applied.

the power supply of my PPC Powerbook just died, so I can't tell if this will work. :-(

Karl-Dieter, could you please test this on a MacOSX PPC ?

[leif]

Replying to dimpase:

the power supply of my PPC Powerbook just died, so I can't tell if this will work. :-(

So now it's just a book, without power... :/

[kcrisman]

Replying to dimpase:

Replying to leif:

If #10508 is not an option yet, we should at least upgrade our dead old LAPACK, since people are having problems installing external packages into Sage which use it, but need a more recent version. (I.e., they get linker errors due to missing symbols.)

Also, someone^TM should provide a cvxopt spkg with the patch already applied.

the power supply of my PPC Powerbook just died, so I can't tell if this will work. :-(

Karl-Dieter, could you please test this on a MacOSX PPC ?

If you tell me exactly what to do and provide and spkg or two, sure. I know I won't have time while at work to make a new spkg etc. May need to wait until Thursday.

Will this potentially impact Cygwin as well? Just curious.

[dimpase]

Replying to kcrisman:

Replying to dimpase:

Replying to leif:

If #10508 is not an option yet, we should at least upgrade our dead old LAPACK, since people are having problems installing external packages into Sage which use it, but need a more recent version. (I.e., they get linker errors due to missing symbols.)

Also, someone^TM should provide a cvxopt spkg with the patch already applied.

the power supply of my PPC Powerbook just died, so I can't tell if this will work. :-(

Karl-Dieter, could you please test this on a MacOSX PPC ?

If you tell me exactly what to do and provide and spkg or two, sure. I know I won't have time while at work to make a new spkg etc. May need to wait until Thursday.

well, apply the patch on this ticket to the cvxopt spkg, and test if it still works. Unless I don't understand something, it will not :-)

Will this potentially impact Cygwin as well? Just curious.

no, it should not.

[kcrisman]

End of very short attempt:

running build_ext
building 'gsl' extension
creating build/temp.macosx-10.4-ppc-2.6
creating build/temp.macosx-10.4-ppc-2.6/C
gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/student/Desktop/sage-4.7.2.alpha2/local/include -I/Users/student/Desktop/sage-4.7.2.alpha2/local/include -I/Users/student/Desktop/sage-4.7.2.alpha2/local/include/python2.6 -c C/gsl.c -o build/temp.macosx-10.4-ppc-2.6/C/gsl.o
gcc -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib -bundle -undefined dynamic_lookup build/temp.macosx-10.4-ppc-2.6/C/gsl.o -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib/gcc-lib/powerpc-apple-darwin6.8/4.0.3/ -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib -lm -llapack -lgsl -lgslcblas -lcblas -lf77blas -lf95 -lgsl -o build/lib.macosx-10.4-ppc-2.6/cvxopt/gsl.so
/usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/ld: can't locate file for: -lcblas
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
Error building/installing cvxopt

real    0m7.116s
user    0m0.953s
sys     0m1.155s

Maybe I needed to do #10508 first? But you didn't say that ;-)

[dimpase]

Replying to kcrisman:

End of very short attempt:

running build_ext
building 'gsl' extension
creating build/temp.macosx-10.4-ppc-2.6
creating build/temp.macosx-10.4-ppc-2.6/C
gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/student/Desktop/sage-4.7.2.alpha2/local/include -I/Users/student/Desktop/sage-4.7.2.alpha2/local/include -I/Users/student/Desktop/sage-4.7.2.alpha2/local/include/python2.6 -c C/gsl.c -o build/temp.macosx-10.4-ppc-2.6/C/gsl.o
gcc -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib -bundle -undefined dynamic_lookup build/temp.macosx-10.4-ppc-2.6/C/gsl.o -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib/gcc-lib/powerpc-apple-darwin6.8/4.0.3/ -L/Users/student/Desktop/sage-4.7.2.alpha2/local/lib -lm -llapack -lgsl -lgslcblas -lcblas -lf77blas -lf95 -lgsl -o build/lib.macosx-10.4-ppc-2.6/cvxopt/gsl.so
/usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/ld: can't locate file for: -lcblas
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
Error building/installing cvxopt

real    0m7.116s
user    0m0.953s
sys     0m1.155s

Maybe I needed to do #10508 first? But you didn't say that ;-)

No, 10508 updates Atlas, but on OSX we don't use Atlas at all?! Well, perhaps I was missing something: then you should start with the ticket 10508 depends on, then 10508 itself, and only then this one.

[kcrisman]

Correct, we don't. What ticket does #10508 depend on? I don't see a list.

Do we have to link to cblas instead of blas? I realize this is pretty naive, but I don't know whether that exists on Mac (yet). We would have to make it, maybe? See ​this ask.sagemath question.

[kcrisman]

What's the status of this ticket, given #12011 upgrading ATLAS?

Do we have to link to cblas instead of blas? I realize this is pretty naive, but I don't know whether that exists on Mac (yet). We would have to make it, maybe? See ​this ask.sagemath question.

See #12519, where Jeroen is pretty clearly that cblas doesn't exist on Mac, or at least not in the current situation (see #12011 also).

[kcrisman]

That is, #12011 is also about upgrading ATLAS now, and #12519 is about cvxopt and blas.

[dimpase]

Replying to kcrisman:

Correct, we don't. What ticket does #10508 depend on? I don't see a list.

Do we have to link to cblas instead of blas? I realize this is pretty naive, but I don't know whether that exists on Mac (yet). We would have to make it, maybe? See ​this ask.sagemath question.

cblas exists on Mac OSX in disguise. The equivalent of -lcblas in compiler/linker options is -framework Accelerate. Although I have no idea how setup.py has to be modified to accommodate this.

[kcrisman]

In fact, at William's answer to that ask.sagemath.org question, he claims cblas even exists 'normally', as /usr/lib/libcblas.dylib (which it does on Snow Leopard, but not on Tiger). I got confused by #12519 because Jeroen was referring to it not being one of the Sage installed libraries, not that it didn't exist at all.

[kcrisman]

Replying to kcrisman:

In fact, at William's answer to that ask.sagemath.org question, he claims cblas even exists 'normally', as /usr/lib/libcblas.dylib (which it does on Snow Leopard, but not on Tiger).

In which latter case I believe we use $SAGE_LOCAL/lib/libgslcblas.dylib for such purposes.

[vbraun]

It seems we should fall back to gslcblas on all old hardware or whenever atlas fails to build. But is it actually full-featured enough? Do all doctests pass if we just use gslcblas for everything?

[kcrisman]

Replying to vbraun:

It seems we should fall back to gslcblas on all old hardware or whenever atlas fails to build. But is it actually full-featured enough? Do all doctests pass if we just use gslcblas for everything?

Apparently. At any rate

find . -name "*blas.dylib" -print

in the home directory on my PPC box yields only the library mentioned above and $SAGE_LOCAL/lib/R/lib/libRblas.dylib. So I guess this is ok, if perhaps "frickin' slow", according to an old linbox spkg-install.

[vbraun]

Replying to kcrisman:

So I guess this is ok, if perhaps "frickin' slow", according to an old linbox spkg-install.

So is your PPC box ;-)

[kcrisman]

So I guess this is ok, if perhaps "frickin' slow", according to an old linbox spkg-install.

So is your PPC box ;-)

Touché! But beautiful; you wouldn't believe how many students come into the office and comment on how "you've got a really cool computer!"

[vbraun]

In the new atlas-3.10.0 the symbol is moved back to liblapack so we can close this ticket as invalid.