Opened 11 years ago
Closed 11 years ago
#8086 closed defect (fixed)
numpy fails to build on OpenSolaris or Solaris 10 64-bit
Reported by: | drkirkby | Owned by: | drkirkby |
---|---|---|---|
Priority: | major | Milestone: | sage-4.5.3 |
Component: | porting: Solaris | Keywords: | |
Cc: | jsp | Merged in: | sage-4.5.3.alpha0 |
Authors: | David Kirkby | Reviewers: | John Palmieri |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
Build environment
- Sun Ultra 27 3.33 GHz Intel W3580 Xeon. Quad core. 8 threads. 12 GB RAM
- OpenSolaris 2009.06 snv_111b X86
- Sage 4.3.1 (with a few packages hacked to work on 64-bit)
- gcc 4.3.4 configured with Sun linker and GNU assembler from binutils version 2.20.
- 64-bit build. SAGE64 was set to yes, plus various other tricks to get -m64 into packages.
The problem
numpy is failing to build. It looks like there is a mix of 32 and 64-bit objects.
numpy-1.3.0.p2/src/setup.py numpy-1.3.0.p2/src/MANIFEST.in numpy-1.3.0.p2/.hgtags Finished extraction **************************************************** Host system uname -a: SunOS hawk 5.11 snv_111b i86pc i386 i86pc **************************************************** **************************************************** CC Version gcc -v Using built-in specs. Target: i386-pc-solaris2.11 Configured with: ../gcc-4.3.4/configure --prefix=/usr/local/gcc-4.3.4-GNU-assembler-Sun-linker --with-as=/usr/local/binutils-2.20/bin/as --with-ld=/usr/ccs/bin/ld --with-gmp=/usr/local --with-mpfr=/usr/local Thread model: posix gcc version 4.3.4 (GCC) **************************************************** Running from numpy source directory. F2PY Version 2 blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in /export/home/drkirkby/sage-4.3.1/local/lib NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /export/home/drkirkby/sage-4.3.1/local/lib NOT AVAILABLE atlas_blas_info: FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/export/home/drkirkby/sage-4.3.1/local/lib'] language = c include_dirs = ['/export/home/drkirkby/sage-4.3.1/local/include'] /export/home/drkirkby/sage-4.3.1/spkg/build/numpy-1.3.0.p2/src/numpy/distutils/command/config.py:361: DeprecationWarning: +++++++++++++++++++++++++++++++++++++++++++++++++ Usage of get_output is deprecated: please do not use it anymore, and avoid configuration checks involving running executable on the target machine. +++++++++++++++++++++++++++++++++++++++++++++++++ DeprecationWarning) customize Sage_FCompiler_1 customize Sage_FCompiler_1 customize Sage_FCompiler_1 using config compiling '_configtest.c': /* This file is generated from numpy/distutils/system_info.py */ void ATL_buildinfo(void); int main(void) { ATL_buildinfo(); return 0; } C compiler: gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC compile options: '-c' gcc: _configtest.c gcc _configtest.o -L/export/home/drkirkby/sage-4.3.1/local/lib -lf77blas -lcblas -latlas -o _configtest ld: fatal: file /export/home/drkirkby/sage-4.3.1/local/lib/libf77blas.so: wrong ELF class: ELFCLASS64 ld: fatal: file /export/home/drkirkby/sage-4.3.1/local/lib/libcblas.so: wrong ELF class: ELFCLASS64 ld: fatal: file /export/home/drkirkby/sage-4.3.1/local/lib/libatlas.so: wrong ELF class: ELFCLASS64 ld: fatal: file processing errors. No output written to _configtest collect2: ld returned 1 exit status ld: fatal: file /export/home/drkirkby/sage-4.3.1/local/lib/libf77blas.so: wrong ELF class: ELFCLASS64 ld: fatal: file /export/home/drkirkby/sage-4.3.1/local/lib/libcblas.so: wrong ELF class: ELFCLASS64 ld: fatal: file /export/home/drkirkby/sage-4.3.1/local/lib/libatlas.so: wrong ELF class: ELFCLASS64 ld: fatal: file processing errors. No output written to _configtest collect2: ld returned 1 exit status failure. removing: _configtest.c _configtest.o Status: 255 Output: FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/export/home/drkirkby/sage-4.3.1/local/lib'] language = c define_macros = [('NO_ATLAS_INFO', 2)] include_dirs = ['/export/home/drkirkby/sage-4.3.1/local/include'] lapack_opt_info: lapack_mkl_info:
Attachments (1)
Change History (13)
comment:1 Changed 11 years ago by
- Description modified (diff)
comment:2 Changed 11 years ago by
comment:3 Changed 11 years ago by
- Status changed from new to needs_review
Added patch file and updated spkg:
http://boxen.math.washington.edu/home/jsp/ports/numpy-1.3.0.p3.spkg
Jaap
comment:4 Changed 11 years ago by
- Status changed from needs_review to needs_info
Hi Jaap,
I'm having a problem with this. I got a message about "could not fork". I even rebooted the machine once, thinking something was wrong, but everything else seems to work.
Is this bit of code in the patch right? It looks wrong to me:
echo "#! /usr/bin/env bash" > gcc echo "`which gcc` -m64 " | sed 's/$/\$@/' >> fake_gcc
Should the first line not write to the file 'fake_gcc' rather than 'gcc'? If so, since it gets copied to $SAGE_LOCAL/bin/gcc, why not write it directly there?
i.e.
echo "#! /usr/bin/env bash" > $SAGE_LOCAL/bin/gcc echo "`which gcc` -m64 " | sed 's/$/\$@/' >> $SAGE_LOCAL/bin/gcc
Also, why use 'sed'? If I understand your intentions correctly,
echo "`which gcc` -m64 \$@"
would achieve the same, but without invoking 'sed'.
I would also add that 'which' is not a POSIX command, whereas 'command -v' gives you the same information, but in a standards complient way. This can be important, especially on Solaris 10, as the return code of 'which' is undefined on there.
drkirkby@hawk:~$ echo "`command -v gcc` -m64 \$@" /usr/local/gcc-4.4.4/bin/gcc -m64 $@
is more portable.
However, I can't seem to get it to work. I'm not sure if I understand exactly what you are aiming to do here. Why do we need a fake gcc?
Dave
comment:5 Changed 11 years ago by
Jaap,
I can't get your patch to work, so have reimplemented this. I've completely removed the file fake_gcc from Numpy, and instead create the fake file dynamically with the correct path.
I've also reported the issue to the Numpy bug database.
http://projects.scipy.org/numpy/ticket/1525
which should have been done years ago when the problem was first realised in OS X.
Here's a package waiting for review, which I believe does fix the problem.
http://boxen.math.washington.edu/home/kirkby/patches/numpy-1.3.0.p4.spkg
Dave
comment:6 Changed 11 years ago by
- Status changed from needs_info to needs_review
comment:7 Changed 11 years ago by
- Description modified (diff)
- Summary changed from numpy fails to build on Open Solaris x64 - 32 / 64-bit mixup to numpy fails to build on OpenSolaris or Solaris 10 64-bit
comment:8 Changed 11 years ago by
- Status changed from needs_review to needs_work
Great news
I've found a much simpler solution to this, which totally avoids using any fake gcc files. Just compile with
CC="gcc -m64" export CC
and it works! I will attach a simpler patch in 15 minutes. I'm marking as "needs work" for now, as I need to do something else just now.
Dave
Changed 11 years ago by
Mercurial patch which solves the Numpy build issues without any sort of hack. I think this would work on OS X too, but I'm not changing the OS X code now.
comment:9 Changed 11 years ago by
- Status changed from needs_work to needs_review
Here's the updated package. I believe this is a decent fix (at last).
http://boxen.math.washington.edu/home/kirkby/patches/numpy-1.3.0.p4.spkg
I've reported this upstream
http://projects.scipy.org/numpy/ticket/1525
Dave
comment:10 Changed 11 years ago by
- Reviewers set to John Palmieri
- Status changed from needs_review to positive_review
The changes look good. The spkg builds on a number of different machines, both 32-bit and 64-bit, and doctests pass.
By the way, with the old spkg, numpy didn't build properly on 64-bit Solaris (either sparc -- t2 -- or x86 -- fulvia), so it's not exclusively an OpenSolaris issue. With the new spkg, it works on fulvia. I haven't gotten t2 (with SAGE64='yes') to build the prerequisite packages yet. I'm working on that, and if there are issues building numpy, I'll revert the positive review.
(Tested on t2 (32-bit), mark2 (32-bit), Mac OS X 10.6 (64-bit), sage.math (both with SAGE64='yes' and with SAGE64 unset), taurus (both with SAGE64='yes' and SAGE64 unset), cicero (32-bit), fulvia (both settings for SAGE64), and iras (64-bit machine? but with SAGE64 unset).)
comment:11 Changed 11 years ago by
Note to the release manager
All changes are committed in the repository of the .spkg file. No library changes need to be made. Just drop in
http://boxen.math.washington.edu/home/kirkby/patches/numpy-1.3.0.p4.spkg
comment:12 Changed 11 years ago by
- Merged in set to sage-4.5.3.alpha0
- Resolution set to fixed
- Status changed from positive_review to closed
I do have a working numpy on your machine.
There is a numpy-1.3.0.p3.spkg in trac waiting to be merged.
Here is a pre version of mine: http://boxen.math.washington.edu/home/jsp/ports/numpy-1.3.0.p3.spkg
Works on Open Solaris.
Jaap