Opened 10 years ago
Closed 10 years ago
#10226 closed enhancement (fixed)
Rewrite ATLAS spkg-install
Reported by: | vbraun | Owned by: | GeorgSWeber |
---|---|---|---|
Priority: | critical | Milestone: | sage-4.7.2 |
Component: | packages: standard | Keywords: | ATLAS spkg |
Cc: | jsp, AlexGhitza, rlm, was, drkirkby, jason, mpatel | Merged in: | sage-4.7.2.alpha0 |
Authors: | Volker Braun | Reviewers: | Dan Drake, David Kirkby |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
I rewrote the ATLAS spkg-install to
- allow customization of architectural defaults via the new
SAGE_ATLAS_ARCH
environment variable. - still build a reasonable library if the automatic tuning process fails to find accurate timings.
SAGE_ATLAS_LIB=path
now searches inpath/libatlas.so
instead ofpath/lib/libatlas.so
so it works for people with atlas in/lib64
, too.- Threading is now enabled by default. Whats the point of trying to squeeze more performance out of modern processors if you run on one core only?
By default, ATLAS will now try twice to get timings, if that fails build with SAGE_ATLAS_ARCH=fast
, and if that fails with SAGE_ATLAS_ARCH=base
.
From the new SPKG.txt
:
The package can be configured via two environment variables: * SAGE_ATLAS_LIB=path If this environment variable is set, path/libatlas.so is used and ATLAS is not compiled. * SAGE_ATLAS_ARCH=arch[,isaext1][,isaext2]...[,isaextN] The given architectural default and instruction set extensions are used instead of the empirical tuning. Available architectures are POWER3, POWER4, POWER5, PPCG4, PPCG5, P5, P5MMX, PPRO, PII, PIII, PM, CoreSolo, CoreDuo, Core2Solo, Core2, Corei7, P4, P4E, Efficeon, K7, HAMMER, AMD64K10h, UNKNOWNx86, IA64Itan, IA64Itan2, USI, USII, USIII, USIV, UnknownUS, MIPSR1xK, MIPSICE9 and instruction set extensions are AltiVec, SSE3, SSE2, SSE1, 3DNow In addition, you can also set - SAGE_ATLAS_ARCH=fast picks defaults for a modern (2-3 year old) CPU of your processor line, and - SAGE_ATLAS_ARCH=base picks defaults that should work for a ~10 year old CPU. For example, SAGE_ATLAS_ARCH=Corei7,SSE3,SSE2,SSE1 would be appropriate for a Core i7 CPU.
I implemented the spkg-install script in python without relying on any external programs (except for make, of course). The configuration-detecting part is in the configuration.py
module. For easier review I will attach the files to this ticket. The complete spkg is here:
Attachments (3)
Change History (96)
comment:1 Changed 10 years ago by
- Status changed from new to needs_review
comment:2 Changed 10 years ago by
- Cc jsp AlexGhitza rlm was added
comment:3 Changed 10 years ago by
- Cc drkirkby added
- Status changed from needs_review to needs_work
This fails to build on t2.math.washington.edu (Solaris on sparc); I've posted the log at http://sage.math.washington.edu/home/palmieri/misc/atlas-3.8.3.p17.log. I get a similar failure on the skynet machines mark (Solaris on sparc) and fulvia (Solaris on x86). I am not seeing this problem on David Kirkby's machine hawk (OpenSolaris on x86).
comment:4 Changed 10 years ago by
Solaris ld only understands -V
and not --version
`, it seems. I've updated the spkg.
comment:5 Changed 10 years ago by
IMHO, it would be good if we could remove the small amount of python that was in this package, we could start building this as soon as Fortran is built. This would allow us to start building ATLAS much earlier (before building Python).
We really should update ATLAS too.
I do wonder if such a large change is appropriate without updating ATLAS. It will make an update even more difficult.
Given ATLAS takes longer than any other package to build on many systems, getting it building early, which means avoiding Python, would seem a good idea IMHO.
Dave
comment:6 Changed 10 years ago by
I'm still having problems with this on t2; now I get the following error message, immediately after the description of the platform and the C compiler:
**************************************************** ld: Software Generation Utilities - Solaris Link Editors: 5.10-1.496 ld: fatal: no files on input command line Cannot execute ld!
With the previous version, the build also failed on hawk eventually, although with a different problem. I'll try to post the log somewhere soon.
We really should update ATLAS too.
We seem to be using the most recent "stable" version, whatever stable means. (I'm using the language from the Atlas website.) It's over a year and a half old, though, with 25 unstable releases since then. Should we try 3.9.25, released in early November? Or is it too risky to try something labeled "unstable" by the developers?
comment:7 follow-up: ↓ 9 Changed 10 years ago by
ATLAS 3.8.3 is the latest stable release. The next release should come soon but until its out we should stick with 3.8.3
ATLAS is also very difficult to build and it would be a nightmare to rewrite the spkg-install in POSIX shell script. This is why I rewrote the mixture of shell script, Perl, and Python in pure Python. Python is actually very well suited to detect the architectural defaults necessary for building ATLAS.
I also take it from jhpalmieri's comment that "ld -V" exits with a non-zero exit code on t2 (where I don't have an account). I'll add a workaround later...
comment:8 Changed 10 years ago by
(I thought that if you had a sage.math account, then you automatically had a t2.math account, same username and same password. But maybe I'm wrong about that.)
comment:9 in reply to: ↑ 7 Changed 10 years ago by
Replying to vbraun:
ATLAS 3.8.3 is the latest stable release. The next release should come soon but until its out we should stick with 3.8.3
Normally I would agree with you, but Clint, who is the ATLAS developer, has advised us not to use the latest stable release, but instead use the development release. He has also said off list that he will not be making many changes to ATLAS due to other commitments. The Sage on gentoo project is using the latest ATLAS without issue
ATLAS is also very difficult to build and it would be a nightmare to rewrite the spkg-install in POSIX shell script.
As it is now, I agree. But there's an awful lot of junk in there. It is a non-trivial task, but really we should update ATLAS.
This is why I rewrote the mixture of shell script, Perl, and Python in pure Python.
Yes, but it seriously increases the time to to build Sage, as numerous packages have to be build before Python is built, but if
- The small bit of Python in ATLAS is removed
- The small bit of Python in the Fortran package is removed
then it will be possible to start the build of ATLAS much earlier. Currently, builds are often building ATLAS for a long time, with nothing else going on.
I also take it from jhpalmieri's comment that "ld -V" exits with a non-zero exit code on t2 (where I don't have an account). I'll add a workaround later...
I can create you an account on t2.math
.
To answer John's point, no the passwords are not shared with t2, as the Linux based NIS is not working happily with the Sun NIS. It is possible to get around, but it is a non-trivial task. William asked someone to set it up, without mentioning this to me. The guy left the machine unusable. The only way was to log in on the console. I sorted that out, but have never looked at the NIS issues.
Apparenlty the latest ATLAS builds in under an hour on t2.math, compared to the 7 hours it takes now. But I think that is only as a 64-bit application, as I don't think ATLAS has the 32-bit tuning parameters for t2.math. But I think sorting that out will be somewhat easier than with the current ATLAS, as the new ATLAS has the code to detect the T2+ processors on t2.math. Dave
comment:10 in reply to: ↑ description Changed 10 years ago by
Replying to vbraun:
I implemented the spkg-install script in python without relying on any external programs (except for make, of course). The configuration-detecting part is in the
configuration.py
module.
I aimed at removing the Python dependency of ATLAS. ;-)
I'm not in principle against using Python for installation scripts, but unless Python gets a prerequisite (or of course is required by the package or its prerequisites anyway)...
Maybe we somehow should support SAGE_ALLOW_SYSTEM_PYTHON_FOR_BUILD
.
comment:11 Changed 10 years ago by
The way parallel makes are "disabled" is certainly wrong for a couple of reasons.
(This was in before "of course", too.)
comment:12 in reply to: ↑ description ; follow-up: ↓ 13 Changed 10 years ago by
Replying to vbraun:
I implemented the spkg-install script in python without relying on any external programs (except for make, of course). The configuration-detecting part is in the
configuration.py
module. For easier review I will attach the files to this ticket. The complete spkg is here:http://www.stp.dias.ie/~vbraun/Sage/spkg/atlas-3.8.3.p17.spkg
Hi,
I had problems compiling Sage-4.6 both on i7_860 and i7-740QM, whereas compilation on i7-950, Xeon-E5530, Quad-9550 ran smoothly. I use environment variables:
export MAKE='make -jN'
export SAGE64='yes'
export SAGE_PARALLEL_SPKG_BUILD='yes'
with N=number of cores (not hw threads).
Ok, so that didn't quite work out on the 860 and 740QM, I mean it took ages to compile, spending lots of time in Atlas. So I applied the p17 change, added the variables mentioned for i7, and I also as obliged to use the performance governor :
sudo cpufreq-set -r -g performance
Otherwise, even the p17 didn't detect the top frequency and only compiled with a flag indicating the lowest frequency available.
So now all my CPUs compile Sage in a reasonably coherent amount of time.
comment:13 in reply to: ↑ 12 Changed 10 years ago by
export MAKE='make -jN'
Parallel make is disabled in the ATLAS spkg
export SAGE64='yes'
Not used by the ATLAS spkg
export SAGE_PARALLEL_SPKG_BUILD='yes'
Might throw off ATLAS timings if you build stuff in the background. But since ATLAS takes such a long time to build it probably does not matter.
sudo cpufreq-set -r -g performance
What governor are you using usually? I'm hoping that "ondemand" is good enough on modern CPU's since the tuning process will kick them into high gear. Any other governor besides performance and ondemand is useless unless you have an old CPU.
comment:14 Changed 10 years ago by
- Milestone set to sage-4.6.1
- Status changed from needs_work to needs_review
#10508: Update ATLAS to version 3.9.32
is not ready yet as it makes linbox crash. Its not yet clear whether this is the fault of atlas-3.9.32, linbox, or ntl but something is wrong. In the meantime, somebody should review the new build system as it is the way to move forward.
This ticket also fixes #9382: atlas not respecting SAGE_FAT_BINARY on i686 systems.
comment:15 Changed 10 years ago by
I've looked over the code here pretty carefully and I anticipate this can get a positive review. I just need to try an actual build and see how it works.
comment:16 follow-up: ↓ 18 Changed 10 years ago by
I'm still puzzled by the logic of making changes which:
- Make it much more difficult to speed up the Sage build process by removing the small amount of Python necessary to start the ATLAS build earlier.
- Implementing changes, some of which are unnecessary if ATLAS is updated.
IMGHO, the two most logical things to do the the ATLAS package are:
1) Update the ATLAS version, which has support for many new CPUs and so allow a lot of the Sage-specfic CPU hacks to be removed. It also has the facility to use a generic set of tuning parameters. 2) Remove the few lines of Python, so ATLAS no longer depends on Python. Then the build of ATLAS can be started much earlier. This would speed up the Sage build, as ATLAS would be building for longer. ATLAS is ofgten the only item being built, so is a bottleneck.
This patch seems a backward step to me.
Dave
comment:17 Changed 10 years ago by
We talked about the ATLAS install script here at bugdays, and I did raise your point that it introduces an extra python dependency. But nobody minded that much. William thinks that, if anything, more scripts should be written in python. The build scripts don't use any Sage modules, so they can easily be executed by any system-wide install of python. Eventually we'll just the system python interpreter, potentially falling back to compiling our own if no python is preinstalled. We already depend on a working gcc and perl installation, so adding python to that list isn't much of a stretch. In any case, the goal of this ticket is to first make ATLAS build reliably.
The new ATLAS version makes linbox segfault, this is why I haven't gone ahead and updated it. But again that task is a lot easier with the new spkg-install script, so I think it makes sense to push this ticket through until we figure out whats wroing with linbox/ATLAS-3.9/NTL. This will be worked on in #10508.
comment:18 in reply to: ↑ 16 Changed 10 years ago by
Replying to drkirkby:
I'm still puzzled by the logic of making changes which:
- Make it much more difficult to speed up the Sage build process by removing the small amount of Python necessary to start the ATLAS build earlier.
- Implementing changes, some of which are unnecessary if ATLAS is updated.
Right now the build process is run by a Python script that calls a shell script that calls a Python script that mostly just runs shell commands and also a shell script that calls a Perl script.
Understanding what's going on is about as easy as understanding that sentence. It's crazy. The proposed solution here is two easy-to-understand Python files. If we upgrade ATLAS, it will be easy to remove the unnecessary bits. And since the files only require Python 2.3 (I think), and there's no problem demanding Python as a build dependency for Sage, so with a little extra work, we can get the ATLAS building very early in the process.
comment:19 follow-up: ↓ 20 Changed 10 years ago by
With the new scripts dropped into the existing .p16 spkg, this builds ATLAS and all doctests pass on sage.math (Ubuntu 8.04 amd64). I'm going to test on t2.math and bsd.math today.
comment:20 in reply to: ↑ 19 Changed 10 years ago by
Replying to ddrake:
With the new scripts dropped into the existing .p16 spkg, this builds ATLAS and all doctests pass on sage.math (Ubuntu 8.04 amd64). I'm going to test on t2.math and bsd.math today.
Volker has now updated the spkg so that it does not get confused by "ld -V" on MacOSX (this is of course very minor). It works on MacOSX 10.6 just fine.
comment:21 Changed 10 years ago by
I built Sage from scratch with an atlas spkg that has these scripts. First, with SAGE_ROOT/spkg/standard/deps
unchanged, and then the second time I removed the Python dependency. Both builds worked and passed doctests.
I would like to see something that shows me that SAGE_FAT_BINARY works in i686 (to make sure that we've fixed #9382) but I don't have access to any suitable machines at the moment. I'll look into this.
comment:22 Changed 10 years ago by
I don't think Volker will forget about these two things, but just to make a note of it:
- the
time.sleep
doesn't work;time
doesn't seem to be imported - to insure that this will work with Python 2.4, you'll need to change the tuples to lists, since that version of Python strangely didn't support
.index()
on tuples.
comment:23 Changed 10 years ago by
You are late Dan, I had the spkg already updated five minutes after you told me about the issues :-)
comment:24 Changed 10 years ago by
- Reviewers set to Dan Drake
- Status changed from needs_review to positive_review
I tested SAGE_FAT_BINARY on skynet/cicero and it works properly. So positive review here, and we can close #9382 when this is merged.
comment:25 Changed 10 years ago by
#10666 is a followup ticket for documenting SAGE_ATLAS_ARCH.
comment:26 Changed 10 years ago by
- Keywords spkg added
- Milestone changed from sage-4.6.2 to sage-4.7
comment:27 Changed 10 years ago by
- Cc jason added
comment:28 Changed 10 years ago by
- Component changed from build to packages
comment:29 Changed 10 years ago by
- Status changed from positive_review to needs_work
I know it's a detail, but I would prefer the ATLAS spkg-install
does NOT print the message
Finished installing ATLAS.
Simply because grepping logs for '^Finished installing '
now gives
[...] Finished installing python-2.6.4.p9.spkg Finished installing fortran-20100629.spkg Finished installing lapack-20071123.p2.spkg Finished installing ATLAS. Finished installing atlas-3.8.3.p17.spkg Finished installing blas-20070724.spkg [...]
comment:30 Changed 10 years ago by
- Status changed from needs_work to positive_review
Ok, done. Updated spkg at the same location.
comment:31 Changed 10 years ago by
Thanks!
comment:32 Changed 10 years ago by
- Merged in set to sage-4.7.alpha1
- Resolution set to fixed
- Status changed from positive_review to closed
comment:33 Changed 10 years ago by
- Merged in sage-4.7.alpha1 deleted
- Resolution fixed deleted
- Status changed from closed to new
On Mac OS X 10.6, 32 bit:
atlas-3.8.3.p17 Machine: Darwin bsd.local 10.6.0 Darwin Kernel Version 10.6.0: Wed Nov 10 18:13:17 PST 2010; root:xnu-1504.9.26~3/RELEASE_I386 i386 Deleting directories from past builds of previous/current versions of atlas-3.8.3.p17 Extracting package /Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg ... -rw-r--r-- 1 buildbot staff 2819924 Mar 8 06:30 /Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg Finished extraction **************************************************** Host system uname -a: Darwin bsd.local 10.6.0 Darwin Kernel Version 10.6.0: Wed Nov 10 18:13:17 PST 2010; root:xnu-1504.9.26~3/RELEASE_I386 i386 **************************************************** **************************************************** CC Version gcc -v Using built-in specs. Target: i686-apple-darwin10 Configured with: /var/tmp/gcc/gcc-5664~105/src/configure --disable-checking --enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin10 --program-prefix=i686-apple-darwin10- --host=x86_64-apple-darwin10 --target=i686-apple-darwin10 --with-gxx-include-dir=/include/c++/4.2.1 Thread model: posix gcc version 4.2.1 (Apple Inc. build 5664) **************************************************** ld: unknown option: -V Cannot execute ld!
comment:34 Changed 10 years ago by
- Status changed from new to needs_work
comment:35 Changed 10 years ago by
On OpenSolaris? 06.2009-32 (hawk):
atlas-3.8.3.p17 Machine: SunOS hawk 5.11 snv_134 i86pc i386 i86pc Deleting directories from past builds of previous/current versions of atlas-3.8.3.p17 Extracting package /export/home/buildbot/build/sage/hawk-1/hawk_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg ... -rw-r--r-- 1 buildbot other 2819924 Mar 8 14:30 /export/home/buildbot/build/sage/hawk-1/hawk_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg Finished extraction **************************************************** Host system uname -a: SunOS hawk 5.11 snv_134 i86pc i386 i86pc **************************************************** **************************************************** CC Version gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/local/gcc-4.5.0/libexec/gcc/i386-pc-solaris2.10/4.5.0/lto-wrapper Target: i386-pc-solaris2.10 Configured with: ../gcc-4.5.0/configure --prefix=/usr/local/gcc-4.5.0 --build=i386-pc-solaris2.10 --enable-languages=c,c++,fortran --with-gmp=/usr/local/gcc-4.5.0 --with-mpfr=/usr/local/gcc-4.5.0 --disable-nls --enable-checking=release --enable-werror=no --enable-multilib -with-system-zlib --enable-bootstrap --with-gnu-as --with-as=/usr/local/binutils-2.20/bin/as --without-gnu-ld --with-ld=/usr/ccs/bin/ld Thread model: posix gcc version 4.5.0 (GCC) **************************************************** ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.1689 sh: line 1: readelf: not found Unknown linker: Configuration: linker_GNU?: False CYGWIN?: False ld: None FreeBSD?: False 32bit?: True linker_Solaris?: False bits: 32bit system: SunOS Darwin?: False machine: i86pc fortran: gfortran PPC?: False 64bit?: False fortran_GNU?: True fortran_g95?: False Intel?: True Linux?: False SPARC?: False processor: i386 Solaris?: True Unable to find one of liblapack.a, libcblas.a, libatlas.a or libf77blas.a in the directory /ATLAS32 Set SAGE_ATLAS_LIB to the parent directory of the directory containing liblapack.a, libcblas.a, libatlas.a and libf77blas.a if you wish to use existing ATLAS libraries. So if for example those four libraries are in /usr/local/atlas/lib64 then set SAGE_ATLAS_LIB to be /usr/local/atlas/lib64 Unset SAGE_ATLAS_LIB to build ATLAS from source. Then type make.
comment:36 Changed 10 years ago by
On SunOS 5.10-32 (fulvia):
atlas-3.8.3.p17 Machine: SunOS fulvia 5.10 Generic_127128-11 i86pc i386 i86pc Deleting directories from past builds of previous/current versions of atlas-3.8.3.p17 Extracting package /home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg ... -rw-r--r-- 1 buildbot sage 2819924 Mar 8 09:30 /home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg Finished extraction **************************************************** Host system uname -a: SunOS fulvia 5.10 Generic_127128-11 i86pc i386 i86pc **************************************************** **************************************************** CC Version gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/local/gcc-4.5.1/x86_64-SunOS-core2-sun-ld/libexec/gcc/i386-pc-solaris2.10/4.5.1/lto-wrapper Target: i386-pc-solaris2.10 Configured with: /usr/local/gcc-4.5.1/src/gcc-4.5.1/configure --enable-languages=c,c++,fortran --with-gnu-as --with-as=/usr/local/binutils-2.20.1/x86_64-SunOS-core2-gcc-4.4.3/bin/as --with-ld=/usr/ccs/bin/ld --with-gmp=/usr/local/gmp-5.0.1/x86_64-SunOS-core2-gcc-4.5.0-abi32 --with-mpfr=/usr/local/mpfr-3.0.0/x86_64-SunOS-core2-gmp-5.0.1-gcc-4.5.0-abi32 --with-mpc=/usr/local/mpc-0.8.2/x86_64-SunOS-core2-mpfr-3.0.0-gmp-5.0.1-gcc-4.5.0-abi32 --prefix=/usr/local/gcc-4.5.1/x86_64-SunOS-core2-sun-ld Thread model: posix gcc version 4.5.1 (GCC) **************************************************** ld: Software Generation Utilities - Solaris Link Editors: 5.10-1.489 [...] STAGE 4: POST-BUILD TUNING make -f Makefile IPostTune 2>&1 | ./xatlas_tee INSTALL_LOG/POSTTUNE.LOG done. STAGE 4: Threading install make -f Makefile IBuildPtlibs 2>&1 | ./xatlas_tee INSTALL_LOG/LIBPTBUILD.LOG ATLAS install complete. Examine ATLAS/bin/<arch>/INSTALL_LOG/SUMMARY.LOG for details. make[3]: Leaving directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build' make clean make[3]: Entering directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build' rm -f *.o x* config?.out *core* make[3]: Leaving directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build' make[2]: Leaving directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build' make[2]: Entering directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build/lib' make[2]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule. rm -f libatlas.so liblapack.so make libatlas.so liblapack.so libf77blas.so libcblas.so liblapack.so make[3]: Entering directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build/lib' ld -melf_i386 -shared -soname libatlas.so -o libatlas.so \ --whole-archive libatlas.a --no-whole-archive -lc -lpthread -lm ld: fatal: file libatlas.so: open failed: No such file or directory make[3]: *** [libatlas.so] Error 1 make[3]: Leaving directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build/lib' make[2]: *** [shared] Error 2 make[2]: Leaving directory `/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build/lib' Unknown linker: Configuration: linker_GNU?: False CYGWIN?: False ld: None FreeBSD?: False 32bit?: True linker_Solaris?: False bits: 32bit system: SunOS Darwin?: False machine: i86pc fortran: gfortran PPC?: False 64bit?: False fortran_GNU?: True fortran_g95?: False Intel?: True Linux?: False SPARC?: False processor: i386 Solaris?: True Copying patches/archinfo_linux.c to src/CONFIG/src/backend/archinfo_linux.c Copying patches/archinfo_x86.c to src/CONFIG/src/backend/archinfo_x86.c Copying patches/probe_comp.c to src/CONFIG/src/probe_comp.c Copying patches/Make.top to src First automatic tuning attempt. Running ../src/configure --prefix=/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/local --with-netlib-lapack=/home/buildbot/build/sage/fulvia-1/fulvia_full/build/sage-4.7.alpha1/local/lib/liblapack.a -Si cputhrchk 0 -Fa alg -fPIC -C if sage_fortran -b 32 -O 2 Finished configuring ATLAS. Finished building ATLAS core. Traceback (most recent call last): File "./spkg-install", line 406, in <module> assert rc==0, 'Building shared ATLAS library failed.' AssertionError: Building shared ATLAS library failed.
comment:37 Changed 10 years ago by
On SunOS 5.10-32 (t2):
atlas-3.8.3.p17 Machine: SunOS t2 5.10 Generic_141414-02 sun4v sparc SUNW,T5240 Deleting directories from past builds of previous/current versions of atlas-3.8.3.p17 Extracting package /scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg ... -rw-r--r-- 1 buildbot buildbot 2819924 Mar 8 06:30 /scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg Finished extraction **************************************************** Host system uname -a: SunOS t2 5.10 Generic_141414-02 sun4v sparc SUNW,T5240 **************************************************** **************************************************** CC Version gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/local/gcc-4.5.1/libexec/gcc/sparc-sun-solaris2.10/4.5.1/lto-wrapper Target: sparc-sun-solaris2.10 Configured with: ../gcc-4.5.1/configure --prefix=/usr/local/gcc-4.5.1 --with-gmp=/usr/local/gcc-4.5.1 --with-mpfr=/usr/local/gcc-4.5.1 --with-mpc=/usr/local/gcc-4.5.1 --enable-languages=c,c++,fortran --disable-nls --enable-checking=release --enable-werror=no --enable-multilib --with-system-zlib --enable-bootstrap --without-gnu-as --with-as=/usr/ccs/bin/as --without-gnu-ld --with-ld=/usr/ccs/bin/ld --with-pkgversion='Built by D. Kirkby gmp-5.0.1 mpfr-3.0.0 mpc-0.8.2' Thread model: posix gcc version 4.5.1 (Built by D. Kirkby gmp-5.0.1 mpfr-3.0.0 mpc-0.8.2) **************************************************** ld: Software Generation Utilities - Solaris Link Editors: 5.10-1.496 ld: fatal: no files on input command line Cannot execute ld!
comment:38 Changed 10 years ago by
Non-reproducable failure on Fedora 14-64 (taurus):
atlas-3.8.3.p17 Machine: Linux taurus 2.6.35.10-74.fc14.x86_64 #1 SMP Thu Dec 23 16:04:50 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux Deleting directories from past builds of previous/current versions of atlas-3.8.3.p17 Extracting package /home/buildbot/build/sage/taurus-1/taurus_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg ... -rw-r--r-- 1 buildbot sage 2819924 Mar 8 09:30 /home/buildbot/build/sage/taurus-1/taurus_full/build/sage-4.7.alpha1/spkg/standard/atlas-3.8.3.p17.spkg Finished extraction **************************************************** Host system uname -a: Linux taurus 2.6.35.10-74.fc14.x86_64 #1 SMP Thu Dec 23 16:04:50 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux **************************************************** **************************************************** CC Version gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/local/gcc-4.5.1/x86_64-Linux-nehalem-fc/libexec/gcc/x86_64-unknown-linux-gnu/4.5.1/lto-wrapper Target: x86_64-unknown-linux-gnu Configured with: /usr/local/gcc-4.5.1/src/gcc-4.5.1/configure --enable-languages=c,c++,fortran --with-gnu-as --with-gnu-as=/usr/local/binutils-2.20.1/x86_64-Linux-nehalem-fc-gcc-4.4.3-rh/bin/as --with-gnu-ld --with-ld=/usr/local/binutils-2.20.1/x86_64-Linux-nehalem-fc-gcc-4.4.3-rh/bin/ld --with-gmp=/usr/local/gmp-5.0.1/x86_64-Linux-nehalem-fc-gcc-4.4.3-rh --with-mpfr=/usr/local/mpfr-3.0.0/x86_64-Linux-nehalem-fc-gmp-5.0.1-gcc-4.4.4-rh --with-mpc=/usr/local/mpc-0.8.2/x86_64-Linux-nehalem-fc-mpfr-3.0.0-gmp-5.0.1-gcc-4.4.4-rh --prefix=/usr/local/gcc-4.5.1/x86_64-Linux-nehalem-fc Thread model: posix gcc version 4.5.1 (GCC) **************************************************** [...] STAGE 2-2-5: GEMV TUNE make -f Makefile INSTALL_LOG/sMVRES pre=s 2>&1 | ./xatlas_tee INSTALL_LOG/sMVTUNE.LOG gemvN : chose routine 3:ATL_gemvN_1x1_1a.c written by R. Clint Whaley Yunroll=32, Xunroll=1, using 87 percent of L1 Performance = 4858.37 (39.48 of copy matmul, 166.04 of clock) gemvT : chose routine 104:ATL_gemvT_4x16_1.c written by R. Clint Whaley Yunroll=4, Xunroll=16, using 87 percent of L1 Performance = 3299.85 (26.81 of copy matmul, 112.78 of clock) STAGE 2-2-6: GER TUNE make -f Makefile INSTALL_LOG/sR1RES pre=s 2>&1 | ./xatlas_tee INSTALL_LOG/sR1TUNE.LOG make[3]: *** [build] Error 255 make[3]: Leaving directory `/home/buildbot/build/sage/taurus-1/taurus_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build' make[2]: *** [build] Error 2 make[2]: Leaving directory `/home/buildbot/build/sage/taurus-1/taurus_full/build/sage-4.7.alpha1/spkg/build/atlas-3.8.3.p17/ATLAS-build' Configuration: linker_GNU?: True CYGWIN?: False ld: GNU FreeBSD?: False 32bit?: False linker_Solaris?: False bits: 64bit system: Linux Darwin?: False machine: x86_64 fortran: gfortran PPC?: False 64bit?: True fortran_GNU?: True fortran_g95?: False Intel?: True Linux?: True SPARC?: False processor: x86_64 Solaris?: False Copying patches/archinfo_linux.c to src/CONFIG/src/backend/archinfo_linux.c Copying patches/archinfo_x86.c to src/CONFIG/src/backend/archinfo_x86.c Copying patches/probe_comp.c to src/CONFIG/src/probe_comp.c Copying patches/Make.top to src First automatic tuning attempt. Running ../src/configure --prefix=/home/buildbot/build/sage/taurus-1/taurus_full/build/sage-4.7.alpha1/local --with-netlib-lapack=/home/buildbot/build/sage/taurus-1/taurus_full/build/sage-4.7.alpha1/local/lib/liblapack.a -Si cputhrchk 0 -Fa alg -fPIC -C if sage_fortran -b 64 -O 1 Finished configuring ATLAS. Traceback (most recent call last): File "./spkg-install", line 402, in <module> assert rc==0, 'Failed to build ATLAS.' AssertionError: Failed to build ATLAS.
comment:39 Changed 10 years ago by
Sorry the atlas spkg was an older version, and I accidentally dropped a couple of compile fixes. I've updated the spkg (same place) with the most recent version.
The OSX issue has been fixed previously. It also definitely works on t2. Is fulvia a supported platform? Sage refuses to compile there unless SAGE_PORT
is set.
comment:40 follow-up: ↓ 41 Changed 10 years ago by
- Status changed from needs_work to needs_info
I tried to build Sage on fulvia but it fails to build mpir. How does the buildbot manage to build Sage for starters?
On hawk (where is that machine, its apparently not on skynet) it seems like SAGE_ATLAS_LIB is set to something but at least one of the libraries is not in the specified path.
[vbraun@eno ~]$ ssh hawk ssh: Could not resolve hostname hawk: Name or service not known
comment:41 in reply to: ↑ 40 Changed 10 years ago by
- Cc mpatel added
Replying to vbraun:
I tried to build Sage on fulvia but it fails to build mpir. How does the buildbot manage to build Sage for starters?
On hawk (where is that machine, its apparently not on skynet)
Mitesh Patel should know the answer to both of these questions.
comment:42 Changed 10 years ago by
'fulvia' is a Solaris 10 x86 machine on Skynet. Sage builds on that fine.
'hawk' is a Sun Ultra 27 running OpenSolaris and is my personal machine.
The buildbot regularly builds Sage on both of these machines.
Dave
comment:43 Changed 10 years ago by
For more on fulvia and the other skynet machines, see http://wiki.sagemath.org/skynet. In particular, this should tell you how to set environment variables to build Sage on fulvia.
comment:44 follow-up: ↓ 51 Changed 10 years ago by
I found a couple more environment variables that might help on http://wiki.sagemath.org/skynet so I'll try again to build on fulvia.
Dave, can you try the atlas spkg? And if it fails again, can you tell me the directory structure of /ATLAS32? Then I'll modify the spkg-install to be not too picky. Do you have an automated script to build a suitable gcc on OpenSolaris? by any chance, then I could use my virtual machine.
For the record, the updated atlas spkg builds fine on OSX and t2.
comment:45 follow-up: ↓ 46 Changed 10 years ago by
Either here or on a followup ticket, the environment variables should be documented in the Sage installation guide. See http://www.sagemath.org/doc/installation/source.html#environment-variables, in particular. Since SAGE_ATLAS_LIB
now has a different usage, this especially needs to be documented. Is this lack of backwards compatibility okay, by the way? For someone like me, if we don't realize that the variable has been changed, then the time to build Sage on t2.math.washington.edu will go from 4 hours to 15, or something like that. (This might also explain the situation with hawk: perhaps SAGE_ATLAS_LIB
is set to the parent directory of .../lib.)
comment:46 in reply to: ↑ 45 Changed 10 years ago by
Replying to jhpalmieri:
Either here or on a followup ticket, the environment variables should be documented in the Sage installation guide.
Oh, I see that SAGE_ATLAS_ARCH has been documented already in #10666. But my comments about SAGE_ATLAS_LIB still stand.
comment:47 Changed 10 years ago by
- Status changed from needs_info to needs_work
I have made a new version of sage-4.7.alpha1 without this ticket which should be tested today on the buildbots (and hopefully released soon). After sage-4.7.alpha1 is released, I can test the updated version of the atlas spkg which can then maybe go in a later alpha.
comment:48 Changed 10 years ago by
Oh, I remember now. I changed the SAGE_ATLAS_LIB
semantics such that one has to specify /path/lib
instead of /path
. This was changed because otherwise the spkg-install has to hardcode /lib
but some OS'es use /lib32
, /lib64
, etc. The header must now be in SAGE_ATLAS_LIB/../include
.
comment:49 Changed 10 years ago by
Though thinking about it, thats also insufficient. For example, Fedora puts the ATLAS shared libraries into /usr/lib64/atlas
, so the headers are not in ../include
. How about we'll have a library/header search path. If it consists of a single entry, it'll try to append /lib
and /include
. Useful yet backward-compatible.
comment:50 Changed 10 years ago by
http://wiki.sagemath.org/solaris has information about building on Solaris on SPARC. The main difference on x86 is that one must build binutils, to get the GNU assembler 'as', then configure gcc to use that assembler, rather than the Sun assembler. But use the Sun linker, as one does on SPARC.
Building on OpenSolaris is the same as on Solaris 10 x86 - just see what's done on skynet.
The directory structure on 'hawk' of /ATLAS32
is
/ATLAS32/lib /ATLAS32/include
That corresponds with the current behavior in Sage, where one sets SAGE_ATLAS_LIB
to be /ATLAS32
.
I don't know if we actually need the include files, in which case there may be an argument for making SAGE_ATLAS_LIB point to /ATLAS32/lib
or /ATLAS32/lib64
or /usr/lib64/atlas
etc. I'm fairly sure one can delete the include files on a system-wide install of ATLAS and Sage will still build with just the ATLAS libraries.
comment:51 in reply to: ↑ 44 Changed 10 years ago by
Replying to vbraun:
Do you have an automated script to build a suitable gcc on OpenSolaris? by any chance, then I could use my virtual machine.
No, but I just wrote this, which should be enough information. If there's something missing, let me know. It's all from memory.
http://wiki.sagemath.org/Solaris86
Dave
comment:52 Changed 10 years ago by
It fails on OpenSolaris in 64-bit mode. I reports not all 4 libraries are present, though I'm not convinced all 4 were needed. Anyway, that might be an installation issue at this end with the 64-bit libraries, as I've not used them much.
Here's the failure message:
**************************************************** Platform detected to be 64 bits system_atlas.py:6: DeprecationWarning: os.popen2 is deprecated. Use the subprocess module. fortran = os.popen2(os.environ['SAGE_LOCAL']+'/bin/'+'which_fortran')[1].read() Unable to find one of liblapack.a, libcblas.a, libatlas.a or libf77blas.a in the directory /ATLAS64/lib
As you know, I think the idea of writing this in Python is a bad one, but if its going to be done, I would have thought it better not to use depreciated code. Ideally one should use something that works in Python 2.6 and well as 3.x, but at least make sure it is not depreciated already, as that will only make it more difficult to maintain as Python changes.
Dave
comment:53 Changed 10 years ago by
Do you know which libraries are necessary? I'm pretty sure we need liblapack, and at least the liblapack.so requires the other three shared libraries.
comment:54 Changed 10 years ago by
drkirkby@hawk:~$ ls /ATLAS32/lib libatlas.a libcblas.a libf77blas.a liblapack.a libatlas.so libcblas.so libf77blas.so
I'm not convinced we need any shared libraries. I don't believe ATLAS builds any, and certainly Mathematica comes with libatlas.a, but no libatlas.so.
Note there is no liblapack.so
, yet this builds fine.
On the 64-bit version,
drkirkby@hawk:~$ ls /ATLAS64/lib libatlas.a libcblas.a libf77blas.a libatlas.so libcblas.so libf77blas.so
but I must have omitted to copy liblapack.a
there, which is why I had to build ATLAS from scratch, which takes about 15 minutes on this machine in 64-bit mode, but a few hours in 32-bit mode, since the ATLAS package does not have the right tuning parameters for 32-bit code.
I've since copied the 64-bit liblapack.a to my global installation directory, so now the directory has
drkirkby@hawk:~/pre-release/64/sage-4.7.alpha1/local/lib$ ls /ATLAS64/lib libatlas.a libcblas.a libf77blas.a liblapack.a libatlas.so libcblas.so libf77blas.so
which is the same number and names as the 32-bit version.
Dave
comment:55 Changed 10 years ago by
For Sage, or any other code that uses blas/lapack, it really does not matter if these libraries exist as static or shared form. Of course static libraries are cruft from the dark ages and shouldn't be used nowadays, but I digress. As long as either is available, the linker will do the right thing and hook up the code with blas/lapack.
With this in mind, I changed the spgk-install to just use whatever is in the SAGE_ATLAS_LIB directory, either static or shared libraries. Fedora ships dynamic atlas libraries ONLY and Sage compiles fine with SAGE_ATLAS_LIB=/usr/lib64/atlas. I also removed the part where headers from SAGE_ATLAS_LIB were symlinked to SAGE_LOCAL_LIB and apparently to no ill effect. Finally, I'm now also looking into SAGE_ATLAS_LIB/lib for libraries in order to be backward compatible.
I've updated the atlas spkg (same place), please give it a try :-)
comment:56 Changed 10 years ago by
- Status changed from needs_work to needs_review
comment:57 Changed 10 years ago by
I'll test this as soon as I can, but for now I have other things that must take priority. It's going to take some effort to test this with the different possibilities for the environment variables.
A couple of points.
- Did you sort out the depreciated Python code?
- Is there revised documentation which explains the role of SAGE_ATLAS_LIB? (I don't see a patch attached to the ticket, and as John remarks, this is quite important).
Dave
comment:58 Changed 10 years ago by
- Reviewers changed from Dan Drake to Dan Drake, David Kirkby
- Status changed from needs_review to needs_info
comment:59 Changed 10 years ago by
- Status changed from needs_info to needs_review
Documentation changes for SAGE_ATLAS_LIB are at #10666.
I haven't changed the os.popen2 call since a) its also in the sage library and b) its probably better to be backward than future compatible if we ever want to attempt to use the system python for executing spkg scripts. But then, I don't care too much about popen vs. subprocess and will be happy to rewrite it if you insist.
comment:60 Changed 10 years ago by
Dave: I just checked and I'm actually using subprocess.Popen instead of os.popen2. The warning that you posted is from the system_atlas.py
script which has never been part of the rewrite. You must be using an old version of the atlas spkg before I rewrote anything in its spkg-install.
comment:61 Changed 10 years ago by
For testing purposes I built a source distribution with the updated atlas spkg and documentation fixes:
comment:62 Changed 10 years ago by
I updated the spkg to also symlink the parallel libraries libptcblas
, libptf77blas
from SAGE_ATLAS_LIB
if they are available.
comment:63 Changed 10 years ago by
Planning to test this on the buildbot...
comment:64 Changed 10 years ago by
Things are looking very good on the buildbot, no problems with ATLAS!
comment:65 Changed 10 years ago by
- Milestone changed from sage-4.7 to sage-4.7.1
comment:66 Changed 10 years ago by
I created an updated version
http://www.stp.dias.ie/~vbraun/Sage/spkg/atlas-3.8.3.p18.spkg
to address #11326 (make ATLAS respect CC environment variable).
comment:67 Changed 10 years ago by
- Description modified (diff)
comment:68 Changed 10 years ago by
- Priority changed from major to critical
comment:69 Changed 10 years ago by
I'll reiterate the point that ATLAS takes a long time to build, which could be speeded up considerably if a couple of minor changes were made to the Fortran package to remove its unnecessary reliance on Python. If we build Fortran earlier, we could start the build of ATLAS much earlier, so reduce the time to build Sage. For much of the time building Sage, ATLAS is the only package being built.
As such, I feel this change is a retrograde step.
IMHO, the best approach for ATLAS would be to
- Update it to use the pre-release/alpha version, as advised by the ATLAS developer and has been used in Gentoo.
- Check what (if any) changes need to be made to ATLAS. I think the new ATLAS will address some of the issues this update attempts to solve.
- Rewrite this package to use a POSIX shell commands
- Rewrite the Fortran package to avoid the very small amount of Python.
- Change the deps file to remove the dependency of ATLAS on Python.
That would allow ATLAS to be built before (or in parallel with Python), so speeding up the Sage build time.
I feel these changes are a retrograde step.
Dave
comment:70 Changed 10 years ago by
- The development version of ATLAS currently does not work for Sage purposes, see #10508.
- Python takes essentially no time to compile vs. ATLAS, especially if ATLAS is going through its tuning process.
- If anything, more spkg-install scripts should be written in Python. Its easier to maintain and avoids may common pitfalls that can make shell scripts not POSIX-compliant. It also gives you portable functions to find out bit width and number of CPU cores, say.
- Nowadays every distribution already comes with Python, we can just use the system Python for building Sage.
comment:71 Changed 10 years ago by
Regarding Python, I agree with Volker's opinion. I would also like to add that I don't find the Sage build time so important. And I'm saying this as the person who probably does the most compiles of a full Sage from source.
comment:72 Changed 10 years ago by
- Description modified (diff)
New stable release ATLAS-3.8.4 is out! Spkg updated to reflect this.
Now would be a good time for somebody to review this so that it can make its way into Sage-4.7.1 ...
comment:73 Changed 10 years ago by
preliminary testing on sage-on-gentoo are all good. Now to test the spkg on a couple of linux box with "vanilla" installs. I have a few other things I want to review so there may be a couple of days delay.
comment:74 Changed 10 years ago by
If the version of ATLAS is updated, many if not all of the Sage defaults for different processors can be removed. I know many of them in Sage were available in the alpha releases ages ago.
The buildbots on "hawk" and "t2.math" both use a preinstalled version of ATLAS, so testing on those machine would have to be done with unsetting SAGE_ATLAS_LIB in the buildbot's configuration.
Dave
comment:75 Changed 10 years ago by
Looks like Volker took care of all the old patches already. I applied to a 4.7.1.alpha1 that I had lying around then rebuilt all ATLAS using packages I could think of: iml, linbox, cvxopt, numpy, scipy and R. I followed that by sage -ba and run the long tests. The only tests that failed were unrelated ones that failed before (german tutorial that were removed from alpha1 in the end because they were failing on this very machine).
So far it looks good to me on amd64.
comment:76 follow-up: ↓ 77 Changed 10 years ago by
It would be good if we had some reports from the build boot. I hesitate to put on positive review on a package like this one which has an enormous platform dependency just based on my tests on 1 or 2 machines. It looks good, people who used the 3.8.3 version were satisfied (but unfortunately didn't always report their success story on the ticket). If we had a round of testing on the bot I would fell more comfortable pushing the positive review button.
comment:77 in reply to: ↑ 76 Changed 10 years ago by
Replying to fbissey:
It would be good if we had some reports from the build boot. I hesitate to put on positive review on a package like this one which has an enormous platform dependency just based on my tests on 1 or 2 machines. It looks good, people who used the 3.8.3 version were satisfied (but unfortunately didn't always report their success story on the ticket). If we had a round of testing on the bot I would fell more comfortable pushing the positive review button.
This is a good idea. In fact, I'd like a way to take any spkg, put it the most recent alpha, and submit it to the buildbot. But for now I would be very happy to see this ATLAS upgrade get into 4.7.1.
comment:78 follow-up: ↓ 79 Changed 10 years ago by
François, Dan: may I take your comments as meaning "if no problems arise on the buildbot, then positive_review for this ticket"?
comment:79 in reply to: ↑ 78 ; follow-up: ↓ 80 Changed 10 years ago by
Replying to jdemeyer:
François, Dan: may I take your comments as meaning "if no problems arise on the buildbot, then positive_review for this ticket"?
Yes.
comment:80 in reply to: ↑ 79 Changed 10 years ago by
comment:81 Changed 10 years ago by
- Merged in set to sage-4.7.1.alpha3
comment:82 Changed 10 years ago by
- Merged in sage-4.7.1.alpha3 deleted
- Status changed from needs_review to needs_work
This fails on the buildbot machine iras
(Linux ia64): see http://build.sagemath.org/sage/builders/SUSE%20ES10-64%20%28iras%29/builds/129/steps/shell_1/logs/atlas
If I recall correctly, an earlier version of the Atlas spkg on this ticket did work correctly.
comment:83 Changed 10 years ago by
- Status changed from needs_work to needs_review
I built Sage-4.7 with my atlas-3.8.4.spkg on iras overnight and it worked fine. Looking at the log file, it seems like atlas just failed to get timings because of system load.
I updated my atlas spkg to force the architecture to IA64Itan2/IA64Itan if the automatic tuning fails, this would give us two more shots at a successful build on Itanium and match the behaviour of the old atlas spkg (4 build attempts altogether). But when I built atlas overnight, the automatic tuning worked on the first try. I did not set SAGE_PARALLEL_SPKG_BUILD=yes
, but the buildbot does. It might be a good idea to not set it on iras if the atlas timing problem surface often.
To summarize, this failure would have happened with the old atlas spkg as well. Can you try again with the updated spkg (same location)?
comment:84 Changed 10 years ago by
Volker, I tried again with the old atlas-3.8.4 (i.e. the package which I originally said didn't work) and after 4 attempts at building Sage, it finally succeeded.
However, I think this is still a regression, because I did not get atlas build failures usually (even with SAGE_PARALLEL_SPKG_BUILD=yes).
I will need to do some more testing.
comment:85 Changed 10 years ago by
I went back and re-checked the differences with the previous atlas-3.8.3 and found that I accidentally dropped some patches that deal with itanium detection. This manifests itself in the build log as Architecture configured as UNKNOWN
whereas it previously was Architecture configured as IA64Itan2
. I've re-added the Itanium patches and updated the atlas-3.8.4.spkg. I'm pretty sure that it'll work now.
comment:86 Changed 10 years ago by
- Status changed from needs_review to needs_info
We really needed that run on the build bot otherwise we wouldn't have caught it. I had a quick look at the spkg Volker, did you update SPKG.txt with what you just did? It doesn't look like it to me. It looks like it could use some clean up so we know exactly what is currently needed versus what was needed in the past.
comment:87 Changed 10 years ago by
- Status changed from needs_info to needs_review
Thanks, I updated the SPKG.txt to properly reflect which patches were removed. The new spkg is in the usual location. No changes except to the SPKG.txt log.
comment:88 Changed 10 years ago by
Sorry to be a pain in the ass but it seems that this spkg-install is looking for the wrong thing under cygwin according to #11494.
comment:89 Changed 10 years ago by
I've updated the atlas spkg to include the Cygwin path fix.
comment:90 Changed 10 years ago by
- Milestone changed from sage-4.7.1 to sage-4.7.2
comment:91 Changed 10 years ago by
- Status changed from needs_review to positive_review
OK so ia64 should be fixed and we have feedback from #11494 that it works on cygwin with windows XP so I think we can switch to positive review. *Pushing the button*
comment:92 Changed 10 years ago by
one more positive "review" (haven't read the actual patch). Had issues with Atlas building on fresh Sage on ubuntu 11.04 x86. Downloaded Volker's new spkg and put it in spkg/standard.
First 'make' run failed again for 'probe/aux' problem. Setting SAGE_ATLAS_ARCH=fast made everything go smooth.
comment:93 Changed 10 years ago by
- Merged in set to sage-4.7.2.alpha0
- Resolution set to fixed
- Status changed from positive_review to closed
This should fix #10051, #9385.
Is the
make_lapack_library()
function inspkg-install
(originally frommake_correct_shared.sh
) necessary on Solaris? It was just bitrot on Linux...