Ticket #10508: atlas-3.10.1.p0.diff

.hgignore

@@ -1 +1 @@
 src
+lapack.*tar
+patches/ATLAS-lib/Makefile.in
+patches/ATLAS-lib/aclocal.m4
+patches/ATLAS-lib/configure$
+patches/ATLAS-lib/config.guess
+patches/ATLAS-lib/config.sub
+patches/ATLAS-lib/install-sh
+patches/ATLAS-lib/ltmain.sh
+patches/ATLAS-lib/m4
+patches/ATLAS-lib/missing

SPKG.txt

@@ -2 +2 @@
 
 == Description ==
 
-This spkg builds ATLAS for Sage. It is build per default on Linux, but should
-also work on OSX and Solaris.
+This spkg builds ATLAS for Sage. It is build per default on Linux and
+Solaris, but should also work on OSX and Cygwin.
 
 == License ==
 
 3-clause BSD
 
 == SPKG Maintainers ==
+
  * David Kirkby
  * William Stein
+ * Volker Braun
 
 == Upstream Contact ==
 
  * Atlas devel mailing list.
-
  * Clint Whaley has frequently answered questions from the Sage project
 
 == Dependencies ==
 
  * Python
  * Fortran
- * Lapack
 
 == Build Instructions/Notes ==
 
  * patches/archinfo_linux.c.patch: Fix Itanium2 support on modern
    RHEL 5 and SLES 10 systems
- * patches/archinfo_x86.c.patch: add cpuid support for Pentium D, E models
- * patches/Make.top.patch: add make install target for shared libs
  * patches/probe_comp.c.patch: work around -m64 issue on Itanium2
- * Tuning info for K732SSE1.tgz, PM32SSE2.tgz and PPCG432.tgz
- * Changes linker flags on Solaris if the Sun linker is used.
- * AN UPDATE TO THE LATEST ATLAS IS WELL OVERDUE, BUT THIS IS NON-TRIVIAL
- * Changing this package, which takes a long time to build, so it (and the
-   Fortran package) did not use Python, would allow the ATLAS build to be
-   started sooner in parallel builds. This should reduce the overall time to
-   build Sage.
- * The package is never installed on Cygwin or OS X.
+ * patches/long_filenames.patch: fix for long filenames (>128 chars)
+ * patches/shell.patch: replace two instances of "test -e" by "test -f",
+   as the former is not portable (in particular for the Solaris shell).
+ * lapack-x.y.z.tgz: The netlib lapack tarball. If you update this,
+   make sure you also update the LAPACK_TARBALL variable in
+   spkg-install.
+ * If you update atlas to a new version make sure that the
+   ATLAS_OSTYPE, ATLAS_MACHTYPE, and ATLAS_ISAEXT variables in
+   spkg-install remain in sync with atlas' CONFIG/include/atlconf.h
+ * The package is never installed on Cygwin or OS X, unless you set
+   SAGE_ATLAS_ARCH.
 
-The package can be configured via two environment variables:
+We are using a dummy autotools/libtools project (see
+patches/ATLAS-lib) to repack the static ATLAS libraries into shared
+libraries.
+
+The package can be configured via three environment variables:
 
  * SAGE_ATLAS_LIB=path
    If this environment variable is set, the libraries libatlas,
@@ -54 +59 @@
    The given architectural default and instruction set extensions are
    used instead of the empirical tuning. Available architectures are
 
-     POWER3, POWER4, POWER5, PPCG4, PPCG5, P5, P5MMX, PPRO, PII, PIII,
-     PM, CoreSolo, CoreDuo, Core2Solo, Core2, Corei7, P4, P4E,
-     Efficeon, K7, HAMMER, AMD64K10h, UNKNOWNx86, IA64Itan, IA64Itan2,
-     USI, USII, USIII, USIV, UnknownUS, MIPSR1xK, MIPSICE9
+     POWER3, POWER4, POWER5, PPCG4, PPCG5, POWER6, POWER7, IBMz9,
+     IBMz10, IBMz196, x86x87, x86SSE1, x86SSE2, x86SSE3, P5, P5MMX,
+     PPRO, PII, PIII, PM, CoreSolo, CoreDuo, Core2Solo, Core2, Corei1,
+     Corei2, Atom, P4, P4E, Efficeon, K7, HAMMER, AMD64K10h, AMDDOZER,
+     UNKNOWNx86, IA64Itan, IA64Itan2, USI, USII, USIII, USIV, UST2,
+     UnknownUS, MIPSR1xK, MIPSICE9, ARMv7
 
    and instruction set extensions are
 
-     AltiVec, SSE3, SSE2, SSE1, 3DNow
+     VSX, AltiVec, AVXMAC, AVXFMA4, AVX, SSE3, SSE2, SSE1, 3DNow, NEON
 
    In addition, you can also set
 
@@ -73 +80 @@
 
    For example,
 
-     SAGE_ATLAS_ARCH=Corei7,SSE3,SSE2,SSE1
+     SAGE_ATLAS_ARCH=Corei2,AVX,SSE3,SSE2,SSE1
 
    would be appropriate for a Core i7 CPU.
 
-== TODO ==
-
- * The build is currently disabled on OSX until we sort out some
-   linking issues on OSX.
-
- * On FreeBSD dynamic libraries don't work since the wrong flags are
-   passed to the linker. This will be fixed soon.
+ * If SAGE_ATLAS_SAVE_ARCHDEF = <path> is given, then a new archdef
+   file is created and saved to the given path.
 
 == ChangeLog ==
 
+=== atlas-3.10.1.p0, lapack-3.4.2 (Jean-Pierre Flori, 26 February 2013) ===
+ * Trac #10508: Update ATLAS to version 3.10.1 and LAPACK to version 3.4.2.
+ * Removed patches integrated upstream.
+ * Add shell.patch to correctly build LAPACK on Solaris.
+ * Replace os.system() by subprocess.call()
+
+=== atlas-3.10.0.p1, lapack-3.4.1 (Volker Braun, September 20 2012) ===
+ * Trac #10508: Fix for long file names.
+ * Disable tuning on Itanium, go straight to "fast".
+
+=== atlas-3.10.0.p0, lapack-3.4.1 (Jeroen Demeyer, 19 September 2012) ===
+ * Trac #10508: re-run autotools in patches/ATLAS-lib to fix timestamps.
+ * Re-download sources to fix timestamps.
+ * Remove backup files (ending with ~).
+ * Uncompress lapack tarball, yielding a smaller spkg.
+
+=== atlas-3.10.0, lapack-3.4.1 (Volker Braun, July 11 2012) ===
+ * Stable release
+ * Now OSX and Cygwin will attempt to build if SAGE_ATLAS_ARCH is set.
+
+=== atlas-3.9.85, lapack-3.4.1 (Volker Braun, July 9 2012) ===
+ * Should be identical to the next stable release
+
+=== atlas-3.9.84, lapack-3.4.1 (Volker Braun, July 8 2012) ===
+ * Update to the newest upstream version
+ * flush before os.system()
+ * completely new shared library build system
+
+=== atlas-3.9.80, lapack-3.4.1 (Volker Braun, June 23rd 2012) ===
+ * Update to the newest upstream version
+ * remove obsoleted patches
+
+=== atlas-3.9.68, lapack-3.4.0 (Volker Braun, Jeroen Demeyer, March 6th 2012) ===
+ * In src/CONFIG/src/Makefile, change "rm -f" to "rm -rf" in
+   the "clean" rule to support Xcode creating directories like
+   xspew.dSYM.
+ * update to latest upstream version
+ * Make atlas respect CC environment variable.
+ * Cygwin library name fix.
+
+=== atlas-3.9.32, lapack-3.3.0 (Volker Braun, December 5th 2010) ===
+ * Introduce the SAGE_ATLAS_ARCH environment variable.
+ * Update to the developer release atlas-3.9.32 according to advice of
+   Clint Whaley (ATLAS developer).
+ * Now includes LAPACK source tarball.
+
 === atlas-3.8.4.p1 (Jeroen Demeyer, 15 January 2012) ===
  * Trac #12312: Completely disable parallel make everywhere
 
@@ -311 +359 @@
 === atlas-3.7.37 (Michael Abshoff, Aug. 15th, 2007) ===
  * new upstream release (3.7.37), initial release
  * {{{make check}}} added in spkg-check
-
-ToDo
-
-* compile threading support conditionally
-

configuration.py

@@ -13 +13 @@
 # strings:
 #
 # system:  Linux, SunOS, Darwin, FreeBSD, CYGWIN
+# release: (Version number of the operating system, output of uname -r)
 # machine: i386, x86_64,   # Linux, Darwin, *BSD
 #          sun4u, i86pc    # SunOS
 # processor: i386, x86_64, powerpc, sparc
 # bits:    32bit, 64bit
 # fortran: g95, gfortran
-# ld:      GNU, Solaris
+# ld:      GNU, Solaris, Darwin
 
 # The following pre-defined boolean values are determined from the
 # strings. The keys are distinguished by a question mark at the
 # end. If possible use these keys as they guard against typos.
 # 
 # Linux?, Solaris?, Darwin?, FreeBSD?, CYGWIN?  # system
+# OS_X_Lion?                                    # release
 # Intel?, PPC?, SPARC?                          # processor
 # 64bit?, 32bit?                                # bit width
 # fortran_g95?, fortran_GNU?                    # sage_fortran
-# linker_GNU?, linker_Solaris?                  # ld
+# linker_GNU?, linker_Solaris?, linker_Darwin?  # ld
 
 
 ######################################################################
@@ -73 +75 @@
         shutil.copy2(filename, destination)
 
 
+def ln(source, destination):
+    """
+    Portable implementation of "ln -sf"
+    """
+    if os.path.exists(destination):
+        os.remove(destination)
+    print 'Linking', source, 'to', destination
+    os.symlink(source, destination)
+
+
+def is_executable_file(fpath):
+    """
+    Check that `fpath` is a file and executable (ie not a directory with +x set)
+    """
+    return os.path.isfile(fpath) and os.access(fpath, os.X_OK)
+
+def which(program):
+    """
+    Portable implementation of "which"
+    """
+    fpath, fname = os.path.split(program)
+    if fpath != '':
+        return program
+    for path in os.environ['PATH'].split(os.pathsep):
+        f = os.path.join(path, program)
+        if is_executable_file(f):
+            return f
+    raise ValueError('The program '+program+' is not in the $PATH.')
+
 class edit_in_place(object):
     """
     Edit a file in-place by search/replacing regular expressions.
@@ -102 +133 @@
 except KeyError:
     conf['system'] = platform.system()
 
+try:
+    conf['release'] = subprocess.check_output(['uname', '-r']).strip()
+except subprocess.CalledProcessError:
+    conf['release'] = ""
+
 conf['Linux?'  ] = (conf['system'] == 'Linux')
 conf['Solaris?'] = (conf['system'] == 'SunOS')
 conf['Darwin?' ] = (conf['system'] == 'Darwin')
+conf['OS_X_Lion?'] = (conf['Darwin?'] and conf['release'].startswith('11.'))
 conf['FreeBSD?'] = (conf['system'] == 'FreeBSD')
 conf['CYGWIN?' ] = (conf['system'] == 'CYGWIN')
 
@@ -114 +151 @@
 
 conf['Intel?'] = (platform.machine() in ('i386', 'i486', 'i586', 'i686', 'x86_64', 
                                          'AMD64', 'i86pc'))
-conf['IA64?']   = (platform.processor() == 'ia64')
+conf['IA64?']   = ((platform.processor() == 'ia64')
+                   or (platform.machine() == 'ia64'))
 conf['PPC?']   = (platform.processor() == 'powerpc')
 conf['SPARC?'] = (platform.processor() == 'sparc')
 
+conf['generic_binary?'] = (os.environ.get('SAGE_FAT_BINARY', 'no') == 'yes')
 
 ######################################################################
 ### bit width
@@ -178 +217 @@
     conf['ld'] = 'GNU'
 elif 'Solaris' in ld_version:
     conf['ld'] = 'Solaris'
+elif 'Apple' in ld_version:
+    conf['ld'] = 'Darwin'
 else:
     print 'Unknown linker: '+ld_version
     conf['ld'] = None
 
 conf['linker_GNU?'] = (conf['ld'] == 'GNU')
 conf['linker_Solaris?'] = (conf['ld'] == 'Solaris')
+conf['linker_Darwin?'] = (conf['ld'] == 'Darwin')
 
 
 if conf['Solaris?'] and conf['linker_GNU?']:

new file patches/ATLAS-lib/Makefile.am

@@ +1 @@
+ACLOCAL_AMFLAGS = -I m4
+
+# Replace with the actual destination directory
+SO_VERSION=3:0:1
+
+# You shouldn't have to customize anything from here on
+
+ATLAS_SERIAL_LIBS=libatlas.la libcblas.la libf77blas.la liblapack.la
+
+ATLAS_PARALLEL_LIBS=libptcblas.la libptf77blas.la libptlapack.la
+
+all: all_parallel
+
+all_serial: $(ATLAS_SERIAL_LIBS)
+
+all_parallel: all_serial $(ATLAS_PARALLEL_LIBS)
+
+clean:
+        $(RM) -r -f .libs *.la *-obj
+
+install: install_parallel
+
+install_serial: $(ATLAS_SERIAL_LIBS)
+        $(MKDIR_P) @libdir@
+        @LIBTOOL@ --mode=install $(INSTALL) -c libatlas.la @libdir@/libatlas.la
+        @LIBTOOL@ --mode=install $(INSTALL) -c libcblas.la @libdir@/libcblas.la
+        @LIBTOOL@ --mode=install $(INSTALL) -c libf77blas.la @libdir@/libf77blas.la
+        @LIBTOOL@ --mode=install $(INSTALL) -c liblapack.la @libdir@/liblapack.la
+        @LIBTOOL@ --finish @libdir@
+
+install_parallel: install_serial $(ATLAS_PARALLEL_LIBS)
+        $(MKDIR_P) @libdir@
+        @LIBTOOL@ --mode=install $(INSTALL) -c libptcblas.la @libdir@/libptcblas.la
+        @LIBTOOL@ --mode=install $(INSTALL) -c libptf77blas.la @libdir@/libptf77blas.la
+        @LIBTOOL@ --mode=install $(INSTALL) -c libptlapack.la @libdir@/libptlapack.la
+        @LIBTOOL@ --finish @libdir@
+
+
+# In presence of multiple definitions of thread related functions, use the ones from *_mut.o
+# Works with all nm output formats (BSD/POSIX/System V)
+libatlas.la: libatlas.a
+        -$(RM) -r -f libatlas-obj
+        $(MKDIR) libatlas-obj
+        cd libatlas-obj && $(AR) x ../$<
+        if [ `$(NM) -g $< | $(EGREP) -w 'ATL_(Set|Reset|Free|Dec)AtomicCount' | $(GREP) -w T | wc -l` -gt 4 ]; then \
+          $(RM) `ls -1 libatlas-obj/ATL_{Set,Reset,Free,Dec}AtomicCount_*.o | $(GREP) -v '_mut.o$$'`; \
+        fi
+        @LIBTOOL@ --tag=CC --mode=link $(CC) @LIBTOOL_TYPE@ \
+            -o $@ libatlas-obj/*.o \
+            @PTHREAD_LIB@ -lm \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+
+libcblas.la: libcblas.a libatlas.la
+        -$(RM) -r -f libcblas-obj
+        $(MKDIR) libcblas-obj
+        cd libcblas-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=CC --mode=link $(CC) @LIBTOOL_TYPE@ \
+            -o $@ libcblas-obj/*.o \
+            -latlas \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+
+libptcblas.la: libptcblas.a libatlas.la
+        -$(RM) -r -f libptcblas-obj
+        $(MKDIR) libptcblas-obj
+        cd libptcblas-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=CC --mode=link $(CC) @LIBTOOL_TYPE@ \
+            -o $@ libptcblas-obj/*.o \
+            @PTHREAD_LIB@ -latlas \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+
+libf77blas.la: libf77blas.a libatlas.la
+        -$(RM) -r -f libf77blas-obj
+        $(MKDIR) libf77blas-obj
+        cd libf77blas-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=F77 --mode=link $(F77) @LIBTOOL_TYPE@ \
+            -o $@ libf77blas-obj/*.o \
+            -latlas \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+libptf77blas.la: libptf77blas.a libatlas.la
+        -$(RM) -r -f libptf77blas-obj
+        $(MKDIR) libptf77blas-obj
+        cd libptf77blas-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=F77 --mode=link $(F77) @LIBTOOL_TYPE@ \
+            -o $@ libptf77blas-obj/*.o \
+            @PTHREAD_LIB@ -latlas \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+libf77refblas.la: libf77refblas.a
+        -$(RM) -r -f libf77refblas-obj
+        $(MKDIR) libf77refblas-obj
+        cd libf77refblas-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=F77 --mode=link $(F77) @LIBTOOL_TYPE@ \
+            -o $@ libf77refblas-obj/*.o \
+            -latlas \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+liblapack.la: liblapack.a libatlas.la libcblas.la libf77blas.la
+        -$(RM) -r -f liblapack-obj
+        $(MKDIR) liblapack-obj
+        cd liblapack-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=F77 --mode=link $(F77) @LIBTOOL_TYPE@ \
+            -o $@ liblapack-obj/*.o \
+            -latlas -lcblas -lf77blas -lm \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+libptlapack.la: libptlapack.a libatlas.la libptcblas.la libptf77blas.la
+        -$(RM) -r -f libptlapack-obj
+        $(MKDIR) libptlapack-obj
+        cd libptlapack-obj && $(AR) x ../$<
+        @LIBTOOL@ --tag=F77 --mode=link $(F77) @LIBTOOL_TYPE@ \
+            -o $@ libptlapack-obj/*.o \
+            @PTHREAD_LIB@ -latlas -lptcblas -lptf77blas -lm \
+            -rpath @libdir@ -version-info $(SO_VERSION)
+
+
+.PHONY: all all_serial all_parallel build install install_serial install_parallel clean

new file patches/ATLAS-lib/README

@@ +1 @@
+This is a stub autotools project to generate the system-dependent
+libtool script. Nothing is compiled here. The Makefile.am unpacks the
+atlas static libraries and repacks them into shared libraries.

new file patches/ATLAS-lib/configure.ac

@@ +1 @@
+AC_INIT([ATLAS],[3.10.1])
+AM_INIT_AUTOMAKE(foreign)
+
+LT_INIT
+
+AC_PROG_CC
+AC_PROG_F77
+
+AC_CONFIG_FILES([Makefile])
+AC_CONFIG_MACRO_DIR([m4])
+
+AC_CHECK_LIB(pthread, pthread_create, [PTHREAD_LIB="-lpthread"])
+AC_SUBST(PTHREAD_LIB)
+
+AC_ARG_ENABLE([static],
+    [AS_HELP_STRING([--enable-static],
+                    [also install static libtool library (default: no)])])
+AS_IF([test "x$enable_static" = "xyes"],
+    [libtool_type=""],
+    [libtool_type=-shared])
+AC_SUBST(LIBTOOL_TYPE, [$libtool_type])
+
+AC_OUTPUT

new file patches/Makefile.patch

@@ +1 @@
+--- src/CONFIG/src/Makefile     2012-06-23 17:27:27.000000000 +0100
++++ new/CONFIG/src/Makefile     2012-06-24 03:05:52.043151077 +0100
+@@ -580,6 +580,6 @@
+ confclean: $(CLEANdep)
+        rm -f *core* *.o config?.out
+ clean : $(CLEANdep)
+-       rm -f *.o x* config?.out *core*
++       rm -rf *.o x* config?.out *core*
+ cleanall: clean
+        $(MAKE) -f Make.top clean

patches/archinfo_linux.c.patch

@@ -1 @@
---- archinfo_linux.c.orig       2011-06-15 10:25:07.837647902 -0700
-+++ archinfo_linux.c    2011-06-15 10:26:16.728647870 -0700
-@@ -100,12 +100,12 @@
+--- src/CONFIG/src/backend/archinfo_linux.c     2012-06-23 17:27:27.000000000 +0100
++++ new/CONFIG/src/backend/archinfo_linux.c     2012-06-24 03:17:47.297954105 +0100
+@@ -117,7 +117,7 @@
+       }
        break;
     case AFIA64:
-       res[0] = '\0';
--      ierr = CmndOneLine(NULL, "fgrep 'Itanium' /proc/cpuinfo", res);
-+      ierr = CmndOneLine(NULL, "fgrep 'IA-64' /proc/cpuinfo", res);
-       if (ierr || res[0] == '\0')
-          ierr = CmndOneLine(NULL, "fgrep \"model name\" /proc/cpuinfo", res);
-       if (!ierr && res[0] != '\0')
+-      res = atlsys_1L(NULL, "fgrep 'Itanium' /proc/cpuinfo", 0, 0);
++      res = atlsys_1L(NULL, "fgrep 'IA-64' /proc/cpuinfo", 0, 0);
+       if (res && res[0] == '\0')
        {
--         if (strstr(res, "Itanium 2") || strstr(res, "McKinley"))
-+         if (strstr(res, "IA-64") || strstr(res, "McKinley"))
-             mach = IA64Itan2;
-          else if (strstr(res, "Itanium")) mach = IA64Itan;
-       }
+          free(res);
+@@ -129,7 +129,7 @@
+       {
+          if (res[0] != '\0')
+          {
+-            if (strstr(res, "Itanium 2") || strstr(res, "McKinley"))
++            if (strstr(res, "IA-64") || strstr(res, "McKinley"))
+                mach = IA64Itan2;
+             else if (strstr(res, "Itanium")) mach = IA64Itan;
+          }

new file patches/atlas-config

@@ +1 @@
+#!/usr/bin/env python
+
+import sys, errno, os, platform, time, glob, subprocess, signal, argparse
+
+parser = argparse.ArgumentParser(
+    description='(Re-)Build ATLAS (http://math-atlas.sourceforge.net) '
+    'according to the SAGE_ATLAS_ARCH environment variable')
+
+parser.add_argument('--unthrottle', nargs=1, type=int, 
+                    help='switch CPU throttling off until PID finishes', 
+                    metavar='PID')
+
+parser.add_argument('--archdef', action='store_const', const=True,
+                    help='build archdef tarball and save it to the current directory')
+
+
+def pid_exists(pid):
+    """
+    Check whether pid exists in the current process table.
+    """
+    if pid < 0:
+        return False
+    try:
+        os.kill(pid, 0)
+    except OSError, e:
+        return e.errno == errno.EPERM
+    else:
+        return True
+
+
+def unthrottle_posix(pid):
+    cpus = list(glob.glob('/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor'))
+    scaling_governor = []
+    for cpu in cpus:
+        with open(cpu, 'r') as f:
+            scaling_governor.append(f.readline())
+    for cpu in cpus:
+        with open(cpu, 'w') as f:
+            f.write('performance')
+
+    def signal_handler(signum, frame):
+        print 'Signal hanadler called with signal', signum
+        for cpu, governor in zip(cpus, scaling_governor):
+            with open(cpu, 'w') as f:
+                f.write(governor)
+        print 'Reverted throttling to the previous behaviour.'
+        sys.exit(0)
+
+    signal.signal(signal.SIGPIPE, signal_handler)
+    signal.signal(signal.SIGINT, signal_handler)
+    signal.signal(signal.SIGTERM, signal_handler)
+    
+    try:
+        while pid_exists(pid):
+            time.sleep(1)
+    except KeyboardInterrupt:
+        pass
+    
+    signal_handler(None, None)
+   
+     
+def unthrottle(pid):
+    if os.name == 'posix':
+        unthrottle_posix(pid)
+    else:
+        print 'I don\'t know how to unthrottle your system ('+platform.system()+')'
+        sys.exit(3)
+
+
+
+def is_throttled_posix():
+    cpus = list(glob.glob('/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor'))
+    for cpu in cpus:
+        with open(cpu, 'r') as f:
+            if f.readline().strip() != 'performance':
+                return True
+    return False
+
+
+def is_throttled():
+    if os.name == 'posix':
+        return is_throttled_posix()
+    else:
+        print 'I don\'t know how to unthrottle your system ('+platform.system()+')'
+        sys.exit(3)
+        
+
+def check_root():
+    if os.name == 'posix':
+        if os.getuid() == 0:
+            return
+        print '\nError: You need to be root to (un)throttle the CPU.\n'
+        sys.exit(1)
+    else:
+        print 'I don\'t know how to probe administrator rights on your system ('+platform.system()+')'
+        sys.exit(3)
+        
+
+def check_nonroot():
+    if os.name == 'posix':
+        if os.getuid() > 0:
+            return
+        print '\nError: You are crazy to run this as root, exiting.\n'
+        sys.exit(2)
+    else:
+        print 'I don\'t know how to probe administrator rights on your system ('+platform.system()+')'
+        sys.exit(3)
+
+
+command = None
+
+def wait_for_command():
+    global command
+    if command is None:
+        return
+    else:
+        command.send_signal(signal.SIGTERM)
+
+import atexit
+atexit.register(wait_for_command)
+
+
+def unthrottle_self():
+    if os.name == 'posix':
+        global command
+        print 'Running sudo atlas-config --unthrottle to turn CPU throttling off.'
+        command = subprocess.Popen(['sudo', os.path.abspath( __file__ ), 
+                                    '--unthrottle', str(os.getpid())])
+    else:
+        print 'I don\'t know how to unthrottle your system ('+platform.system()+')'
+        sys.exit(2)
+
+    print 'Waiting for CPU throttling to be turned off...'
+    while is_throttled():
+        time.sleep(1)
+        
+
+
+if __name__ == '__main__':
+    args = parser.parse_args()
+
+    if args.unthrottle:
+        check_root()
+        unthrottle(args.unthrottle[0])
+        
+    check_nonroot()
+    unthrottle_self()
+    print 
+    if os.environ.get('SAGE_FAT_BINARY', 'no') == 'yes':
+        print 'Building ATLAS with SAGE_FAT_BINARY (generic archdefs).'
+    elif os.environ.has_key('SAGE_ATLAS_ARCH'):
+        print 'Building ATLAS with SAGE_ATLAS_ARCH =', os.environ['SAGE_ATLAS_ARCH']
+    else:
+        print 'Building ATLAS without specifying architecture.'
+    print 'This may take many hours during which you should leave the computer otherwise'
+    print 'idle to obtain accurate timings.'
+    if args.archdef:
+        os.environ['SAGE_ATLAS_SAVE_ARCHDEF'] = os.getcwd()
+        print 'The resulting <archdef>.tar.bz2 will be saved in '+os.getcwd()
+    print
+    print 'You have 5 seconds to interrupt...'
+    time.sleep(1)
+    print 'You have 4 seconds to interrupt...'
+    time.sleep(1)
+    print 'You have 3 seconds to interrupt...'
+    time.sleep(1)
+    print 'You have 2 seconds to interrupt...'
+    time.sleep(1)
+    print 'You have 1 second to interrupt...'
+    time.sleep(1)
+
+    sys.stdout.flush()
+    sys.stderr.flush()
+    os.system('sage -f atlas')
+
+    
+    
+    
+        
+        
+        

new file patches/long_filenames.patch

@@ +1 @@
+--- old/bin/extract.c   2012-07-10 20:22:50.000000000 +0100
++++ src/bin/extract.c   2012-09-20 17:28:33.989175811 +0100
+@@ -21,10 +21,10 @@
+ #include <assert.h>
+ #include <stdarg.h>
+ 
+-#define NAMLEN 128
++#define NAMLEN 1024
+ #define LNLEN  2048
+-#define HANLEN 50
+-#define SUBLEN 128
++#define HANLEN 1024
++#define SUBLEN 1024
+ 
+ #define F_nFlags     16
+ #define F_Case        0

patches/probe_comp.c.patch

@@ -1 @@
---- probe_comp.c.orig   2011-06-15 10:20:18.336648039 -0700
-+++ probe_comp.c        2011-06-15 10:22:01.323647990 -0700
-@@ -532,6 +532,8 @@
+--- src/CONFIG/src/probe_comp.c 2012-06-23 17:27:27.000000000 +0100
++++ new/CONFIG/src/probe_comp.c 2012-06-24 03:20:46.118154762 +0100
+@@ -607,6 +607,8 @@
        return(sp);
     else if (OS == OSAIX)
        sp = (ptrbits == 64) ? "-maix64" : "-maix32";

new file patches/shell.patch

@@ +1 @@
+diff -dNru src.orig/makes/Make.Flp src/makes/Make.Flp
+--- src.orig/makes/Make.Flp     2013-01-08 19:15:41.000000000 +0100
++++ src/makes/Make.Flp  2013-03-27 18:01:17.224597923 +0100
+@@ -756,7 +756,7 @@
+ 
+ $(FLAdir)/lapack_$(ARCH).a : $(FLAdir)/make.inc
+        cd $(FLAdir) ; $(PMAKE) lib
+-       if [ -e $(FLAdir)/liblapack.a ]; then \
++       if [ -f $(FLAdir)/liblapack.a ]; then \
+            ln -s $(FLAdir)/liblapack.a $(FLAdir)/lapack_$(ARCH).a ; \
+         fi
+ 
+diff -dNru src.orig/makes/Make.ttune src/makes/Make.ttune
+--- src.orig/makes/Make.ttune   2013-01-08 19:15:42.000000000 +0100
++++ src/makes/Make.ttune        2013-03-27 18:01:17.020597929 +0100
+@@ -26,7 +26,7 @@
+        - rm -f $(sumf)
+        echo "#ifndef ATLAS_SUMM_H" > $(sumf)
+        echo "   #define ATLAS_SUMM_H" >> $(sumf)
+-       if [ -e res/aff.h ]; then \
++       if [ -f res/aff.h ]; then \
+            cat res/aff.h >> $(sumf) ; \
+         else \
+            echo "#define ATL_TAFFINITY 0" >> $(sumf) ; \

spkg-install

@@ -9 +9 @@
 ### Import stuff
 ######################################################################
 
-import os, sys, shutil, time
-from configuration import conf, cp, try_run, edit_in_place
+import os, sys, shutil, time, glob
+from configuration import conf, cp, ln, which, try_run, edit_in_place
 
 
 ######################################################################
@@ -18 +18 @@
 ### update ATLAS or LAPACK to a newer version
 ######################################################################
 
+PATCH_DIR = os.path.join(conf['SPKG_DIR'], 'patches')
+
+# the current lapack source tarball
+LAPACK_TARFILE = os.path.join(conf['SPKG_DIR'], 'lapack-3.4.2.tar')
+
 # temporary directory to build everything in
 BUILD_DIR = os.path.join(conf['SPKG_DIR'], 'ATLAS-build')
+BUILD_LIB_DIR = os.path.join(conf['SPKG_DIR'], 'ATLAS-lib')
 
 # constants from src/CONFIG/include/atlconf.h
 # Note: must be lists, not tuples, for Python-2.4 support
-ATLAS_OSTYPE = ['UNKNOWN', 'Linux', 'SunOS', 'SunOS4', 'OSF1', 
-                'IRIX', 'AIX', 'Win9x', 'WinNT', 'HPUX', 'FreeBSD', 'OSX']
+ATLAS_OSTYPE = (  # static char *osnam
+    'UNKNOWN', 'Linux', 'SunOS', 'SunOS4', 'OSF1', 
+    'IRIX', 'AIX', 'Win9x', 'WinNT', 'Win64', 
+    'HPUX', 'FreeBSD', 'OSX')
+ATLAS_MACHTYPE = (  # static char *machnam
+    'UNKNOWN', 'POWER3', 'POWER4', 'POWER5', 'PPCG4', 'PPCG5',
+    'POWER6', 'POWER7', 'IBMz9', 'IBMz10', 'IBMz196',
+    'x86x87', 'x86SSE1', 'x86SSE2', 'x86SSE3',
+    'P5', 'P5MMX', 'PPRO', 'PII', 'PIII', 'PM', 'CoreSolo',
+    'CoreDuo', 'Core2Solo', 'Core2', 'Corei1', 'Corei2', 'Atom', 'P4', 'P4E',
+    'Efficeon', 'K7', 'HAMMER', 'AMD64K10h', 'AMDDOZER', 'UNKNOWNx86',
+    'IA64Itan', 'IA64Itan2',
+    'USI', 'USII', 'USIII', 'USIV', 'UST2', 'UnknownUS',
+    'MIPSR1xK', 'MIPSICE9', 'ARMv7')
 
-ATLAS_MACHTYPE = ['UNKNOWN', 'POWER3', 'POWER4', 'POWER5', 'PPCG4', 'PPCG5',
-                  'P5', 'P5MMX', 'PPRO', 'PII', 'PIII', 'PM', 'CoreSolo',
-                  'CoreDuo', 'Core2Solo', 'Core2', 'Corei7', 'P4', 'P4E', 'Efficeon', 'K7',
-                  'HAMMER', 'AMD64K10h', 'UNKNOWNx86', 'IA64Itan', 'IA64Itan2',
-                  'USI', 'USII', 'USIII', 'USIV', 'UnknownUS', 'MIPSR1xK', 'MIPSICE9']
-
-ATLAS_ISAEXT = ['', 'AltiVec', 'SSE3', 'SSE2', 'SSE1', '3DNow']
+ATLAS_ISAEXT = (  # static char *ISAXNAM
+    'None', 'VSX', 'AltiVec', 'AVXMAC', 'AVXFMA4', 'AVX', 'SSE3', 'SSE2', 'SSE1',
+    '3DNow', 'NEON' )
 
 # we need to disable parallel builds
 os.environ['MAKE'] += ' -j1'
+MAKE = os.environ['MAKE']
 
+######################################################################
+### Some auxiliary functions to facilitate IO and error checking
+######################################################################
+
+# Run shell command "command", but flush stdout and stderr before doing
+# this. Also echo commands which are executed.
+def system_with_flush(command):
+    print 'Running', command
+    sys.stdout.flush()
+    sys.stderr.flush()
+    import subprocess
+    return subprocess.call(command, shell=True)
+
+def assert_success(rc, good=None, bad=None):
+    if rc == 0:  
+        if good is not None:
+            print good
+        return
+    print '-'*60
+    import traceback
+    traceback.print_stack(file=sys.stdout)
+    print '-'*60
+    if bad is not None:
+        print 'Error: ', bad
+    sys.exit(rc)
 
 ######################################################################
 ### Skip building ATLAS on specific systems
 ######################################################################
 
 # On Cygwin we simply require that the system-wide lapack is installed.
-# This includes BLAS and is enough to build the rest of Sage.
-if conf['CYGWIN?']:
-    lib = '/usr/lib/libblas.dll.a'
-    if not os.path.exists(lib):
-        print '*'*75
-        print 'On Cygwin you must install the standard LAPACK Cygwin package'
-        print 'via the Cygwin setup.exe program in the "Math" category.'
-        print '*'*75 
-        sys.exit(1)
-    cp(lib, os.path.join(conf['SAGE_LOCAL'], 'lib'))
+# This includes BLAS and LAPACK and is enough to build the rest of Sage.
+if conf['CYGWIN?'] and not os.environ.has_key('SAGE_ATLAS_ARCH'):
+    libraries = ['/usr/lib/libblas.dll.a', '/usr/lib/liblapack.dll.a']
+    for lib in libraries:
+        if not os.path.exists(lib):
+            print '*'*75
+            print 'Could not locate required file: "' + lib + '".'
+            print 'On Cygwin you must install the following standard LAPACK Cygwin packages'
+            print 'via the Cygwin setup.exe program in the "Math" category:'
+            print '* lapack,'
+            print '* liblapack-devel.'
+            print 'Alternatively you can try building your own ATLAS by setting'
+            print 'SAGE_ATLAS_ARCH to something sensible although that is not'
+            print 'officially supported.'
+            print '*'*75
+            sys.exit(1)
+    for lib in libraries:
+        cp(lib, os.path.join(conf['SAGE_LOCAL'], 'lib'))
     sys.exit(0)
 
 
-
-if conf['Darwin?']:
-    print 'Skipping build of ATLAS on OSX'
+if conf['Darwin?'] and not os.environ.has_key('SAGE_ATLAS_ARCH'):
+    print 'Skipping build of ATLAS on OS X, using system library instead.'
+    print 'You can try building your own ATLAS by setting SAGE_ATLAS_ARCH'
+    print 'to something sensible although that is not officially supported.'
+    if conf['PPC?']:   # OSX 10.4 PPC linker needs help to find the accelerate blas
+        veclib_dir = '/System/Library/Frameworks/Accelerate.framework/' + \
+            'Versions/A/Frameworks/vecLib.framework/Versions/A'
+        for lib in [ 'libBLAS.dylib', 'libLAPACK.dylib']:
+            ln(os.path.join(veclib_dir, lib),
+               os.path.join(conf['SAGE_LOCAL'], 'lib', lib))
     sys.exit(0)
 
 
@@ -68 +123 @@
 
 if os.environ.has_key('SAGE_ATLAS_LIB'):
     ATLAS_LIB = os.environ['SAGE_ATLAS_LIB']
-    library_extensions = ['.a', '.so', '.dylib']
     libraries = ['libatlas', 'liblapack', 'libcblas', 'libf77blas']
     libraries_optional = ['libptcblas', 'libptf77blas']
 
@@ -77 +131 @@
             return False
         if not os.path.isdir(path):
             return False
-        def library_exists(lib):
-            library_base = os.path.join(path, lib)
-            return any(os.path.exists(library_base+ext) for ext in library_extensions)
-        return all(library_exists(lib) for lib in libraries)
+        filenames = os.listdir(path)
+        for lib in libraries:
+            if not any(fname.startswith(lib) for fname in filenames):
+                print 'Cannot find '+lib+'.* in '+path
+                return False
+        return True
 
-    paths = [ os.path.join(ATLAS_LIB, 'lib'), 
-              ATLAS_LIB, 
-              None ]   # None indicates error
+    paths = [ ATLAS_LIB, os.path.join(ATLAS_LIB, 'lib64'), os.path.join(ATLAS_LIB, 'lib') ]
+    ATLAS_LIB = None
     for path in paths:
         if is_atlas_lib_path(path):
             ATLAS_LIB = path
             break
 
-    if path is None:
+    if ATLAS_LIB is None:
         print 'Unable to find one of liblapack, libcblas, libatlas or libf77blas'
         print 'in the directory', ATLAS_LIB
         print 'Set SAGE_ATLAS_LIB to the directory containing liblapack, libcblas,'
-        print 'libatlas and libf77blas (either .a or .so extensions) if you wish to'
-        print 'use existing ATLAS libraries. For more details, see'
+        print 'libatlas and libf77blas (either .a, .so, .dylib extensions) if'
+        print 'you wish to use existing ATLAS libraries. For more details, see'
         print 'http://sagemath.org/doc/installation/source.html#environment-variables'
         print 'Unset SAGE_ATLAS_LIB to build ATLAS from source.'
         print 'Then type make.'
@@ -120 +175 @@
 
     SAGE_LOCAL_LIB = os.path.join(conf['SAGE_LOCAL'], 'lib')
     def symlinkOSlibrary(library_basename):
-        for ext in library_extensions:
-            lib = library_basename + ext
-            source = os.path.join(ATLAS_LIB, lib)
-            destination = os.path.join(SAGE_LOCAL_LIB, lib)
-            if not os.path.exists(source):
-                continue
-            # ln -sf source destination
+        filenames = [ fname for fname in os.listdir(path) 
+                      if fname.startswith(library_basename) ]
+        for fname in filenames:
+            source = os.path.join(ATLAS_LIB, fname)
+            destination = os.path.join(SAGE_LOCAL_LIB, fname)
             print 'Symlinking '+source+' -> '+destination
             try:
                 os.remove(destination)   
@@ -145 +198 @@
 ### Patch source
 ######################################################################
 
-# Apply a TEMPORARY fix to allow ATLAS to build with
-# gcc 4.4.0 on Solaris. Implemented 16th June 2009, by David Kirkby.
-# One would expect to remove this within a couple of months,
-# once the underlying issue in ATLAS is resolved. The patch
-# forces GuessSmallNB() in src/tune/blas/gemm/mmsearch.c
-# to return 28 as suggested by Clint Whaley.
-# Changed on July 19th 2009 (see trac 6558) to be more selective and only
-# apply the fix on sun4v machines which are based on the Sun T1, T2 and T2+
-# processors (codenamed Niagra). This is because there are no known problems
-# on other Sun architectures such as sun4u, or any x86 based Solaris system. 
-if conf['Solaris?'] and conf['machine'] == 'sun4v':
-    cp('patches/mmsearch-with-temp-Solaris-fix.c',
-       'src/tune/blas/gemm/mmsearch.c')
+# apply all patches
+for fname in glob.glob(os.path.join(PATCH_DIR,'*.patch')):
+    rc = system_with_flush('patch -p0 --input '+os.path.join(PATCH_DIR, fname)) 
+    assert_success(rc, bad='Applying '+fname+' failed.', good='Applied '+fname+'.')
 
-# add dynamic libs make install targets to Make.top
-cp('patches/Make.top',
-   'src')
+# patches/Makefile
+# fix for Xcode creating directories like xspew.dSYM
 
+# patches/archinfo_linux.c
 # add PPC4 7447 CPU and better Itanium2 detection:
-cp('patches/archinfo_linux.c',
-   'src/CONFIG/src/backend/archinfo_linux.c')
 
-# add Core2Duo and Dunnington CPUids
-cp('patches/archinfo_x86.c',
-   'src/CONFIG/src/backend/archinfo_x86.c')
+# patches/archinfo_x86.c
+# add Core2Duo and Dunnington CPUid
 
+# patches/probe_comp.c
 # work around "-m64" cflag issue on Itanium
-cp('patches/probe_comp.c', 
-   'src/CONFIG/src/probe_comp.c')
 
-# add K7, Pentium M and non-AltiVec G4 profiles
-cp('patches/*tgz', 
-   'src/CONFIG/ARCHS')
+# add extra architectural defaults
+# currently IA64Itan264 and x86SSE264SSE2
+cp('patches/ARCHS/*.tar.bz2', 'src/CONFIG/ARCHS')
 
 # hardcoded gcc in SpewMakeInc.c
 edit_in_place('src/CONFIG/src/SpewMakeInc.c') \
@@ -185 +225 @@
     .close()
 
 
+# override throttling check if architecture is specified
+edit_in_place('src/CONFIG/src/config.c') \
+    .replace('if \(mach == MACHOther\)', 'if (mach != MACHOther) thrchk=0; else') \
+    .close()
+
+shutil.copytree(os.path.join(PATCH_DIR, 'ATLAS-lib'), BUILD_LIB_DIR)
+
 ######################################################################
 ### configure
 ######################################################################
@@ -194 +241 @@
     # ATLAS_ISAEXT) from environment variables:
     arch = None
     isa_ext = None
-    if os.environ.get('SAGE_FAT_BINARY', 'no') == 'yes' and conf['Intel?']:
-        print 'Sage "fat" binary mode set: Building SSE2 only Hammer binary'
+    archdef_dir = None
+    if conf['generic_binary?']:
+        print 'Sage "fat" binary mode set: Building "base" binary'
         print 'NOTE: This can result in a Sage that is significantly slower at certain numerical'
         print 'linear algebra since full FAT binary support has not been implemented yet.'
-        arch = 'HAMMER'
-        isa_ext = ('SSE2', 'SSE1')
+        arch, isa_ext, archdef_dir = configure_base()
     elif os.environ.has_key('SAGE_ATLAS_ARCH'):
         opts = os.environ['SAGE_ATLAS_ARCH'].split(',')
         arch = opts[0]
+        if arch not in ATLAS_MACHTYPE + ('fast', 'base'):
+            print 'SAGE_ATLAS_ARCH architecture must be one of '+ ', '.join(ATLAS_MACHTYPE)
+            sys.exit(1)
         if len(opts)>0:
             isa_ext = opts[1:]
-    return (arch, isa_ext)
+        if not all(ext in ATLAS_ISAEXT for ext in isa_ext):
+            print 'SAGE_ATLAS_ARCH ISA extension must be one of '+ ', '.join(ATLAS_ISAEXT)
+            sys.exit(1)
+    return (arch, isa_ext, archdef_dir)
 
 
-def configure(arch=None, isa_ext=None):
+def configure(arch=None, isa_ext=None, archdef_dir=None):
     """
     Configure for ``arch``.
 
@@ -219 +272 @@
     - ``isa_ext`` -- ``None`` or a sublist of ``ATLAS_ISAEXT``
     """
     if arch is None:
-        arch, isa_ext = configure_options_from_environment()
+        arch, isa_ext, archdef_dir = configure_options_from_environment()
     if arch=='fast':
-        return configure_fast()
-    if arch=='base':
-        return configure_base()
+        arch, isa_ext, archdef_dir = configure_fast()
+    if arch=='generic' or arch=='base':
+        arch, isa_ext, archdef_dir = configure_base()
 
-    print 'Running configure with arch =', arch, 'and isa extensions', isa_ext
+    print 'Running configure with arch = '+str(arch)+ \
+        ', isa extensions '+str(isa_ext)+ \
+        ', archdef dir '+str(archdef_dir)
     os.chdir(conf['SPKG_DIR'])
     if os.path.isdir(BUILD_DIR):
         shutil.rmtree(BUILD_DIR)
     os.mkdir(BUILD_DIR)
     os.chdir(BUILD_DIR)
 
+    # We need to provide full pathes to FC and CC to ATLAS configure script,
+    # so that it does not use 'find' and travel around the filesystem to find
+    # them.
+    # We first split the compiler executable names from potential options, e.g.
+    # as in 'gcc -m64', then use 'which' to locate them, and finally add the
+    # options back.
+    FC = os.environ['FC']
+    FCsplit = FC.find(' ')
+    if  FCsplit != -1:
+        FCbin, FCopt = FC[:FCsplit], FC[FCsplit:]
+    else:
+        FCbin, FCopt = FC, ''
+
+    CC = os.environ['CC']
+    CCsplit = CC.find(' ')
+    if  CCsplit != -1:
+        CCbin, CCopt = CC[:CCsplit], CC[CCsplit:]
+    else:
+        CCbin, CCopt = CC, ''
+
     cmd = '../src/configure'
     cmd += ' --prefix=' + conf['SAGE_LOCAL']
-    cmd += ' --with-netlib-lapack=' + os.path.join(conf['SAGE_LOCAL'],'lib','liblapack.a')
-    cmd += ' --cc="' + os.environ['CC'] + '"'
+    cmd += ' --with-netlib-lapack-tarfile=' + LAPACK_TARFILE 
+    cmd += ' --cc="' + CC + '"'
 
-    # -Si cputhrchk 0: Ignore/heed CPU throttle probe
-    cmd += ' -Si cputhrchk 0'
+    ## -Si latune 1: enable lapack tuning 
+    ## typically adds 3-4 hours of install time
+    cmd += ' -Si latune 0'
 
     # -Fa alg -fPIC: set flags so we can build dynamic libraries
     cmd += ' -Fa alg -fPIC'
@@ -247 +323 @@
     #cmd += ' -t 0'
 
     # set fortran compiler
-    cmd += ' -C if sage_fortran'
+    cmd += ' -C if "' + which(FCbin) + FCopt + '"'
 
     # set C compiler
-    cmd += ' -C xc "' + os.environ['CC'] + '"'
-    cmd += ' -C ic "' + os.environ['CC'] + '"'
-    cmd += ' -C dm "' + os.environ['CC'] + '"'
-    cmd += ' -C sm "' + os.environ['CC'] + '"'
-    cmd += ' -C dk "' + os.environ['CC'] + '"'
-    cmd += ' -C sk "' + os.environ['CC'] + '"'
+    cmd += ' -C acg "' + which(CCbin) + CCopt + '"'
 
     # set bit width
     cmd += ' -b ' + conf['bits'][0:2]
 
     # set OS type
-    system = None
     try:
-       system = ATLAS_OSTYPE.index(conf['system'])
-    except KeyError:
-       if conf['Darwin?']: system = ATLAS_OSTYPE.index('OSX')
-       if conf['CYGWIN?']: system = ATLAS_OSTYPE.index('WinNT')
-    if not system is None:
-       cmd += ' -O '+str(system)
+        if conf['Darwin?']:
+            atlas_osnam = 'OSX'
+        elif conf['CYGWIN?']:
+            atlas_osnam = 'WinNT'
+        else:
+            atlas_osnam = conf['system']
+        atlas_system = ATLAS_OSTYPE.index(atlas_osnam)
+        cmd += ' -O '+str(atlas_system)
+    except ValueError:
+        pass
 
     # set machine architecture
-    if not arch is None:
-       cmd += ' -A '+str(ATLAS_MACHTYPE.index(arch))
+    if arch is not None:
+        cmd += ' -A '+str(ATLAS_MACHTYPE.index(arch))
 
     # set cpu instruction set extensions
-    if not isa_ext is None:
+    if isa_ext is not None:
         isa_extension = sum(1 << ATLAS_ISAEXT.index(x) for x in isa_ext)
         cmd += ' -V '+str(isa_extension)
 
-    print 'Running', cmd
-    rc = os.system(cmd)
+    # use a predefined archdef file in the archdef_dir directory
+    if archdef_dir is not None:
+        cmd += ' -Ss ADdir '+str(archdef_dir)
+
+    rc = system_with_flush(cmd)
     return rc
 
 
 def configure_fast():
-   if conf['Intel?']:
-      print 'Fast configuration on Intel compatible CPUs.'
-      arch = 'HAMMER'
-      isa_ext = ('SSE3', 'SSE2', 'SSE1')
-   elif conf['SPARC?']:
-      print 'Fast configuration on SPARC.'
-      arch = 'USIV'
-      isa_ext = ()
-   elif conf['PPC?']:
-      print 'Fast configuration on PPC.'
-      arch = 'POWER5'
-      isa_ext = ('AltiVec', )
-   elif conf['IA64?']:
-      print 'Fast configuration on Itanium.'
-      arch = 'IA64Itan2'
-      isa_ext = ()
-   else:
-      raise NotImplementedError, 'I don\'t know a "fast" configuration for your cpu.'
-   return configure(arch, isa_ext)
+    if conf['Intel?'] and conf['64bit?']:
+        print 'Fast configuration on Intel x86_64 compatible CPUs.'
+        arch = 'P4E'
+        isa_ext = ('SSE3', 'SSE2', 'SSE1')
+        archdef_dir = None
+    elif conf['Intel?'] and conf['32bit?']:
+        print 'Fast configuration on Intel i386 compatible CPUs.'
+        arch = 'x86SSE3'
+        isa_ext = ('SSE3', 'SSE2', 'SSE1')
+        archdef_dir = None
+    elif conf['SPARC?']:
+        print 'Fast configuration on SPARC.'
+        arch = 'USIV'
+        isa_ext = ()
+        archdef_dir = None
+    elif conf['PPC?']:
+        print 'Fast configuration on PPC.'
+        arch = 'PPCG5'
+        isa_ext = ('AltiVec', )
+        archdef_dir = None
+    elif conf['IA64?']:
+        print 'Fast configuration on Itanium.'
+        arch = 'IA64Itan2'
+        isa_ext = ()
+        archdef_dir = None
+    else:
+        raise NotImplementedError, 'I don\'t know a "fast" configuration for your cpu.'
+    return (arch, isa_ext, archdef_dir)
 
 
 def configure_base():
-   if conf['Intel?']:
-      print 'Base configuration on Intel compatible CPUs.'
-      arch = 'P4'
-      isa_ext = ('SSE2', 'SSE1')
-   elif conf['SPARC?']:
-      print 'Base configuration on SPARC.'
-      arch = 'USIII'
-      isa_ext = ()
-   elif conf['PPC?']:
-      print 'Base configuration on PPC.'
-      arch = 'POWER4'
-      isa_ext = ()
-   elif conf['IA64?']:
-      print 'Base configuration on Itanium.'
-      arch = 'IA64Itan'
-      isa_ext = ()
-   else:
-      raise NotImplementedError, 'I don\'t know a "base" configuration for your cpu.'
-   return configure(arch, isa_ext)
+    if conf['Intel?'] and conf['64bit?']:
+        print 'Generic configuration on Intel x86_64 compatible CPUs.'
+        arch = 'x86SSE2'
+        isa_ext = ('SSE2', 'SSE1')
+        archdef_dir = PATCH_DIR
+    elif conf['Intel?'] and conf['32bit?']:
+        print 'Generic configuration on Intel i386 compatible CPUs.'
+        arch = 'x86x87'
+        isa_ext = ('3DNow',)
+        archdef_dir = None
+    elif conf['SPARC?']:
+        print 'Base configuration on SPARC.'
+        arch = 'USIII'
+        isa_ext = ()
+        archdef_dir = None
+    elif conf['PPC?']:
+        print 'Base configuration on PPC.'
+        arch = 'PPCG4'
+        isa_ext = ()
+        archdef_dir = None
+    elif conf['IA64?']:
+        print 'Base configuration on Itanium.'
+        arch = 'IA64Itan'
+        isa_ext = ()
+        archdef_dir = None
+    else:
+        raise NotImplementedError, 'I don\'t know a "base" configuration for your cpu.'
+    return (arch, isa_ext, archdef_dir)
 
 
 
@@ -334 +429 @@
 ### make
 ######################################################################
 
-def make_core():
+def make_atlas():
     os.chdir(BUILD_DIR)
-    rc = os.system('$MAKE')
+    rc = system_with_flush(MAKE)
     return rc
 
-def make_atlas_library():
-    os.chdir(os.path.join(BUILD_DIR, 'lib'))
 
-    # don't put the path into the soname or we can't relocate it
-    edit_in_place('Makefile') \
-        .replace('-soname \$\(LIBINSTdir\)/lib', '-soname lib') \
-        .close()
+def configure_atlas_library():
+    os.chdir(BUILD_LIB_DIR)
+    static_library_dir = os.path.join(BUILD_DIR, 'lib')
+    for static_lib in glob.glob(os.path.join(static_library_dir, 'lib*.a')):
+        shutil.copy(static_lib, BUILD_LIB_DIR)
+    cmd = './configure'
+    cmd += ' --prefix=' + conf['SAGE_LOCAL']
+    cmd += ' --libdir=' + os.path.join(conf['SAGE_LOCAL'],'lib')
+    cmd += ' --disable-static'
+    return system_with_flush(cmd)
+    
 
-    if conf['linker_Solaris?']:
-        print "The Makefile generated in ATLAS for building shared libraries"
-        print "assumes the linker is the GNU linker, which it not true in"
-        print "your setup. (It is generally considered better to use the"
-        print "Sun linker in /usr/ccs/bin rather than the GNU linker from binutils)"
-        print "The linker flags in `pwd`/Makefile will be changed. "
-        print "'-shared' will be changed to '-G'"
-        print "'-soname' will be changed to '-h'"
-        print "'--whole-archive' will be changed to '-zallextract'"
-        print "'--no-whole-archive' will be changed to '-zdefaultextract'"
-        edit_in_place('Makefile') \
-            .replace('-shared', '-G') \
-            .replace('-soname', '-h') \
-            .replace('--whole-archive', '-z allextract') \
-            .replace('--no-whole-archive', '-z defaultextract') \
-            .close()
-        if conf['Intel?']:
-            print "Change ldflag -melf_x86_64 to -64 as needed for Sun ld"
-            print "on 64-bit builds of ATLAS on x64 hardware"
-            edit_in_place('Make.inc').replace('-melf_x86_64', '-64').close()
-            print "Remove the linker flag -melf_i386 as needed for Sun ld"
-            print "on 32-bit builds of ATLAS on x86/x64 hardware"
-            edit_in_place('Make.inc').replace('-melf_i386', '').close()
-        
-    rc = os.system('$MAKE shared cshared')
-    return rc
+def make_atlas_library(target=None):
+    os.chdir(BUILD_LIB_DIR)
+    if target is None:
+        return system_with_flush(MAKE)
+    else:
+        return system_with_flush(MAKE + ' ' + target)
 
 
-def make_lapack_library():
-    os.chdir(os.path.join(BUILD_DIR, 'lib'))
-    sage_local_lib_dir = ' -L' + os.path.join(conf['SAGE_LOCAL'],'lib')
+######################################################################
+### make and save archdef
+######################################################################
 
-    libraries = []
-    if conf['Linux?'] or conf['FreeBSD?']:
-        if conf['fortran_g95?']:
-            fortran_dir = '-L'+conf['fortran_g95_dir']
-            fortran_lib = '-lf95'
-        else:
-            fortran_dir = ''
-            fortran_lib = '-lgfortran'
-        libraries = ['liblapack', 'libf77blas']
-        # call gcc instead of ld in order to find libgfortran.so
-        cmd = os.environ['CC'] + ' ' + fortran_dir + sage_local_lib_dir + \
-            ' -shared -Wl,-soname,{0}.so ' + \
-            '  -Wl,-whole-archive {0}.a -Wl,-no-whole-archive' + \
-            ' -o {0}.so -lc -lm ' + fortran_lib
-
-    if conf['Solaris?']:
-        if conf['64bit?']:
-            linker_flag_64 = '-64'
-        else:
-            linker_flag_64 = ''
-        libraries = ['libatlas', 'libf77blas', 'libcblas']
-        cmd = '/usr/ccs/bin/ld ' + linker_flag_64 + sage_local_lib_dir + \
-            ' -G -h {0}.so -o {0}.so  -zallextract {0}.a -zdefaultextract -lc -lm -lgfortran'
-        
-    rc = 0
-    for LIB in libraries:
-        cmd_LIB = cmd.format(LIB)
-        print 'Running', cmd_LIB
-        rc_LIB = os.system(cmd_LIB)
-        if rc_LIB!=0:
-            print 'Failed to build ATLAS library '+LIB+'.so'
-        rc = max(rc, abs(rc_LIB))
-        
-    # liblapack.so causes problems with R on Solaris. 
-    if conf['Solaris?']:
-        try:
-            os.remove('liblapack.so')
-        except OSError:
-            pass
-
-    return rc
-
+def build_and_save_archdef():
+    try:
+        ARCHDEF_SAVE_DIR = os.environ['SAGE_ATLAS_SAVE_ARCHDEF']
+    except KeyError:
+        return
+    os.chdir(os.path.join(BUILD_DIR, 'ARCHS'))
+    rc = system_with_flush(MAKE + ' ArchNew')
+    assert_success(rc, bad='Making archdef failed.', good='Finished building archdef.')
+    rc = system_with_flush(MAKE + ' tarfile')
+    assert_success(rc, bad='Making archdef tarfile failed.', good='Finished building archdef tarfile.')
+    for tarfile in glob.glob('*.tar.bz2'):
+        cp(tarfile, ARCHDEF_SAVE_DIR)
+    
 
 ######################################################################
 ### build atlas and lapack
 ######################################################################
 
-def build(arch=None, isa_ext=None):
-    rc = configure(arch, isa_ext)    
-    assert rc==0, 'Configure failed!'
-    print 'Finished configuring ATLAS.'
-    rc = make_core()
+def build(arch=None, isa_ext=None, archdef_dir=None):
+    """
+    Configure/build with given architectural information.
+
+    Return ``0`` if successfull.
+    """
+    rc = configure(arch, isa_ext, archdef_dir)    
+    assert_success(rc, bad='Configure failed.', good='Finished configuring ATLAS.')
+    return make_atlas()
+
+def build_tuning():
+    """
+    Configure/build by going through the full tuning process.
+
+    Return ``0`` if successfull.
+    """
+    rc = configure()
     if rc!=0:
-        print 'ATLAS failed to build, possibly because of a loaded system.'
+        print 'Configure failed, possibly because you have CPU throttling enabled.'
+        print 'Skipping tuning attempts.'
+        return rc
+    print 'First attempt: automatic tuning.'
+    rc = make_atlas()
+    if rc==0:
+        return rc
+    print 'ATLAS failed to build, possibly because of throttling or a loaded system.'
+    print 'Waiting 1 minute...'
+    sys.stdout.flush()
+    time.sleep(60)
+    print 'Second attempt: Re-running make.'
+    return make_atlas()
+
+
+#
+# Workaround for specific platforms: Disable tuning and go straight to
+# fast/base architectural defaults
+#
+skip_tuning = conf['IA64?']    # Itanium is dead and tuning is broken
+
+
+rc = None
+if os.environ.has_key('SAGE_ATLAS_ARCH') or conf['generic_binary?']:
+    print 'Building using specific architecture.'
+    rc = build()
+else:
+    print 'Configuring ATLAS.'
+    if skip_tuning:
+        print 'Skipping tuning attempts (skip_tuning = True).'
+        rc = 1    # Fake failed tuning attempts
     else:
-        print 'Finished building ATLAS core.'
-    return rc
+        rc = build_tuning()
+    if rc!=0:
+        print 'Third attempt: use "fast" options.'
+        rc = build(arch='fast')
+    if rc!=0:
+        print 'Fourth attempt: use "base" options.'
+        rc = build(arch='base')
 
+assert_success(rc, bad='Failed to build ATLAS.', good='Finished building ATLAS core.')
 
-print 'First attempt: automatic tuning.'
-rc = build()
-if rc!=0:
-    print 'Waiting 5 minutes...'
-    time.sleep(5*60)
-    print 'Second attempt: Re-running make.'
-    rc = make_core()
-    if rc!=0:
-        print 'Waiting 5 minutes...'
-        time.sleep(5*60)
-        try:
-            print 'Third attempt: use "fast" options.'
-            rc = build(arch='fast')
-            if rc!=0:
-                print 'Fourth attempt: use "base" options.'
-                rc = build(arch='base')
-        except NotImplementedError:
-            pass
+build_and_save_archdef()
 
-assert rc==0, 'Failed to build ATLAS.'
-print 'Finished building ATLAS core.'
+rc = configure_atlas_library()
+assert_success(rc, bad='Configuring shared ATLAS library failed.',
+               good='Finished configuring shared ATLAS library.')
 
 rc = make_atlas_library()
-assert rc==0, 'Building shared ATLAS library failed.'
-print 'Finished building shared ATLAS library.'
-
-rc = make_lapack_library()
-assert rc==0, 'Building LAPACK+ATLAS library failed.'
-print 'Finished building LAPACK+ATLAS library.'
+if rc!=0:
+    print 'Failed to build serial+parallel shared libraries, possibly because your'
+    print 'system does not support both. Trying to build serial libraries only.'
+    rc = make_atlas_library('all_serial')
+    have_parallel_libs = False
+else:
+    have_parallel_libs = True
+assert_success(rc, bad='Building shared ATLAS library failed.',
+               good='Finished building shared ATLAS library.')
 
 
 ######################################################################
 ### install
 ######################################################################
 
-def install():
-    library_extensions = ['.a', '.so', '.dylib']
-    libraries = ['libatlas', 'liblapack', 'libcblas', 'libf77blas', 'libptcblas', 'libptf77blas']
-    for lib in libraries:
-        for ext in library_extensions:
-            try:
-                filename = os.path.join(conf['SAGE_LOCAL'], 'lib', lib+ext)
-                os.remove(filename)
-                print('Removed file from previous installation: '+filename)
-            except OSError:
-                pass
-    os.chdir(BUILD_DIR)
-    rc = os.system('$MAKE install')
-    os.chdir('..')
-    return rc
+if have_parallel_libs:
+    rc = make_atlas_library('install')
+else:
+    rc = make_atlas_library('install_serial')
+assert_success(rc, bad='Installing the shared ATLAS library failed.',
+               good='Finished installing shared ATLAS library.')
 
-rc = install()
-assert rc==0, 'Make install for ATLAS failed.'
-# will automatically print "Finished installing atlas-<version>.spkg"
 
+cp(os.path.join(PATCH_DIR, 'atlas-config'), 
+   os.path.join(conf['SAGE_LOCAL'], 'bin'))