Opened 6 years ago

Last modified 5 years ago

#15433 closed task

Port to OSX 10.9 — at Version 31

Reported by: ohanar Owned by:
Priority: blocker Milestone: sage-5.13
Component: porting Keywords:
Cc: yomcat, jhpalmieri Merged in:
Authors: R. Andrew Ohana Reviewers:
Report Upstream: N/A Work issues:
Branch: u/ohanar/10.9-port Commit: 7547fceeb85afc0b6b932cb2487e61fb94fef405
Dependencies: Stopgaps:

Description (last modified by ohanar)

Sage fails to build on OSX 10.9.

Issues:

  1. scipy doesn't play nicely with the new accelerate framework headers
  2. R depends on CoreData? which uses non-standard sqlite3 modules (and this affects us since we set *_LIBRARY_PATH and install a custom copy of sqlite3)
  3. g++ and 10.9's cdefs.h header are currently incompatible, leading to a broken libstc++
  4. accelerate framework no longer pretends to be atlas for linking purposes

To Apply:

  1. Apply trac15433.patch to the Sage Library repository
  2. Put gcc-4.7.3.p1.spkg, sqlite-3.7.17.p1.spkg, and scipy-0.12.0.p1.spkg into spkg/standard.

Change History (32)

comment:1 Changed 6 years ago by ohanar

The attached branch should fix everything except for the polybori issue. This should be checked on older OSX versions, since I've made some changes that are not 10.9 specific (I've checked these changes against bsd.math, and they work fine there).

As for the polybori issue, I'm not experiencing it anymore and I don't know what fixed it for me. I was going to setup a 10.9 VM, but unfortunately Apple refuses to let me download 10.9 on the machine I'm currently using -- however, I should be able to get a copy sometime in the few days.

I will update spkgs/add mercurial patches if we can fix this in time for 5.13, but for now it is much easier for me to work off of the git version.

comment:2 Changed 6 years ago by yomcat

  • Cc yomcat added

comment:3 Changed 6 years ago by vbraun

I'm getting the following:

$ make
cd build && \
"../build/pipestatus" \
	"env SAGE_PARALLEL_SPKG_BUILD='' ./install all 2>&1" \
	"tee -a ../logs/install.log"
./install: line 503: syntax error near unexpected token `then'
./install: line 503: `if { uname -sr | grep 'Darwin' } &>/dev/null; then'
make: *** [build] Error 2

$ uname -sr
Linux 3.11.7-200.fc19.x86_64

comment:4 Changed 6 years ago by jhpalmieri

  • Cc jhpalmieri added

comment:5 Changed 6 years ago by git

  • Commit changed from 7a0879bde6a2e557b7a4056eba398eeaef60f54f to 3ad56a494d4f8085189695546d4ff7480155608a

Branch pushed to git repo; I updated commit sha1. New commits:

3ad56a4build/install: add missing ;

comment:6 follow-up: Changed 6 years ago by jhpalmieri

Here's a stupid git question: what command do I give to check out this branch? I mean, if I'm on an OS X Mavericks system and I don't have a working copy of Sage with the git devel tools, I can't just do ./sage -dev …, but there must be some git command using the branch name or something. So what is it?

comment:7 Changed 6 years ago by jhpalmieri

My guess about polybori, by the way, is that it was fixed by the changes to gcc.

comment:8 in reply to: ↑ 6 Changed 6 years ago by ohanar

Replying to jhpalmieri:

Here's a stupid git question: what command do I give to check out this branch?

Assuming you have git installed and in your path, you can do

$ git clone git://trac.sagemath.org/sage.git -b u/ohanar/10.9-port

to make a clone of the git repository, or if you already have a clone, you can do

$ git fetch git://trac.sagemath.org/sage.git u/ohanar/10.9-port
$ git checkout -b u/ohanar/10.9-port FETCH_HEAD

from within the repository.

comment:9 follow-up: Changed 6 years ago by zabrocki

How do I resolve this problem?

sh: line 1: 59700 Trace/BPT trap: 5       /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk / -find ar 2> /dev/null
ar: error: unable to find utility "ar", not a developer tool or in PATH
make[3]: *** [libsymmetrica.a] Error 72
make[3]: Target `all' not remade because of errors.
Error building Symmetrica.

because I do see

-bash:/Applications/sage-5.13.beta1 $ /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk / -find ar
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar
-bash:/Applications/sage-5.13.beta1 $ echo $PATH
.:/usr/bin:/bin:/usr/sbin

comment:10 in reply to: ↑ 9 Changed 6 years ago by ohanar

Replying to zabrocki:

How do I resolve this problem?

Looks like more or less the same problem as polybori. That said, symmetrica was updated in 5.13.beta2, so you may want to see if you still experience the issue with a more recent beta.

comment:11 Changed 6 years ago by jhpalmieri

I don't know how the new build system works, but if you modify a file like build/pkgs/gcc/spkg-install, for example, should you also modify build/pkgs/gcc/package-version.txt so that the new version gets built? I have a version of Sage/git from early October, and when I cloned this branch and ran make, gcc was not rebuilt — it didn't even try, the log file was not modified — while scipy, for example, just returned the message Package scipy-0.12.0.p0 is already installed.

Building the whole thing from scratch worked well, though, and all tests passed.

Regarding the patch to gcc/spkg-install, for consistency's sake among other things, would it be a good idea to check the OS X version number the same way it's done in prereqs? That is, DARWIN_VERSION=`uname -r | cut '-d.' -f1`?

comment:12 Changed 6 years ago by zabrocki

The directory name is sage-5.13.beta1 is probably misleading since it is the name of the 'git' copy of sage. I did a checkout of the branch and it stops at the end of the compile of symmetrica with a make -k. If I just run make I don't get as far.

comment:13 Changed 6 years ago by ohanar

I agree that gcc should have a patch version bump, but I don't think that scipy should: scipy issue is purely a build-time issue, unlike gcc's.

For gcc/spkg-install I just tried to be consistent with what was already there.

comment:14 Changed 6 years ago by git

  • Commit changed from 3ad56a494d4f8085189695546d4ff7480155608a to 119372bfd995f1d06aaf11e774456a8abb176cfe

Branch pushed to git repo; I updated commit sha1. New commits:

119372bgcc: version bump to force rebuild on OSX 10.9

comment:15 Changed 6 years ago by ohanar

  • Authors set to R. Andrew Ohana
  • Status changed from new to needs_review

jhpalmieri: the change to gcc resolved the polybori and symmetrica issues on my second machine as well

zabrocki: try rebuilding with the latest commit

I'm marking this as needs review, although if someone could test on OSX 10.4, that would be great.

comment:16 Changed 6 years ago by yomcat

No issues at all building on 10.9 for me -- all test passed.

comment:17 Changed 6 years ago by zabrocki

Andrew, you saved me again! Brilliant! It worked for me too. Successful build and all tests pass. I had only to make all because files compiled pre-fix were messing things up (or instead, I just untar'ed another copy and compiled that version).

comment:18 Changed 6 years ago by git

  • Commit changed from 119372bfd995f1d06aaf11e774456a8abb176cfe to 9fb89a2e18a84261b3bb61a0bd9dcc6f3cf25b18

Branch pushed to git repo; I updated commit sha1. New commits:

9fb89a2build/install: add comment explaining why we don't build sqlite on OSX

comment:19 follow-up: Changed 6 years ago by jpflori

Random remark: I would have put the logic for installing or not sqlite3 in the spkg-install script of sqlite3, just as we do for cephes for example.

comment:20 in reply to: ↑ 19 Changed 6 years ago by jhpalmieri

Replying to jpflori:

Random remark: I would have put the logic for installing or not sqlite3 in the spkg-install script of sqlite3, just as we do for cephes for example.

Atlas also. That makes sense to me.

comment:21 Changed 6 years ago by git

  • Commit changed from 9fb89a2e18a84261b3bb61a0bd9dcc6f3cf25b18 to cfc31f333b25626941353545126dbc4c6dbb01f1

Branch pushed to git repo; I updated commit sha1. New commits:

cfc31f3sqlite: move platform disabling to spkg-install like atlas and cephes

comment:22 Changed 6 years ago by jhpalmieri

Now you also need to change the instructions in src/bin/sage about running sage -i sqlite3: that will have no effect on OS X. Note also that sqlite3 appears twice, once in usage and once in usage_advanced. Maybe

  • src/bin/sage

    diff --git a/src/bin/sage b/src/bin/sage
    index 7eca9b0..df51f76 100755
    a b usage() { 
    2525    echo "  -python [...]       -- run the Python interpreter"
    2626    echo "  -R [...]            -- run Sage's R with given arguments"
    2727    echo "  -singular [...]     -- run Sage's singular with given arguments"
     28    test -x "$SAGE_LOCAL/bin/sqlite3" || \
    2829    echo "  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments"
    2930    echo "  -root               -- print the Sage root directory"
    3031    echo "  --nodotsage         -- run Sage without using the user's .sage directory:"
    usage_advanced() { 
    102103    echo "  -scons [...]        -- run Sage's scons"
    103104    echo "  -sh [...]           -- run \$SHELL ($SHELL) with Sage environment variables"
    104105    echo "  -singular [...]     -- run Sage's singular with given arguments"
    105     echo "  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments"
    106106    test -x "$SAGE_LOCAL/bin/sqlite3" || \
    107     echo "                         (not installed currently, run sage -i sqlite3)"
     107    echo "  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments"
    108108    echo "  -twistd [...]       -- run Twisted server"
    109109
    110110    echo

Or you could do

  • src/bin/sage

    diff --git a/src/bin/sage b/src/bin/sage
    index 7eca9b0..5be445b 100755
    a b usage() { 
    2626    echo "  -R [...]            -- run Sage's R with given arguments"
    2727    echo "  -singular [...]     -- run Sage's singular with given arguments"
    2828    echo "  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments"
     29    test -x "$SAGE_LOCAL/bin/sqlite3" || \
     30    echo "                         (NOTE: not installed on OS X)"
    2931    echo "  -root               -- print the Sage root directory"
    3032    echo "  --nodotsage         -- run Sage without using the user's .sage directory:"
    3133    echo "                         create and use a temporary .sage directory instead"
    usage_advanced() { 
    104106    echo "  -singular [...]     -- run Sage's singular with given arguments"
    105107    echo "  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments"
    106108    test -x "$SAGE_LOCAL/bin/sqlite3" || \
    107     echo "                         (not installed currently, run sage -i sqlite3)"
     109    echo "                         (NOTE: not installed on OS X)"
    108110    echo "  -twistd [...]       -- run Twisted server"
    109111
    110112    echo

comment:23 Changed 6 years ago by git

  • Commit changed from cfc31f333b25626941353545126dbc4c6dbb01f1 to 3f36a9e459741b609d176251285f969485c03478

Branch pushed to git repo; I updated commit sha1. New commits:

3f36a9eremove --sqlite3 option if sqlite is not installed

comment:24 follow-up: Changed 6 years ago by jhpalmieri

Would it be a good idea on OS X to create a symlink at local/bin/sqlite3 pointing to /usr/bin/sqlite3 (or whatever the output of command -v sqlite3 is)? Then sage --sqlite3 would still work.

comment:25 in reply to: ↑ 24 Changed 6 years ago by was

Replying to jhpalmieri:

Would it be a good idea on OS X to create a symlink at local/bin/sqlite3 pointing to /usr/bin/sqlite3 (or whatever the output of command -v sqlite3 is)? Then sage --sqlite3 would still work.

See trac #12627 which would fix this.

comment:26 follow-up: Changed 6 years ago by jhpalmieri

As I noted at #12627, that ticket is somewhat controversial, and I'm not sure it's a good idea to get it merged soon. Something much more targeted, like making a symlink for sqlite3 only on OS X, seems more palatable.

comment:27 Changed 6 years ago by was

REMARKS:

  • In build/pkgs/gcc/spkg-install and build/pkgs/sqlite/spkg-install ¶, change "OSX 10.9" to "OS X 10.9". Also "posed" --> "posted".
  • If trac #12627 gets in (which it should if rebased), then this would be deleted:
     if test -x "$SAGE_LOCAL/bin/sqlite3" && [ "$1" = '-sqlite3' -o "$1" = '--sqlite3' ]; then 
    

I am for having #12627 merged, since I can easily imagine users, scripts, workflows, whatever that have "sage --sqlite3" in them, and would be broken by the above change.

Modulo testing (on 10.4, 10.5, 10.6, 10.7, 10.8, 10.9), and the above comment regarding #12627, I think this patch looks excellent.

comment:28 in reply to: ↑ 26 Changed 6 years ago by was

Replying to jhpalmieri:

As I noted at #12627, that ticket is somewhat controversial, and I'm not sure it's a good idea to get it merged soon. Something much more targeted, like making a symlink for sqlite3 only on OS X, seems more palatable.

OK, good point -- symlinking temporarily seems reasonable, since fully supporting OS X 10.9 asap is important.

comment:29 Changed 6 years ago by git

  • Commit changed from 3f36a9e459741b609d176251285f969485c03478 to 2766ab35a1e88af4f36e46b2eb29eb9893707cb2

Branch pushed to git repo; I updated commit sha1. New commits:

2766ab3gcc: fix comment typos in install script
a64c2d7symlink system sqlite3 on OS X

comment:30 Changed 6 years ago by git

  • Commit changed from 2766ab35a1e88af4f36e46b2eb29eb9893707cb2 to 7547fceeb85afc0b6b932cb2487e61fb94fef405

Branch pushed to git repo; I updated commit sha1. New commits:

7547fceremove a couple of now unused lines

Changed 6 years ago by ohanar

comment:31 Changed 6 years ago by ohanar

  • Description modified (diff)

Ok, since this seems to be all but positively reviewed now, I've backported this to mercurial land.

Note: See TracTickets for help on using tickets.