Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#12161 closed defect (fixed)

Make Sage App on Mac work right all the time on OS X 10.7 Lion, and OS X 10.6 Snow Leopard also

Reported by: kcrisman Owned by: tbd
Priority: blocker Milestone: sage-5.0
Component: distribution Keywords:
Cc: iandrus, was Merged in: sage-5.0.beta6
Authors: Ivan Andrus Reviewers: Georg S. Weber
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by iandrus)

This ask.sagemath.org question has no fewer than eight users who have had trouble using the Mac app bundle from Snow Leopard on Lion.

Most of the errors seem to look like

$ '/Applications/Sage-4.7.1-OSX-64bit-10.6.app/Contents/Resources/sage/'/sage --notebook
Traceback (most recent call last):
  File "/Applications/Sage-4.7.1-OSX-64bit-10.6.app/Contents/Resources/sage//local/bin/sage-notebook", line 3, in <module>
    import os, sys, socket
  File "/Applications/Sage-4.7.1-OSX-64bit-10.6.app/Contents/Resources/sage/local/lib/python/socket.py", line 46, in <module>
    import _socket
ImportError: No module named _socket

when the app is started. Then we told people to try

/Applications/Sage-4.7.1-OSX-64bit-10.6.app/Contents/Resources/sage/sage

or the equivalent, and that pretty much uniformly led to an IPython import error.

However, nearly everyone was able to solve the problem by one of

  • Installing the new Xcode
  • Installing IPython
  • Upgrading the system Python

Note that none of these things should have anything to do with Sage running from the app!

Anyway, it seems to all be the same or very similar problems, so I'm making this a ticket. I wouldn't be surprised if it was a deployment target thing, even.

Possible solutions that would close this:

  • Getting the app bundle to build right on Lion with Xcode 4 and distributing that
  • Getting the Xcode 3/Snow Leopard one to work right on Lion

Home base is #11881; we probably shouldn't release a Sage 5.0 that doesn't have a working app bundle on Lion.

Apply trac_12161-re-source-sage-env.patch to the EXTCODE repository.

Attachments (1)

trac_12161-re-source-sage-env.patch (1.0 KB) - added by iandrus 7 years ago.

Download all attachments as: .zip

Change History (23)

comment:1 Changed 7 years ago by kcrisman

  • Description modified (diff)

comment:2 Changed 7 years ago by GeorgSWeber

diff --git a/sage/ext/mac-app/start-sage.sh b/sage/ext/mac-app/start-sage.sh --- a/sage/ext/mac-app/start-sage.sh +++ b/sage/ext/mac-app/start-sage.sh @@ -53,6 +53,7 @@

./sage --notebook >> "$SAGE_LOG" 2>> "$SAGE_LOG"

else

echo Starting Notebook in Terminal >> "$SAGE_LOG"

+ export SAGE_ENV_SOURCED=""

sage-native-execute osascript \

-e 'tell app "Terminal"' \ -e ' activate' \

comment:3 follow-up: Changed 7 years ago by GeorgSWeber

Sorry,

I do know how to properly attach patches to trac, of course. But I don't know either how soon I get around to find enough time to do so. And the one-liner in the comment above (add to the extcode hg repo, or in a "live install" to the file "start-sage-sh" under /Applications/Sage?-xyz/Contents/Resources/sage) was simply too tempting to be kept entirely unposted.

The story: In this "start-sage.sh" when called in a fresh install, the directory "$DOT_SAGE/sage_notebook.sagenb" is checked for existence and will not exist. Then (and only then) some different means to start up the Notebook server is chosen, by some call to Terminal. Under OS X Lion, it seems this way leads to some strange re-sorting of the entries in PATH variable (which had been properly set before in "start-sage.sh"). As a result, /usr/bin/python is called instead of Sage's python --- hence the failures to find "_socket" and "IPython"!

The one line added cares for when sage-env is sourced a second time (which is the case), the environment variables are set again (which wouldn't be done if SAGE_ENV is kept set to "1"), including PATH now being what we want.

Once the Notebook server has started once up (where the user has to set in the terminal the admin password and such), the one directory checked for exists, and the a different codepath is taken (which works without changes).

comment:4 Changed 7 years ago by GeorgSWeber

Corrections:

In a live install "start-sage.sh" lives under /Applications/Sage?-4.7.1-OSX-64bit-10.6.app/Contents/Resources/ (or the like, but not lower).

The environment variable alluded to is not named SAGE_ENV, but SAGE_ENV_SOURCED (as in the diff).

comment:5 in reply to: ↑ 3 Changed 7 years ago by iandrus

Replying to GeorgSWeber:

Sorry,

I do know how to properly attach patches to trac, of course. But I don't know either how soon I get around to find enough time to do so. And the one-liner in the comment above (add to the extcode hg repo, or in a "live install" to the file "start-sage-sh" under /Applications/Sage?-xyz/Contents/Resources/sage) was simply too tempting to be kept entirely unposted.

I'm happy to make a patch (it will be up in a few minutes). In fact if this fixes it, I'll reach out through the internet and give you a big kiss (or cup of coffee whichever you prefer :-)

The story: In this "start-sage.sh" when called in a fresh install, the directory "$DOT_SAGE/sage_notebook.sagenb" is checked for existence and will not exist. Then (and only then) some different means to start up the Notebook server is chosen, by some call to Terminal. Under OS X Lion, it seems this way leads to some strange re-sorting of the entries in PATH variable (which had been properly set before in "start-sage.sh"). As a result, /usr/bin/python is called instead of Sage's python --- hence the failures to find "_socket" and "IPython"!

The one line added cares for when sage-env is sourced a second time (which is the case), the environment variables are set again (which wouldn't be done if SAGE_ENV is kept set to "1"), including PATH now being what we want.

I would have thought is was impossible to pass environment variables via the applescript. I just checked it however and it depends on if Terminal.app was already started. If it was then the variables aren't "passed". If the call to osascript opens Terminal.app though, it does inherit the variables from the environment. This is just weird, but does make sense in a way.

Once the Notebook server has started once up (where the user has to set in the terminal the admin password and such), the one directory checked for exists, and the a different codepath is taken (which works without changes).

This explains why I (and others) could never reproduce this. It was because we all had sage notebooks already. And when I did test with no notebook, my PATH was such that it didn't cause problems.

I just hope this is the only problem.

comment:6 follow-up: Changed 7 years ago by iandrus

  • Authors set to Georg S. Weber
  • Status changed from new to needs_review

I ran some tests and this does solve what we suppose the problem to be. We won't really know until someone who reported the bug tries it though.

Perhaps unsetting of SAGE_ENV_SOURCED should be done in sage-native-execute. I don't think sage-native-execute was ever intended to run a sage command. :-)

comment:7 in reply to: ↑ 6 Changed 7 years ago by nbruin

Replying to iandrus:

Perhaps unsetting of SAGE_ENV_SOURCED should be done in sage-native-execute. I don't think sage-native-execute was ever intended to run a sage command. :-)

See also #9386 for other problems with sage-native-execute

comment:8 Changed 7 years ago by jdemeyer

The commit message "unset SAGE_ENV_SOURCED in the case of a new user" doesn't make sense to me.

Changed 7 years ago by iandrus

comment:9 Changed 7 years ago by kcrisman

  • Reviewers set to Ivan Andrus

How should this be reviewed? It sounds like Georg is the author here (any reason that we don't "export" in the actual patch? I assume that in the .sh file that doesn't matter or something?) and Ivan confirms that this solves the problem, and Georg confirms that Ivan's confirmation is right. Modulo someone with the problem actually trying it.

There is also the weird thing about one of the posters on the ask.sagemath question actually having IPython installed in his 'usual' install, yet getting that error.

Anyway, do the author and reviewer think this is enough for positive review? It would be really, really good to be able to merge this quickly and then post some beta binaries for newbies to test - maybe even mentioning this at the ask.sagemath question which started it all. Not having this work consistently in 5.0 would be bad news.

comment:10 follow-up: Changed 7 years ago by GeorgSWeber

Hi,

from a mere "formal/technical" point of view, I would be happy naming Ivan the "author" and myself then "reviewing" it ;-)

As for the commit message,

"trac #12161: work around some PATH issue, when using the Terminal on OS X 10.7 Lion, on first start of Sage for a new user"

might be an alternative choice. As for the testing/reviewing, this needs someone with an OS X 10.7 Lion install, of course. I thought about creating some "sage-4.8+trac12161-OSX-64bit-10.6-x86_64-Darwin-app.dmg" (note the attribution to this trac ticket) and upload it e.g. to "http://sage.math.washington.edu/home/weberg/bdist/", eventually even giving eager OS X 10.7 users a pointer to that "unofficial" location. But in the end, the "correct" solution for reviewing would be to set up some OS X 10.7 machine amongst the buildbots --- we want that anyway, since William seems to be inclined to support the LLVM-backend-based gcc, that Apple uses in newer XCode (additionally to the FSF-backend based gcc).

@Jeroen:

How is the status of adding an OS X 10.7 build bot/system to the Sage deployment machine park? Has this task been started (or even thought of) yet?

Cheers, Georg

comment:11 Changed 7 years ago by was

@Jeroen: How is the status of adding an OS X 10.7 build bot/system to the Sage deployment machine park? Has this task been started (or even thought of) yet?

I visited my 10.7 online machine, and it is fubar'd; I think the hardware is faulty. I have a 10.6.8 imac machine in the Sage lab, and Andrew Ohana is planning to put 10.7 on it. If that works, it can be the machine.

comment:12 Changed 7 years ago by was

Oh, Andrew is planning to do this tomorrow...

comment:13 in reply to: ↑ 10 ; follow-up: Changed 7 years ago by iandrus

  • Authors changed from Georg S. Weber to Ivan Andrus
  • Reviewers changed from Ivan Andrus to Georg S. Weber

Replying to kcrisman:

How should this be reviewed? It sounds like Georg is the author here (any reason that we don't "export" in the actual patch? I assume that in the .sh file that doesn't matter or something?)

It's already exported in sage-env. If it wasn't then it wouldn't be a problem.

There is also the weird thing about one of the posters on the ask.sagemath question actually having IPython installed in his 'usual' install, yet getting that error.

It might be related to mismatching PATH and PYTHON_PATH

Anyway, do the author and reviewer think this is enough for positive review? It would be really, really good to be able to merge this quickly and then post some beta binaries for newbies to test - maybe even mentioning this at the ask.sagemath question which started it all. Not having this work consistently in 5.0 would be bad news.

Indeed.

Replying to GeorgSWeber:

Hi,

from a mere "formal/technical" point of view, I would be happy naming Ivan the "author" and myself then "reviewing" it ;-)

Fine by me.

As for the commit message,

"trac #12161: work around some PATH issue, when using the Terminal on OS X 10.7 Lion, on first start of Sage for a new user"

might be an alternative choice.

To be fair, at the time the patch had a rather nonsensical commit message which I have since changed. I'm happy to change it again if desired.

comment:14 in reply to: ↑ 13 Changed 7 years ago by kcrisman

To be fair, at the time the patch had a rather nonsensical commit message which I have since changed. I'm happy to change it again if desired.

The commit message is fine, I checked that, Ivan is right about the history of this comment.

comment:15 Changed 7 years ago by kcrisman

Just bumping because this has been a hot topic on sage-devel the last few days - assuming that is the same problem as this one, which it might not be.

comment:16 Changed 7 years ago by iandrus

  • Description modified (diff)

comment:17 Changed 7 years ago by GeorgSWeber

  • Status changed from needs_review to positive_review

Good goodness, I finally found the time to test the necessary bits and pieces (building Sage-4.8 from scratch and bdist it as "app", then apply the patch here and bdist that as another "app" --- both times on a OS X 10.6 installation --- and then test these two apps on a vanilla OS X 10.7 installation, i.e. without resp. an empty ".sage" folder, that the first app barfs (with the _socket syndrom) and that the second app does work fine).

It was reported that under certain circumstances, the first app might barf also on some OS X 10.6 installations, but I couldn't verify this up to now.

Cheers, Georg

comment:18 Changed 7 years ago by GeorgSWeber

  • Priority changed from major to blocker
  • Summary changed from Make Sage App on Mac work right all the time on OS X Lion to Make Sage App on Mac work right all the time on OS X 10.7 Lion, and OS X 10.6 Snow Leopard also

Update:

Well, I *could* verify that indeed, at least on one of my own OS X 10.6 installations (the one that is located on my QuadCore PowerBook, i.e. starting native in 64bit resp. pressing "6" and "4" on bootup on any 64bit capable OS X installation should show the problem with some high probability), when $HOME/".sage" is nuked, the first app just doesn't come up! The symptom has already been described elsewhere in sage-devel, it hangs as on OS X 10.7, but the error message in the terminal does not show "_socket", but something about "Symbol not found: _PyUnicodeUCS4_AsUTF8String"!

I thus added OS X 10.6."Snow Leopard" to the title line, and enhanced the priority of this patch to "blocker"!!

Cheers, Georg

comment:19 follow-up: Changed 7 years ago by kcrisman

You rock.

comment:20 in reply to: ↑ 19 Changed 7 years ago by iandrus

Replying to kcrisman:

You rock.

I concur. Thanks a ton Georg.

comment:21 Changed 7 years ago by jdemeyer

  • Merged in set to sage-5.0.beta6
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:22 Changed 7 years ago by kcrisman

Just to follow up: Apparently there is still the missing IPython issue, presumably after one has already tried to use the app bundle. Will this fix that as well? See this ask.sagemath.org question, where the user has already tried all the tricks from the original question about this.

Note: See TracTickets for help on using tickets.