upgrade ipython to 0.10.2

[jason]

Our current version of ipython is pretty out of date, and it's causing some problems with our singlecell project. It would be good to upgrade to 0.10.2.

New spkg:

• ​http://sage.math.washington.edu/home/leif/Sage/spkgs/ipython-0.10.2.p0.spkg

[jason]

Ccing fbissey, since he seems to always be helpful on these upgrades.

[jason]

Fernando said we shouldn't upgrade to 0.11 whenever it comes out (wait for 0.12), so chances are it will be a while before we can upgrade past 0.10.2, so it's probably worth the effort to upgrade to 0.10.2 (as compared to just waiting until 0.12).

[fbissey]

Thx, 0.10.2 works well in sage-on-gentoo, it seems to be a no brainer as far as I can see. I am taking notes about 0.11 (still it will be tested).

I probably won't be able to work on a new spkg until Monday. You put it for 4.7.1 is it somewhat urgent?

[jason]

No, it's not urgent. I'm glad you think it'll be easy.

Fernando said that moving to 0.11/0.12 will probably be major, as it is a major architectural shift and rewrite.

[fbissey]

At long last I made a new spkg. I removed all the patches from the current spkg. One issue has definitely been addressed upstream and I don't suffer from any drawback using it on sage-on-gentoo even on my iMac. So it needs testing to check that everything is all right with sage -gdb and backtrace, particularly on OS X.

[jhpalmieri]

A few quick comments:

• in SPKG.txt, the descriptions of the patches should be deleted, since they are no longer used.
• if you feel like it, you could change SPKG.txt so the lines are all < 80 characters. (There are two areas: several lines in the "description" section, and one line in your part of the Changelog.)
• along the lines of the spkg-install file in #11665, it might be best to build IPython first, using python setup.py build, then if there were no errors, remove the old version (along with printing a message that you're doing it), and then finally install the new one with python setup.py install.

I'm testing on a few systems now. What exactly should I test with sage -gdb on OS X?

[jhpalmieri]

I'm having some issues with sage -gdb on OS X. I'm trying again, both with your spkg and with another version which just reinstates the relevant patch. I'll report on results when I have some.

[jhpalmieri]

It looks as though we still need the patch. With your spkg, I get this message when running sage -gdb on my OS X box:

. done
Reading symbols for shared libraries warning: Could not find object file "/Applications/sage_builds/11460/sage-4.7.1/spkg/build/python-2.6.4.p11/src/build/temp.macosx-10.6-i386-2.6/Applications/sage_builds/11460/sage-4.7.1/spkg/build/python-2.6.4.p11/src/Modules/_hashopenssl.o" - no debug information available for "/Applications/sage_builds/11460/sage-4.7.1/spkg/build/python-2.6.4.p11/src/Modules/_hashopenssl.c".

.... done
Traceback (most recent call last):
  File "/Applications/sage_builds/11460/sage-4.7.1/local/bin/sage-ipython", line 18, in <module>
    import IPython
  File "/Applications/sage_builds/11460/sage-4.7.1/local/lib/python2.6/site-packages/IPython/__init__.py", line 58, in <module>
    __import__(name,glob,loc,[])
  File "/Applications/sage_builds/11460/sage-4.7.1/local/lib/python2.6/site-packages/IPython/ipstruct.py", line 17, in <module>
    from IPython.genutils import list2dict2
  File "/Applications/sage_builds/11460/sage-4.7.1/local/lib/python2.6/site-packages/IPython/genutils.py", line 114, in <module>
    import IPython.rlineimpl as readline
  File "/Applications/sage_builds/11460/sage-4.7.1/local/lib/python2.6/site-packages/IPython/rlineimpl.py", line 49, in <module>
    if status == 0 and len(result) > 0:
NameError: name 'status' is not defined
>>> 

With a version including the patch, it gets past this point; instead, it ends up like this:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/Applications/sage_builds/11460/sage-4.7.1/local/lib/python2.6/site-packages/IPython/ipmaker.pyc in force_import(modname, force_reload)
     61         reload(sys.modules[modname])
     62     else:
---> 63         __import__(modname)
     64         
     65 

/Applications/sage_builds/11460/sage-4.7.1/ipy_profile_sage.py in <module>()
     14     from sage.misc.interpreter import attached_files 
     15 
---> 16     branch = sage.misc.misc.branch_current_hg_notice(sage.misc.misc.branch_current_hg())
     17     if branch:
     18         print branch

/Applications/sage_builds/11460/sage-4.7.1/local/lib/python2.6/site-packages/sage/misc/misc.pyc in branch_current_hg()
   2051     i = s.rfind('->')
   2052     if i == -1:
-> 2053         raise RuntimeError, "unable to determine branch?!"
   2054     s = s[i+2:]
   2055     i = s.find('-')

RuntimeError: unable to determine branch?!
Error importing ipy_profile_sage - perhaps you should run %upgrade?
WARNING: Loading of ipy_profile_sage failed.

Two comments:

• This also doesn't look great, but it's exactly the same behavior as with IPython 0.9.
• With the patched version, at least once I've been able to run sage -gdb without any errors at all.

So I think the patch should be there, at least to maintain the status quo. I'm posting a link to a new spkg, based on yours, which reinstates the patch. I'm attaching the hg patch for the Mercurial repo for the spkg, so you can see what's been changed. I'm also posting logs of what happens when you run sage -gdb with the current 0.9.1.p0 spkg, with your 0.10.2 spkg, and with my 0.10.2.p0 spkg. (Notice how much longer the logs are for spkgs which use the patch: they seem to process many more libraries. The logs for 0.9.1.p0 and 0.10.2.p0 are identical, except for the paths involved.)

[jhpalmieri]

hg patch for ipython spkg; for review only, do not apply

[jhpalmieri]

Actually, attaching the logs is a bad idea, because they're big. Here are links to them:

• ​http://sage.math.washington.edu/home/palmieri/misc/ipython-0.9.1.p0.gdb.log
• ​http://sage.math.washington.edu/home/palmieri/misc/ipython-0.10.2.gdb.log
• ​http://sage.math.washington.edu/home/palmieri/misc/ipython-0.10.2.p0.gdb.log

(The log for 0.10.2.p0 happens to be for a time when I didn't get the runtime error.)

[fbissey]

Ok, thanks for the feedback. I posted a spkg but I didn't consider it finalized, I guess I should have switched to need_info rather than need_review. Your changes look good, removing after building is a good idea. I should do something similar in sympy (of course sympy has the added difficulty of importing itself in setup.py - but I know how to deal with that).

I wonder why I don't the patch in my OS X gentoo prefix install. The fault may actually be somewhere else and the current patch in ipython just a work around. But that would be a long investigation.

[jhpalmieri]

I don't know how much Sage doctesting really tests IPython — I know it does some, because of some tests in misc for docstrings and things like that — so I don't know how informative this is, but with my version, all tests pass on sage.math and on an OpenSolaris machine (David Kirkby's box 'hawk').

[jhpalmieri]

So what do you think needs to be done to review this properly?

[fbissey]

I'll do a test build against 4.7.1 on OS X (10.5.8) I guess we mutually reviewed our changes so apart from doing as many tests as we can, and we have done a bit, a run through the build bot would be good.

[fbissey]

Build from scratch on OS X 10.5.8, all long tests passed.

[jhpalmieri]

Okay, I'm building Sage from scratch using this spkg on various ​skynet machines: cicero, fulvia, iras, lena, mark, and taurus. I'll report back when I have results.

[jhpalmieri]

All tests passed on all of the machines, except for fulvia, on which I can no longer build Sage for reasons unknown to me (and unrelated to this ticket — the build fails well before it reaches ipython).

[fbissey]

I am switching this to positive review then.

[jhpalmieri]

Okay, sounds good to me.

[fbissey]

As I mentioned on sage-devel I'll have a look at ipython-0.11 when it becomes available in Gentoo. Strangely enough it is not there yet, I suspect the maintainer is on holidays.

[leif]

Had to fix the heading of an old changelog entry (having the wrong version number).

Corrected spkg at new location.

[was]

The Ipython initialization appears to be importing sage.all. If you make the following change, then do "sage -br", you get a less-helpful-than-before message.

blastoff:sage wstein$ hg diff
diff --git a/sage/all.py b/sage/all.py
--- a/sage/all.py
+++ b/sage/all.py
@@ -21,6 +21,8 @@
 
 from __future__ import with_statement
 
+1/0
+
 ###############################################################################
 #
 #   SAGE: System for Algebra and Geometry Experimentation