Rich output and the IPython Notebook

[vbraun]

This framework separates the construction of rich output (images, mathjax, 3d scenes, ...) from the viewer details. Displaying rich output is handled by interchangeable plugins that encapsulate the differences between various notebooks, the command line, etc.

As a side effect, mathjax output in the ipython notebook works now.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​1bac881

WIP

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​9e1c355

WIP

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b902167

WIP

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​14c147e

WIP

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3411efa

test

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​fc4e54a	WIP
​830e463	rebase, minor fixes
​07adf45	save builtins._
​f78a5d3	WIP: SageNB jmol

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​7296703	WIP
​ed9bc27	rebase, minor fixes
​f8f8386	save builtins._
​a88a042	WIP: SageNB jmol
​5dc0090	Make SageNB j(s)mol work

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​db74693	make sagenb temp filen world-readable
​a13e0f1	Add canvas3d graphics support
​7633dc3	hook up the Java3d (sage3d) viewer

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​037d0eb

Finish lightwave support

[jdemeyer]

Regarding changes of the form

- def show(self, filename=None, dpi=DEFAULT_DPI, figsize=None, axes=None, **kwds):
+ def show(self, **kwds):

I also tried once to make the same simplification as you but people complained.

See discussion starting at 25:ticket:16533

[vbraun]

On the plus side the filename keyword is removed, too: you actually have to call save() to save now. We could add a deprecation workaround but really that was never the intent of the show() method nor match its documentation.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​867cf2a

WIP

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​7747261

Rich output framework

[vbraun]

This is now feature complete and all doctests pass. I will not rebase the branch any more, all future additions will be incremental.

The only thing missing is some documentation.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7dc42d9

More documentation

[vbraun]

Documentation is complete now, ready for review.

[ohanar]

Some comments for now:

• It seems like the show method should maybe get moved to the SageObject class. It pops up everywhere, is more or less the same (other than supporting some deprecated functionality in a small handful of places) and as far as I can tell should work for every child of SageObject.
• Similarly, in graphics.py, a bit of refactoring could be done by creating a common parent for Graphics and GraphicsArray (maybe something like GraphicsBase)
• In sage.repl.formatter there is presumably a debugging print statement still left in SagePlainTextFormatter.
• In the module docstring for sage.repl.image you mention the class SageImage, which you presumably renamed to just be Image. I didn't see any other instances of this, but it warrants a double check.
• In sage.repl.interpreter it would be good to at least have trivial docstrings for the new methods as well as # not tested doctests -- hopefully it reduces the number of coverage complaints (even though it is a bit silly in this case).
• For the display magic, you deprecate some old values, maybe add a proper deprecation? (or maybe this doesn't work well with magics, I don't know)
• The only the first pdf assertion in validate method for the doctest backend is being tested at the moment.
• The display_immediately method on the IPython notebook backend currently does return something.
• The display_immediately doctest for the test backend is actually testing the base backend.
• For the various backends, wouldn't it make more sense for the supported_types to return frozensets rather than sets (since really the supported types aren't mutable). You do this later in the display manager.
• For the display manager, why don't you just make a it have a unique instance (either by using a metaclass, or by using __new__) rather than just assert that there is only one instance?

Once I get a chance to test it and give it a second pass I'll add some more comments. It would also be good to get someone familiar with SageNB to take a look at this, since I have no idea what hacks SageNB depends on and can't really review the SageNB backend.

[vbraun]

1, 2) I agree on the first two, but I don't want to do any further refactoring on this ticket. It is already way too big...

3) is actually never used, thouugh it might be handy to have a plain IPython formatter. I've clarified the documentation.

6) its still not clear to me what the best syntax is, so for now I think its easiest to just allow different combinations. But I don't like "simple" (not expressive, better: "plain") and "typeset" (probably better: "mathjax").

7) was supposed to be OutputImageDvi, fixed.

10) The backend supported_output can return any iterable. The display manager then turns that into a nice frozen set and performs checks and type demotion. Nobody interacts with the backends directly, the display manager intentionally has no public accessor for the current backend.

11) Just for simplicity, the display manager's type isn't global so you already have to work to try to make a second instance. You aren't going to accidentally call the ctor.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2d915f0

Fix Andrews comments

[ohanar]

A few more comments:

1. I think it would be better to have a latex output type rather than a mathjax output type -- different backends might use different renders for latex, (e.g. the command line currently uses the pdflatex command + a pdf viewer). This should work fine with ipython which supports the text/latex MIME type (which it renders with mathjax).

2. Sure, we can hold off on the show stuff for another ticket, this is already quite big. (Note for future ticket) From talking with William, he convinced me that show should be an alias for pretty_print, which should always tries to use a rich output for an object (falling back on the plain text), while the plain print function should always use the plain text for the object.

3. I don't really like right now how the graphics and text are treated very differently at the moment -- the graphics object are responsible for creating their own output types, while the display manager/backend is responsible for creating text output types. I would define both ascii art and latex output as rich output rather than plain text output, and would then move the logic for an object's rich output to the _rich_repr_ method of the object itself (in this case it would go into SageObject._rich_repr_).

4. I'm getting a lot of doctests failing under sage.plot (I can attach a log if you aren't getting them). A lot of them are fixed with changing tachyon.png.save to tachyon.png.save_as on line 244 of sage.plot.plot3d.base (this also fixes 3d plots in SageNB for me).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​be577ff

Rename Lightwave -> Wavefront obj

[vbraun]

1) sounds good, I can make the change

2) agree but thats yet another ticket for sure. a better name for the viewer='...' keyword option that works also for plain text / latex / 2d animation formats / ... would be nice, too. (see also: #17783)

3) our text formatters are special in that they also apply to non-Sage objects, namely tall Python list and alike. So we can't just move them to SageObject._rich_repr_

4) I see, I have Java installed and you don't. Fixed.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​af73513

Fix saving 3d preview image without java

[vbraun]

I've opened #17821 to collect the ideas for refactoring show / pretty_print

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f5b31e4

Rename mathjax -> latex in rich output types

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​82b4bc1

use the plot3d viewer keyword

[ohanar]

Regarding point 3:

But they don't work consistently, which is precisely the problem with trying to handle the all text with the display manager (the object itself is always going to know best how to represent itself). For instance, the tall list stuff only works for lists and tuples, not sets or dicts or even Sage's Set_object.

I would rather see something like a _call_plain_repr method that checks whether an object is a builtin type that we are giving a our own repr method to, and then call our repr method in that case (rather than the the built in one).

Regardless, there would be a lot of redoing of stuff to make something like that work, so it is probably best to put in another ticket anyway.

---

I'm still getting doctest errors on my machine (which as you can tell, doesn't have too much on it). I'll attach a log in a moment.

[vbraun]

Imho the ascii art is the right way to do it, boxes that are aware of height/width and baseline. Then assemble containers etc. Maybe make ascii art the default at one point. I hesitate to kludge too much on top of the plain text, this would lead to massive changes to the Sage libray doctests.

[ohanar]

Sure, in that case in the _call_rich_repr method you could do something special for builtin python objects.

[vbraun]

Ok, your libpng puts out a different sub-version of png and no imagemagick/ffmpeg. I'll fix it when I have some time.

There is already a framework around _ascii_art_() methods, so I would recommend to build on top of that.

[kcrisman]

Apparently fixes #17858 - I haven't personally confirmed this.

[vbraun]

The failing tests for PNG output actually don't work in the released Sage either, e.g.

sage: sum([dodecahedron(center=[2.5*x, 0, 0], color=(1, 0, 0, x/10)) for x in range(2)]).show(viewer='tachyon')

produces no output. It does create a 0-byte png file, and this ticket is the first to test whether it starts with the PNG magic. The tachyon source file is invalid, but tachyon apparently returns 0 on error.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2c34e3a

Check for Tachyon errors, remove the non-blocking tachyion option

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cc9a5af

Normalize plot data (like texture opacity) to float

[vbraun]

The problem was that tachyon doesn't understand rational input like "Opacity 1/10". Fixed by normalizing everything to float before handing it to plotting backends.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​37ae94a

Mark all animations that require ImageMagick as #optional

[vbraun]

This should fix all of your doctest failures

[ohanar]

I've attached other various doctest failures throughout the library. They look trivial (just missed).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​29853da

fix remaining doctest failures

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​57469c6	correct file permissions
​6c6becd	mark more animation doctess as optional ImageMagick

[charpent]

Replying to git:

[ ... ]

A couple data point for this commit (6.6 beta1 built from scratch + this trac branch) : 1) code builds OK, but doc building fails (" make doc-clean && make " succeeds). 2) in the Ipython notebook :

• %display typeset gives typeset (MathJax?) output. At last ! This was my motivation for testing this branch.
• %display ascii_art fails for not having an output stream
• %display simple restores default output (whew...)
• no 3D viewer available but tachyon."jmol" and "canvas3d" fail silently, "java3d" gives a deprecation warning pointing to "wavefront", which fails silently.

Hoping this helps.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​38c6db0

Fix ascii art in the IPython notebook

[vbraun]

I fixed the ascii art issue.

I'm aware that there is still no 3d graphics in the notebook at this point, but that should go to another ticket.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7c17bce

Fix documentation build

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fcbc588

fix the last missing optional ImageMagick

[vbraun]

Doctests now also pass without ImageMagick

[charpent]

I got 3 failures after "ID 215-02-28-17-11-36-39ff4408", testlong got three failures in src/sage/plot/plot3d/base.pyx (see below) , all bound to 3D graphics. The rest got no failures.

I got no build failure after integrating the last commit(s), but I did not rebuild from scratch.

Is that enough for "positive review" ? Do you need a scratch build (long since serial : I didn't integrate #17839...) ?

HTH.

The three errors :

charpent@asus16-ec:/usr/local/sage-6.6$ sage -t --long --warn-long 24.8 src/sage/plot/plot3d/base.pyx
Running doctests with ID 2015-02-28-19-11-29-0b70177c.
Git branch: t/17234/rich_output_and_the_ipython_notebook
Doctesting 1 file.
sage -t --long --warn-long 24.8 src/sage/plot/plot3d/base.pyx
**********************************************************************
File "src/sage/plot/plot3d/base.pyx", line 1422, in sage.plot.plot3d.base.Graphics3d.save_image
Failed example:
    G.save_image(gif)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/sage-6.6/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 492, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/sage-6.6/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 854, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.plot.plot3d.base.Graphics3d.save_image[5]>", line 1, in <module>
        G.save_image(gif)
      File "sage/plot/plot3d/base.pyx", line 1434, in sage.plot.plot3d.base.Graphics3d.save_image (build/cythonized/sage/plot/plot3d/base.c:16182)
        Image.open(png).save(filename)
      File "build/bdist.linux-x86_64/egg/PIL/Image.py", line 2006, in open
        raise IOError("cannot identify image file")
    IOError: cannot identify image file
**********************************************************************
File "src/sage/plot/plot3d/base.pyx", line 1423, in sage.plot.plot3d.base.Graphics3d.save_image
Failed example:
    assert open(gif).read().startswith('GIF')
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/sage-6.6/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 492, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/sage-6.6/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 854, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.plot.plot3d.base.Graphics3d.save_image[6]>", line 1, in <module>
        assert open(gif).read().startswith('GIF')
    AssertionError
**********************************************************************
File "src/sage/plot/plot3d/base.pyx", line 1474, in sage.plot.plot3d.base.Graphics3d.save
Failed example:
    cube().save(tmp_filename(ext='.gif'))
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/sage-6.6/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 492, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/sage-6.6/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 854, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.plot.plot3d.base.Graphics3d.save[5]>", line 1, in <module>
        cube().save(tmp_filename(ext='.gif'))
      File "sage/plot/plot3d/base.pyx", line 1480, in sage.plot.plot3d.base.Graphics3d.save (build/cythonized/sage/plot/plot3d/base.c:16543)
        self.save_image(filename)
      File "sage/plot/plot3d/base.pyx", line 1434, in sage.plot.plot3d.base.Graphics3d.save_image (build/cythonized/sage/plot/plot3d/base.c:16182)
        Image.open(png).save(filename)
      File "build/bdist.linux-x86_64/egg/PIL/Image.py", line 2006, in open
        raise IOError("cannot identify image file")
    IOError: cannot identify image file
**********************************************************************
2 items had failures:
   1 of   7 in sage.plot.plot3d.base.Graphics3d.save
   2 of   8 in sage.plot.plot3d.base.Graphics3d.save_image
    [297 tests, 3 failures, 44.46 s]
----------------------------------------------------------------------
sage -t --long --warn-long 24.8 src/sage/plot/plot3d/base.pyx  # 3 doctests failed
----------------------------------------------------------------------
Total time for all tests: 44.6 seconds
    cpu time: 1.3 seconds
    cumulative wall time: 44.5 seconds

[vbraun]

I don't know what "ID 215-02-28-17-11-36-39ff4408" is, but these failures might be the same ones that I fixed in cc9a5af.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a173e92

Do not change permissions on doctest sample files

[vbraun]

Passed tests on the buildbot!

[charpent]

a173e92 builds without a hitch and passes (p)testlong with no failure ; the Ipython notebook is functional and displays Mathjax and asci art ==> positive review.

The (temporary) loss of 3D plotting is a (small) nuisance, though. Are there already tickets to watch for this functionality ?

I also note that plot3d(..) or p1=plot3d(...) ; p1 does display in the Ipython notebook, as long as no argument of plot3d has to be passed to show() and friends. Is this "vestigial" (or "embryonic") 3D displaying the intended behaviour ?

[kcrisman]

The (temporary) loss of 3D plotting is a (small) nuisance, though.

I assume you mean only in the IPython notebook, not in the sagenb?

[charpent]

Replying to kcrisman:

The (temporary) loss of 3D plotting is a (small) nuisance, though.

I assume you mean only in the IPython notebook, not in the sagenb?

Yes, indeed. My curiosity was towards the notebook functionality : the Ipython notebook, wit its one-file container, is a real boon.

But, for me at least, not having typeset math (or at least decent ascii_art display) is a pain. So I'm eager to see an Ipython notebook with typesetting functionality.

[gagern]

The changes committed here interfere heavily with my work from #7298 for animations displayed in the HTML5 <video> tag, and raise new questions for #16650 about APNG integration. I'd be very happy if the people involved in landing this code here could have a look at those tickets as well, advise how to proceed there and perhaps re-evaluate some design decisions made here.

[novoselt]

Volker - what is the relation between stuff here and EMBEDDED_MODE? The latter is still spread over the source.

[vbraun]

This ticket wasn't about removing the remaining 55 occurrences of EMBEDDED_MODE, thats another job... It should be removed, but its going to take time.

[iandrus]

I would like to use this to typeset the output in Emacs, but I need some sort of delimiters on the latex, so that I don't try to typeset backtraces, the result of print statements, etc. In fact, it would be nice to have both plain text and latex outputs properly delimited so that I can switch between the two. e.g. display the plain text until the latex rendering is finished.

Does it make sense to add a OutputDelimitedLatex? class and update BackendIPythonCommandline? I run an ipython shell from inside Emacs.

[iandrus]

Replying to iandrus:

Does it make sense to add a OutputDelimitedLatex? class and update BackendIPythonCommandline? I run an ipython shell from inside Emacs.

I have a very experimental branch at u/iandrus/emacs-typeset-output with the basics of what I'm thinking of. It's definitely not finished, but if the general approach seems reasonable, I'll polish it up and open a ticket etc.

[vbraun]

IMHO emacs-mode should define its own backend, possibly by inheriting from the ipython command line backend. Thats why there are pluggable backends. Then you can format OutputLatex in precisely the way the emacs-mode needs without stepping on anybody else's toes.

[iandrus]

Replying to vbraun:

IMHO emacs-mode should define its own backend, possibly by inheriting from the ipython command line backend. Thats why there are pluggable backends. Then you can format OutputLatex in precisely the way the emacs-mode needs without stepping on anybody else's toes.

Thanks. I opened #18115 with the new approach. It seems to work, but I've probably forgotten something.