Opened 3 years ago

Closed 3 years ago

# Fix HTML output

Reported by: Owned by: vbraun major sage-6.7 notebook novoselt, kcrisman, jhpalmieri Volker Braun Andrey Novoseltsev N/A 553d2bb (Commits)

### Description

Html output also suffers from the EMBEDDED_MODE syndrome, it should actually return html expressions (instead of the empty string) and not rely on print()ing HTML to the screen doing anything.

The current behavior is:

sage: n=7
sage: L = map(None,[p for p in prime_range(n+1) if p%4==1],[p for p in prime_range(n+1) if p%4==3])
sage: L = [['',l[1]] if l[0] is None else l for l in L]
sage: T = [['$p\equiv 1\\text{ mod }(4)$','$p\equiv 3\\text{ mod }(4)$']]
sage: output = html(table(T+L,header_row=True))   # print() to stdout???
<html>
<div class="notruncate">
<table  class="table_form">
<tbody>
<tr>
<th><script type="math/tex">p\equiv 1\text{ mod }(4)</script></th>
<th><script type="math/tex">p\equiv 3\text{ mod }(4)</script></th>
</tr>
<tr class ="row-a">
<td><script type="math/tex">5</script></td>
<td><script type="math/tex">3</script></td>
</tr>
<tr class ="row-b">
<td></td>
<td><script type="math/tex">7</script></td>
</tr>
</tbody>
</table>
</div>
</html>
sage: repr(output)     # what?
''
sage: type(output)
<class 'sage.misc.html.HTMLExpr'>


### comment:2 Changed 3 years ago by novoselt

My thinking was that it should call display manager. Or the idea is that html(x) makes some object and when I do show(html(x)) then display hook will be called with HTML input? So far I've done this: https://github.com/novoselt/sage/commit/8667c1e3fa44de724b24697bb3f406f6814ffead

And actually, since calling html(x) before from the middle of the code resulted in HTML output, we really should not change it without deprecation, all my interacts will be broken otherwise and probably any others that tried to be pretty...

### comment:3 Changed 3 years ago by vbraun

Yes, the existing HTMLExpr should have _rich_repr_... I'm working on this.

### comment:5 Changed 3 years ago by vbraun

• Branch set to u/vbraun/fix_html_output

### comment:6 Changed 3 years ago by vbraun

What about interacts? Obviously the current @interact doesn't do anything desirable outside of SageNB. We could let html(x) actually do show(html(x)) (*) inside an @interact in SageNB.

(*): Really just show(table(...)) instead of html(table(...)) since a table now defaults to HTML rich output if the backend supports HTML.

New commits:

 ​d8d97c8 Reasonable html() implementation

### comment:7 Changed 3 years ago by git

• Commit changed from d8d97c8ac64efa8ad57225972d4eea5865370ca0 to f2d400c50e489cff9c496078aaf83ac017e861f1

Branch pushed to git repo; I updated commit sha1. New commits:

 ​4e2e74f fix doctests ​f2d400c match table and html.table keyword arguments

### comment:8 Changed 3 years ago by git

• Commit changed from f2d400c50e489cff9c496078aaf83ac017e861f1 to e9828f8050251eabff57e97cd419768f6a1e3f58

Branch pushed to git repo; I updated commit sha1. New commits:

 ​a65bbfe use header_row in the interact library ​e9828f8 revert to old html behavior inside SageNB interacts

### comment:9 Changed 3 years ago by vbraun

• Authors set to Volker Braun
• Status changed from new to needs_review

### comment:10 follow-up: ↓ 11 Changed 3 years ago by novoselt

Well, @interact does something extremely desirable in SageMathCell (to the point that I would have little use for it if there were no interacts), but the implementation is different and I have no clue about it yet. Personally I was using stuff like html("Something in a prettier font than print command") and html("Layout code for interact")

### comment:11 in reply to: ↑ 10 Changed 3 years ago by kcrisman

Well, @interact does something extremely desirable in SageMathCell (to the point that I would have little use for it if there were no interacts),

Agreed.

but the implementation is different and I have no clue about it yet. Personally I was using stuff like html("Something in a prettier font than print command") and html("Layout code for interact")

Yes, exactly, and especially to get LaTeX of things that might vary in an interact. I am pretty sure we are not the only ones using it for this, either.

### comment:12 Changed 3 years ago by vbraun

Is there a hook for checking whether we are running in the sage cell server? There is no sage.plot.plot.EMBEDDED_MODE...

What might be worth a thought is to have every top-level statement in an @interact-function show its rich output, e.g.

@interact
def f():
plot(sin)         # shows plot
plt = plot(sin)   # does not show


though better not on this ticket...

### comment:13 Changed 3 years ago by novoselt

There used to be a special version of EMBEDDED_MODE which is still in the deployed version, corresponding to https://github.com/novoselt/sage/tree/sagecell however I switched to the new display backend in https://github.com/novoselt/sage/tree/celldev which has minimal changes from Sage-6.6 + new show. I will switch to this branch (currently running on http://sagecell.sagedev.org/ ) next week while switching hosting, so any investigation/fixing should be based on that.

### comment:14 Changed 3 years ago by git

• Commit changed from e9828f8050251eabff57e97cd419768f6a1e3f58 to 1f75112c7b835b8c7cdfdde992d8b86e4e7ab73e

Branch pushed to git repo; I updated commit sha1. New commits:

 ​1f75112 also support arguments that are not literal

### comment:15 Changed 3 years ago by novoselt

Karl-Dieter - any plans to adjust !SageNB to use new display framework "properly"? Once it is done, we can get rid of EMBEDDED_MODE shrubbery completely, I think.

### comment:16 Changed 3 years ago by kcrisman

I accept pull requests but don't understand this new framework at all and don't have time to learn it for some time yet.

### comment:17 Changed 3 years ago by vbraun

IMHO there isn't really any need to change SageNB, backend_sagenb.py can just keep translating output into whatever SageNB expects. To get rid of the EMBEDDING_MODE global we need to tie up a couple of loose ends on the Sage side first, e.g. pager...

### comment:18 Changed 3 years ago by novoselt

What is the motivation for replacing header with header_row?

### comment:19 Changed 3 years ago by novoselt

• Reviewers set to Andrey Novoseltsev

Otherwise looks good to me and should be merged for widespread testing!

### comment:20 follow-up: ↓ 21 Changed 3 years ago by vbraun

table() uses header_row, so I'm thinking that html.table() should match that. We could change table to use header instead, which is shorter. The only downside is that probably more people use table than html.table... I would be fine with either way, though.

### comment:21 in reply to: ↑ 20 Changed 3 years ago by kcrisman

table() uses header_row, so I'm thinking that html.table() should match that. We could change table to use header instead, which is shorter. The only downside is that probably more people use table than html.table... I would be fine with either way, though.

I think that the decision was made at the time John et al. made table() to just leave html.table() alone, so that there was no need for deprecation... the intent would be for people to eventually all switch to table(). Which I was hoping to do for my NT text this time around, but unfortunately the cell server doesn't support it (yet) so I am sticking with html.table(). Anyway, could be worth briefly asking about that, or saving it for another ticket.

(Also, there is no header column option in html.table() which there is in table(), I believe, so there isn't the need to change it that way either.)

### comment:22 follow-up: ↓ 23 Changed 3 years ago by vbraun

With this ticket + updating the cell server to the new display backend just table() should work and display a html table. Should I just deprecate html.table altogether?

### comment:23 in reply to: ↑ 22 ; follow-up: ↓ 29 Changed 3 years ago by novoselt

With this ticket + updating the cell server to the new display backend just table() should work and display a html table. Should I just deprecate html.table altogether?

Yes please, I would never think that such functions will behave differently!

### comment:24 follow-up: ↓ 27 Changed 3 years ago by kcrisman

With this ticket + updating the cell server to the new display backend just table() should work and display a html table. Should I just deprecate html.table altogether?

John, this make sense to you? (I feel like you were involved in the table stuff in some way.)

That said, Andrey, I'm still having trouble getting tables to work in embedded Sage cells. Is that possibly related to this, even after your update?

### comment:25 Changed 3 years ago by novoselt

Embedded cells are still running 6.5 version (using sagecell.sagemath.org, right?). I am currently busy setting up new servers which will be running 6.6. DNS will probably switch tomorrow if I can resolve some problems quickly.

### comment:26 Changed 3 years ago by kcrisman

I guess? Thanks for clarifying that.

### comment:27 in reply to: ↑ 24 ; follow-up: ↓ 28 Changed 3 years ago by jhpalmieri

With this ticket + updating the cell server to the new display backend just table() should work and display a html table. Should I just deprecate html.table altogether?

John, this make sense to you? (I feel like you were involved in the table stuff in some way.)

Maybe because of this at the top of misc/tables.py?

AUTHORS:

- John H. Palmieri (2012-11)


First, html.table(...) just called table after renaming a keyword, so deprecating html.table is fine. Second, at a glance, the changes to table.py all look okay to me.

### comment:28 in reply to: ↑ 27 Changed 3 years ago by kcrisman

John, this make sense to you? (I feel like you were involved in the table stuff in some way.)

Maybe because of this at the top of misc/tables.py?

Haha! You are imputing more diligence to me than I have time for right now - it really was just a vague sense that you were involved...

First, html.table(...) just called table after renaming a keyword, so deprecating html.table is fine. Second, at a glance, the changes to table.py all look okay to me.

Gr8.

### comment:29 in reply to: ↑ 23 ; follow-up: ↓ 30 Changed 3 years ago by novoselt

With this ticket + updating the cell server to the new display backend just table() should work and display a html table. Should I just deprecate html.table altogether?

Yes please, I would never think that such functions will behave differently!

After that we got sidetracked.

Volker - any plans for deprecation? (If not - I approve the current version for inclusion and beta testing before 6.7 is out!)

### comment:30 in reply to: ↑ 29 ; follow-up: ↓ 31 Changed 3 years ago by kcrisman

With this ticket + updating the cell server to the new display backend just table() should work and display a html table. Should I just deprecate html.table altogether?

Just to be clear, there should be a way to get html output without something crazy in sagenb and sage cell. Since html() usually historically actually displayed the html, if that isn't happening any more a very loud deprecation should say exactly how to make that happen (perhaps not using show), though my preference would be for it to continue as has been.

### comment:31 in reply to: ↑ 30 Changed 3 years ago by novoselt

Just to be clear, there should be a way to get html output without something crazy in sagenb and sage cell. Since html() usually historically actually displayed the html, if that isn't happening any more a very loud deprecation should say exactly how to make that happen (perhaps not using show), though my preference would be for it to continue as has been.

show(table) will show HTML table in the cell server. There are two independent steps - generating code for LaTeX/HTML/MathJax/whatever and displaying it. Separating them makes a lot of sense because you may want to use the code as a building block for something else before displaying and because displaying is backend specific and it should be as localized as possible for easy maintenance including seldom breaks rather than with every version bump.

So I am for making latex(x) and html(x) return the code without displaying it and requiring show or pretty_print or whatever else to actually display it. Table display in SageMathCell has been broken for a while with little complains, so I am not sure about necessity of deprecations there and I am not sure how I can keep printing something directly and displaying something returned as well.

### comment:32 Changed 3 years ago by kcrisman

On a related note, has anyone tried html output in sagenb recently? I've been using Sage cell so much the past few months I don't know what that state is with a bleeding-edge Sage (which I hope to update to next week).

### comment:33 Changed 3 years ago by novoselt

No idea about sagenb, I only use html for interacts and since they are incompatible, it does not work to develop them there and then embed into cells.

After discovering that I have broken all my interacts (since they use html to place controls), I am a little less radical in changing the behaviour of html(x) ;-) I still would like it to mean "return HTML code for x and let me decide what to do later", but deprecation period is mandatory, I am just not clear how to implement it. Perhaps this will work:

• let html(x) print/display x and also through a deprecation warning
• make users call some no_html_autoprinting function that will suppress the deprecation AND printing
• after a year or so stop autoprinting, remove the warning, and deprecate no_html_autoprinting

Thoughts?

### comment:34 Changed 3 years ago by vbraun

Nothing changed with html output, so SageNB works but the IPython notebook does not work without this ticket (and cannot work).

How about switch html() to return html code outside of @interact, and preserve the old behavior inside @interact for now. This is implemented in the branch, though the cell server probably changes @interact.

Then we can add a warning and you can get the new behavior with @interact(html_autoprint=True).

So localize it to just @interact, and add some syntactic sugar on top.

Last edited 3 years ago by vbraun (previous) (diff)

### comment:35 Changed 3 years ago by novoselt

html("code for interact location")

@interact ...

So I need it to work outside of interact. And I am not going to make use of sagenb code if I can avoid it - only adds extra headache. I had a two-liner hack to catch HTML in a cell-specific way for 6.6 and I am fine doing it again for 6.7, but eventually it has to stop and go through the backend framework.

I don't think I like the idea of different behaviour for interacts anyway - either we should always print unassigned calls to html (why else would it be called if not for displaying stuff?) or never (except for the last line, of course). Since I imagine that catching all unassigned uses is tricky, the best solution would be to force users always use show(html(x)) if they want to force HTML version of x, which probably will not be necessary too often since show(x) will give the same results. For those who want to inject code on their own show(html("<b>bold</b>")) is a perfectly fine syntax, I think.

### comment:37 Changed 3 years ago by kcrisman

• Status changed from needs_review to needs_info

Not sure what the current status is here...

### comment:38 Changed 3 years ago by novoselt

OK, I think a nice way to proceed here is:

• make HTMLExpr take strings (presumably with HTML code) and turn them into HTMLExpr objects that can be sensibly printed/displayed depending on the backend
• deprecate html: keep its current behaviour of displaying HTML code that was passed to it, but recommend usage of pretty_print(HTMLExpr(...)) instead
• once the deprecation is over, html is a great name for the function that will try to convert its input into HTMLExpr in a smart way, similar to how latex produces LatexExpr and perhaps relying on _html_ methods which we may or may not have already.

Regarding tables:

• html.table should be gone
• table should have HTML pretty-printing.

Thoughts/objections?

### comment:39 Changed 3 years ago by vbraun

Sounds good to me. The ipython notebook should use the new behavior immediately, of course.

### comment:40 Changed 3 years ago by git

• Commit changed from 1f75112c7b835b8c7cdfdde992d8b86e4e7ab73e to 4d0ba82735abf28f7dfc5de99a35cb3b003b1b18

Branch pushed to git repo; I updated commit sha1. New commits:

 ​47cc2ab wip: deprecate html.table ​4d0ba82 Merge Sage-6.9.beta3 into t/18292/fix_html_output

### comment:41 Changed 3 years ago by git

• Commit changed from 4d0ba82735abf28f7dfc5de99a35cb3b003b1b18 to b2419c9b7b72253d80650e91ed695ff8b108c406

Branch pushed to git repo; I updated commit sha1. New commits:

 ​b2419c9 turn off the new html() behavior in the SageNB backend

### comment:42 Changed 3 years ago by vbraun

On a side note: I prefer HTMLFragment over HTMLExpr. The former is standard nomenclature, e.g. https://developer.mozilla.org/en-US/docs/Web/API/DocumentFragment

### comment:43 Changed 3 years ago by novoselt

HTMLFragment is great, but it seems to me that there is no deprecation warning when using html - is this correct?

I am also not convinced that html should have "correct" behaviour in IPython notebook right away - as long as we support multiple frontends they should behave as close to each other as possible. If I type (or copy-paste from a different frontend)

def f():
html("text")

f()


I want to have the same thing happening in SageNB/IPython/Cell/command line - see "text" and get a warning that I should replace html to pretty_print(HTMLFragment("text")) to avoid warning and make sure that my code will work the same in the future.

### comment:44 Changed 3 years ago by git

• Commit changed from b2419c9b7b72253d80650e91ed695ff8b108c406 to 6def164370b96868c373f85ed81fe861cc2e336e

Branch pushed to git repo; I updated commit sha1. New commits:

 ​6def164 deprecate html()

### comment:45 Changed 3 years ago by novoselt

Add SageMathCloud? to the list of frontends. Or backends - how do I distinguish who is who?!

New commits:

 ​6def164 deprecate html()

### comment:46 Changed 3 years ago by novoselt

Had to double check that you are not behind my shoulder...

### comment:47 Changed 3 years ago by vbraun

html() in the IPython notebook is currently completely dysfunctional, there is no point in keeping it broken for a year.

### comment:48 Changed 3 years ago by vbraun

PS: SMC does not use the rich output stuff, perhaps after SD70...

### comment:49 Changed 3 years ago by novoselt

I don't advocate having it broken, but I do advocate consistency. The current behaviour is flawed in other interfaces, but it does a consistent thing and many people (at least me ;-)) have used it extensively. I want them to be able to copy old code into IPython notebook and have it working the same way as before (and now) in a different interface. Otherwise, while the deprecation is in effect, it is not possible to use html and switch interfaces at all.

The fact that SMC does not use rich output is irrelevant - very few people know what does it really mean. But the user-visible behaviour should be as much the same as possible.

### comment:50 Changed 3 years ago by vbraun

PPS: I always think of the "frontend" as the UI part that is NOT done here... e.g. the IPython notebook. The corresponding backend is the piece in Sage that prepares the output for the frontend.

### comment:51 Changed 3 years ago by vbraun

I'm just saying the old behavior is never going to be consistent in the IPython notebook because it just can't work.

But we can use opt-in instead of opt-out...

### comment:52 Changed 3 years ago by novoselt

Why can't it work? What is the problem with html(x) calling pretty_print(HTMLFragment(x)) instead of print(x) inside of itself and warning that this is going to stop? This will work for anything that supports pretty_print and will be consistent with the current beahviour without relying on custom hacks for each system!

### comment:53 Changed 3 years ago by vbraun

Not that easy, pretty print will try to display something "reasonable" which will not be plain html code outside of the browser:

sage: pretty_print(HtmlFragment('<b>a</b>'))
\newcommand{\Bold}[1]{\mathbf{#1}}\verb|<b>a</b>|


This is because pretty_print tries to use the rich output (no match on the command line) and then falls back to latex.

The HtmlFragment could provide a different representation, but it can't change the fact that the commandline cannot render html.

### comment:54 Changed 3 years ago by novoselt

OK, the command line is a bit of a pathological example, although it would be nice to at least avoid latex wrapping of html.

All other backends, however, do use browsers and can show html properly: we want users to use pretty_print(HTMLFragment(...)) and all I propose is that meanwhile html will effectively do exactly that. For that matter, if its current logic goes to HTMLFragment its code can be literally

def html(*args, **kwds):
pretty_print(HTMLFragment(*args, **kwds))


### comment:55 follow-up: ↓ 56 Changed 3 years ago by kcrisman

Would show(html(*args, **kwds)) do the same thing as pretty_print(HTMLFragment(*args, **kwds))? I can't imagine someone stumbling upon the latter without a full training course or an incredible dedication to reading docs.

### comment:56 in reply to: ↑ 55 Changed 3 years ago by novoselt

Would show(html(*args, **kwds)) do the same thing as pretty_print(HTMLFragment(*args, **kwds))? I can't imagine someone stumbling upon the latter without a full training course or an incredible dedication to reading docs.

Eventually html(x) should produce in a smart way an HTML representation of x and RETURN it, so that it can be manipulated/displayed later. Right now we have

sage: r = html(x^2)
<html><font color='black'>x^2</font></html>


which is neither a good representation, nor something returned. However, html("some text") was useful so far to take a string of HTML code and show it. I propose to keep this behaviour for backward compatibility, but show a deprecation and instruct users to call pretty_print(HTMLFragment("some text")) instead. In a year html(x^2) will return a string $x^2$ which actually has a proper type so that it is inserted into HTML output and rendred using MathJax?.

### comment:57 Changed 3 years ago by kcrisman

As long as html(x^2) or show(html(x^2)) allows people in browser-based front ends to see something typeset. However, I have to say that pretty_print(HTMLFragment()) is possibly the most enigmatic syntax anyone could construct for "put this in my thing in html". Even if %html or whatever exists too. There has to be some alias that could be done for this.

### comment:58 Changed 3 years ago by novoselt

What is enigmatic about it? pretty_print is the standard way to display rich output. HTMLFragment (easily discoverable by tab completion) is a way of constructing objects from plain HTML code similar to LatexExpr thatt constructs objects from plain LaTeX code.

The current behavior of html showing stuff (by just printing to stdout which has to be intercepted by frontends) is not at all similar to what latex does.

### comment:59 Changed 3 years ago by vbraun

For starters show is terribly named, it in no way conveys what it does. Nobody understands the difference between

sage: plot(sin)


and

sage: show(plot(sin))


pretty_print is much clearer, it works like print but pretty.

### comment:60 follow-up: ↓ 62 Changed 3 years ago by kcrisman

pretty_print may be standard but how on earth is a non-coding mathematician or teacher supposed to come up with that? I've never heard of it outside of Sage in "real-life" conversation about stuff. Maybe print_pretty or print_fancy or print_cool - at least that is available with tab-completion! I would never come upon that in a million years - it's also not typeset_nicely or anything like that. I hope it is obvious that no one outside of developer circles would know what "rich output" even means. They just want a way to make stuff render with MathJax, if they know what that is.

I agree that the html behavior from earlier is not consistent with e.g. latex but that doesn't mean it's bad... but I know I don't have energy to fight that, and long-run it is probably ok. But there needs to be an easy way to say "print this as html", and this is not very easy to intuit. (And for the record, HT<tab> will give a different list from ht<tab>, but you already knew that, which I just had to test because I couldn't remember if it did.)

### comment:61 Changed 3 years ago by novoselt

Someone who knows what MathJax is and can write raw HTML is sophisticated enough to read some basic tutorial that better mention existence of pretty_print. It may be also a good idea to have a section of the tutorial specifically on displaying thing nicely.

pretty_print tries its best to show a nice representation of an object, but some of them have many, i.e. a plot, a LaTeX formula, and HTML. So it may be nice to specify what exactly do you want to get and pretty_print(latex()) or pretty_print(html()) look quite natural to me. Yet latex and html should NOT display the result automatically in case I want to make some modifications/glue it with other pieces etc! And those who are willing to provide raw LaTeX or HTML code can, I think, figure out LatexExpr and HTMLFragment. Starting with capital is consistent with class names in Sage, although there is nothing wrong in having a lower-case synonym, I suppose.

### comment:62 in reply to: ↑ 60 Changed 3 years ago by vbraun

pretty_print may be standard but how on earth is a non-coding mathematician or teacher supposed to come up with that?

How on earth is somebody supposed to come up with show? And even if you find the many (and often misguided) examples in the docs, can you tell me what it does?

### comment:63 follow-up: ↓ 64 Changed 3 years ago by kcrisman

How on earth is somebody supposed to come up with show? And even if you find the many (and often misguided) examples in the docs, can you tell me what it does?

Nope, I can't, but at least it's in a lot of places so people can find it. So if pretty_print starts showing up lots of places (perhaps replacing a misguided show), they we should be golden. There are various tutorial-esque parts of the Sage doc/tutorial as well (perhaps where html used to appear as an example).

Someone who knows what MathJax? is and can write raw HTML is sophisticated enough to read some basic tutorial that better mention existence of pretty_print. It may be also a good idea to have a section of the tutorial specifically on displaying thing nicely.

They might not know about MathJax?, just that there is a way to get things to look nice. If there were sufficient places where it was all introduced (including the fragment business) that would be an acceptable solution, I think.

That proves my point, I think: it's not a term in general non-technical usage.

Seriously, is print_pretty a possibility as an alias, or is that just pointless?

Here is an example of what I mean. You just did

-.. skip
-
-::
-
-    sage: html.table( [ (i,det(A^i)) for i in [0..4] ] )
-    <html>...</html>
+    sage: table( [ (i,det(A^i)) for i in [0..4] ] )
+      0   1
+      1   -2
+      2   4
+      3   -8
+      4   16


But now anyone using a notebook interface won't know how to make it "pretty". Yet elsewhere in your diff you have pretty_print(table(input)). This would have been a great opportunity to help teach people about this. In one place you even left the wording

 Even better, we can put a header line on it to make it really clear what
we are doing, by adding lists. We've seen keywords like header=True
when doing some of our plotting and limits. What do you think will
-happen if you put dollar signs around the labels in the header?
+happen if you put dollar signs around the labels in the header? ::


which makes absolutely no sense without the html/MathJax showing up in the notebook!

### comment:64 in reply to: ↑ 63 ; follow-up: ↓ 67 Changed 3 years ago by kcrisman

which makes absolutely no sense without the html/MathJax showing up in the notebook!

which I see does apparently work with "rich output" but it still would be useful to make clear what is going on, which was the case with html.table(). So that they know it will look different in the command line. And what if someone WANTS the "command line style output" from table in the notebook - is that even possible? (I assume so, but they won't learn it from this change.)

### comment:65 Changed 3 years ago by vbraun

We could play tricks with the destructor of the HtmlFragment and only show the deprecation if it is not pretty printed:

sage: def foo():
....:    html('foo')
sage: foo()
foo
/home/vbraun/Code/sage/src/bin/sage-ipython:2: DeprecationWarning:  html(...) will change soon to return HTML instead of printing it. Instead use pretty_print(html(...)) for strings or just pretty_print(...) for math.
See http://trac.sagemath.org/18292 for details.
# -*- coding: utf-8 -*-


vs.

sage: def bar():
....:     pretty_print(html('bar'))
sage: bar()
bar


See attached branch for implementation

### comment:66 Changed 3 years ago by git

• Commit changed from 6def164370b96868c373f85ed81fe861cc2e336e to 9a4bc8d4f2cfe6f16567de1c82efb45c9d194023

Branch pushed to git repo; I updated commit sha1. New commits:

 ​9a4bc8d hack to have our cake and eat it, too

### comment:67 in reply to: ↑ 64 ; follow-up: ↓ 69 Changed 3 years ago by vbraun

And what if someone WANTS the "command line style output" from table in the notebook - is that even possible?

Nobody wants ascii art in the browser, just like nobody wants html code printed to the console. Just do the right thing for the corresponding UI.

### comment:68 follow-up: ↓ 80 Changed 3 years ago by novoselt

I guess it sacrifices html(...) on the last line and that's why you had to adjust doctests, but it is livable and I think is a great option. Karl-Dieter - are you happy with this one?

Regarding interacts, which are also supported by SMCs (SageMathCell and SageMathCloud ;-)) - what is "the right way" to adjust sage.interacts.decorator.interact to other UIs?

Do we need math_parse? It seems to me that the string $x._latex_()$ should be a perfectly valid HTML for our UI, i.e. MathJax will process math. The advantage of not fiddling with LaTeX code beyond adding dollar signs, apart from simpler code, is that such a string can be used directly in LaTeX via SageTeX.

Typo: def eval(self, s, locals=None): has description for globals argument.

Why do we need this test:

+        OutputHtml = display_manager.types.OutputHtml
+        if OutputHtml in display_manager.supported_output():
+            return OutputHtml(self._html_())


isn't the point of all this display framework to make these decisions WITHOUT object's interference?

Is it really necessary to include <html> tag for SageNB?

### comment:69 in reply to: ↑ 67 ; follow-up: ↓ 78 Changed 3 years ago by kcrisman

And what if someone WANTS the "command line style output" from table in the notebook - is that even possible?

Nobody wants ascii art in the browser, just like nobody wants html code printed to the console. Just do the right thing for the corresponding UI.

What about cut-and-paste? Esp. browser-to-stuff. html in console perhaps more annoying but again could be useful for cut-and-paste of something... actually, maybe it would be easier to make an html table by firing up Sage and having it output the correct html for a complicated table of function values :) The user should have the option of getting it somehow.

I guess it sacrifices html(...) on the last line and that's why you had to adjust doctests, but it is livable and I think is a great option. Karl-Dieter - are you happy with this one?

Maybe? I'm not sure what the question is. You mean about documentation? I would think there is somewhere else where this comes up in the tutorial or something, but it would be good to mention there. What about here - what do you suggest replacing this with? Maybe with first the "new" output of html and then showing how to actually use it?

As a different question, what happens with iframe in this new behavior? I know that some people have used this to embed a variety of interesting things (videos, Geogebra) in the sagenb. Would this just be almost-inaccessible or would we just need to have some update on the corresponding place... Note that I'm not making this up, here is a Geogebra forum request for exactly this. So it's "used in the wild".

### comment:70 Changed 3 years ago by novoselt

New recommended behavior will be pretty_print(html(stuff)) and it will dump the code in the command line and do the useful thing in all other cases. On the other hand, print(html(stuff)) will print the raw code in all cases. And until the deprecation is over even html(stuff) will work from inside loops/functions/cells to display stuff without necessity of explicit pretty_print.

According to the code iframe should still work as before (hopefully with deprecation warnings).

### comment:71 Changed 3 years ago by vbraun

Note that I haven't fixed all doctests but it should work as before...

The <html>'s are necessary as thats what SageNB is looking for afaik.

### comment:72 Changed 3 years ago by kcrisman

Okay, sounds reasonable though as usual we probably need twice as many tutorials as we will ever get around to writing, sigh. I'm already dreading an email I will anticipate regarding this, but I know I couldn't make it to the SD you all are probably at so I can't contribute properly with regard to this.

### comment:73 Changed 3 years ago by novoselt

Only I am at SageDays ;-) But as annoying as it is to fix breaks in old code, "the right way" makes it easier to maintain interfaces - the day when changing Sage version in cell server will mean just compiling the new version is still in the future and this ticket is a necessary step for it. Git transition also was a bit scary two years ago, but I definitely appreciate it now.

### comment:74 Changed 3 years ago by novoselt

On the command line and SageNB I am getting double output from html and no deprecation warning (after checking out the branch here and sage -b - do I need to do something else?)

### comment:75 Changed 3 years ago by vbraun

You don't get any warning if it hits the displayhook

sage: html('foo')
foo


The action of the displayhook generates output like calling pretty_print after all, so I'd call that a feature: It'll keep working after switching html() to return the fragment.

There isn't really a clean way to avoid double printing while actually returning the HtmlFragment...

### comment:76 Changed 3 years ago by git

• Commit changed from 9a4bc8d4f2cfe6f16567de1c82efb45c9d194023 to 11ebe8ad644133332558f297d074cedd7e5d4ebf

Branch pushed to git repo; I updated commit sha1. New commits:

 ​11ebe8a Do not return the fragment in html() for now

### comment:77 Changed 3 years ago by vbraun

This version should behave nicely, sadly html() still doesn't return the html fragment. But if everybody heeds the warning and switches to pretty_print(html(...)) then everything will work later.

### comment:78 in reply to: ↑ 69 Changed 3 years ago by vbraun

it would be easier to make an html table by firing up Sage and having it output the correct html for a complicated table of function values

Sage is not a HTML authoring tool. Any attempts at making Sage a HTML authoring tool will be to the detriment of users that want to use Sage for math. And you can always have your web browser display the source if you really are curious how its done.

Once the deprecation period is over and html() actually returns a HtmlFragment you can use it like a string (combine, convert, save, print, ...) to work with the HTML source, but I still wouldn't want to advertise Sage as a HTML authoring tool.

As a different question, what happens with iframe in this new behavior?

Nothing, works like before (i.e. doesn't work for sites that prevent iframing like google.com)

### comment:79 Changed 3 years ago by kcrisman

it would be easier to make an html table by firing up Sage and having it output the correct html for a complicated table of function values

Sage is not a HTML authoring tool. Any attempts at making Sage a HTML authoring tool will be to the detriment of users that want to use Sage for math. And you can always have your web browser display the source if you really are curious how its done.

Heehee, that's what I thought you'd say - hence the smiley face. But there are always unauthorized uses for any computation :)

Once the deprecation period is over and html() actually returns a HtmlFragment you can use it like a string (combine, convert, save, print, ...) to work with the HTML source, but I still wouldn't want to advertise Sage as a HTML authoring tool.

No, of course not.

As a different question, what happens with iframe in this new behavior?

Nothing, works like before (i.e. doesn't work for sites that prevent iframing like google.com)

Great, thanks - I think Andrey implied this as well.

### comment:80 in reply to: ↑ 68 Changed 3 years ago by vbraun

Regarding interacts, which are also supported by SMCs (SageMathCell and SageMathCloud ;-)) - what is "the right way" to adjust sage.interacts.decorator.interact to other UIs?

Perhaps the best solution would be to add a hook to the rich output backend, what do you think? It is very much related to the UI frontend. Though ideally do this on a separate ticket.

Do we need math_parse?

I potentially agree but don't want to change it on this ticket. I haven't tried whether your suggestion will work.

Why do we need this test:

+        OutputHtml = display_manager.types.OutputHtml
+        if OutputHtml in display_manager.supported_output():
+            return OutputHtml(self._html_())


isn't the point of all this display framework to make these decisions WITHOUT object's interference?

The commandline can't display HTML output, so that tests is false. Not returning anything from _rich_repr_ means that the ordinary _repr_ is used, which here means ascii art table.

### comment:81 Changed 3 years ago by git

• Commit changed from 11ebe8ad644133332558f297d074cedd7e5d4ebf to 553d2bba92e0cda60ed9ce7579c11d079963ea8b

Branch pushed to git repo; I updated commit sha1. New commits:

 ​553d2bb fix the remaining doctests

### comment:82 follow-up: ↓ 83 Changed 3 years ago by kcrisman

Lest this all seem of academic importance, see this SO question.

### comment:83 in reply to: ↑ 82 Changed 3 years ago by vbraun

Lest this all seem of academic importance, see this SO question.

And that poor soul just wants a good-looking table. But table doesn't produce decent tables in browsers and html.table is too married to SageNB to be of any use. Whereas with this ticket + SMC integration he'd just call table and get a nice-looking table.

### comment:84 Changed 3 years ago by kcrisman

Correct, that was my point.

### comment:85 Changed 3 years ago by novoselt

• Status changed from needs_info to positive_review

Loose ends:

• some hook for interact implementations
• get rid of math_parse, I'll open a ticket and do it

### comment:86 Changed 3 years ago by vbraun

• Branch changed from u/vbraun/fix_html_output to 553d2bba92e0cda60ed9ce7579c11d079963ea8b
• Resolution set to fixed
• Status changed from positive_review to closed

### comment:87 Changed 3 years ago by kcrisman

• Commit 553d2bba92e0cda60ed9ce7579c11d079963ea8b deleted

Hah! I was right about all this silly stuff like "pretty_print". Real-life users don't get all these conventions.

I love "about 8 weeks of my work were gobbled up". Even if this is really a "bug" in the cell server, I don't think anyone was crying out for html to suddenly return a string.

(And note that even if there is a regex that can solve all this user's woes, that is not really the correct solution, any more than the fact that my library's recent decision to completely change their web hierarchy has as a solution "email so-and-so for the new URL for that page". We can't be handing out regex commands like candy all day long.)

Last edited 3 years ago by kcrisman (previous) (diff)

### comment:88 Changed 3 years ago by novoselt

I am crying for a) consistency and b) being able to construct HTML representations of objects.

When you call latex(x), you do NOT get x displayed, instead you get LaTeX code for x which you can use to cook up LaTeX representation of a more complicated object. I want to have the same for html(x).

### comment:89 Changed 3 years ago by kcrisman

And I didn't push back too hard here because the sagenb continues to allow

html("hi there $x^2$")


to correctly render, without a deprecation, or so the above discussion (and my testing just now) suggests. But now the Sage cell apparently doesn't? Though I don't see the deprecation warning... so why is it showing up in the embedded version for this user? (We can open a new ticket or discuss at the ask.sagemath question, if need be.)

### comment:90 Changed 3 years ago by novoselt

html here returns something and that something is automatically displayed as the last thing in the cell. Try putting some stuff after html in the same cell - you should get deprecation. Same with the cell server: html by itself has no warning (and will work like that forever), but if it is followed by something there is a warning and it has to be addressed in a year or so.

### comment:91 Changed 3 years ago by kcrisman

If html() returns html, I thought that the legendary "display hook" would make that html display as well. Why does it matter if something is after the stuff in html? I could very easily imagine someone with zero examples that trigger deprecation warning in that case suddenly being surprised!

### comment:92 Changed 3 years ago by vbraun

The displayhook displays the result of the last expression:

• A cell containing
1/3  # <-- first expression
2/3  # <-- second (=last) expression

outputs 2/3
• A cell containing
latex(1/3)
latex(2/3)

outputs \frac{2}{3}
• A cell containing
html(1/3)
html(2/3)

should output mathjax for 2/3.

### comment:93 Changed 3 years ago by kcrisman

Thank you, that is SUPER helpful and it's a wonder that I never saw that clear of an explanation before.

So at least one workaround that should work into the future is, as Andrey says, just putting html(stuff) in the last expression always. Maybe this could be in the documentation for html()? Or would that be considered gauche?

### comment:94 Changed 3 years ago by novoselt

That's not at all a workaround if you have code that tries to display multiple HTML bits in different places. And there is (or rather there will be) nothing special about html(x): it returns HTML code similar to latex(x) returning a LaTeX expression, plot(x) returning a graphical object, and, for that matter, str(x) returning a string. All of them just return, but not display by themselves in case you have different plans (e.g. constructing a more complicated object from pieces or saving to a file). If you want to see any, call print or pretty_print. This also helps with the issue of what pretty_print(x) should mean when x can be typeset/plotted/etc - it will do some default, but if you prefer something specific do pretty_print(plot(x)) or pretty_print(html(x)) depending on your needs.

### comment:96 Changed 2 years ago by kcrisman

Something's still up, at least in cell server -

from sage.misc.html import HtmlFragment
pretty_print(HtmlFragment((interacts.calculus.riemann_sum())))


doesn't work. Nor does anything else I tried with the interact library. Also, I'm not sure we need to ask people to import HtmlFragment but I suppose that battle's already lost. Any ideas?

### comment:97 Changed 2 years ago by novoselt

Note that cell server is running 6.9.beta6 (unless you are experimenting with the test one I've set up recently). I am fine with putting HtmlFragment into global space ;-)

### comment:98 Changed 2 years ago by kcrisman

I guess the point being that I can't figure out a way to use things like interacts.calculus.riemann_sum(). Given that it is the only way to now put HTML in notebooks (I guess?) I think HtmlFragment in global namespace is good but I don't know how others would feel about that.

### comment:99 Changed 2 years ago by vbraun

You are supposed to build HtmlFragments with the html() function, it should definitely not be in the global namespace:

sage: sage.misc.html._old_and_deprecated_behavior = False
sage: html('test')
test
sage: type(_)
<class 'sage.misc.html.HtmlFragment'>


### comment:100 Changed 2 years ago by kcrisman

That's fine. I don't actually care about HtmlFragment per se, but I do want the interact library to work in the cell server - but maybe there's no workaround other than building a new display hook there?

Note: See TracTickets for help on using tickets.