Opened 4 years ago

Closed 3 years ago

# KeyError raised by GraphLatex.dot2tex_picture when edge_labels=True

Reported by: Owned by: slabbe tbd major sage-6.2 packages: optional dot2tex, graph drawing stumpc5, ncohen Sébastien Labbé Christian Stump N/A 2634500 (Commits) 26345009b7d0063c3cbb24b5cbf54f8a911b28fb

The following is ok :

sage: G = DiGraph()
sage: G.set_latex_options(format='dot2tex')
sage: view(G)


but with edges_labels=True a KeyError is raised :

sage: G.set_latex_options(edge_labels=True)
sage: view(G)
ERROR    Failed to process input
Traceback (most recent call last):
File "/Users/slabbe/Applications/sage-5.2/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 2939, in main
s =  conv.convert(dotdata)
File "/Users/slabbe/Applications/sage-5.2/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 794, in convert
return self.do_preview_preproc()
File "/Users/slabbe/Applications/sage-5.2/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 1167, in do_preview_preproc
hp,dp,wt = pp.texdims[name]
KeyError: '3333880'


The error is also raised by dot2tex_picture method:

sage: G = DiGraph()
sage: G.latex_options().dot2tex_picture()
'\n\\begin{tikzpicture}[>=latex,line join=bevel,]\n%%\n\\node (1) at (6bp,7bp) [draw,draw=none] {$1$};\n  \\node (0) at (6bp,57bp) [draw,draw=none] {$0$};\n  \\draw [black,->] (0) ..controls (6bp,44.053bp) and (6bp,33.103bp)  .. (1);\n%\n\\end{tikzpicture}\n'
.dot2tex_picture()
sage: G.set_latex_options(edge_labels=True)
sage: G.latex_options().dot2tex_picture()
ERROR    Failed to process input
Traceback (most recent call last):
File "/Users/slabbe/Applications/sage-5.2/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 2939, in main
s =  conv.convert(dotdata)
File "/Users/slabbe/Applications/sage-5.2/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 794, in convert
return self.do_preview_preproc()
File "/Users/slabbe/Applications/sage-5.2/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 1167, in do_preview_preproc
hp,dp,wt = pp.texdims[name]
KeyError: '010'
''


This is due to dot2tex currently not supporting '\verb' properly.

The attached patch works around this by replacing \verb by \text, and stripping out tricky characters like ^ and _.

### comment:1 Changed 4 years ago by slabbe

• Description modified (diff)

### comment:2 Changed 4 years ago by slabbe

I managed to fix the bug with the following hack :

• ## src/dot2tex/dot2tex.py

diff --git a/src/dot2tex/dot2tex.py b/src/dot2tex/dot2tex.py
 a To see what happened, run dot2tex with t for name,item in usededges.items(): edge = item break hp,dp,wt = pp.texdims[name] xmargin, ymargin = self.get_margins(edge) labelcode = '<<'\

but I still don't know what was this loop for... because the output is OK (with edge labels).

Last edited 4 years ago by slabbe (previous) (diff)

### comment:3 Changed 4 years ago by slabbe

I just created an issue on the dot2tex google code project :

### comment:4 Changed 4 years ago by nthiery

• Description modified (diff)
• Status changed from new to needs_review

### comment:5 Changed 4 years ago by nthiery

Hi Sébastien,

I had not noticed this ticket. A quick and dirty workaround has been in the Sage-Combinat queue for some time. I did some further cleanup, but it's still nothing but a workaround. Shall we get it into Sage? If you think the patch could use more work, do you mind taking it over?

Cheers,

Nicolas

### comment:6 Changed 4 years ago by nthiery

While I was at it, I folded in a couple tests I had put in a later patch in the Sage-Combinat queue.

### comment:8 Changed 4 years ago by nthiery

• Description modified (diff)

### comment:9 Changed 4 years ago by nthiery

The change verb -> text in sage.misc.latex was breaking quite a few doctests. I made it a separate ticket: #14256. This one should not depend on it.

On the other hand, I added three tiny things that were in a later patch in the queue:

• shape=plain text whenever there are labels on the vertices (latex or not)
• trim "." from keys (I guess I must have had a problem with them at some point)
• a doctest fix.
Last edited 4 years ago by nthiery (previous) (diff)

### comment:11 Changed 4 years ago by nthiery

• Authors set to Nicolas M. Thiéry

### comment:12 Changed 4 years ago by nthiery

Hi Nathann!

Would you have time to review this one?

Thanks!

Nicolas

### comment:13 Changed 4 years ago by ncohen

This is what happens when I install graphviz

[...]
make[3]: *** No rule to make target ../../plugin/pango/libgvplugin_pango.la', needed by dot_builtins'.  Stop.
make[3]: *** Waiting for unfinished jobs....
mv -f .deps/dot_builtins.Tpo .deps/dot_builtins.Po
mv -f .deps/dot.Tpo .deps/dot.Po
make[3]: Leaving directory /home/ncohen/.Sage/spkg/build/graphviz-2.16.1.p0/src/cmd/dot'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory /home/ncohen/.Sage/spkg/build/graphviz-2.16.1.p0/src/cmd'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory /home/ncohen/.Sage/spkg/build/graphviz-2.16.1.p0/src'
make: *** [all] Error 2
Error building Graphviz

real	0m53.650s
user	0m45.475s
sys	0m8.313s
************************************************************************
Error installing package graphviz-2.16.1.p0
************************************************************************
explaining the problem and including the relevant part of the log file
/home/ncohen/.Sage/spkg/logs/graphviz-2.16.1.p0.log
Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/home/ncohen/.Sage/spkg/build/graphviz-2.16.1.p0 and type 'make' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
(cd '/home/ncohen/.Sage/spkg/build/graphviz-2.16.1.p0' && '/home/ncohen/.Sage/sage' -sh)
When you are done debugging, you can type "exit" to leave the subshell.
************************************************************************


Nathann

### comment:14 follow-ups: ↓ 15 ↓ 16 Changed 4 years ago by nthiery

Hi!

Yes, the graphviz spkg is broken (and as far as I can remember has always been). But most linux distro have a graphviz package, so that's not too bad. The doc already says to install graphviz out of Sage.

We should probably remove this broken spkg from "experimental" ...

Cheers,

Nicolas

### comment:15 in reply to: ↑ 14 Changed 4 years ago by ncohen

Yes, the graphviz spkg is broken (and as far as I can remember has always been). But most linux distro have a graphviz package, so that's not too bad. The doc already says to install graphviz out of Sage.

......

who the hell put that in the doc without removing the package ?...

Nathann

### comment:16 in reply to: ↑ 14 ; follow-up: ↓ 17 Changed 4 years ago by ncohen

We should probably remove this broken spkg from "experimental" ...

Indeed. The only thing to do is update the doc to remove any mention of this graphviz package, and ask Jeroen in a ticket to remove it from experimental.

And it will be quickly reviewed, I swear.

Nathann

### comment:17 in reply to: ↑ 16 ; follow-up: ↓ 18 Changed 4 years ago by nthiery

We should probably remove this broken spkg from "experimental" ...

Indeed. The only thing to do is update the doc to remove any mention of this graphviz package, and ask Jeroen in a ticket to remove it from experimental.

I was already careful *not* to mention the spkg in the doc (but if you see how to improve the graphviz installation instructions, please go ahead in a separate ticket; I'll review it).

As for removing the spkg, this is now: #14398.

### comment:18 in reply to: ↑ 17 ; follow-up: ↓ 22 Changed 4 years ago by ncohen

I was already careful *not* to mention the spkg in the doc (but if you see how to improve the graphviz installation instructions, please go ahead in a separate ticket; I'll review it).

Is written explicitly anywhere that there is no spkg and that the users should install it by themselves ?

Nathann

### comment:19 Changed 4 years ago by jdemeyer

• Status changed from needs_review to needs_work
# optional - requires dot2tex and graphviz


should be

# optional - dot2tex graphviz


### comment:20 follow-up: ↓ 25 Changed 4 years ago by ncohen

(and while you're editing the patch, you included a http link which Sphinx does not understand as one)

Nathann

### comment:21 Changed 4 years ago by ncohen

5 doctests fail on my machine. The error message do not seem to say that there is something wrong with my install (dot or graphviz which I installed through my distro).

sage -t --long dot2tex_utils.py
**********************************************************************
File "dot2tex_utils.py", line 11, in sage.graphs.dot2tex_utils
Failed example:
output = dot2tex.dot2tex(graph, format="positions", prog="neato") # optional - requires dot2tex and graphviz
Expected nothing
Got:
ERROR    Failed to process input
Traceback (most recent call last):
File "/home/ncohen/.Sage/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 2928, in main
s =  conv.convert(dotdata)
File "/home/ncohen/.Sage/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 848, in convert
return self.output()
File "/home/ncohen/.Sage/local/lib/python2.7/site-packages/dot2tex/dot2tex.py", line 2566, in output
positions[node.name] = map(int, pos.split(','))
ValueError: invalid literal for int() with base 10: '87.137'
**********************************************************************


Nathann

### comment:22 in reply to: ↑ 18 ; follow-up: ↓ 23 Changed 4 years ago by nthiery

I was already careful *not* to mention the spkg in the doc (but if you see how to improve the graphviz installation instructions, please go ahead in a separate ticket; I'll review it).

Is written explicitly anywhere that there is no spkg and that the users should install it by themselves ?

It's written that the soft can be downloaded from the graphviz website. This seems explicit enough to me, but if you want to improve that, go ahead (but I would not speak of spkg at all).

### comment:23 in reply to: ↑ 22 ; follow-up: ↓ 24 Changed 4 years ago by ncohen

It's written that the soft can be downloaded from the graphviz website.

Where ?

Nathann

### comment:24 in reply to: ↑ 23 ; follow-up: ↓ 27 Changed 4 years ago by nthiery

It's written that the soft can be downloaded from the graphviz website.

Where ?

Nathann

See the method layout_graphviz in generic_graph.

### comment:25 in reply to: ↑ 20 Changed 4 years ago by nthiery

(and while you're editing the patch, you included a http link which Sphinx does not understand as one)

Done. And added the file to the index.rst. And misc cleanup of the docstrings.

### comment:26 Changed 4 years ago by nthiery

• Status changed from needs_work to needs_review

### comment:27 in reply to: ↑ 24 ; follow-up: ↓ 29 Changed 4 years ago by ncohen

• Status changed from needs_review to needs_work

Cool.

See the method layout_graphviz in generic_graph.

Do we both agree that this is nowhere sufficient ?

Here is what happens when some code that requires graphviz (and which does not use layout_graphviz) is executed wihout graphviz installed

sage: sage: import dot2tex                                                    # optional - dot2tex graphviz ## line 10 ##
sage: sage: output = dot2tex.dot2tex(graph, format="positions", prog="neato") # optional - dot2tex graphviz ## line 11 ##
ERROR    Could not locate Graphviz binaries
ERROR    Failed to create xdotdata. Is Graphviz installed?
ERROR    Failed to parse the input data. Is it a valid dot file?
Try to input xdot data directly. Example:
dot -Txdot file.dot | dot2tex > file.tex

If this does not work, check that you have an updated version of PyParsing and
Graphviz. Users have reported problems with old versions. You can also run
dot2tex in debug mode using the --debug option:
dot2tex --debug file.dot
A file dot2tex.log will be written to the current directory with detailed
information useful for debugging.
An exception has occurred, use %tb to see the full traceback.

SystemExit: 1

To exit: use 'exit', 'quit', or Ctrl-D


The error message should redirect the user toward a way to install graphviz I guess. And I have never met "SystemExit" By the way, do we really need pyparsing ? I thought this did not belong to Sage, and I needed it a couple of days ago.

I reset this ticket to needs_work because the doctests from two messages above still do not pass, even with graphviz installed.

Nathann

### comment:28 Changed 4 years ago by nthiery

5 doctests fail on my machine. The error message do not seem to say that there is something wrong with my install (dot or graphviz which I installed through my distro).

This is a bug in dot2tex. It's fixed in the dev branch since two years. I was waiting the official release to make a new spkg, but it seems to be taking some time ...

Oh well, I uploaded my dev spkg and updated the instructions in #7004. And updated a bit the docstring in the patch as well.

Note: I had to wipe by hand the dot2tex directory before reinstalling dot2tex for things to work:

rm -rf /opt/sage-5.9.beta2/local/lib/python2.7/site-packages/dot2tex


### comment:29 in reply to: ↑ 27 ; follow-up: ↓ 31 Changed 4 years ago by nthiery

See the method layout_graphviz in generic_graph.

Do we both agree that this is nowhere sufficient ?

Here is what happens when some code that requires graphviz (and which does not use layout_graphviz) is executed wihout graphviz installed

sage: sage: import dot2tex                                                    # optional - dot2tex graphviz ## line 10 ##
sage: sage: output = dot2tex.dot2tex(graph, format="positions", prog="neato") # optional - dot2tex graphviz ## line 11 ##
ERROR    Could not locate Graphviz binaries
ERROR    Failed to create xdotdata. Is Graphviz installed?
ERROR    Failed to parse the input data. Is it a valid dot file?
Try to input xdot data directly. Example:
dot -Txdot file.dot | dot2tex > file.tex

If this does not work, check that you have an updated version of PyParsing and
Graphviz. Users have reported problems with old versions. You can also run
dot2tex in debug mode using the --debug option:
dot2tex --debug file.dot
A file dot2tex.log will be written to the current directory with detailed
information useful for debugging.
An exception has occurred, use %tb to see the full traceback.

SystemExit: 1

To exit: use 'exit', 'quit', or Ctrl-D


The error message should redirect the user toward a way to install graphviz I guess. And I have never met "SystemExit"

I agree this is ugly, but that's a problem with dot2tex, not with Sage (feel free to report!). Any Sage code using dot2tex should call assert_have_doc2tex.

By the way, do we really need pyparsing ? I thought this did not belong to Sage, and I needed it a couple of days ago.

No clue.

### comment:30 Changed 4 years ago by nthiery

• Status changed from needs_work to needs_review

### comment:31 in reply to: ↑ 29 Changed 4 years ago by ncohen

I agree this is ugly, but that's a problem with dot2tex, not with Sage (feel free to report!). Any Sage code using dot2tex should call assert_have_doc2tex.

This one doesn't ?

Nathann

### comment:32 Changed 4 years ago by ncohen

Oh. dot2tex is a Python spkg. Explains a lot, sorry :-P

Nathann

### comment:33 follow-up: ↓ 37 Changed 4 years ago by ncohen

Anyway why do you set this ticket to needs_review. I said twice that the doctests do not pass even with graphviz installed.

Nathann

### comment:34 Changed 4 years ago by ncohen

• Description modified (diff)
• Status changed from needs_review to needs_work

### comment:35 follow-up: ↓ 39 Changed 4 years ago by ncohen

See the method layout_graphviz in generic_graph.

Do we both agree that this is nowhere sufficient ?

Nathann

### comment:36 Changed 4 years ago by slabbe

Nicolas, could you add the following doctest to your patch to make sure the issue do not come up again in the future?

We make sure :trac:13624 is fixed::

sage: G = DiGraph()
sage: G.set_latex_options(format='dot2tex')
sage: G.set_latex_options(edge_labels=True)
sage: G.latex_options().dot2tex_picture()  # optional - dot2tex graphviz
\begin{tikzpicture}
first_line?
...
last_line?
\end{tikzpicture}


I don't really know where to put the information about where and how to download graphviz and dot2tex. In fact, there are many entry points for the user:

• G.set_latex_options
• G.latex_options
• G.latex_options().dot2tex_picture
• G.latex_options().set_option <- it might be strange, but a lot of doc is in here
• G.latex_options().set_options
• G.latex_options().latex

It is hard to know where the user will first have the idea that installing graphviz + dot2tex could be nice. Personnaly, I found once that information here :

• G.layout_graphviz?

Here is what it says :

This requires "graphviz" and the "dot2tex" spkg. Here are some
installation tips:

* Install graphviz >= 2.14 so that the programs dot, neato, ...
http://graphviz.org.

http://trac.sagemath.org/sage_trac/ticket/7004 and install it
with "sage -i dot2tex-2.8.?.spkg"


This does not speak about any graphviz spkg. In fact, I did not know there was one. When I saw that doc in the past, I was then able to install graphviz properly.

The information about the dot2tex spgk is obsolete. I suggest to replace it by :

   * Get the name of the most recent version of dot2tex from
the sage command optional_packages(). Then,
install it with sage -i dot2tex-2.8.?.spkg.


### comment:37 in reply to: ↑ 33 ; follow-up: ↓ 42 Changed 4 years ago by nthiery

Anyway why do you set this ticket to needs_review. I said twice that the doctests do not pass even with graphviz installed.

Have you installed the new spkg as I pointed out in my previous comment?

### comment:38 follow-up: ↓ 40 Changed 4 years ago by nthiery

Ok guys, I have used up my time on this patch. I need to move on. Feel free to takeover.

Nathann: if you want patches to get in Sage quickly, don't keep expanding tickets beyond their original scope.

### comment:39 in reply to: ↑ 35 ; follow-up: ↓ 41 Changed 4 years ago by nthiery

See the method layout_graphviz in generic_graph.

Do we both agree that this is nowhere sufficient ?

Certainly not great. Certainly out of scope for this ticket.

### comment:40 in reply to: ↑ 38 Changed 4 years ago by slabbe

Ok guys, I have used up my time on this patch. I need to move on. Feel free to takeover.

Sorry, I will propose a patch soon which apply on yours that will implement what I suggested. I can't do it now as the version of Sage I have now (sage-5.7.beta4) is too old: I get a conflict with the patch.

Sébastien

### comment:41 in reply to: ↑ 39 Changed 4 years ago by ncohen

Do we both agree that this is nowhere sufficient ?

Certainly not great. Certainly out of scope for this ticket.

Then find somebody else to review it. I refuse to be part of this mess.

Nathann

### comment:42 in reply to: ↑ 37 Changed 4 years ago by slabbe

Ils sont fous ces romains...

Have you installed the new spkg as I pointed out in my previous comment?

I think it is better not to update the old ticket #7004. Here is the link to the new spkg:

sage -f http://sage.math.washington.edu/home/nthiery/dot2tex-2.8.7-dev.spkg


### comment:43 follow-up: ↓ 44 Changed 4 years ago by slabbe

I just added a patch which fixes the broken doctests seen by the patchbot. I also added a doctest that make sure the issue of this ticket is solved. I also fixed an # optional - dot2tex, graphviz which was not up to date.

Finally I realize that some optional test are broken. Like this one that we can see in the doc of layout_graphviz. This may be considered in another ticket.

sage: g = digraphs.ButterflyGraph(2)
sage: g.plot(layout = "graphviz", prog = "neato") # optional - dot2tex, graphviz
Traceback (most recent call last):
...
ValueError: invalid literal for int() with base 10: '67.297'


### comment:44 in reply to: ↑ 43 Changed 4 years ago by nthiery

• Status changed from needs_work to needs_review

I just added a patch which fixes the broken doctests seen by the patchbot. I also added a doctest that make sure the issue of this ticket is solved. I also fixed an # optional - dot2tex, graphviz which was not up to date.

Thanks Sébastien! I looked at your reviewer patch, and it looks good to me. Let's see what the patchbot says. Back to needs review for the first patch.

Finally I realize that some optional test are broken. Like this one that we can see in the doc of layout_graphviz. This may be considered in another ticket.

sage: g = digraphs.ButterflyGraph(2)
sage: g.plot(layout = "graphviz", prog = "neato") # optional - dot2tex, graphviz
Traceback (most recent call last):
...
ValueError: invalid literal for int() with base 10: '67.297'


+1 since the tests were already failing before this ticket. This is now #14408 which is about updating the dot2tex spkg.

### comment:45 Changed 4 years ago by novoselt

For the record, current installation instructions for dot2tex are outdated (I tried the example from the description and it failed at first).

With the current patch at #14382, the example from the description does not give errors.

### Changed 4 years ago by slabbe

Adding a doctest to make sure the initial issue is fixed

### comment:46 follow-up: ↓ 47 Changed 4 years ago by slabbe

I just updated my patch which adds a doctest. Even if #14382 finally fix the problem, I think it could be merged anyway.

I removed the part that was fixing dot2tex optional doctests errors. I will move this part in another patch that I am going to post on #14408 where similar issues were discussed.

### comment:47 in reply to: ↑ 46 Changed 4 years ago by novoselt

Even if #14382 finally fix the problem,

Feel free to give it a try and set to positive review - it is not big ;-)

### comment:48 Changed 4 years ago by jdemeyer

• Milestone changed from sage-5.11 to sage-5.12

### comment:49 Changed 3 years ago by vbraun_spam

• Milestone changed from sage-6.1 to sage-6.2

### comment:50 follow-up: ↓ 51 Changed 3 years ago by tscrim

• Status changed from needs_review to needs_info

Currently in 6.2.rc0, the example given in the description works (probably from #14382):

sage: G = DiGraph()
sage: G.set_latex_options(format="dot2tex", edge_labels=True)
sage: latex(G)

\begin{tikzpicture}[>=latex,line join=bevel,]
%%
\node (node_1) at (11.000000bp,79.000000bp) [draw,draw=none] {$333$};
\node (node_0) at (11.000000bp,7.000000bp) [draw,draw=none] {$88$};
\draw [black,->] (node_1) ..controls (11.000000bp,61.481000bp) and (11.000000bp,39.548000bp)  .. (node_0);
\definecolor{strokecol}{rgb}{0.0,0.0,0.0};
\pgfsetstrokecolor{strokecol}
\draw (38.000000bp,43.000000bp) node {$\text{\texttt{my{\char\_}label}}$};
%
\end{tikzpicture}


So how much of this ticket should we keep or should we just close as a "works-for-me"?

Last edited 3 years ago by tscrim (previous) (diff)

### comment:51 in reply to: ↑ 50 Changed 3 years ago by slabbe

So how much of this ticket should we keep or should we just close as a "works-for-me"?

As I suggested in a previous comment, I think we could merge the patch trac_13624-review-sl.patch which simply adds a doctest.

Sébastien

### comment:52 Changed 3 years ago by slabbe

• Authors changed from Nicolas M. Thiéry to Sébastien Labbé
• Branch set to u/slabbe/13624
• Commit set to 26345009b7d0063c3cbb24b5cbf54f8a911b28fb
• Status changed from needs_info to needs_review

I just converted my old hg patch to a git branch. It should be easy to review as it only adds a doctest since the problem was fixed by #14382.

New commits:

 ​2634500 adding a doctest to make sure 13624 is fixed

### comment:53 Changed 3 years ago by stumpc5

• Reviewers set to Christian Stump
• Status changed from needs_review to positive_review

### comment:54 Changed 3 years ago by vbraun

• Branch changed from u/slabbe/13624 to 26345009b7d0063c3cbb24b5cbf54f8a911b28fb
• Resolution set to fixed
• Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.