Upgrade Sphinx to version 1.1.2

[davidm]

Just made a spkg and a change to sage_autodoc in order to upgrade sphinx to the latest version. Spkg and patch attached.

---

New spkg:

• ​http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-1.1.2.p0.spkg

Apply

• trac_10620-scripts-hgignore.patch​ to the scripts repo
• trac_10620-sageinspect.patch​ to the main Sage library
• trac_10620-sagenb-sageinspect.patch​ to the sagenb repo

[mhansen]

In 4.6.1, Sphinx was upgraded to 1.0.4. Any new spkg should be based on the one found in #10350.

[jason]

Should this ticket be set to "needs review"?

I guess by now sphinx 1.0.7 is released.

[kini]

We might want to wait for 1.1, depending on how long it will take, since 1.1 includes MathJax? (hurrah!).

[jhpalmieri]

The patch for sage_autodoc needs a lot more work: the file sage_autodoc should be kept in line with the file autodoc.py in sphinx/ext. Should we somehow autogenerate this file (using patch) from autodoc.py?

[jhpalmieri]

In fact, maybe we should include sage_autodoc.py in the Sphinx spkg: make a copy of autodoc.py, patch it, and write it in the sphinx/ext directory. That would make it much easier to keep sage_autodoc.py in sync with autodoc.py. Then we just have to make this change in SAGE_ROOT/devel/sage/doc/common/conf.py:

doc/common/conf.py

@@ -13 +13 @@
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sage_autodoc',  'sphinx.ext.graphviz',
+extensions = ['sphinx.ext.sage_autodoc',  'sphinx.ext.graphviz',
               'sphinx.ext.inheritance_diagram', 'sphinx.ext.todo',
               'sphinx.ext.intersphinx']
 # also of interest:

[jason]

#10715 is also an "upgrade sphinx" ticket that might be considered a dup of this ticket.

[leif]

And the spkg has to be rebased on that of #11659 (1.0.4.p7).

[leif]

Replying to leif:

And the spkg has to be rebased on that of #11659 (1.0.4.p7).

Oh, just remembered I had prepared a follow-up Sphinx 1.0.4.p8 spkg at #11665 myself (currently needing review).

[jhpalmieri]

Version 1.0.8 of Sphinx has just been released.

[jhpalmieri]

Okay, I have a new spkg, actually, two. The first ("p0") is just a straightforward upgrade of the previous version. However, we should probably update sage_autodoc.py in light of changes to the Sphinx extension autodoc.py, on which it is based. I can provide a patch to the Sage library modifying sage_autodoc.py if necessary.

A different approach: the second spkg ("p1") autogenerates sage_autodoc.py from autodoc.py, and saves sage_autodoc.py in sphinx/ext instead of in sage/doc/common. So using this one requires a change to sage/doc/common/conf.py, which I've provided. This approach seems as though it will be easier to maintain, or at least it will be easier to keep sage_autodoc.py in line with autodoc.py.

With both of these approaches, I get some warnings about cross-references, which are why the other entries are in the patch for the Sage library. I'm seeing lots of broken cross-references, though: take a look at the reference manual for categories/examples/finite_weyl_groups.py:

    Only the following basic operations are implemented:
     - :meth:`one`
     - :meth:`product`
     - :meth:`simple_reflection`
     - :meth:`Element.has_right_descent`.
{{{
None of these produces a link.  I don't know what's going on.
}}}

[jhpalmieri]

patch for sphinx pkg, for review only

[jhpalmieri]

patch for sphinx pkg, for review only

[jhpalmieri]

By the way, we could use the new spkg (either version) and keep the Sage library intact; this would use the old, and now somewhat out-dated, sage_autodoc.py, but it does not have the cross-reference problems.

[hivert]

Replying to jhpalmieri:

Thanks for building the new spkg !

By the way, we could use the new spkg (either version) and keep the Sage library intact; this would use the old, and now somewhat out-dated, sage_autodoc.py, but it does not have the cross-reference problems.

Concerning cross references, I've been doing some work in #9128. So my suggestion is to upgrade the spkg (I can probably find the time to review it this week) and the keep the cross reference problem for another ticket. What to you think ?

[jhpalmieri]

Replying to hivert:

Replying to jhpalmieri:

Thanks for building the new spkg !

By the way, we could use the new spkg (either version) and keep the Sage library intact; this would use the old, and now somewhat out-dated, sage_autodoc.py, but it does not have the cross-reference problems.

Concerning cross references, I've been doing some work in #9128. So my suggestion is to upgrade the spkg (I can probably find the time to review it this week) and the keep the cross reference problem for another ticket. What to you think ?

It makes me uncomfortable to have sage_autodoc.py so out-of-synch with autodoc.py, and as far as I can tell, changes to autodoc.py, and hence to the updated sage_autodoc.py, are what is causing the problem with the cross-references. So I think just merging the spkg would only be acceptable if the work on #9128 (or elsewhere?) includes updating sage_autodoc.py. What do you think of the idea of putting that file in the Sphinx directory (as in the "p1" version of the spkg) instead of keeping it in the Sage library?

[jdemeyer]

There is now a version 1.1.2 of Sphinx. Probably we should update this spkg.

[jdemeyer]

I updated the "straightforward" spkg, I have no clue what autodoc is, so I left that alone.

[jhpalmieri]

sage_autodoc is the Sage version of the Sphinx extension autodoc (found in sphinx/ext/); it is used to extract the reference manual from the source code. We've modified the original to handle various things (perhaps cython code, lazy methods, cached methods?). So sage_autodoc.py, which lives in devel/sage/doc/common, used to be a simple modification of the Sphinx file autodoc.py, but now the two files differ more dramatically as autodoc.py has been patched in ways that sage_autodoc.py hasn't, and vice versa.

It might be good, to take full advantage of upgrades to Sphinx, to try to remedy this by creating sage_autodoc.py by patching autodoc.py in the Sphinx spkg. Unfortunately, when I tried this approach, the new sage_autodoc module didn't work perfectly, probably because of recent changes in autodoc. This could be dealt with on another ticket. Following Leif's general suggestion about this kind of thing, perhaps we should add a "To do" item to the "Special Build Instructions" section of SPKG.txt. I've prepared a new spkg (based on yours) with this change. The spkg also removes the patch to 'autodoc.py', since I don't think we use that anymore: we use 'sage_autodoc.py' instead.

[jdemeyer]

The following should be added to the scripts .hgignore:

sphinx-apidoc

[jhpalmieri]

I think this is okay now. It's a little more complicated than I thought. First, Sage does use autodoc, in the 'sphinxify' routine when doing introspection. So I restored the autodoc patch. Second, with the new spkg, I was getting doctest failures: one piece of output in sageinspect.py was wrong. I think that the new output is actually better: it's processing a docstring using sphinxify and coming up with nicer output. So I think it's okay to change the output for the doctest.

So there are three or four patches, one for the scripts repo (hgignore), one for the Sage library (doctest), and one for the Sphinx spkg. The patch for the Sage library should come with a corresponding patch for sagenb, but I don't know what to do about that. Submit a ticket for the flask notebook? I'll include one for the current sagenb repo, in case that's the right thing to do.

[jhpalmieri]

patch for sphinx 1.1.2 spkg; for review only

[jhpalmieri]

scripts repo

[jhpalmieri]

sage library repo

[jhpalmieri]

sagenb repo

[jdemeyer]

Replying to jhpalmieri:

I'll include one for the current sagenb repo, in case that's the right thing to do.

Yes, this is the right thing to do (I have discussed with the sagenb developers some time ago).

[jdemeyer]

Looks good to me, but see the encoding issue at #10112.