Opened 11 years ago

Closed 10 years ago

#6586 closed enhancement (fixed)

[with spkg, positive review] update Sphinx to version 0.6.2 or 0.6.3

Reported by: jhpalmieri Owned by: mabshoff
Priority: major Milestone: sage-4.2
Component: packages: standard Keywords:
Cc: mhansen, sage-combinat Merged in: sage-4.2.alpha0
Authors: John Palmieri Reviewers: John Cremona, Tim Dumol, Mitesh Patel
Report Upstream: Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jhpalmieri)

As the summary says. The spkg can be found at

http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.3.p1.spkg

After installing it and before building any docs, you will probably need to delete the directory SAGE_ROOT/devel/sage/doc/output -- this holds cached information which may not be compatible between different versions of Sphinx.

Sphinx 0.6.3 also uses Jinja2. Sage includes Jinja, not Jinja2. As long as you're connected to a network, the installation of Sphinx 0.6.3 downloads Jinja2 and installs it -- it can coexist with Jinja, so it shouldn't mess anything up. If you're not connected to a network, and for putting in the Sage tar ball, there is an spkg for Jinja2:

http://sage.math.washington.edu/home/palmieri/SPKG/jinja2-2.1.1.p0.spkg

For producing a new Sage tarball, since we're adding a new spkg, I think we need to modify the files spkg/install and spkg/standard/deps; I've attached new versions of these.

Finally, the patch below ("inheritance_example.patch") is not required. It provides an illustration of a new capability of Sphinx 0.6.3 (and I think it's quite nice), but it's not something that needs to be installed or reviewed for this ticket.

Attachments (5)

inheritance_example.patch (1.6 KB) - added by mpatel 10 years ago.
Example inheritance diagram for "The Steenrod algebra". This is just for fun.
install (6.1 KB) - added by jhpalmieri 10 years ago.
new version the file spkg/install (incorporating jinja2)
deps (12.4 KB) - added by jhpalmieri 10 years ago.
new version of the file spkg/standard/deps (incorporating jinja2)
install.patch (250 bytes) - added by jhpalmieri 10 years ago.
diff between old and new install
deps.patch (1.4 KB) - added by jhpalmieri 10 years ago.
diff between old and new deps

Download all attachments as: .zip

Change History (67)

comment:1 follow-up: Changed 11 years ago by jhpalmieri

Oh, I forgot: you need to apply the patch at #6585 for the reference manual to build. This version of Sphinx also prints a lot of new warnings, too.

comment:2 Changed 11 years ago by mvngu

  • Summary changed from [with spkg; needs review] update Sphinx to version 0.6.2 to [with spkg, needs work] update Sphinx to version 0.6.2

I installed the updated SPKG with

./sage -f http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.2.p0.spkg

and applied the patch at #6585. When doing

./sage -docbuild reference html

I got the following errors:

mvngu@sage sage-4.1.1.alpha0-sage.math.washignton.edu-x86_64-Linux]$ ./sage -docbuild reference html
Traceback (most recent call last):
  File "/scratch/mvngu/sage-4.1.1.alpha0-sage.math.washignton.edu-x86_64-Linux/devel/sage/doc/common/builder.py", line 667, in <module>
    getattr(get_builder(name), type)()
  File "/scratch/mvngu/sage-4.1.1.alpha0-sage.math.washignton.edu-x86_64-Linux/devel/sage/doc/common/builder.py", line 348, in _wrapper
    for module_name in self.get_modified_modules():
  File "/scratch/mvngu/sage-4.1.1.alpha0-sage.math.washignton.edu-x86_64-Linux/devel/sage/doc/common/builder.py", line 412, in get_modified_modules
    env = self.get_sphinx_environment()
  File "/scratch/mvngu/sage-4.1.1.alpha0-sage.math.washignton.edu-x86_64-Linux/devel/sage/doc/common/builder.py", line 403, in get_sphinx_environment
    return BuildEnvironment.frompickle(config, env_pickle)
  File "/scratch/mvngu/sage-4.1.1.alpha0-sage.math.washignton.edu-x86_64-Linux/local/lib/python2.6/site-packages/Sphinx-0.6.2-py2.6.egg/sphinx/environment.py", line 210, in frompickle
    env = pickle.load(picklefile)
AttributeError: 'module' object has no attribute 'RedirStream'

comment:3 Changed 11 years ago by jhpalmieri

I think I got this error, too. Once I deleted doc/output, things worked for me. (I think there is something stored in the old version of the documentation which confuses the new version of Sphinx.)

comment:4 Changed 11 years ago by jhpalmieri

  • Summary changed from [with spkg, needs work] update Sphinx to version 0.6.2 to [with spkg, needs review] update Sphinx to version 0.6.2

comment:5 Changed 11 years ago by mvngu

  • Cc sage-combinat added

The upgraded Sphinx SPKG can't handle binary encodings in the file:

sage/combinat/partition.py

In particular, the following sections from that source file:
line:2233

sage: p = loads('x\x9c\x85\x8e\xbd\xaa\xc2@\x10\x85\x89\xff.>\xc4\x94\xda\x04\x15|\x04\xb1\xb1\x90\x0b[\x87I\x18\x935\xc9\xae\xb33\xda\t\xd7\xc2\xf76"biw\x0e\x9c\x9f\xef\xbfW\x08\x96\x94\x16\xa1\xcd\x9dGM\xcf\x18\xd5\xa9\x0b\xde\x1c>Jv\x91PIt\xbf\xcd|m8Y\xdc\xb9w\xe3\xfe\xdc&\xf5\xbb\x1d\x9d/%u^\xa9\xa4hZ\xac)\xfb\x18\x1e\xd8d\xfd\xf8\xe3\xa1\x1df\x1e[\xe2\x91\xdd|\x97!\x1ca\xb5\x84\n\xaf\xdd\x02\xbc\xbe\x05\x1a\x12\x01\xad\xd0C\x88@|\xc1\x064\xc0\x9a\xc7v\x16\xf2\x13\x15\x9a\x15\r\x8a\xf0\xe47\xf9;ixj\x13_u \xd8\x81\x98K\x9e>\x01\x13iVH')


line:2294

sage: p = loads('x\x9c\x85\x8e\xbd\x0e\x820\x14\x85\x03\x8a?\x8d\x0f\xd1Q\x97\x06y\x07\xe3\xaa&\x9d\xc9\xa5\xb9\x96\n\xb4^Z\xdcLt\xf0\xbd\xc5 qt;\'9?\xdf#V\x1e4\n\xe5\x9a\xc2X\x08\xe2\nm0\xc18\xcb\x0e\xa3\xf2\xfb\x16!\xa0\x0f\xbbcn+F\xd1\xe6I\xf1\x9d&k\x19UC\xbb5V{al@\x8d-k\xa0\xc2|44\x95Q\xf6:Q"\x93\xdcB\x834\x93\xe9o\x99\xbb3\xdf\xa6\xbc\x84[\xbf\xc0\xf5\xf7\x87\x7f 8R\x075\x0f\x8eg4\x97+W\\P\x85\\\xd5\xe0=-\xfeC\x0fIFK\x19\xd9\xb2g\x80\x9e\x81u\x85x\x03w\x0eT\xb1')


line:2839

sage: dmp = 'x\x9ck`J.NLO\xd5K\xce\xcfM\xca\xccK,\xd1+H,*\xc9,\xc9\xcc\xcf\xe3\n\x80\xb1\x8a\xe3\x93\x81DIQbf^I1W!\xa3fc!Sm!\xb3F(7\x92x!Km!k(GnbE<\xc8\x88B6\x88\xb9E\x99y\xe9\xc5z@\x05\xa9\xe9\xa9E\\\xb9\x89\xd9\xa9\xf10N!{(\xa3QkP!Gq(c^\x06\x90c\x0c\xe4p\x96&\xe9\x01\x00\xc2\xe53\xfd'

When these lines are removed, the HTML version of the reference manual builds OK. So a possible fix for line 2233 is to replace everything in the TEST block starting from that line by:

sage: p = PartitionsGreatestLE(10,2)
sage: p == loads(dumps(p))
True

We can do something similar for line 2294; replace everything in the TEST block starting from line 2294 by:

sage: p = PartitionsGreatestEQ(10,2)
sage: p == loads(dumps(p))
True

As for the TEST block starting from line 2839, I have no idea how to fix it so that the reference manual would successfully builds. Someone on the sage-combinat team to the rescue?

comment:6 in reply to: ↑ 1 Changed 11 years ago by jhpalmieri

Replying to jhpalmieri:

Oh, I forgot: you need to apply the patch at #6585 for the reference manual to build. This version of Sphinx also prints a lot of new warnings, too.

I think that this patch fixes the issue with binary encodings; at least it did for me.

comment:7 follow-up: Changed 11 years ago by mvngu

With the patch at #6585 and the SPKG, building the reference manual on sage.math takes much longer than previously.

comment:8 in reply to: ↑ 7 Changed 11 years ago by jhpalmieri

Replying to mvngu:

With the patch at #6585 and the SPKG, building the reference manual on sage.math takes much longer than previously.

I agree: on my mac, it took 28 minutes to build the html version with the old one, and 43 minutes with the new one. With the --jsmath option on, the old one took 6 minutes and the new one took 15 minutes. I was getting similar comparisons on sage.math. (This is with me deleting devel/sage/doc/output between each build, to level the playing field.)

It's strange: for the tutorial and developer's guide, building is about the same or slightly faster with the new version. I also made a modified version of the developer's guide by duplicating some sections (so that the resulting PDF file was 559 pages); the new version was slightly faster there, too. Maybe it's all of the math in the reference manual? Maybe it's the large number of files? Maybe it's the autogeneration of files?

comment:9 Changed 10 years ago by mpatel

Cursory profiling with #6187's "testreference" target indicates that the rewritten autodoc extension is significantly slower than before. I think autodoc now also uses far more memory. Attempting to build the full reference manual on my machine, Sphinx "stalled" at 97% of modules read. I think the kernel was too busy swapping everything but Sphinx to disk.

Perhaps there's a more efficient, parallelizable representation. Can we subclass and/or Cythonize sphinx.ext.autodoc.Documenter?

comment:10 Changed 10 years ago by mpatel

Minor progress: It seems that a custom autodoc-skip-member method now overrides Sphinx's default behavior, which is to skip private methods (__init__, etc.). To keep these out of the reference manual, we might use, e.g.,

def skip_NestedClass(app, what, name, obj, skip, options):
    """docstring"""
    skip_nested = str(obj).find("sage.misc.misc") != -1 and name.find("MainClass.NestedClass") != -1
    return skip or skip_nested

in conf.py. Unfortunately, this does not significantly reduce the build time or memory usage.

comment:11 Changed 10 years ago by mpatel

Apparently, the culprit is sphinx.pycode's Python source analyzer, which sphinx.ext.autodoc uses to extract attribute docstrings. From Sphinx-0.6.2-py2.6.egg/sphinx/pycode/__init__.py:

    The docstrings can either be in special '#:' comments before the assignment
    or in a docstring after it.

According to the analyzer, there are just two places to find such docstrings in the Sage library:

  • combinat/designs/block_design.py
  • schemes/elliptic_curves/ell_field.py

Still, resource use is high, because there is a lot of caching going on. We can reduce it with weak references:

--- __init__.py.orig    2009-08-01 09:58:27.303927239 -0700
+++ __init__.py 2009-08-01 11:35:17.477865562 -0700
@@ -120,10 +120,11 @@ class PycodeError(Exception):
             res += ' (exception was: %r)' % self.args[1]
         return res
 
-
+import weakref
 class ModuleAnalyzer(object):
     # cache for analyzer objects -- caches both by module and file name
-    cache = {}
+#    cache = {}
+    cache = weakref.WeakValueDictionary()
 
     @classmethod
     def for_string(cls, string, modname, srcname='<string>'):

This does reduce build time and memory usage, but I don't know if the output is identical.

comment:12 Changed 10 years ago by mpatel

For sage -docbuild reference html --jsmath, at least, the results are close:

$ du -s output_v0*
95550   output_v051
97392   output_v062
97388   output_v062_mod

The differences are in environmental.pickle or stem from using a function as a default argument in

  • combinat/words/word_content.py
  • rings/polynomial/groebner_fan.py

comment:13 Changed 10 years ago by mpatel

Clearly, it's time for a break.

comment:14 Changed 10 years ago by jhpalmieri

Nice observation. Making the change with weak_references does indeed speed things up:

0.5.1     28 minutes
0.6.2     43 minutes
0.6.2     33 minutes   (version using weakref)

With no other changes, it includes all of the private methods, as already pointed out (and I assume it did before using weakref -- this may account for some of the increased time, since it's producing more output), but it also includes (for some classes, anyway), an entry for __weakref)_. See the documentation for sage/algebras/steenrod_algebra.py, for instance: it lists as an attribute of SteenrodAlgebraFactory?

__weakref__
   list of weak references to the object (if defined)

comment:15 Changed 10 years ago by mpatel

Actually, noticing __weakref__ for that module is what led to the comment about autodoc-skip-member. I think this was before I "patched" Sphinx, but I may be mistaken. I'm also not sure about why __weakref__ appears spontaneously. Perhaps a super class?

comment:16 Changed 10 years ago by mpatel

See #6664 for the autodoc-skip-member issue.

comment:17 follow-up: Changed 10 years ago by mpatel

Also, I've asked about the merits of the weakref workaround on sphinx-dev.

Changed 10 years ago by mpatel

Example inheritance diagram for "The Steenrod algebra". This is just for fun.

comment:18 Changed 10 years ago by mpatel

The new inheritance diagram extension has potential. I've attached a minimal example. Please do not merge it.

The options are limited, however. For example, I could not find a directive that auto-generates a diagram for each module. Also, it seems there's no way to color-code the nodes/classes, e.g., to indicate their modular provenance. Maybe it's possible to do one or more of these ourselves, in builder.py, conf.py, or by patching Sphinx.

comment:19 Changed 10 years ago by mpatel

I should add that this requires a local installation of GraphViz.

comment:20 in reply to: ↑ 17 ; follow-up: Changed 10 years ago by mpatel

Replying to mpatel:

Also, I've asked about the merits of the weakref workaround on sphinx-dev.

The thread. Preliminary tests indicate that the proposed alternative does not reduce memory use significantly.

comment:21 in reply to: ↑ 20 Changed 10 years ago by mpatel

Replying to mpatel:

Replying to mpatel:

Also, I've asked about the merits of the weakref workaround on sphinx-dev.

The thread. Preliminary tests indicate that the proposed alternative does not reduce memory use significantly.

Here, actually.

comment:22 follow-up: Changed 10 years ago by mpatel

There's been no further activity about the workaround on sphinx-dev. Since weakrefs appear to work for us, should we add a patch to the spkg?

comment:23 in reply to: ↑ 22 ; follow-up: Changed 10 years ago by mvngu

Replying to mpatel:

There's been no further activity about the workaround on sphinx-dev. Since weakrefs appear to work for us, should we add a patch to the spkg?

Hmm... could you make an spkg with that workaround?

comment:24 in reply to: ↑ 23 Changed 10 years ago by jhpalmieri

Replying to mvngu:

Replying to mpatel:

There's been no further activity about the workaround on sphinx-dev. Since weakrefs appear to work for us, should we add a patch to the spkg?

Hmm... could you make an spkg with that workaround?

I can certainly do it, but not until Monday.

comment:25 Changed 10 years ago by mpatel

Does the problem described here recur with the new spkg?

comment:26 Changed 10 years ago by jhpalmieri

The new spkg is at

http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.2.p1.spkg

(The old one is still there; it's "...p0.spkg", as listed in the description above.)

comment:27 Changed 10 years ago by jhpalmieri

By the way, note that since this includes the private methods in the reference manual, there are a lot of new warnings when building that document. (People haven't written all of the docstrings for their __init__ methods, etc., to conform with proper reST format, since they didn't need to before.) These should be dealt with in a separate ticket: otherwise, we might feel obliged to keep rebasing this ticket until it was merged.

comment:28 Changed 10 years ago by mpatel

Does #6664 suppress the private methods?

comment:29 Changed 10 years ago by jhpalmieri

Does #6664 suppress the private methods?

Yes, but do we *want* to suppress the private methods? I actually prefer that they be included in the reference manual.

comment:30 Changed 10 years ago by jhpalmieri

Uh oh, it seems as though with this spkg, changes to pyx files are not detected. That is, build the documentation (say, with the jsmath option). Then edit a .py file and a .pyx file (I chose sage/misc/latex.py and sage/matrix/matrix0.pyx). Run 'sage -docbuild reference print_modified_modules': the .py file will be listed, but not the .pyx file. I can't figure out why this is happening :(

comment:31 Changed 10 years ago by mpatel

I also don't know why. builder.py uses Sphinx's get_outdated_files() method to get an iterable of modified modules. However, this method's code is the same in v0.5.1 and v0.6.2. I did find that the newer Sphinx does not include .so files for .pyx files in environment.pickle's dependencies dictionary. This may "explain" the observed behavior.

comment:32 Changed 10 years ago by jhpalmieri

I played around with the Sphinx code for a while, and I could not figure out what's going on. Well, I believe that "matrix0.so" was listed as a dependency of "matrix0.pyx" in version 0.5.1, so changing the .pyx file resulted in the modification time for the .so file to be wrong, triggering a rebuild of the docs, but in 0.6.2, the .so file no longer appears to be a dependency of the .pyx file. I've posted a message to sphinx-dev about how dependencies are determined; we'll see if there are any helpful responses.

comment:33 Changed 10 years ago by jhpalmieri

Okay, I now have a patch which seems to recognize changes to pyx files. The dependency information is stored in the pickle, so to test this, first delete SAGEDOC/output/doctrees/. The new spkg is available from http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.2.p2.spkg.

Please test thoroughly. (One thing to watch out for: I thought it wasn't working because I was changing sage/matrix/action.pyx and seeing no notification when the reference manual was built, but then I realized that action.pyx isn't even included in the reference manual. Doh.)

comment:34 Changed 10 years ago by jhpalmieri

In case anyone downloaded the version posted yesterday, there is a slightly modified version today: the author of Sphinx posted a patch for the Cython problem to the Sphinx mercurial site, and so the version today uses that patch instead. (Mine was similar to his, but why not use the official one?)

I've replaced the old one with the new one, so the link is the same: http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.2.p2.spkg.

comment:35 Changed 10 years ago by jhpalmieri

  • Summary changed from [with spkg, needs review] update Sphinx to version 0.6.2 to [with spkg, needs review] update Sphinx to version 0.6.2 or 0.6.3

Okay, here's another one. Sphinx was just updated to version 0.6.3, and this is supposed to fix our problems. In my testing, it doesn't have the Cython problem that the original version of 0.6.2 had; it incorporates the patch in the most recent 0.6.2 spkg. The 0.6.3 spkg is a bit slower than 0.5.1 and also than our version of 0.6.2 with the unauthorized weakref patch. It's not nearly as slow and doesn't seem to use nearly as much memory as the unpatched 0.6.2. Here's the spkg: http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.3.p0.spkg.

Some timings in minutes for "sage -docbuild reference html --jsmath", with the patch at #6664 applied:

                    Mac      sage.math  ubuntu
Sphinx 0.5.1         6           5        10
Sphinx 0.6.2.p2      6           6        10
Sphinx 0.6.3         8           9        11

"Mac" is my Intel iMac running OS X 10.5; "sage.math" is sage.math, and "ubuntu" is an old ubuntu box I have access to.

As I said above, the memory usage seems to stay within reason for all of these, but I'm not using anything sophisticated to test this: with the terrible version, my computer would become very sluggish, and with the versions tested here, that doesn't happen.

comment:36 follow-up: Changed 10 years ago by mpatel

For Sage 4.1.2.alpha0's HTML reference manual, at least, the new spkg seems to work just as Georg indicated. I'll try to do more testing soon.

By the way, it might be useful to update spkg-install along the lines of #6598 and this example. For example,

#!/usr/bin/env bash

if [ -z "$SAGE_LOCAL" ]; then
   echo "SAGE_LOCAL undefined ... exiting";
   echo "Maybe run 'sage -sh'?"
   exit 1
fi

success() {
    if [ $? -ne 0 ]; then
        echo "Error building Sphinx: '$1'"
        exit 1
    fi
}

set -e

echo "Removing old version..."
rm -rf "$SAGE_LOCAL"/lib/python/site-packages/Sphinx-*
success 'deleting previous version'

[...]

comment:37 in reply to: ↑ 36 Changed 10 years ago by jhpalmieri

Replying to mpatel:

By the way, it might be useful to update spkg-install along the lines of #6598 and this example. For example,

Okay, try this one: http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.3.p1.spkg. (Putting "set -e" in other places made the script exit without printing the custom error message. With this one, if, for example, I make the directory "$SAGE_LOCAL"/lib/python/site-packages/Sphinx-* undeletable, I get the appropriate message.)

comment:38 Changed 10 years ago by mpatel

Thanks very much for pointing that out. I may need to update the spkgs at #5447...

comment:39 Changed 10 years ago by timdumol

Forgive me if I'm mistaken, but doesn't Sphinx>=0.6 depend on Jinja2?

Incompatible changes:

  • Templating now requires the Jinja2 library, which is an enhanced version of the old Jinja1 engine.

http://sphinx.pocoo.org/changes.html#release-0-6-mar-24-2009

comment:40 follow-up: Changed 10 years ago by mpatel

  • Authors set to John Palmieri

Yes. I think site-packages/Jinja2-2.1.1-py2.6-linux-x86_64.egg, say, is installed automatically (via network) by python setup.py install in spkg-install.

Should we make it a separate spkg, to supplement or replace jinja-1.2.p0.spkg?

comment:41 Changed 10 years ago by mpatel

  • Reviewers set to Mitesh Patel

I checked that Sphinx now incorporates changes to included .py and .pyx files.

For the HTML builder, both jsmath and pngmath modes appear to work as expected.

The PDF reference manual's table of contents is missing. This may stem from #6885 and my invoking LaTeX's batch mode with b.

comment:42 in reply to: ↑ 40 Changed 10 years ago by timdumol

Replying to mpatel:

Yes. I think site-packages/Jinja2-2.1.1-py2.6-linux-x86_64.egg, say, is installed automatically (via network) by python setup.py install in spkg-install.

Should we make it a separate spkg, to supplement or replace jinja-1.2.p0.spkg?

I think it would probably best to make a separate spkg so that installation will not require network access. I've made an spkg that I can post if you want.

Also, the spkg for this states that it depends on Jinja>=1.1, which is no longer the case.

comment:43 Changed 10 years ago by jhpalmieri

Okay, I've updated the file SPKG.txt (in http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.3.p1.spkg) to say that it depends on Jinja >= 2. If you want to post a link to an spkg for jinja, that would be great.

comment:44 Changed 10 years ago by timdumol

comment:45 Changed 10 years ago by jhpalmieri

A few issues with the Jinja spkg: the source is under version control but it shouldn't be. When I install it, I get jinja2-2.1.1, not jinja2-2.2, so the name is wrong. Also, the part of the spkg-install file which is supposed to remove the old version wasn't removing the right thing (it said "Jinja-2*", but the directory is actually "Jinja2-2.1.1*").

Anyway, here's a modified spkg, much smaller since src is not in the mercurial repository. This is an updated version of the version 1 jinja spkg, with the new src dropped in, replacing the old src: http://sage.math.washington.edu/home/palmieri/SPKG/jinja2-2.1.1.p0.spkg.

Should this have its own ticket? Will it screw up anything from #6568? I don't know anything about Jinja, so I don't know what to do to test it; all I see is that the docs are built correctly with the new versions of Sphinx and Jinja.

comment:46 Changed 10 years ago by timdumol

Thanks for the fixes.

Jinja and Jinja2 are totally different, and so it shouldn't affect any current templating routines. In any case, once Sphinx 0.6.3 is included, we can remove the dependency on Jinja by replacing all instances of import jinja with import jinja2, etc. The API's are pretty much unchanged, and from what I know, none of the changed interfaces are currently used.

comment:47 Changed 10 years ago by cremona

I'm not at all sure what you people are doing here, so perhaps I should have kept away, but (in trying to help with #6820) I tried installing the new sphinx spkg using

./sage -f http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.2.p0.spkg

and now

john@ubuntu%sage -docbuild reference html
Traceback (most recent call last):
  File "/home/john/sage-4.1.1/devel/sage/doc/common/builder.py", line 667, in <module>
    getattr(get_builder(name), type)()
  File "/home/john/sage-4.1.1/devel/sage/doc/common/builder.py", line 348, in _wrapper
    for module_name in self.get_modified_modules():
  File "/home/john/sage-4.1.1/devel/sage/doc/common/builder.py", line 412, in get_modified_modules
    env = self.get_sphinx_environment()
  File "/home/john/sage-4.1.1/devel/sage/doc/common/builder.py", line 403, in get_sphinx_environment
    return BuildEnvironment.frompickle(config, env_pickle)
  File "/home/john/sage-4.1.1/local/lib/python2.6/site-packages/Sphinx-0.6.2-py2.6.egg/sphinx/environment.py", line 210, in frompickle
    env = pickle.load(picklefile)
AttributeError: 'module' object has no attribute 'RedirStream'

This is in the main branch, so a vanilla 4.1.1 build. Should that be 4.1.2.alpha0?

comment:48 Changed 10 years ago by timdumol

Hey.

The first spkg had problems. There are newer spkgs. http://sage.math.washington.edu/home/palmieri/SPKG/sphinx-0.6.3.p1.spkg is the latest.

comment:49 Changed 10 years ago by jhpalmieri

  • Description modified (diff)

To cremona: sorry about that. When you switch versions of Sphinx, you have to delete the old output in the docs directory -- it caches some information there, apparently in ways which are not compatible between versions. So after you install the spkg, delete the directory SAGE_ROOT/devel/sage/doc/output and try again. I should change the summary on this ticket to reflect this...

To timdumol: I see, jinja and jinja2 can coexist on the same system, so for now we can have both. That's the best solution now, I agree. (I tried replacing all of the "import jinja" lines with "import jinja2", but there was a problem with one of them: in sage.ext.gen_interpreters, the line from jinja.datastructure import ComplainingUndefined doesn't translate, because there is no jinja2.datastructure.)

comment:50 Changed 10 years ago by cremona

Thanks -- that worked! (Except that I cannot read and deleted all SAGE_ROOT/devel/sage/doc and not just the output directory, so had to copy the rest from another clone...)

I like the percentage meter and the fact that the output overwrites one line rather than running off the screen. Minor point here: it does not overwrite long lines when they are followed by short ones so you see strange garbage characters on the end of te line like this:

reading sources... [ 49%] sage/misc/explain_pickle      _densensephismsmpecies

Of course this is a triviality.

Otherwise all went well and (from the point of view of an ordinary user/developer) I'm happy with this spkg. (There are lots of warnings and at least one error in the docstrings, but that is another matter). I have no objection to this getting a positive review, assuming tha the other issues discussed above have been dealt with.

Changed 10 years ago by jhpalmieri

new version the file spkg/install (incorporating jinja2)

Changed 10 years ago by jhpalmieri

new version of the file spkg/standard/deps (incorporating jinja2)

Changed 10 years ago by jhpalmieri

diff between old and new install

Changed 10 years ago by jhpalmieri

diff between old and new deps

comment:51 Changed 10 years ago by jhpalmieri

  • Description modified (diff)

comment:52 Changed 10 years ago by mpatel

Both the latest Sphinx and Jinja2 packages appear to install without network activity. The changes to install and deps look good to me, but I haven't done a complete installation from scratch.

To the extent that it counts, my review is positive.

comment:53 Changed 10 years ago by timdumol

  • Reviewers changed from Mitesh Patel to Mitesh Patel, Tim Dumol
  • Summary changed from [with spkg, needs review] update Sphinx to version 0.6.2 or 0.6.3 to [with spkg, partially positive review, needs review] update Sphinx to version 0.6.2 or 0.6.3

Documentation builds perfectly. Sphinx 0.6.3 takes only 2 minutes more sys time than 0.5.1 to build all the docs, for me.

Haven't done a complete install yet, but I'll do so once I finish downloading the 4.1.2.alpha2 tarball.

comment:54 Changed 10 years ago by mpatel

  • Reviewers changed from Mitesh Patel, Tim Dumol to John Cremona, Tim Dumol, Mitesh Patel
  • Summary changed from [with spkg, partially positive review, needs review] update Sphinx to version 0.6.2 or 0.6.3 to [with spkg, positive review] update Sphinx to version 0.6.2 or 0.6.3

I updated spkg/install and spkg/standard/deps, replaced the old Sphinx spkg with the new one in spkg/standard, and copied the Jinja2 package to spkg/standard. With these changes, sage -bdist 100 generates the expected binary distributions on sage.math from both binary and compiled source distributions. Moreover, sage -sdist 200 makes a source distribution that compiles as expected.

There's a failed test:

mpatel@sage sage-200$ ./sage -t devel/sage/sage/misc/trace.py 
sage -t  "devel/sage/sage/misc/trace.py"                    
**********************************************************************
File "/scratch/mpatel/source/sage-200/devel/sage/sage/misc/trace.py", line 61:
    sage: print s.before[s.before.find('-'):]
Expected:
    ---...
    ipdb> c
    2 * 5
Got:
    --Call--
    > /scratch/mpatel/source/sage-200/local/lib/python2.6/site-packages/sage/rings/arith.py(1944)factor()
       1943 
    -> 1944 def factor(n, proof=None, int_=False, algorithm='pari', verbose=0, **kwds):
       1945     """
    
    ipdb> c
    2 * 5
    <BLANKLINE>
**********************************************************************
1 items had failures:
   1 of  10 in __main__.example_1
***Test Failed*** 1 failures.
For whitespace errors, see the file /scratch/mpatel/source/sage-200/tmp/.doctest_trace.py
         [3.7 s]
exit code: 1024
 
----------------------------------------------------------------------
The following tests failed:


        sage -t  "devel/sage/sage/misc/trace.py"
Total time for all tests: 3.7 seconds

I think it's not related to this ticket.

comment:55 follow-up: Changed 10 years ago by mpatel

Problem: Somehow, Sphinx 0.6.3 breaks the sidebar toggle (cf. #6507) in the live docs. In Firebug's console:

unexpected end of XML source
<span class="math">('#searchbox').show(0);

comment:56 follow-up: Changed 10 years ago by mpatel

Patching Sphinx 0.6.3 should fix this and perhaps also a similar problem reported on sage-devel:

  • sphinx/themes/basic/layout.html

    old new  
    8181              {{ _('Enter search terms or a module, class or function name.') }}
    8282              </p>
    8383          </div>
    84           <script type="text/javascript">$('#searchbox').show(0);</script>
     84          <script type="text/javascript">jQuery('#searchbox').show(0);</script>
    8585          {%- endif %}
    8686          {%- endblock %}
    8787        </div>

comment:57 Changed 10 years ago by mpatel

Or perhaps not.

comment:58 in reply to: ↑ 56 Changed 10 years ago by timdumol

mpatel: The problem with the sidebar is caused by compression. Line comments ("") are used in the sidebar code, therefore when the code is compressed, the entire function is commented out. Changing the line comments to block comments ("/* */") fixed the problem for me.

comment:59 Changed 10 years ago by timdumol

A patch for the doc sidebar problem is now up at #7126.

comment:60 Changed 10 years ago by mpatel

If this comment at #7126 is correct, we should still patch sphinx/themes/basic/layout.html. I can do this.

comment:61 in reply to: ↑ 55 Changed 10 years ago by timdumol

Replying to mpatel:

Problem: Somehow, Sphinx 0.6.3 breaks the sidebar toggle (cf. #6507) in the live docs. In Firebug's console:

unexpected end of XML source
<span class="math">('#searchbox').show(0);

#7141 adds a patch that fixes this.

comment:62 Changed 10 years ago by mhansen

  • Merged in set to sage-4.2.alpha0
  • Resolution set to fixed
  • Status changed from positive_review to closed

Merged both the jinja2 and sphinx in spkg.

Note: See TracTickets for help on using tickets.