Upgrade to Sphinx 1.7.5

[jdemeyer]

Tarballs:

• ​https://files.pythonhosted.org/packages/41/32/915efa0e95ef7c79458175b09d9ea9ffc34f4d6791ff84c9b113f3439178/Sphinx-1.7.5.tar.gz

• ​https://files.pythonhosted.org/packages/77/32/439f47be99809c12ef2da8b60a2c47987786d2c6c9205549dd6ef95df8bd/packaging-17.1.tar.gz

• ​https://github.com/sagemath/sagenb/releases/download/1.0.3/sagenb-1.0.3.tar.bz2

Upstream bugs:

• ​https://github.com/sphinx-doc/sphinx/issues/4978 (fixed in release 1.7.5)
• ​https://github.com/sphinx-doc/sphinx/issues/5040 (fixed in master; patch added to Sage)

[jdemeyer]

New commits:

​f5dd6db	added sphinxcontrib_websupport
​8ecd93c	Minor fixes to sphinxcontrib_websupport
​41f114e	Upgrade to Sphinx 1.7.1

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​83682b2

Upgrade to Sphinx 1.7.1

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​4fc721a

Upgrade to Sphinx 1.7.1

[rws]

Can you please give a link to the sphinxcontrib_websupport package, or is it supposed to be installed by other means? An upgrade attempt failed here because the mirrors didn't have it.

[jdemeyer]

Replying to rws:

Can you please give a link to the sphinxcontrib_websupport package, or is it supposed to be installed by other means?

It's linked on the dependency #24972

[rws]

**********************************************************************
File "src/sage/misc/sphinxify.py", line 51, in sage.misc.sphinxify.sphinxify
Failed example:
    sphinxify('**Testing**\n`monospace`')
Expected:
    '...<div class="docstring"...<strong>Testing</strong>\n<span class="math"...</p>\n\n\n</div>'
Got:
    '<div class="docstring">\n    \n  <p><strong>Testing</strong>\n<span class="math notranslate">monospace</span></p>\n\n\n</div>'
**********************************************************************
File "src/sage/misc/sphinxify.py", line 53, in sage.misc.sphinxify.sphinxify
Failed example:
    sphinxify('`x=y`')
Expected:
    '...<div class="docstring">\n    \n  <p><span class="math">x=y</span></p>\n\n\n</div>'
Got:
    '<div class="docstring">\n    \n  <p><span class="math notranslate">x=y</span></p>\n\n\n</div>'
**********************************************************************

[jdemeyer]

I did not consider that Sphinx might affect doctests too.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7419e02

Fix sphinxify doctests

[embray]

Replying to jdemeyer:

I did not consider that Sphinx might affect doctests too.

It definitely can :) There are several places in Python 3 where I've had to make sure to ignore some python 3 specific DeprecationWarnings from Sphinx at import time or else it broke some tests. So I'm not too surprised that a Sphinx upgrade might also affect some docbuild-related tests.

[rws]

It would be great if patchbot could test "unsafe" tickets but I remember about difficulties.

[embray]

Replying to rws:

It would be great if patchbot could test "unsafe" tickets but I remember about difficulties.

I already have a patch for patchbot that will allow it to test tickets with new packages. It's been running on my Cygwin patchbot for a while now, but it does occasionally run into hiccups requiring me to manually restart the patchbot (after cleaning things up a bit).

I especially am not 100% sure why this happens sometimes, since the patchbot creates a copy of SAGE_LOCAL for "unsafe" tickets before doing much else.

[jdemeyer]

Speaking of "unsafe" tickets, I'm not convinced that "unsafe" tickets are on average less safe than "safe" tickets. We certainly have had "safe" tickets messing up a patchbot installation.

[embray]

Yes, it's a pretty flimsy distinction.

[rws]

Replying to embray:

I especially am not 100% sure why this happens sometimes, since the patchbot creates a copy of SAGE_LOCAL for "unsafe" tickets before doing much else.

Please note ​https://github.com/sagemath/sage-patchbot/issues/34

[vbraun]

PDF docs don't build...

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​5bbd500	Upgrade to Sphinx 1.7.4
​e49afe3	Fix sphinxify doctests

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5741161

Run TeX for docbuild non-interactively

[jdemeyer]

Replying to vbraun:

PDF docs don't build...

Indeed... unfortunately it's a genuine problem with Sphinx. I reported it at ​https://github.com/sphinx-doc/sphinx/issues/4978

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​3e86444	Run TeX for docbuild non-interactively
​ae63489	Upgrade to Sphinx 1.7.5
​f4ca3d2	Fix sphinxify doctests

[slelievre]

The Sphinx problem with Brazilian is fixed [0] but now there is a Sphinx problem with Russian [1].

• [0] ​https://github.com/sphinx-doc/sphinx/issues/4978
• [1] ​https://github.com/sphinx-doc/sphinx/issues/5037

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​3e86444	Run TeX for docbuild non-interactively
​7842049	Upgrade to Sphinx 1.7.5
​867372c	Fix sphinxify doctests

[jdemeyer]

This took several iterations to get right, but now the PDF docs build for me.

[fbissey]

Upstream greatly simplified the patch for PR5400 but I'll admit I am unclear if it can be used alone or need some other PR to go with it.

[jdemeyer]

Are you referring to ​https://github.com/sphinx-doc/sphinx/pull/5045?

Anyway, the current branch seems to work so I'd rather keep it.

[fbissey]

Replying to jdemeyer:

Are you referring to ​https://github.com/sphinx-doc/sphinx/pull/5045?

Anyway, the current branch seems to work so I'd rather keep it.

No. 5040 has two commits. Your current patch is the first one. It is described as a temporary solution while the second commit is more "future proofing". But indeed the first commit you are using is supposed to work. Let's send this to the bots as is.

[jdemeyer]

Replying to fbissey:

Your current patch is the first one.

It actually contains both commits.

[fbissey]

Cumulative patches on the same files confuse me no end.

[vbraun]

sage -t --long src/sage/misc/sphinxify.py  # 2 doctests failed
sage -t --long local/lib/python2.7/site-packages/sagenb/misc/sphinxify.py  # 2 doctests failed

[fbissey]

Trivial but we should have seen it in reviews

fbissey@moonloop ~ $ sage -t --long /usr/lib64/python2.7/site-packages/sage/misc/sphinxify.py 
too many failed tests, not using stored timings
Running doctests with ID 2018-06-10-20-16-29-93e55012.
Using --optional=optional,sage
Doctesting 1 file.
sage -t --long /usr/lib64/python2.7/site-packages/sage/misc/sphinxify.py
**********************************************************************
File "/usr/lib64/python2.7/site-packages/sage/misc/sphinxify.py", line 51, in sage.misc.sphinxify.sphinxify
Failed example:
    sphinxify('**Testing**\n`monospace`')
Expected:
    '<div class="docstring"...<strong>Testing</strong>\n<span class="math notranslate"...</p>\n\n\n</div>'
Got:
    '<div class="docstring">\n    \n  <p><strong>Testing</strong>\n<span class="math notranslate nohighlight">monospace</span></p>\n\n\n</div>'
**********************************************************************
File "/usr/lib64/python2.7/site-packages/sage/misc/sphinxify.py", line 53, in sage.misc.sphinxify.sphinxify
Failed example:
    sphinxify('`x=y`')
Expected:
    '<div class="docstring">\n    \n  <p><span class="math notranslate">x=y</span></p>\n\n\n</div>'
Got:
    '<div class="docstring">\n    \n  <p><span class="math notranslate nohighlight">x=y</span></p>\n\n\n</div>'
**********************************************************************
1 item had failures:
   2 of  10 in sage.misc.sphinxify.sphinxify
    [9 tests, 2 failures, 1.02 s]
----------------------------------------------------------------------
sage -t --long /usr/lib64/python2.7/site-packages/sage/misc/sphinxify.py  # 2 doctests failed
----------------------------------------------------------------------
Total time for all tests: 1.0 seconds
    cpu time: 1.0 seconds
    cumulative wall time: 1.0 seconds
fbissey@moonloop ~ $ sage -t --long /usr/lib64/python2.7/site-packages/sagenb/misc/sphinxify.py 
too many failed tests, not using stored timings
Running doctests with ID 2018-06-10-20-17-44-197b0e5d.
Using --optional=optional,sage
Doctesting 1 file.
sage -t --long /usr/lib64/python2.7/site-packages/sagenb/misc/sphinxify.py
**********************************************************************
File "/usr/lib64/python2.7/site-packages/sagenb/misc/sphinxify.py", line 68, in sagenb.misc.sphinxify.sphinxify
Failed example:
    sphinxify('**Testing**\n`monospace`')
Expected:
    '...<div class="docstring"...<strong>Testing</strong>\n<span class="math"...</p>\n\n\n</div>'
Got:
    '<div class="docstring">\n    \n  <p><strong>Testing</strong>\n<span class="math notranslate nohighlight">monospace</span></p>\n\n\n</div>'
**********************************************************************
File "/usr/lib64/python2.7/site-packages/sagenb/misc/sphinxify.py", line 70, in sagenb.misc.sphinxify.sphinxify
Failed example:
    sphinxify('`x=y`')
Expected:
    '...<div class="docstring">\n    \n  <p><span class="math">x=y</span></p>\n\n\n</div>'
Got:
    '<div class="docstring">\n    \n  <p><span class="math notranslate nohighlight">x=y</span></p>\n\n\n</div>'
**********************************************************************
1 item had failures:
   2 of  10 in sagenb.misc.sphinxify.sphinxify
    [14 tests, 2 failures, 0.74 s]
----------------------------------------------------------------------
sage -t --long /usr/lib64/python2.7/site-packages/sagenb/misc/sphinxify.py  # 2 doctests failed
----------------------------------------------------------------------
Total time for all tests: 1.2 seconds
    cpu time: 0.7 seconds
    cumulative wall time: 0.7 seconds

That unfortunately requires a new version of sagenb. @Dima could we get a 1.0.3 for this.

[dimpase]

OK, so that's just doctests in sagenb, right? No problem, I'll fix these. ​https://github.com/sagemath/sagenb/issues/445

[jdemeyer]

Replying to fbissey:

Trivial but we should have seen it in reviews

Right. I just did not think about running doctests after fixing the PDF documentation stuff.

[fbissey]

Replying to dimpase:

OK, so that's just doctests in sagenb, right? No problem, I'll fix these. ​https://github.com/sagemath/sagenb/issues/445

The commit works for me. We just have to reproduce it in sage's sphinxify.py.

[dimpase]

here is the new sagenb ​https://github.com/sagemath/sagenb/releases/download/1.0.3/sagenb-1.0.3.tar.bz2

Could you bump up the version in build/pkgs/sagenb/ etc to include it on this ticket?

[fbissey]

Replying to dimpase:

here is the new sagenb ​https://github.com/sagemath/sagenb/releases/download/1.0.3/sagenb-1.0.3.tar.bz2

Could you bump up the version in build/pkgs/sagenb/ etc to include it on this ticket?

Definitely. I am leaving it to Jeroen as the ticket author, unless he indicates he is too busy to handle it in which case I will step up.

[jdemeyer]

Replying to fbissey:

I am leaving it to Jeroen as the ticket author, unless he indicates he is too busy to handle it in which case I will step up.

It's literally my job to do this...

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​e664816	Run TeX for docbuild non-interactively
​1a073de	Upgrade to Sphinx 1.7.5
​4c0bc88	Upgrade sagenb
​e398cf8	Fix sphinxify doctests

[fbissey]

Works for me.

[chapoton]

Damn. Could we get rid of the "import __builtin__" in sagenb/misc/sphinxify.py and sagenb/misc/support.py ??

This is breaking the compatibility of sagenb with python3...

[jdemeyer]

Replying to chapoton:

Damn. Could we get rid of the "import __builtin__" in sagenb/misc/sphinxify.py and sagenb/misc/support.py ??

Perhaps, but not in this ticket please.

[fbissey]

Replying to jdemeyer:

Replying to chapoton:

Damn. Could we get rid of the "import __builtin__" in sagenb/misc/sphinxify.py and sagenb/misc/support.py ??

Perhaps, but not in this ticket please.

Totally agree with Jeroen, another ticket please.

That being said, historically we have migrated sphinxify.py from sagenb into sage to reduce dependency on sagenb (that's what I remember at least). If you want to work on that, I would really suggest to try eliminating the sphinxify.py of sagenb and use sage's one instead.

[vbraun]

File "src/doc/common/conf.py", line 627, in doc.common.conf.call_intersphinx
Failed example:
    for line in open(thematic_index).readlines():
        if "padics" in line:
            sys.stdout.write(line)
Expected:
    <li><a class="reference external" href="../reference/padics/sage/rings/padics/tutorial.html#sage-rings-padics-tutorial" title="(in Sage Reference Manual: p-Adics ...)"><span>Introduction to the -adics</span></a></li>
Got:
    <li><a class="reference external" href="../reference/padics/sage/rings/padics/tutorial.html#sage-rings-padics-tutorial" title="(in Sage Reference Manual: p-Adics)"><span>Introduction to the -adics</span></a></li>
**********************************************************************
1 item had failures:
   1 of   4 in doc.common.conf.call_intersphinx
    [3 tests, 1 failure, 0.61 s]

[fbissey]

I have seen it too in the vbraun branch and it confused me quite a bit because I don't remember seeing it the first time round but it should have been there. It seems like before this ticket the title field ends up with v as in title="(in Sage Reference Manual: p-Adics v) not just for this particular one that's tested but for all of them. After this ticket it is reduced to just title="(in Sage Reference Manual: p-Adics) and the v has disappeared everywhere.

[jdemeyer]

It's supposed to say v8.3.beta5 so that's a regression somehow.

[jdemeyer]

I opened #25570 for the version number regression.

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. This was a forced push. New commits:

​3b57916	Fix version number in built documentation
​b321995	Run TeX for docbuild non-interactively
​8a71e29	Upgrade to Sphinx 1.7.5
​a22da72	Upgrade sagenb
​4e066ba	Fix sphinxify doctests