py3: support <type '...'> expected output in doctests on Python 3

[embray]

On Python 3 all classes, including built-in types, are represented as <class '...'> instead of <type '...'> as Python 2 uses for built-in types.

This normalizes the expected doctest output on Python 3 to account for this difference.

Whenever Sage switches to Python 3 as a default it would be simple to also reverse the sense of this normalization, and update the doctests to expect <class '...'>.

This replaces #24228, #24229, #24230, and #24233.

[jdemeyer]

Replying to embray:

Whenever Sage switches to Python 3 as a default it would be simple to also reverse the sense of this normalization, and update the doctests to expect <class '...'>.

Wouldn't it make sense to do that now, i.e. to replace <type by <class ? That would be more future-proof.

[embray]

Replying to jdemeyer:

Replying to embray:

Whenever Sage switches to Python 3 as a default it would be simple to also reverse the sense of this normalization, and update the doctests to expect <class '...'>.

Wouldn't it make sense to do that now, i.e. to replace <type by <class ? That would be more future-proof.

*nod* that was my original plan, but I decided:

a) This would be a less disruptive change for now--just a small change in one module vs. a small change in many

b) For the time being Python 2 is still the default for Sage, so changing all the existing tests (particularly "EXAMPLE" docs) could confuse readers.

[jdemeyer]

Lots of breakage on the patchbot.

[jdemeyer]

Replying to embray:

a) This would be a less disruptive change for now--just a small change in one module vs. a small change in many

If you plan a disruptive change anyway when we switch to Python 3, that argument makes little sense. Why is a disruptive change later better than a disruptive change now.

b) For the time being Python 2 is still the default for Sage, so changing all the existing tests (particularly "EXAMPLE" docs) could confuse readers.

Fair enough.

[embray]

I believe most of the (relevant) failures are actually due to #24257, where they've since been fixed. I can rebase this on top of that....

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​6a9e6cb	Remove 'from __future__ import unicode_literals from some modules.
​1add357	Add a doctest output fixup to replace <type '...'> with <class '...'> so that
​e12b67f	Fix docstring--the normalization is currently not performed on Python 2

[embray]

A similar, though not exactly the same difference I've noticed is that on Python 3 unbound methods of classes (which are just normal functions) are repr'd with their __qualname__ which includes the class name. E.g.

>>> class Foo:
...     def a(self): pass
...
>>> Foo.a
<function Foo.a at 0x7fe3d04c9b20>

So some tests that happen to rely on such output fail on python 3.

The tests could be rewritten, but it might be nice to automatically account for this difference. In principle there's a little advantage to having the __qualname__ in the output since we can also also see in what class the function was defined, but in the meantime we don't lose anything over what we already have by ignoring it.

I could either add that to this ticket, or add a new one. It might be worth adding to this one since the same issue applies to nested classes.

[embray]

Replying to embray:

I could either add that to this ticket, or add a new one. It might be worth adding to this one since the same issue applies to nested classes.

I implemented a workaround for this, but in implementation it's distinct enough from this issue that I'll just open a separate ticket for this once I'm confident that it's ready.

[embray]

I don't think this actually needs any new work--it's been working fine for me on my python 3 branch for some time. The previous patchbot failures were due to #24257 which is now closed.

[embray]

Would be nice to get this done--I have a number of other test output normalizations that I've added since this one.

[jdemeyer]

Please add a test showing how normalize_type_repr works when given a more complicated expression. Something like:

sage: L = [Integer, float]
sage: normalize_type_repr(repr(L))

[embray]

Sure--let me make sure that actually works in the first place (it should--I think I've already encountered some examples like that)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6760db0

Add an additional test for normalize_type_repr

[jdemeyer]

Patchbot complains about docbuild.

[jdemeyer]

If I understand things correctly, this ticket ensures that both the following doctests would pass on Python 3:

sage: int
<class 'int'>

and

sage: int
<type 'int'>

but on Python 2 only

sage: int
<type 'int'>

would be accepted. Is that right?

I would very much prefer to accept <class 'int'> on Python 2 too. That way, we could at least make new doctests more future-proof by using <class instead of <type.

[chapoton]

I fixed the doc

---

New commits:

​476a27b	Merge branch 'u/embray/python3/type-repr-fixups' in 8.2.b2
​514ed93	trac 24258 make doc build

[chapoton]

Replying to jdemeyer:

If I understand things correctly, this ticket ensures that both the following doctests would pass on Python 3:

sage: int
<class 'int'>

and

sage: int
<type 'int'>

but on Python 2 only

sage: int
<type 'int'>

would be accepted. Is that right?

I would very much prefer to accept <class 'int'> on Python 2 too. That way, we could at least make new doctests more future-proof by using <class instead of <type.

We can easily make new doctests future-proof by simply using <... 'int'>. There is a patchbot plugin that checks for that.

[embray]

Replying to jdemeyer:

I would very much prefer to accept <class 'int'> on Python 2 too. That way, we could at least make new doctests more future-proof by using <class instead of <type.

I thought I already addressed this, but it doesn't make enormous sense to write new doctests against Python 3-style output as long as Sage's default python remains python 2--that's what the documentation should be written against. When Sage switches to python 3 as the default python, then it makes sense to make mass replacements on the documentation too, and not until then.

I wouldn't be opposed to making the output checker flexible in both cases, but I don't think there's any good reason to right now either.

[chapoton]

This seems not to work on

sage -t --long src/sage/rings/universal_cyclotomic_field.py  # 1 doctest failed

with sage 8.4.b4.

[embray]

Replying to chapoton:

This seems not to work on

sage -t --long src/sage/rings/universal_cyclotomic_field.py  # 1 doctest failed

with sage 8.4.b4.

It's because the <type ...> repr is split onto multiple lines, which is not supported. You could either change the regexp to be multi-line (but this introduces more complications), or just fix that test to not split <type ...> across lines.