Py3: ValueError in graph_generators doctests with plantri optional package

[vklein]

sage -i plantri
sage -t --long src/sage/graphs/graph_generators.py

**********************************************************************
File "src/sage/graphs/graph_generators.py", line 1641, in sage.graphs.graph_generators.GraphGenerators.?
Failed example:
    for i in range(12, 23):                                             # optional plantri
        L = len(list(graphs.triangulations(i, minimum_connectivity=5))) # optional plantri
        print("{}   {:3d}".format(i,L))                                 # optional plantri
Exception raised:
    Traceback (most recent call last):
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.graph_generators.GraphGenerators.?[7]>", line 2, in <module>
        L = len(list(graphs.triangulations(i, minimum_connectivity=Integer(5)))) # optional plantri
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/graph_generators.py", line 1746, in triangulations
        for G in graphs._read_planar_code(sp.stdout):
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/graph_generators.py", line 1150, in _read_planar_code
        G.set_embedding(embed_g)
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 2444, in set_embedding
        self._check_embedding_validity(embedding, boolean=False)
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 2551, in _check_embedding_validity
        raise ValueError("{} and {} are not neighbors but {} is in the list associated with {}".format(u, v, u, v))
    ValueError: 10 and 6 are not neighbors but 10 is in the list associated with 6
**********************************************************************
File "src/sage/graphs/graph_generators.py", line 1673, in sage.graphs.graph_generators.GraphGenerators.?
Failed example:
    [len(g) for g in graphs.triangulations(9, minimum_degree=4, minimum_connectivity=3, dual=True)]  # optional plantri
Exception raised:
    Traceback (most recent call last):
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.graph_generators.GraphGenerators.?[11]>", line 1, in <module>
        [len(g) for g in graphs.triangulations(Integer(9), minimum_degree=Integer(4), minimum_connectivity=Integer(3), dual=True)]  # optional plantri
      File "<doctest sage.graphs.graph_generators.GraphGenerators.?[11]>", line 1, in <listcomp>
        [len(g) for g in graphs.triangulations(Integer(9), minimum_degree=Integer(4), minimum_connectivity=Integer(3), dual=True)]  # optional plantri
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/graph_generators.py", line 1746, in triangulations
        for G in graphs._read_planar_code(sp.stdout):
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/graph_generators.py", line 1150, in _read_planar_code
        G.set_embedding(embed_g)
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 2444, in set_embedding
        self._check_embedding_validity(embedding, boolean=False)
      File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/generic_graph.py", line 2551, in _check_embedding_validity
        raise ValueError("{} and {} are not neighbors but {} is in the list associated with {}".format(u, v, u, v))
    ValueError: 10 and 9 are not neighbors but 10 is in the list associated with 9

Additionally we get the following errors with benzene and buckygen packages:

sage -i buckygen
sage -i benzene

sage -t --long src/sage/features/graph_generators.py
too many failed tests, not using stored timings
Running doctests with ID 2019-08-19-15-57-32-a8119f22.
Git branch: develop
Using --optional=benzene,buckygen,build,cmake,dochtml,gap_packages,libsemigroups,memlimit,mpir,plantri,primecount,python2,sage
Doctesting 1 file.
sage -t --long src/sage/features/graph_generators.py
**********************************************************************
File "src/sage/features/graph_generators.py", line 68, in sage.features.graph_generators.Buckygen
Failed example:
    Buckygen().is_present()  # optional: buckygen
Exception raised:
    Traceback (most recent call last):
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features.graph_generators.Buckygen[1]>", line 1, in <module>
        Buckygen().is_present()  # optional: buckygen
      File "sage/misc/cachefunc.pyx", line 2311, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (build/cythonized/sage/misc/cachefunc.c:12712)
        self.cache = f(self._instance)
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/__init__.py", line 129, in is_present
        res = self._is_present()
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/__init__.py", line 353, in _is_present
        return self.is_functional()
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/graph_generators.py", line 101, in is_functional
        if lines.find(expected) == -1:
    TypeError: argument should be integer or bytes-like object, not 'str'
**********************************************************************
File "src/sage/features/graph_generators.py", line 90, in sage.features.graph_generators.Buckygen.is_functional
Failed example:
    Buckygen().is_functional()  # optional: buckygen
Exception raised:
    Traceback (most recent call last):
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features.graph_generators.Buckygen.is_functional[1]>", line 1, in <module>
        Buckygen().is_functional()  # optional: buckygen
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/graph_generators.py", line 101, in is_functional
        if lines.find(expected) == -1:
    TypeError: argument should be integer or bytes-like object, not 'str'
**********************************************************************
File "src/sage/features/graph_generators.py", line 116, in sage.features.graph_generators.Benzene
Failed example:
    Benzene().is_present()  # optional: benzene
Exception raised:
    Traceback (most recent call last):
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features.graph_generators.Benzene[1]>", line 1, in <module>
        Benzene().is_present()  # optional: benzene
      File "sage/misc/cachefunc.pyx", line 2311, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (build/cythonized/sage/misc/cachefunc.c:12712)
        self.cache = f(self._instance)
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/__init__.py", line 129, in is_present
        res = self._is_present()
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/__init__.py", line 353, in _is_present
        return self.is_functional()
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/graph_generators.py", line 150, in is_functional
        if not lines.startswith(expected):
    TypeError: startswith first arg must be bytes or a tuple of bytes, not str
**********************************************************************
File "src/sage/features/graph_generators.py", line 138, in sage.features.graph_generators.Benzene.is_functional
Failed example:
    Benzene().is_functional()  # optional: benzene
Exception raised:
    Traceback (most recent call last):
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features.graph_generators.Benzene.is_functional[1]>", line 1, in <module>
        Benzene().is_functional()  # optional: benzene
      File "/home/vklein/odk/sage/local/lib/python3.7/site-packages/sage/features/graph_generators.py", line 150, in is_functional
        if not lines.startswith(expected):
    TypeError: startswith first arg must be bytes or a tuple of bytes, not str
**********************************************************************
4 items had failures:
   1 of   3 in sage.features.graph_generators.Benzene
   1 of   3 in sage.features.graph_generators.Benzene.is_functional
   1 of   3 in sage.features.graph_generators.Buckygen
   1 of   3 in sage.features.graph_generators.Buckygen.is_functional
    [18 tests, 4 failures, 0.08 s]
----------------------------------------------------------------------
sage -t --long src/sage/features/graph_generators.py  # 4 doctests failed
----------------------------------------------------------------------
Total time for all tests: 0.1 seconds
    cpu time: 0.1 seconds
    cumulative wall time: 0.1 seconds

sage -t --long src/sage/graphs/graph_generators.py 
...
   3 items had failures:
   2 of  41 in sage.graphs.graph_generators.GraphGenerators.?
   6 of  11 in sage.graphs.graph_generators.GraphGenerators.fullerenes
   3 of   8 in sage.graphs.graph_generators.GraphGenerators.fusenes
    [139 tests, 11 failures, 8.87 s]
----------------------------------------------------------------------
sage -t --long src/sage/graphs/graph_generators.py  # 11 doctests failed
----------------------------------------------------------------------
Total time for all tests: 8.9 seconds
    cpu time: 8.8 seconds
    cumulative wall time: 8.9 seconds

[dcoudert]

I did the following test, corresponding to the first failing doctest.

sage: def test(version=2):
....:     import subprocess
....:     command = "plantri -m5c5 16"
....:     enc_kwargs = {} if version == 2 else {'encoding': 'latin-1'}
....:     sp = subprocess.Popen(command, shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True,**enc_kwargs)
....:     code_input = sp.stdout
....:     print(code_input.read(15))
....:     s = '[ '
....:     s2 = '[ '
....:     order = ord(code_input.read(1))
....:     i = 0
....:     while i < order:
....:         c = code_input.read(1)
....:         if not ord(c):
....:             i += 1
....:             s += ']  [ '
....:             s2 += ']  [ '
....:         else:
....:             s += str(ord(c)) + ' '
....:             s2 += c + ' '
....:     print(s[:-3])
....:     return s2[:-3]

with Python 2, I get:

sage: test(2)
>>planar_code<<
[ 2 3 4 5 6 ]  [ 1 6 7 8 3 ]  [ 1 2 8 9 10 4 ]  [ 1 3 10 11 5 ]  [ 1 4 11 12 6 ]  [ 1 5 12 13 7 2 ]  [ 2 6 13 14 8 ]  [ 2 7 14 9 3 ]  [ 3 8 14 15 10 ]  [ 3 9 15 11 4 ]  [ 4 10 15 16 12 5 ]  [ 5 11 16 13 6 ]  [ 6 12 16 14 7 ]  [ 7 13 16 15 9 8 ]  [ 9 14 16 11 10 ]  [ 11 15 14 13 12 ] 
'[ \x02 \x03 \x04 \x05 \x06 ]  [ \x01 \x06 \x07 \x08 \x03 ]  [ \x01 \x02 \x08 \t \n \x04 ]  [ \x01 \x03 \n \x0b \x05 ]  [ \x01 \x04 \x0b \x0c \x06 ]  [ \x01 \x05 \x0c \r \x07 \x02 ]  [ \x02 \x06 \r \x0e \x08 ]  [ \x02 \x07 \x0e \t \x03 ]  [ \x03 \x08 \x0e \x0f \n ]  [ \x03 \t \x0f \x0b \x04 ]  [ \x04 \n \x0f \x10 \x0c \x05 ]  [ \x05 \x0b \x10 \r \x06 ]  [ \x06 \x0c \x10 \x0e \x07 ]  [ \x07 \r \x10 \x0f \t \x08 ]  [ \t \x0e \x10 \x0b \n ]  [ \x0b \x0f \x0e \r \x0c ] '

and with Python 3:

sage: test(3)
>>planar_code<<
[ 2 3 4 5 6 ]  [ 1 6 7 8 3 ]  [ 1 2 8 9 10 4 ]  [ 1 3 10 11 5 ]  [ 1 4 11 12 6 ]  [ 1 5 12 10 7 2 ]  [ 2 6 10 14 8 ]  [ 2 7 14 9 3 ]  [ 3 8 14 15 10 ]  [ 3 9 15 11 4 ]  [ 4 10 15 16 12 5 ]  [ 5 11 16 10 6 ]  [ 6 12 16 14 7 ]  [ 7 10 16 15 9 8 ]  [ 9 14 16 11 10 ]  [ 11 15 14 10 12 ] 
'[ \x02 \x03 \x04 \x05 \x06 ]  [ \x01 \x06 \x07 \x08 \x03 ]  [ \x01 \x02 \x08 \t \n \x04 ]  [ \x01 \x03 \n \x0b \x05 ]  [ \x01 \x04 \x0b \x0c \x06 ]  [ \x01 \x05 \x0c \n \x07 \x02 ]  [ \x02 \x06 \n \x0e \x08 ]  [ \x02 \x07 \x0e \t \x03 ]  [ \x03 \x08 \x0e \x0f \n ]  [ \x03 \t \x0f \x0b \x04 ]  [ \x04 \n \x0f \x10 \x0c \x05 ]  [ \x05 \x0b \x10 \n \x06 ]  [ \x06 \x0c \x10 \x0e \x07 ]  [ \x07 \n \x10 \x0f \t \x08 ]  [ \t \x0e \x10 \x0b \n ]  [ \x0b \x0f \x0e \n \x0c ] '

If you check carefully, you can see that each occurence of \r that you can find in the output with Python 2 is replaced by \n in Python 3. However, occurrences of \n that we see in py2 are the same in py3.

Have you already seen that ? Do you know what we can do ?

[dcoudert]

Got it !

I tried adding this fix inside method _read_planar_code but it breaks the doctest of that method as StringIO has no method reconfigure.

I also added the fix to the methods using buckygen and benzene.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e9af570

trac #28108: fix doctests with benzene and buckygen

[dcoudert]

While testing the fix with benzene, I realized it needs the same fix than what was done in #27948 for plantri. I also did it for buckygen. This way, all tests pass for method fusenes (using benzene).

However, I'm unable to install buckygen and so to test method fullerenes.

[vklein]

e9af570 fix the 15 doctests errors. Looks good to me.

[dcoudert]

Thanks.