#28298 closed enhancement (worksforme)

py3: final polishing of the last failing doctests

Reported by: chapoton Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: python3 Keywords:
Cc: vklein, embray, jmantysalo, gh-timokau, jhpalmieri, dcoudert, etn40ff, arojas Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by chapoton)

After #26212, there remains only a few failing doctests (not using any optional package).

Fixed:

  • 1 in src/sage/tests/books/computational-mathematics-with-sagemath/graphtheory_doctest.py #28300
  • 1 in src/sage/tests/books/computational-mathematics-with-sagemath/sol/graphtheory_doctest.py #28300
  • (A) 1 in src/sage/combinat/root_system/non_symmetric_macdonald_polynomials.py #28305
  • (D) 1 in src/sage/tests/cmdline.py #28320
  • (C) 1 in src/sage/symbolic/expression.pyx #28219 #28412
  • (K) 2 in src/sage/crypto/block_cipher/present.py #28403
  • (B) 7 in src/sage/combinat/root_system/weyl_group.py #27967 #28351
  • (E) 1 at line 597 in src/sage/libs/eclib/interface.py #28454 #28472
  • (G) src/sage/rings/polynomial/polynomial_rational_flint.pyx #28334 #28649
  • (H) 1 in src/sage/libs/singular/function.pyx #28622
  • (I) 2 in src/sage/numerical/backends/generic_backend.pyx #28622
  • (J) 4 in src/sage/repl/attach.py #28622

Those that are not yet handled are the following.

*On several machines*:

  • (F) 1 in src/sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field.py (not seen since some time. random ? could be ignored ?)

but not on these machines: macOS mojave (several different machines)

OS X only:

  • (K) 2 in src/sage/symbolic/expression.py #28489
  • (L) 1 in src/sage/numerical/linear_tensor_element.pyx #28559

Table summary:

Issue ref. patchbot ubuntu, debian OS X
(F) fail ?
(K) fail (intermittent)
(L) fail (intermittent)

Change History (62)

comment:1 Changed 15 months ago by dcoudert

  • Cc dcoudert added

comment:2 Changed 15 months ago by chapoton

  • Description modified (diff)

comment:3 Changed 15 months ago by chapoton

  • Description modified (diff)

comment:4 Changed 15 months ago by etn40ff

  • Cc etn40ff added

comment:5 Changed 15 months ago by jhpalmieri

  • Description modified (diff)

comment:6 Changed 15 months ago by jhpalmieri

Are all of the failures "on some machines" actually on the same machine? Can these be reproduced by anyone else?

comment:7 Changed 15 months ago by chapoton

the flint one has been reported by several people on sage-release

comment:8 Changed 15 months ago by jhpalmieri

The flint one is strange, since flint doesn't depend on Python. So either the error occurs also in Python 2, or there is something strange going on with the flint interface.

Last edited 15 months ago by jhpalmieri (previous) (diff)

comment:9 Changed 15 months ago by klee

  • Description modified (diff)

comment:10 Changed 15 months ago by gh-mwageringel

Maybe this is a stupid question, but are we sure that the errors that are unique to the patchbot are machine-specific (i.e. reproducible on that machine) or is there a chance this could be fixed by doing a clean install?

comment:11 Changed 15 months ago by chapoton

I have just "make distclean" that precise patchbot machine. It is currently running, and we will see the results in 2 hours. This machine has a fairly stable configuration, and the remaining failing doctests have been there since many releases.

Some of these errors pass when each file is doctested alone. Some look like a mis-synchronized (pexpect ?) interface.

Of course, we will have to make the decision when we claim "all doctests pass", in spite of the remaining quantum noise..

Last edited 15 months ago by chapoton (previous) (diff)

comment:12 Changed 15 months ago by chapoton

  • Description modified (diff)

comment:13 Changed 15 months ago by chapoton

  • Description modified (diff)

comment:14 Changed 15 months ago by chapoton

  • Description modified (diff)

comment:15 Changed 15 months ago by chapoton

  • Description modified (diff)

comment:16 Changed 15 months ago by klee

  • Description modified (diff)

comment:17 follow-up: Changed 15 months ago by chapoton

Please add your machine results in the table, to help understand the remaining errors.

comment:18 Changed 15 months ago by jhpalmieri

  • Description modified (diff)

comment:19 Changed 15 months ago by jhpalmieri

I have access to three different OS X Mojave machines: an iMac Pro, an iMac, and a MacBook Air, and I see the same failures on all three. I've updated the table correspondingly.

Last edited 15 months ago by jhpalmieri (previous) (diff)

comment:20 Changed 15 months ago by gh-mwageringel

  • Description modified (diff)

Same results on two CentOS machines.

comment:21 follow-up: Changed 15 months ago by jhpalmieri

Frédéric, if you have a machine with the flint problem, can you try to track it down? Eric G thought it dated back at least to 8.8.rc3, but I don't see that in his report on sage-release. Emmanuel first reported it for 8.9.beta2 and provided a report for 8.9.beta1 without this failure, so maybe it happened between 8.9.beta1 and 8.9.beta2.

Can you use git bisect to try to locate the problem? Could it have been #28048? (I don't see why, but I'm looking at the tickets merged in 8.9.beta2, trying to see what could possibly be related. Could accents in SAGE_LOCAL be causing this?)

comment:22 in reply to: ↑ 17 Changed 15 months ago by egourgoulhon

Replying to chapoton:

Please add your machine results in the table, to help understand the remaining errors.

For my laptop (Ubuntu 18.04.2, Core i7-6700HQ, 16 GB RAM), the results are identical to those marked as "laptop ch".

comment:23 Changed 15 months ago by klee

I guess the OS of the machine is important to understand the failures. Then what OS are the reference patchbot and "laptop ch"?

comment:24 in reply to: ↑ 21 ; follow-up: Changed 15 months ago by jhpalmieri

Replying to jhpalmieri:

Frédéric, if you have a machine with the flint problem, can you try to track it down? Eric G thought it dated back at least to 8.8.rc3, but I don't see that in his report on sage-release. Emmanuel first reported it for 8.9.beta2 and provided a report for 8.9.beta1 without this failure, so maybe it happened between 8.9.beta1 and 8.9.beta2.

Oh, I see, the relevant folder wasn't in python3-known-passing-test.txt until 8.9.beta2, so it may have been failing for some time before that.

comment:25 in reply to: ↑ 24 Changed 15 months ago by egourgoulhon

Replying to jhpalmieri:

Replying to jhpalmieri:

Frédéric, if you have a machine with the flint problem, can you try to track it down? Eric G thought it dated back at least to 8.8.rc3, but I don't see that in his report on sage-release. Emmanuel first reported it for 8.9.beta2 and provided a report for 8.9.beta1 without this failure, so maybe it happened between 8.9.beta1 and 8.9.beta2.

Oh, I see, the relevant folder wasn't in python3-known-passing-test.txt until 8.9.beta2, so it may have been failing for some time before that.

Yes I confirm. I've still a working version of Python3 8.8.rc3 (on Ubuntu 18.04) and the flint issue is already there.

comment:26 Changed 15 months ago by jhpalmieri

  • Description modified (diff)

I don't have any solutions, but let's track the flint problem at #28334.

comment:27 Changed 15 months ago by jhpalmieri

  • Description modified (diff)

comment:28 Changed 15 months ago by jhpalmieri

  • Description modified (diff)

comment:29 Changed 14 months ago by chapoton

  • Description modified (diff)

comment:30 Changed 14 months ago by jhpalmieri

  • Description modified (diff)

I expect (K) to fail on all machines due to the nature of the problem, but I've only tested on OS X.

comment:31 Changed 14 months ago by jhpalmieri

  • Description modified (diff)

comment:32 Changed 14 months ago by chapoton

Got a patchbot report from another machine, with similar results as for the reference machine:

https://patchbot.sagemath.org/log/0/Ubuntu/18.04/x86_64/4.15.0-54-generic/petitbonum/2019-08-29%2012:45:45?short

namely

sage -t --long src/sage/symbolic/expression.pyx  # 1 doctest failed
sage -t --long src/sage/combinat/root_system/weyl_group.py  # 7 doctests failed
sage -t --long src/sage/rings/polynomial/polynomial_rational_flint.pyx  # 1 doctest failed
sage -t --long src/sage/libs/eclib/interface.py  # 1 doctest failed
sage -t --long src/sage/crypto/block_cipher/present.py  # 2 doctests failed
sage -t --long src/sage/repl/attach.py  # 4 doctests failed
sage -t --long src/sage/libs/singular/function.pyx  # 1 doctest failed
sage -t --long src/sage/numerical/backends/generic_backend.pyx  # 2 doctests failed

comment:33 Changed 14 months ago by arojas

  • Cc arojas added

comment:34 Changed 14 months ago by chapoton

  • Description modified (diff)

one fixed in 8.9.b9

comment:35 Changed 14 months ago by chapoton

  • Description modified (diff)

rather two, in fact

comment:36 Changed 14 months ago by klee

  • Description modified (diff)

comment:37 Changed 14 months ago by charpent

At this point, it begins to be useful to widen the scope of our checks. I proceeded to make ptestalllong on two different machines (but very close), with a few optional packages, both running 8.9.beta9.

Please let me know if this isn't the right place to post this kind of reports...

TL;DR:

  • make ptestalllong gives failures on 22 packages total (20 on one machine, 19 on the other);
  • Most of these failure cannot be reproduced when running the test standalone (whether MAKE is defined or not). Three file persist to fail when ran standalone:
    • src/sage/combinat/root_system/weyl_group.py (known)
    • src/sage/libs/eclib/interface.py (also known)
    • src/sage/tests/gap_packages.py (not known but optional package, fails only on one of my machines).

I have no explanation for the intermachines differences; one is a Debian testing desktop (core i7, 8 GB RAM), the other is a Debian testing laptop (core i7, 16 GB RAM).

The large number of transient failures let me think that some parts of the libraries are not as reentrant as we might wish them to be...

Details of failing files:

| Test file                                               | Laptop | Desktop | Standalone |
|---------------------------------------------------------+--------+---------+------------|
| src/sage/graphs/generators/smallgraphs.py               | Y      | N       | N          |
| src/sage/combinat/posets/posets.py                      | Y      | Y       | N          |
| src/sage/combinat/root_system/weyl_group.py             | Y      | Y       | Y          |
| src/sage/plot/animate.py                                | Y      | Y       | N          |
| src/sage/doctest/forker.py                              | Y      | N       | N          |
| src/sage/libs/eclib/interface.py                        | Y      | Y       | Y          |
| src/sage/combinat/matrices/hadamard_matrix.py           | Y      | Y       | N          |
| src/sage/rings/polynomial/polynomial_rational_flint.pyx | Y      | Y       | N          |
| src/sage/symbolic/integration/external.py               | Y      | Y       | N          |
| src/sage/misc/latex.py                                  | Y      | Y       | N          |
| src/sage/coding/databases.py                            | Y      | Y       | N          |
| src/sage/misc/persist.pyx                               | Y      | Y       | N          |
| src/sage/databases/oeis.py                              | Y      | Y       | N          |
| src/sage/tests/gap_packages.py                          | Y      | N       | Y          |
| src/sage/repl/load.py                                   | Y      | Y       | N          |
| src/sage/graphs/graph_latex.py                          | Y      | Y       | N          |
| src/sage/combinat/designs/ext_rep.py                    | Y      | N       | N          |
| src/sage/misc/remote_file.py                            | Y      | Y       | N          |
| src/sage/databases/findstat.py                          | Y      | Y       | N          |
| src/sage/interfaces/magma_free.py                       | N      | Y       | N          |
| src/sage/interfaces/mathematica.py                      | N      | Y       | N          |

Details on the failures on request (including the full logs...).

HTH,

Last edited 14 months ago by charpent (previous) (diff)

comment:38 follow-up: Changed 14 months ago by egourgoulhon

  • Description modified (diff)

comment:39 in reply to: ↑ 38 Changed 14 months ago by egourgoulhon

Replying to egourgoulhon:

Description modified according to the report https://groups.google.com/d/msg/sage-release/M1JjXhXfsU4/TuPETvGUAgAJ

See however https://trac.sagemath.org/ticket/28334#comment:11 for a possible further update of the summary table.

comment:40 Changed 14 months ago by jhpalmieri

  • Description modified (diff)

I may have accidentally found a fix for the eclib/interface.py problem. See #28454. Needs testing and, more importantly, some understanding of whether this is a valid fix.

comment:41 Changed 14 months ago by chapoton

  • Description modified (diff)

comment:42 Changed 14 months ago by chapoton

  • Description modified (diff)

comment:43 Changed 14 months ago by jhpalmieri

  • Description modified (diff)

comment:44 Changed 14 months ago by jhpalmieri

  • Description modified (diff)

comment:45 Changed 13 months ago by chapoton

  • Description modified (diff)

comment:46 Changed 13 months ago by jhpalmieri

  • Description modified (diff)

comment:47 Changed 13 months ago by jhpalmieri

  • Description modified (diff)

comment:48 follow-up: Changed 13 months ago by jhpalmieri

Regarding (H), (I), (J): I don't see them ordinarily on OS X, even when running make ptestlong, but I do see them on my OS X patchbot. Is there something different about patchbots which could lead to these failures?

comment:49 Changed 13 months ago by jhpalmieri

  • Description modified (diff)

comment:50 Changed 13 months ago by chapoton

  • Description modified (diff)

comment:51 in reply to: ↑ 48 Changed 12 months ago by jhpalmieri

  • Description modified (diff)

Replying to jhpalmieri:

Regarding (H), (I), (J): I don't see them ordinarily on OS X, even when running make ptestlong, but I do see them on my OS X patchbot. Is there something different about patchbots which could lead to these failures?

I really think this is the case. I've opened #28622 for this, not that I have any ideas how to fix it.

comment:52 Changed 12 months ago by jhpalmieri

These failures seem to also occur outside of the patchbot, but they are very repeatable when using the patchbot, as described at #28622, so I hope if we can fix the patchbot problems, it will either fix them outside of the patchbot or at least give us some clues.

comment:53 follow-up: Changed 12 months ago by jhpalmieri

I still don't know if anyone else has seen (K) or (L). (F) maybe hasn't been seen for a long time. Things are looking good!

comment:54 Changed 12 months ago by gh-timokau

If all remaining failures are transient, maybe its time to switch the betas over to py3 and do some more extensive testing?

comment:55 in reply to: ↑ 53 Changed 12 months ago by gh-mwageringel

Replying to jhpalmieri:

I still don't know if anyone else has seen (K) or (L).

I can reproduce (K) on macOS reliably, but have not seen (L).

comment:56 Changed 12 months ago by jhpalmieri

  • Description modified (diff)

comment:57 Changed 11 months ago by chapoton

  • Description modified (diff)

comment:58 Changed 11 months ago by jhpalmieri

I haven't seen (L) in a while. I haven't heard any other reports of it, either. So I would put a very low priority on it.

comment:59 Changed 11 months ago by klee

I propose to clear up (F), (K), and (L) here. They are all intermittent failures with obscure causes, and do not seem likely to get any solution in the near future, and just blocking sage on python 3. They can change to regular tickets. Any one object?

comment:60 Changed 11 months ago by chapoton

These are not blocking anything. The switch to python3 is happening.

comment:61 Changed 10 months ago by embray

  • Milestone changed from sage-8.9 to sage-9.1

Ticket retargeted after milestone closed

comment:62 Changed 10 months ago by chapoton

  • Milestone changed from sage-9.1 to sage-duplicate/invalid/wontfix
  • Resolution set to worksforme
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.