Opened 4 months ago

Closed 4 months ago

#33219 closed defect (fixed)

defining back ffmpeg as an external feature (which are not doctested by default)

Reported by: vbraun Owned by:
Priority: blocker Milestone: sage-9.5
Component: packages: optional Keywords:
Cc: mkoeppe, slabbe, mjo Merged in:
Authors: Sébastien Labbé Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: 6aaf5e2 (Commits, GitHub, GitLab) Commit: 6aaf5e2440112e7574d717dce578f4fb3b0441ff
Dependencies: Stopgaps:

Status badges

Description

On the kucalc buildbot (debian 9 x86_64):

[...]
sage: sig_on_count() # check sig_on/off pairings (virtual doctest) ## line 700 ##
0
sage: a = animate([sin(x + float(k)) for k in srange(0,2*pi,0.7)],
               xmin=0, xmax=2*pi, figsize=[2,1]) ## line 787 ##
sage: a.show()       # optional -- ImageMagick ## line 789 ##
sage: a.show(iterations=3)    # optional -- ImageMagick ## line 794 ##
sage: a.show(delay=50)        # optional -- ImageMagick ## line 798 ##
sage: a.show(format="ogg")         # optional -- ffmpeg ## line 802 ##
sage: a.show(format="webm")        # optional -- ffmpeg ## line 803 ##
sage: a.show(format="mp4")         # optional -- ffmpeg ## line 804 ##
------------------------------------------------------------------------
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/cysignals/signals.cpython-39-x86_64-linux-gnu.so(+0x6c28)[0x7fdba3db8c28]
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/cysignals/signals.cpython-39-x86_64-linux-gnu.so(+0x6e08)[0x7fdba3db8e08]
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/cysignals/signals.cpython-39-x86_64-linux-gnu.so(+0x91fa)[0x7fdba3dbb1fa]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fdbabeed390]
/lib/x86_64-linux-gnu/libpthread.so.0(waitpid+0x6b)[0x7fdbabeecf7b]
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/libpython3.9.so.1.0(+0x1ccd19)[0x7fdbac2c5d19]
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/libpython3.9.so.1.0(+0xe57e1)[0x7fdbac1de7e1]
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x8ffe)[0x7fdbac16a9ee]
/var/lib/buildbot/slave/sage_git/build/local/var/lib/sage/venv-python3.9.9/lib/libpython3.9.so.1.0(+0x67a88)[0x7fdbac160a88]
[...]
**********************************************************************
----------------------------------------------------------------------
sage -t --long --random-seed=0 src/sage/plot/animate.py  # Timed out
----------------------------------------------------------------------

Change History (9)

comment:1 Changed 4 months ago by mjo

  • Dependencies set to #33045
  • Milestone changed from sage-9.6 to sage-duplicate/invalid/wontfix

We know. The timeout is a separate issue (#33045). Please try Sébastien's branch there.

Feature checks at runtime won't easily be able to detect a process that hangs. It's Yet Another fundamental problem with runtime checks.

comment:2 Changed 4 months ago by mkoeppe

If trying to use this feature can hang, then we should probably declare this feature as unsafe again by adding it back to external_features, https://github.com/sagemath/sage/blob/develop/src/sage/doctest/external.py#L336

comment:3 Changed 4 months ago by slabbe

  • Authors set to Sébastien Labbé
  • Branch set to u/slabbe/33219
  • Commit set to a276d0b7dce1c7d4b71b3015c9bed1a3cd5c2685
  • Dependencies #33045 deleted
  • Milestone changed from sage-duplicate/invalid/wontfix to sage-9.5
  • Priority changed from major to blocker
  • Status changed from new to needs_review

Since #32174, we are automatically testing "safe" external features including ffmpeg. It turns out that those doctests are broken (hang, doctests time out) on machines I don't have access to including the buildbots on which Volker tests tickets.

I have been testing those optional tests and others on my machine for some time now and reporting on sage-release, but indeed, not on a big variety of machines. Therefore, those optional doctests are not guarrenty to work on all machines.

There are two solutions to this:

  • put back ffmpeg to the list of optional doctests that we never test (as it was before #32174)
  • try to fix the issue, this is #33045

The current branch implements the former. Needs review!

#33045 can be dealt with later once we find a way to test that it fixes the issue on a buildbot.


New commits:

a276d0b33219: defining back ffmpeg as an external feature
Last edited 4 months ago by slabbe (previous) (diff)

comment:4 Changed 4 months ago by slabbe

  • Summary changed from ffmpeg/imagemagick feature checks still broken to defining back ffmpeg as an external feature (which are not doctested by default)

comment:5 Changed 4 months ago by mkoeppe

  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

comment:6 Changed 4 months ago by vbraun

  • Status changed from positive_review to needs_work
sage -t --long --warn-long 50.8 --random-seed=123 src/sage/doctest/external.py
**********************************************************************
File "src/sage/doctest/external.py", line 381, in sage.doctest.external.AvailableSoftware
Failed example:
    external_software
Expected:
    ['cplex',
     'gurobi',
     'internet',
     'latex',
     'latex_package_tkz_graph',
     'lualatex',
     'macaulay2',
     'magma',
     'maple',
     'mathematica',
     'matlab',
     'octave',
     'pdflatex',
     'scilab',
     'xelatex']
Got:
    ['cplex',
     'ffmpeg',
     'gurobi',
     'internet',
     'latex',
     'latex_package_tkz_graph',
     'lualatex',
     'macaulay2',
     'magma',
     'maple',
     'mathematica',
     'matlab',
     'octave',
     'pdflatex',
     'scilab',
     'xelatex']
**********************************************************************
1 item had failures:
   1 of   3 in sage.doctest.external.AvailableSoftware
    [39 tests, 1 failure, 0.02 s]
----------------------------------------------------------------------
sage -t --long --warn-long 50.8 --random-seed=123 src/sage/doctest/external.py  # 1 doctest failed
----------------------------------------------------------------------

comment:7 Changed 4 months ago by mkoeppe

  • Branch changed from u/slabbe/33219 to u/mkoeppe/33219

comment:8 Changed 4 months ago by mkoeppe

  • Commit changed from a276d0b7dce1c7d4b71b3015c9bed1a3cd5c2685 to 6aaf5e2440112e7574d717dce578f4fb3b0441ff
  • Status changed from needs_work to positive_review

New commits:

6aaf5e2src/sage/doctest/external.py: Update doctest output

comment:9 Changed 4 months ago by vbraun

  • Branch changed from u/mkoeppe/33219 to 6aaf5e2440112e7574d717dce578f4fb3b0441ff
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.