Opened 5 years ago

Last modified 4 years ago

#24217 new defect

AlarmInterrupt in destructor

Reported by: Volker Braun Owned by:
Priority: major Milestone: sage-8.1
Component: c_lib Keywords:
Cc: Jeroen Demeyer Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

I think there is a general logic error with AlarmInterrupt in that it can be raised in a destructor, where Python will then ignore it. This causes doctest timeout like:

sage: try:  # long time
    alarm(1)
    while True:
        D._save(s, fn)
except (AlarmInterrupt, OSError, AttributeError):
    # OSError could happen due to a double close() in
    # Python's tempfile module.
    # AttributeError could happen due to interrupting
    # in _TemporaryFileWrapper.__init__
    # (see https://trac.sagemath.org/ticket/22423)
    pass ## line 201 ##
Exception cysignals.signals.AlarmInterrupt: AlarmInterrupt() in <bound method _TemporaryFileWrapper.__del__ of <closed file '<fdopen>', mode 'w+b' at 0xc8023390>> ignored
------------------------------------------------------------------------
/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x40c2)[0xf69a20c2]
/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x411c)[0xf69a211c]
/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x6d2f)[0xf69a4d2f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf76f2cb0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf76f2c99]
/lib/i386-linux-gnu/libpthread.so.0(fsync+0x1c)[0xf74b960c]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x13a373)[0xf7607373]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7407)[0xf75bf067]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x6afc1)[0xf7537fc1]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x484ca)[0xf75154ca]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_CallFunctionObjArgs+0x78)[0xf7503f28]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4ac6)[0xf75bc726]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ed4)[0xf75bdb34]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x2e)[0xf75c1e2e]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x60f0)[0xf75bdd50]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x6afc1)[0xf7537fc1]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x484ca)[0xf75154ca]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0xa8bcd)[0xf7575bcd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5c34)[0xf75bd894]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ed4)[0xf75bdb34]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x6afc1)[0xf7537fc1]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x484ca)[0xf75154ca]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0xa9035)[0xf7576035]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0xa5909)[0xf7572909]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5c34)[0xf75bd894]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ed4)[0xf75bdb34]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x2e)[0xf75c1e2e]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyRun_FileExFlags+0x77)[0xf75e5457]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xdd)[0xf75e69bd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyRun_AnyFileExFlags+0x67)[0xf75e70e7]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(Py_Main+0xcfd)[0xf75fdd2d]
python(main+0x27)[0x5655d617]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)[0xf72ac276]
python(+0x655)[0x5655d655]
------------------------------------------------------------------------
Attaching gdb to process id 27022.
Cannot find gdb installed
GDB is not installed.
Install gdb for enhanced tracebacks.
------------------------------------------------------------------------
**********************************************************************
----------------------------------------------------------------------
sage -t --long local/lib/python2.7/site-packages/sagenb/storage/filesystem_storage.py  # Timed out
----------------------------------------------------------------------

Not sure what the right solution is, though. Ideally, the signal would be delivered as soon as the dtor is completed.

Change History (6)

comment:1 in reply to:  description Changed 5 years ago by Jeroen Demeyer

Replying to vbraun:

Python will then ignore it.

There is nothing that cysignals can do about that. I consider it a Python bug: if __del__ can't handle exceptions, it shouldn't check for signals either. See https://bugs.python.org/issue31388

I don't see a solution without patching Python somehow.

comment:2 Changed 5 years ago by Jeroen Demeyer

It would be convenient if Python somehow had a hook to deal with unraisable exceptions. But it doesn't: it just deletes the exception and there is (AFAIK) no way to figure out that it did that.

comment:3 Changed 5 years ago by Nils Bruin

One approach taken by some multi-threaded applications (ECL comes to mind) is that the main thread is dedicated to handling asynchronous signals and work is done in other threads. In principle, it might be possible to do something similar in sage (probably with just 2 threads in total, normally). Then the signal handling thread can set up all kinds of flags that other threads can respond to whenever they are ready. This "cure" may be worse than the problem, though.

comment:4 in reply to:  3 ; Changed 5 years ago by Jeroen Demeyer

Replying to nbruin:

One approach taken by some multi-threaded applications (ECL comes to mind) is that the main thread is dedicated to handling asynchronous signals and work is done in other threads.

The problem is not signal handling, the problem is that Python throws away exceptions.

comment:5 in reply to:  4 Changed 5 years ago by Nils Bruin

Replying to jdemeyer:

The problem is not signal handling, the problem is that Python throws away exceptions.

That's one way of looking at it, but it seems that the exceptions that are really problematic when they're thrown away are the ones stemming from asynchronous signals; at least that's the kind that triggers the reported behaviour in the ticket. It may be the case that splitting threads allows you to work around the behaviour in python without patching it.

Personally I think I can live with knowing that I shouldn't fully trust results from a sage session that has experienced an interruption. That would be my default assumption for computational software anyway. I'm not really advocating that we explore this option, but it seems a possibility.

comment:6 in reply to:  3 Changed 4 years ago by Jeroen Demeyer

Replying to nbruin:

One approach taken by some multi-threaded applications (ECL comes to mind) is that the main thread is dedicated to handling asynchronous signals and work is done in other threads. In principle, it might be possible to do something similar in sage (probably with just 2 threads in total, normally). Then the signal handling thread can set up all kinds of flags that other threads can respond to whenever they are ready.

This is actually more or less how Python internally handles interrupts. It's not literally multi-threaded, but I don't think that this matters. When an interrupt occurs, it just sets a flag that the interpreter checks at suitable times.

So two possible fixes that I see (both involve patching Python):

  1. Python should only check for signals when it can actually handle them.
  1. Python should have a mechanism to deal with unraisable exceptions: say, a callback method __unraisable__ on the exception object. That way, we can re-enable the interrupt flag when a KeyboardInterrupt exception is ignored.

I think that 2. can realistically be implemented, but I don't feel like fighting with Python upstream to make them accept that change (they are extremely conservative and it's hard to push any non-trivial changes).

Note: See TracTickets for help on using tickets.