Opened 5 years ago

Closed 5 years ago

#20624 closed defect (fixed)

maximum recursion depth exceeded in MonoDictEraser

Reported by: vbraun Owned by:
Priority: major Milestone: sage-7.3
Component: misc Keywords: random_fail
Cc: SimonKing Merged in:
Authors: Nils Bruin Reviewers: Volker Braun
Report Upstream: N/A Work issues:
Branch: 58db8ae (Commits) Commit: 58db8ae83a2d9e2148a31c1711c1879517685fc7
Dependencies: Stopgaps:

Description

This has recently started to appear randomly:

sage -t --long src/sage/functions/other.py
**********************************************************************
File "src/sage/functions/other.py", line 364, in sage.functions.other.Function_ceil.__init__
Failed example:
    a(x=4.0)
Expected:
    5
Got:
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431d8e3748> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431d8e36e0> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bd61c338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bd61c338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431d8e34d8> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bc227338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bc227338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431d8e3540> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bd61c338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bd61c338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.misc.weak_dict.WeakValueDictEraser object at 0x7f431d8e33a0> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bd61c338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.TripleDictEraser object at 0x7f43bd61c338> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.misc.weak_dict.WeakValueDictEraser object at 0x7f431d8e33a0> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.misc.weak_dict.WeakValueDictEraser object at 0x7f4311ebd8e8> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431bd74268> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431bd7d2d0> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f431d8e36e0> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f4315cfce30> ignored
    Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <sage.structure.coerce_dict.MonoDictEraser object at 0x7f4315d19880> ignored
    5
**********************************************************************
1 item had failures:
   1 of  22 in sage.functions.other.Function_ceil.__init__
    [496 tests, 1 failure, 4.74 s]

Change History (6)

comment:1 Changed 5 years ago by nbruin

Ah goodie! This brings back fond memories from the ticket from hell #10963. Specifically: comment:242. Note that there the (ignored) errors in the Eraser classes were a symptom of an overloaded call stack caused by other (basically) infinite recursions.

I'm curious to see what's causing this. If it's actually a too-deep recursion in a deletion somewhere, I would be very surprised.

comment:2 Changed 5 years ago by nbruin

Oh boy, it turns out BuiltinFunction does some shuffling around of _eval*_ methods, so looking at the source doesn't give a clear picture. We really have to use introspection.

ceil._evalf_(self,x,**kwds) -> return self.eval_(x) [throws away keywords!] ceil._eval_(self,*args) -> self._evalf_try_(*args) or self._eval0_(*args) if evalf_try returns None. ceil._evalf_try_(self,*args) -> does some tests, but can end up calling self._evalf_(*args,parent=p). Does all of this inside a try...except Exception:pass

whenever you run

ceil(x+2/5)(x=4.0)

the _evalf_try_/_evalf_ loop will run out the python call stack, the error will be caught by the try/except, evalf_try will then call ceil._eval0_(x) which simply tries x.ceil() or some alternatives.

Strong evidence that this happens:

sage: sys.setrecursionlimit(<N>)
sage: %prun ceil(x+2/5)(x=4.0)
    <N/4-epsilon>/1    0.003    0.000    0.003    0.003 {method '_evalf_or_eval_' of 'sage.symbolic.function.BuiltinFunction' objects}
        1    0.000    0.000    0.004    0.004 <string>:1(<module>)
    <N/4-epsilon>1    0.000    0.000    0.003    0.003 other.py:501(_evalf_)

So the change is that apparently something has become collectible and that now it can happen the garbage collector runs when the call stack is nearly full, so that the Eraser classes don't have space to run.

The real solution is to stop BuiltinFunction from mangling methods. It makes it very hard for people to write decent code. It's a horrible design. Just looking at the implementation of ceil, everything seems to be fine (although why would you implement evalf as eval?. With the magic in BuiltinFunction?, perhaps just deleting evalf does the trick)

The commit that seems to have introduced this behavior in BuiltinFunction is 6d107297. I think a comprehensive audit of _evalf_/_eval_ implementations on classes that inherit from BuiltinFunction is in order, because this sort of problem is likely occurring elsewhere to.

Last edited 5 years ago by nbruin (previous) (diff)

comment:3 Changed 5 years ago by nbruin

  • Branch set to u/nbruin/maximum_recursion_depth_exceeded_in_monodicteraser

comment:4 Changed 5 years ago by nbruin

  • Authors set to Nils Bruin
  • Commit set to 58db8ae83a2d9e2148a31c1711c1879517685fc7
  • Status changed from new to needs_review

Just removing the _evalf_ implementations seems to do the trick. If someone wants to add doctests, go ahead.


New commits:

58db8aetrac 20624: remove _evalf_ that leads to infinite recursion due to changes on #14766

comment:5 Changed 5 years ago by vbraun

  • Reviewers set to Volker Braun
  • Status changed from needs_review to positive_review

thanks!

comment:6 Changed 5 years ago by vbraun

  • Branch changed from u/nbruin/maximum_recursion_depth_exceeded_in_monodicteraser to 58db8ae83a2d9e2148a31c1711c1879517685fc7
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.