sage leaves a lot of stackframes alive at startup

[mderickx]

This is a really really bad thing and probably related to a lot of the memory leaks shown up recently. All the ipython frames are probably ok on the stack. But most of the other frames come from a call stack frame which is unneccesarily alive.

----------------------------------------------------------------------
| Sage Version 4.6.1.alpha3, Release Date: 2010-12-05                |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
sage: import gc
sage: import inspect
sage: l1=gc.get_objects()
sage: frames=[x for x in l1 if inspect.isframe(x)]
sage: len(frames)
26
sage: for i in frames: inspect.getframeinfo(i) 
....: 
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/Shell.py', lineno=81, function='mainloop', code_context=['        self.IP.mainloop(banner)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=1577, function='mainloop', code_context=['                self.interact(banner)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=1813, function='interact', code_context=['                more = self.push(line)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=2121, function='push', code_context=['        return more\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=2045, function='runsource', code_context=['            return False\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=2093, function='runcode', code_context=['        return outflag\n'], index=0)
Traceback(filename='<ipython console>', lineno=1, function='<module>', code_context=None, index=None)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=5800, function='_init_qqbar', code_context=["    ZZX_x = ZZ['x'].gen()\n"], index=0)
Traceback(filename='coerce_actions.pyx', lineno=184, function='sage.structure.coerce_actions.detect_element_action (sage/structure/coerce_actions.c:4184)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=184, function='sage.structure.coerce_actions.detect_element_action (sage/structure/coerce_actions.c:4184)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=184, function='sage.structure.coerce_actions.detect_element_action (sage/structure/coerce_actions.c:4184)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=237, function='sage.structure.coerce_actions.ModuleAction.__init__ (sage/structure/coerce_actions.c:4713)', code_context=None, index=None)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=5253, function='_interval_fast', code_context=['        return v\n'], index=0)
Traceback(filename='coerce_actions.pyx', lineno=278, function='sage.structure.coerce_actions.ModuleAction.__init__ (sage/structure/coerce_actions.c:5174)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=278, function='sage.structure.coerce_actions.ModuleAction.__init__ (sage/structure/coerce_actions.c:5174)', code_context=None, index=None)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=2177, function='__init__', code_context=['        self._value = self._descr._interval_fast(64)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=2854, function='__init__', code_context=['        AlgebraicNumber_base.__init__(self, QQbar, x)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/bin/sage-ipython', lineno=52, function='<module>', code_context=["ipy_sage.mainloop(sys_exit=1, banner='')\n"], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/Shell.py', lineno=1233, function='start', code_context=['    return shell(user_ns = user_ns)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/Shell.py', lineno=78, function='__init__', code_context=['                               debug=debug,shell_class=shell_class)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/ipmaker.py', lineno=782, function='make_IPython', code_context=['    return IP\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/ipmaker.py', lineno=66, function='force_import', code_context=['        __import__(modname)\n'], index=0)
Traceback(filename='ipy_profile_sage.py', lineno=33, function='<module>', code_context=["    _ip.expose_magic('prun', lambda self, s: old_prun(preparse(s)))\n"], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/all_cmdline.py', lineno=35, function='<module>', code_context=['sage.misc.session.init()\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/all.py', lineno=313, function='<module>', code_context=["sage.structure.sage_object.register_unpickle_override('sage.categories.category_types', 'ModularAbelianVarieties', ModularAbelianVarieties)\n"], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/combinat/species/generating_series.py', lineno=269, function='factorial_gen', code_context=['def factorial_gen():\n'], index=0)

Apply nothing

[mderickx]

ps. The unnecessary frames are there since at least sage 4.4.4

[davidloeffler]

The vast majority of the stack frames are saved by the coercion model for use in its introspection mechanism. Adding the line

get_coercion_model().exception_stack()

at the end of sage/all.py kills all but 9 of the 26 stack frames listed above. Of the remaining 9, 8 are to do with IPython. The other one is some silly generator in the combinatorics module that is there to compute factorials; I don't know why they don't call the general-purpose factorial function.

[mderickx]

This shows that #10548 is actually the cause of the factorial generator.

sage: import gc
sage: import inspect
sage: frames=[x for x in gc.get_objects() if inspect.isframe(x)]
sage: l=objgraph.find_backref_chain(frames[0],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)])
sage: l=objgraph.find_backref_chain(frames[8],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)])
sage: map(type,l)
[<type 'module'>, <type 'dict'>, <type 'sage.structure.coerce.CoercionModel_cache_maps'>, <type 'list'>, <type 'tuple'>, <type 'traceback'>, <type 'frame'>, <type 'frame'>, <type 'frame'>, <type 'frame'>]

I'm going to add a patch to remove all the other stack frames in the way david sugested so this ticket can be closed. (since when there is a fix for 10548 it should also remove last stackframe)

[kini]

After applying the two-patch series at #10548, this patch tests fine whether or not the last line is commented out -- that is, the issue itself seems to be fixed by #10548 independently.

I will merge this patch into your doctest patch at #10548 and upload it there.

[mderickx]

There is now only the factorial remaining, but now for a different reason then before:

sage: import gc sage: import inspect sage: l1=gc.get_objects() sage: frames=[x for x in l1 if inspect.isframe(x)] sage: len(frames) 9 sage: l=objgraph.find_backref_chain(frames[8],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)]) sage: map(type,l) [<type 'module'>, <type 'dict'>, <class 'sage.combinat.species.stream.Stream_class'>, <type 'dict'>, <type 'generator'>, <type 'frame'>]

[mderickx]

sage: sage: import gc
sage: sage: import inspect
sage: sage: l1=gc.get_objects()
sage: sage: frames=[x for x in l1 if inspect.isframe(x)]
sage: sage: len(frames)
9
sage: sage: l=objgraph.find_backref_chain(frames[8],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)])
sage: sage: map(type,l)
[<type 'module'>, <type 'dict'>, <class 'sage.combinat.species.stream.Stream_class'>, <type 'dict'>, <type 'generator'>, <type 'frame'>]

[mderickx]

I don't consider the remaining frame corresponding to the factorial generator a bug. It is there because appearantly someone thinks that the factorial generator should be available as a generator instead of a function in some module.

[burcin]

Replying to mderickx:

I don't consider the remaining frame corresponding to the factorial generator a bug. It is there because appearantly someone thinks that the factorial generator should be available as a generator instead of a function in some module.

I agree that the main problem was solved by #10548 and this can be closed.