Opened 9 years ago

Closed 9 years ago

#10570 closed defect (duplicate)

sage leaves a lot of stackframes alive at startup

Reported by: mderickx Owned by: rlm
Priority: minor Milestone: sage-duplicate/invalid/wontfix
Component: memleak Keywords:
Cc: jdemeyer Merged in:
Authors: Reviewers: Burcin Erocal
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by mderickx)

This is a really really bad thing and probably related to a lot of the memory leaks shown up recently. All the ipython frames are probably ok on the stack. But most of the other frames come from a call stack frame which is unneccesarily alive.

----------------------------------------------------------------------
| Sage Version 4.6.1.alpha3, Release Date: 2010-12-05                |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
sage: import gc
sage: import inspect
sage: l1=gc.get_objects()
sage: frames=[x for x in l1 if inspect.isframe(x)]
sage: len(frames)
26
sage: for i in frames: inspect.getframeinfo(i) 
....: 
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/Shell.py', lineno=81, function='mainloop', code_context=['        self.IP.mainloop(banner)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=1577, function='mainloop', code_context=['                self.interact(banner)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=1813, function='interact', code_context=['                more = self.push(line)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=2121, function='push', code_context=['        return more\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=2045, function='runsource', code_context=['            return False\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/iplib.py', lineno=2093, function='runcode', code_context=['        return outflag\n'], index=0)
Traceback(filename='<ipython console>', lineno=1, function='<module>', code_context=None, index=None)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=5800, function='_init_qqbar', code_context=["    ZZX_x = ZZ['x'].gen()\n"], index=0)
Traceback(filename='coerce_actions.pyx', lineno=184, function='sage.structure.coerce_actions.detect_element_action (sage/structure/coerce_actions.c:4184)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=184, function='sage.structure.coerce_actions.detect_element_action (sage/structure/coerce_actions.c:4184)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=184, function='sage.structure.coerce_actions.detect_element_action (sage/structure/coerce_actions.c:4184)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=237, function='sage.structure.coerce_actions.ModuleAction.__init__ (sage/structure/coerce_actions.c:4713)', code_context=None, index=None)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=5253, function='_interval_fast', code_context=['        return v\n'], index=0)
Traceback(filename='coerce_actions.pyx', lineno=278, function='sage.structure.coerce_actions.ModuleAction.__init__ (sage/structure/coerce_actions.c:5174)', code_context=None, index=None)
Traceback(filename='coerce_actions.pyx', lineno=278, function='sage.structure.coerce_actions.ModuleAction.__init__ (sage/structure/coerce_actions.c:5174)', code_context=None, index=None)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=2177, function='__init__', code_context=['        self._value = self._descr._interval_fast(64)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/rings/qqbar.py', lineno=2854, function='__init__', code_context=['        AlgebraicNumber_base.__init__(self, QQbar, x)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/bin/sage-ipython', lineno=52, function='<module>', code_context=["ipy_sage.mainloop(sys_exit=1, banner='')\n"], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/Shell.py', lineno=1233, function='start', code_context=['    return shell(user_ns = user_ns)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/Shell.py', lineno=78, function='__init__', code_context=['                               debug=debug,shell_class=shell_class)\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/ipmaker.py', lineno=782, function='make_IPython', code_context=['    return IP\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/IPython/ipmaker.py', lineno=66, function='force_import', code_context=['        __import__(modname)\n'], index=0)
Traceback(filename='ipy_profile_sage.py', lineno=33, function='<module>', code_context=["    _ip.expose_magic('prun', lambda self, s: old_prun(preparse(s)))\n"], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/all_cmdline.py', lineno=35, function='<module>', code_context=['sage.misc.session.init()\n'], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/all.py', lineno=313, function='<module>', code_context=["sage.structure.sage_object.register_unpickle_override('sage.categories.category_types', 'ModularAbelianVarieties', ModularAbelianVarieties)\n"], index=0)
Traceback(filename='/Applications/sage-4.6.rc0/local/lib/python2.6/site-packages/sage/combinat/species/generating_series.py', lineno=269, function='factorial_gen', code_context=['def factorial_gen():\n'], index=0)

Apply nothing

Attachments (1)

trac_10570.patch (1.2 KB) - added by mderickx 9 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 Changed 9 years ago by mderickx

ps. The unnecessary frames are there since at least sage 4.4.4

comment:2 Changed 9 years ago by davidloeffler

The vast majority of the stack frames are saved by the coercion model for use in its introspection mechanism. Adding the line

get_coercion_model().exception_stack()

at the end of sage/all.py kills all but 9 of the 26 stack frames listed above. Of the remaining 9, 8 are to do with IPython. The other one is some silly generator in the combinatorics module that is there to compute factorials; I don't know why they don't call the general-purpose factorial function.

comment:3 Changed 9 years ago by mderickx

This shows that #10548 is actually the cause of the factorial generator.

sage: import gc
sage: import inspect
sage: frames=[x for x in gc.get_objects() if inspect.isframe(x)]
sage: l=objgraph.find_backref_chain(frames[0],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)])
sage: l=objgraph.find_backref_chain(frames[8],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)])
sage: map(type,l)
[<type 'module'>, <type 'dict'>, <type 'sage.structure.coerce.CoercionModel_cache_maps'>, <type 'list'>, <type 'tuple'>, <type 'traceback'>, <type 'frame'>, <type 'frame'>, <type 'frame'>, <type 'frame'>]

I'm going to add a patch to remove all the other stack frames in the way david sugested so this ticket can be closed. (since when there is a fix for 10548 it should also remove last stackframe)

Changed 9 years ago by mderickx

comment:4 Changed 9 years ago by mderickx

  • Status changed from new to needs_review

comment:5 Changed 9 years ago by kini

  • Authors set to Maarten Derickx
  • Description modified (diff)
  • Reviewers set to Keshav Kini
  • Status changed from needs_review to positive_review

After applying the two-patch series at #10548, this patch tests fine whether or not the last line is commented out -- that is, the issue itself seems to be fixed by #10548 independently.

I will merge this patch into your doctest patch at #10548 and upload it there.

comment:6 Changed 9 years ago by mderickx

  • Status changed from positive_review to needs_work

There is now only the factorial remaining, but now for a different reason then before:

sage: import gc sage: import inspect sage: l1=gc.get_objects() sage: frames=[x for x in l1 if inspect.isframe(x)] sage: len(frames) 9 sage: l=objgraph.find_backref_chain(frames[8],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)]) sage: map(type,l) [<type 'module'>, <type 'dict'>, <class 'sage.combinat.species.stream.Stream_class'>, <type 'dict'>, <type 'generator'>, <type 'frame'>]

comment:7 Changed 9 years ago by mderickx

sage: sage: import gc
sage: sage: import inspect
sage: sage: l1=gc.get_objects()
sage: sage: frames=[x for x in l1 if inspect.isframe(x)]
sage: sage: len(frames)
9
sage: sage: l=objgraph.find_backref_chain(frames[8],inspect.ismodule,max_depth=15,extra_ignore=[id(frames)])
sage: sage: map(type,l)
[<type 'module'>, <type 'dict'>, <class 'sage.combinat.species.stream.Stream_class'>, <type 'dict'>, <type 'generator'>, <type 'frame'>]

comment:8 follow-up: Changed 9 years ago by mderickx

  • Description modified (diff)
  • Milestone changed from sage-4.7 to sage-duplicate/invalid/wontfix
  • Priority changed from critical to minor
  • Status changed from needs_work to needs_review

I don't consider the remaining frame corresponding to the factorial generator a bug. It is there because appearantly someone thinks that the factorial generator should be available as a generator instead of a function in some module.

comment:9 in reply to: ↑ 8 Changed 9 years ago by burcin

  • Cc jdemeyer added
  • Status changed from needs_review to positive_review

Replying to mderickx:

I don't consider the remaining frame corresponding to the factorial generator a bug. It is there because appearantly someone thinks that the factorial generator should be available as a generator instead of a function in some module.

I agree that the main problem was solved by #10548 and this can be closed.

comment:10 Changed 9 years ago by jdemeyer

  • Authors Maarten Derickx deleted
  • Reviewers changed from Keshav Kini to Burcin Erocal

comment:11 Changed 9 years ago by jdemeyer

  • Resolution set to duplicate
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.