Opened 12 years ago
Closed 12 years ago
#10548 closed defect (fixed)
The coercion model is keeping references to tracebacks which causes memory leaks.
Reported by: | Maarten Derickx | Owned by: | Robert Bradshaw |
---|---|---|---|
Priority: | major | Milestone: | sage-4.7 |
Component: | coercion | Keywords: | |
Cc: | David Loeffler | Merged in: | sage-4.7.alpha4 |
Authors: | Robert Bradshaw | Reviewers: | Maarten Derickx |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
When I was doing some computations on modular forms, I noticed that the ModularSymbols_clear_cache() was not doing what it claims to do.
sage: import gc sage: m=ModularSymbols(Gamma1(29),sign=1) sage: m=[] sage: ModularSymbols_clear_cache() sage: gc.collect() 57 sage: [x for x in gc.get_objects() if isinstance(x,sage.modular.modsym.ambient.ModularSymbolsAmbient_wtk_g1)] [Modular Symbols space of dimension 49 for Gamma_1(29) of weight 2 with sign 1 and over Rational Field] sage:
So even after garbage collection the modular symbols element is still in memory.
When testing please also make sure that the last stackframe from #10570 is also gone.
Apply 10548-coerce-traceback.2.patch and trac_10548-coerce-traceback-doctest.v2.patch
Attachments (5)
Change History (27)
comment:1 Changed 12 years ago by
comment:2 Changed 12 years ago by
According to heapy the aditional memory comes from;
dict of sage.modular.modsym.manin_symbols.ManinSymbol?
mderickx@mod:~$ sage ---------------------------------------------------------------------- | Sage Version 4.6, Release Date: 2010-10-30 | | Type notebook() for the GUI, and license() for information. | ---------------------------------------------------------------------- sage: import gc sage: from guppy import hpy; hp=hpy() sage: M=ModularSymbols(Gamma1(97),sign=1) sage: get_memory_usage() 848.1953125 sage: M=[] sage: get_memory_usage() 848.1953125 sage: ModularSymbols_clear_cache() sage: get_memory_usage() 848.1953125 sage: hp.heap() Partition of a set of 445281 objects. Total size = 68518056 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 169534 38 28541776 42 28541776 42 str 1 125494 28 10674696 16 39216472 57 tuple 2 1624 0 3964480 6 43180952 63 dict of module 3 25817 6 3098040 5 46278992 68 types.CodeType 4 24634 6 2956080 4 49235072 72 function 5 2798 1 2949584 4 52184656 76 dict (no owner) 6 2660 1 2727392 4 54912048 80 dict of type 7 9408 2 2634240 4 57546288 84 dict of sage.modular.modsym.manin_symbols.ManinSymbol 8 2660 1 2388040 3 59934328 87 type 9 4293 1 1010536 1 60944864 89 list <895 more rows. Type e.g. '_.more' to view.> sage:
comment:3 Changed 12 years ago by
Description: | modified (diff) |
---|
comment:4 Changed 12 years ago by
It's not an error of the ModularSymbols_clear_cache() function
sage: ModularSymbols(Gamma1(29),sign=1,use_cache=False) Modular Symbols space of dimension 49 for Gamma_1(29) of weight 2 with sign 1 and over Rational Field sage: None sage: gc.collect() 54 sage: [x for x in gc.get_objects() if isinstance(x,sage.modular.modsym.ambient.ModularSymbolsAmbient_wtk_g1)] [Modular Symbols space of dimension 49 for Gamma_1(29) of weight 2 with sign 1 and over Rational Field]
comment:5 Changed 12 years ago by
See the attached chain.png picture created with objgraph. There is a reference chain to the homset module. And the homeset module will not get destroyed so we have to find out where this chain comes from.
comment:6 Changed 12 years ago by
Might this be an instance of #715? Modular symbols spaces are Parents, so various maps in and out of them are probably getting cached by the coercion model machinery.
Changed 12 years ago by
comment:7 Changed 12 years ago by
Cc: | David Loeffler added |
---|
Sorry I failed in adding the mentioned chain.png before. It shows a reference chain explaining why ManinSymbol?(37,35) is still in the memory. It seems that it has at least to do something with code in the homset module. I don't know enough details about the coercion and category framework and their interaction to say that it has something to do with 715. But the description of 715 really looks like it could be the cause.
If it is coercion, it might also be related to: #10570
comment:8 Changed 12 years ago by
comment:9 Changed 12 years ago by
Description: | modified (diff) |
---|
comment:10 Changed 12 years ago by
Here is some code run on sage 4.6.1 showing that it is really the coercion framework.
sage: import gc sage: import objgraph sage: import inspect sage: m=ModularSymbols(Gamma1(29),use_cache=False) sage: m=[] sage: ModularSymbols_clear_cache() sage: gc.collect() 54 sage: a=[x for x in gc.get_objects() if isinstance(x,sage.modular.modsym.ambient.ModularSymbolsAmbient_wtk_g1)] sage: l=objgraph.find_backref_chain(a[0],inspect.ismodule,max_depth=15,extra_ignore=[id(a)]) sage: map(type,l) [<type 'module'>, <type 'dict'>, <type 'sage.structure.coerce.CoercionModel_cache_maps'>, <type 'list'>, <type 'tuple'>, <type 'traceback'>, <type 'frame'>, <type 'frame'>, <type 'frame'>, <type 'frame'>, <class 'sage.modular.modsym.ambient.ModularSymbolsAmbient_wtk_g1_with_category'>]
comment:12 Changed 12 years ago by
Component: | modular forms → coercion |
---|---|
Owner: | changed from Craig Citro to Robert Bradshaw |
Summary: | ModularSymbols_clear_cache() not clearing everything? → The coercion model is keeping references to tracebacks which causes memory leaks. |
comment:13 Changed 12 years ago by
Description: | modified (diff) |
---|
Changed 12 years ago by
Attachment: | 10548-coerce-traceback.patch added |
---|
comment:14 Changed 12 years ago by
Status: | new → needs_review |
---|
The attached patch fixes the issue described in the title, but does not fix #715.
Changed 12 years ago by
Attachment: | 10548-coerce-traceback.2.patch added |
---|
rebased against 4.7.alpha1
Changed 12 years ago by
Attachment: | 10548-coerce-traceback-doctest.patch added |
---|
comment:16 Changed 12 years ago by
Status: | positive_review → needs_work |
---|
comment:17 Changed 12 years ago by
Status: | needs_work → needs_review |
---|
Robert, (or someone else), can you review the doctest I added?
comment:18 Changed 12 years ago by
Authors: | → Robert Bradshaw |
---|---|
Description: | modified (diff) |
Reviewers: | → Maarten Derickx |
comment:19 Changed 12 years ago by
Status: | needs_review → positive_review |
---|
The doctest looks good to me.
comment:20 Changed 12 years ago by
Description: | modified (diff) |
---|
I folded in the doctest from the current patch at #10570, also by Maarten.
comment:21 Changed 12 years ago by
Description: | modified (diff) |
---|
comment:22 Changed 12 years ago by
Merged in: | → sage-4.7.alpha4 |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
Ps. you might want to use smaller numbers then 97 to test because ModularSymbols?(Gamma1(97),sign=1) takes a while. It behaves the same, but with smaller memory usage for smaller numbers.