Opened 4 years ago

Last modified 3 years ago

#18905 needs_work defect

fix more leaks found in #18897

Reported by: dimpase Owned by:
Priority: major Milestone: sage-8.0
Component: memleak Keywords:
Cc: jmantysalo Merged in:
Authors: Simon King Reviewers:
Report Upstream: N/A Work issues:
Branch: public/18905 (Commits) Commit: e952388409f97f3e40b9e4d56ab3e42b6cb13b88
Dependencies: Stopgaps:

Description (last modified by SimonKing)

In #18897 one leak is fixed, but there are more left, see comments 27 and later:

sage: L = [1,1,0,1,0,1,2,1,1,0,0,1,2,0,0,1,1,0,2,0,0,0,0,0,1]
sage: def test(L, dim):
....:     import gc
....:     from collections import Counter
....:     gc.collect()
....:     pre={id(c) for c in gc.get_objects()}
....:     m = matrix(dim, L)
....:     for p in range(2,102):
....:         m.change_ring(GF(nth_prime(p))).eigenvalues()
....:     gc.collect()
....:     post=Counter(type(o) for o in gc.get_objects() if id(o) not in pre)
....:     return [(k,v) for (k,v) in post.iteritems() if v>10]
....: 
sage: test(L, 5)
[(<class 'sage.rings.algebraic_closure_finite_field.AlgebraicClosureFiniteField_pseudo_conway_with_category'>,
  100),
...
 (<class 'sage.rings.homset.RingHomset_quo_ring_with_category'>, 100),
...
 (<type 'sage.rings.finite_rings.integer_mod.NativeIntStruct'>, 100),
 (<type 'sage.rings.finite_rings.integer_mod.Int_to_IntegerMod'>, 200),
...
 (<class 'sage.rings.homset.RingHomset_generic_with_category'>, 100),
...]

Attachments (4)

test.png (134.7 KB) - added by SimonKing 4 years ago.
Reference chain to finite field after creating its algebraic closure
test2.png (105.4 KB) - added by SimonKing 4 years ago.
Another reference chain, towards NativeIntStruct?
test_3.png (152.2 KB) - added by slabbe 4 years ago.
Ref graph for Algebraic closure of Finite Field of size 181
test_P_14058.png (219.6 KB) - added by SimonKing 4 years ago.
Backref graph for a polynomial ring over finite field

Download all attachments as: .zip

Change History (49)

comment:1 Changed 4 years ago by SimonKing

  • Dependencies set to #18897
  • Description modified (diff)

comment:2 follow-ups: Changed 4 years ago by pbruin

This is probably because the algebraic_closure() method of finite fields is cached (see #14990), creating circular references between finite fields and their algebraic closures. The following simpler code exhibits a similar memory leak:

import gc                                                                                   
from collections import Counter
gc.collect()
pre = {id(c) for c in gc.get_objects()}
for p in prime_range(100):
    GF(p).algebraic_closure()
gc.collect()
post = Counter(type(o) for o in gc.get_objects() if id(o) not in pre)
print([(k,v) for (k,v) in post.iteritems() if v>10])

comment:3 in reply to: ↑ 2 Changed 4 years ago by SimonKing

Replying to pbruin:

This is probably because the algebraic_closure() method of finite fields is cached (see #14990), creating circular references between finite fields and their algebraic closures.

Circular references shouldn't be problematic, unless one of the objects involved has a __del__ method. However:

sage: K = GF(5)
sage: hasattr(K, '__del__')
False
sage: hasattr(K.algebraic_closure(), '__del__')
False

And indeed:

sage: while 1:
....:     print get_memory_usage()
....:     for p in range(2, 102):
....:         A = GF(nth_prime(p)).algebraic_closure()
....:         
<constant amount of memory>

comment:4 Changed 4 years ago by SimonKing

Oops, that's a bad test. Of course it had a constant memory consumption even if memory could not be freed. Sorry.

sage: import gc
sage: _ = gc.collect()
sage: while 1:                                  
....:     print gc.collect()
....:     for p in range(2, 102):
....:         A = GF(nth_prime(p)).algebraic_closure()
....:         
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

That's a leak.

comment:5 in reply to: ↑ 2 ; follow-ups: Changed 4 years ago by slabbe

Replying to pbruin:

This is probably because the algebraic_closure() method of finite fields is cached (see #14990), creating circular references between finite fields and their algebraic closures. The following simpler code exhibits a similar memory leak:

import gc                                                                                   
from collections import Counter
gc.collect()
pre = {id(c) for c in gc.get_objects()}
for p in prime_range(100):
    GF(p).algebraic_closure()
gc.collect()
post = Counter(type(o) for o in gc.get_objects() if id(o) not in pre)
print([(k,v) for (k,v) in post.iteritems() if v>10])

Like noticed in #18897, when this code is runned the first time, there are many stuff left. But the second time, the list is empty.

comment:6 Changed 4 years ago by SimonKing

And the leak indeed only occurs if we store the algebraic closure. In a freshly started session (the previous result has also been in a freshly started session):

sage: _ = gc.collect()
sage: while 1:                                  
....:     print gc.collect()
....:     for p in range(2, 102):
....:         A = GF(nth_prime(p))
....:         
0
8396
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969
7969

comment:7 in reply to: ↑ 5 Changed 4 years ago by slabbe

Like noticed in #18897, when this code is runned the first time, there are many stuff left. But the second time, the list is empty.

...ok, but that's the same thing for the bug presented in the description of the ticket. Sorry.

comment:8 in reply to: ↑ 5 Changed 4 years ago by SimonKing

Replying to slabbe:

Like noticed in #18897, when this code is runned the first time, there are many stuff left. But the second time, the list is empty.

That's just because there is only one base ring. Do the same while iterating over GF(p), and you'll see how things accumulate.

At #18897, a binary tree was duely deallocated, however it was forgotten to dereference the root node. But here, we have objects that could be deallocated, but aren't. It is a totally different kind of leak.

Changed 4 years ago by SimonKing

Reference chain to finite field after creating its algebraic closure

comment:9 Changed 4 years ago by SimonKing

attachment:test.png results from the following code:

sage: import objgraph, gc
sage: K = GF(31)
sage: n = id(K)
sage: A = K.algebraic_closure()
sage: del A,K
sage: gc.collect()
0
sage: L = [c for c in gc.get_objects() if id(c)==n]
sage: objgraph.show_backrefs(L[0],filename="/home/king/Sage/work/memleak/test.png")
Graph written to /tmp/objgraph-r5RhSM.dot (21 nodes)
Image generated as /home/king/Sage/work/memleak/test.png

Apparently (since all other references are weak or circular), the references that prevent deallocation come from sage.rings.finite_rings.integer_mod.NativeIntStruct.

Last edited 4 years ago by SimonKing (previous) (diff)

comment:10 Changed 4 years ago by SimonKing

It seems that there is a NativeIntStruct created for each finite field. If the NativeIntStruct is small enough, then it has a (multiplication?) table, which holds references to all elements of the finite field. And they have, of course, references to their parent. But what is referencing the NativeIntStruct?

comment:11 Changed 4 years ago by SimonKing

It seems that the reference somehow comes from constructing the algebraic closure, as we have

sage: K = GF(31)
sage: del K
sage: L = [c for c in gc.get_objects() if isinstance(c, sage.rings.finite_rings.integer_mod.NativeIntStruct) and len(c._get_table())==31]
sage: len(L)
0

So, the NativeIntStruct can be garbage collected when we do not construct the algebraic closure.

How can one find a reference chain from the algebraic closure to the NativeIntStruct?

Changed 4 years ago by SimonKing

Another reference chain, towards NativeIntStruct?

comment:12 Changed 4 years ago by SimonKing

attachment:test2.png shows the result of

sage: K = GF(31)
sage: A = K.algebraic_closure()
sage: del A,K
sage: L = [c for c in gc.get_objects() if isinstance(c, sage.rings.finite_rings.integer_mod.NativeIntStruct) and len(c._get_table())==31]
sage: filter = lambda x: (x is not L) and (not isinstance(x, sage.rings.finite_rings.integer_mod.IntegerMod_int))
sage: objgraph.show_backrefs(L[0],filter=filter,filename="/home/king/Sage/work/memleak/test2.png")
Graph written to /tmp/objgraph-4qP1R3.dot (16 nodes)
Image generated as /home/king/Sage/work/memleak/test2.png

The picture somehow looks familiar: There is a coerce map involved, namely !Int_to_IntegerMod. Coerce maps are supposed to have a weak reference to the domain and a strong reference to the codomain. Since the domain of !Int_to_IntegerMod presumably is the ring of integers and can't be deallocated anyway, a strong reference to the codomain means trouble...

comment:13 Changed 4 years ago by slabbe

I was also playing with objgraph...

def test(L, dim):
    import objgraph
    import gc
    from collections import Counter
    gc.collect()
    pre={id(c) for c in gc.get_objects()}
    m = matrix(dim, L)
    for p in range(2,102):
        m.change_ring(GF(nth_prime(p))).eigenvalues()
    gc.collect()
    O = gc.get_objects()
    post=Counter(type(o) for o in O if id(o) not in pre)
    T = [k for (k,v) in post.iteritems() if v==100]
    D = dict((type(o),o) for o in O if type(o) in T)
    for i,v in enumerate(sorted(D.values())):
        print v
        objgraph.show_backrefs(v,filename="test_{}.png".format(i))
        print "--"

I get :

sage: L = [1,1,0,1,0,1,2,1,1,0,0,1,2,0,0,1,1,0,2,0,0,0,0,0,1]
sage: test(L, 5)
Principal ideal (547) of Integer Ring
Graph written to /tmp/objgraph-e803N8.dot (20 nodes)
Image generated as test_0.png
--
Set of Homomorphisms from Integer Ring to Finite Field of size 181
Graph written to /tmp/objgraph-I_uL5V.dot (12 nodes)
Image generated as test_1.png
--
<class 'sage.rings.finite_rings.conway_polynomials.PseudoConwayLattice'>
Graph written to /tmp/objgraph-oMuwMY.dot (13 nodes)
Image generated as test_2.png
--
Algebraic closure of Finite Field of size 181
Graph written to /tmp/objgraph-M_9gmt.dot (14 nodes)
Image generated as test_3.png
--
Finite Field of size 547
Graph written to /tmp/objgraph-ZostIO.dot (25 nodes)
Image generated as test_4.png
--
Set of Homomorphisms from Finite Field of size 181 to Univariate Polynomial Ring in x over Finite Field of size 181
Graph written to /tmp/objgraph-diDGKn.dot (12 nodes)
Image generated as test_5.png
--
<sage.rings.finite_rings.integer_mod.NativeIntStruct object at 0x7f366d1e3590>
Graph written to /tmp/objgraph-JxmEbJ.dot (30 nodes)
Image generated as test_6.png
--

I will attach test_3.png for the algebraic closure right now.

Last edited 4 years ago by slabbe (previous) (diff)

Changed 4 years ago by slabbe

Ref graph for Algebraic closure of Finite Field of size 181

comment:14 Changed 4 years ago by SimonKing

Let me try to recall why the current "weak-referencing coerce maps" are made as they are, to understand why we have a strong reference chain to Int_to_IntegerMod.

  • We have a backtracking algorithm to find coerce maps by transitivity of "registered" coerce maps. Hence, registered coerce maps need to be stored in the codomain.
  • Since the maps are stored in a container that is stored as an attribute of the codomain, there is a strong reference chain from the codomain to the coerce map.
  • A strong reference to the codomain should not prevent the domain from garbage collection. Hence, IN COERCION, we use a weak reference from the map to the domain, and we cut the reference from the map to its parent.
  • A strong reference to the domain should not prevent the codomain from garbage collection. That's what seems to fail in the current example.

What I do not understand: Why is that more than a strong reference CYCLE (which would not prevent garbage collection) from the codomain to the map and back?

comment:15 Changed 4 years ago by SimonKing

Aha. The coercion model caches the coerce maps, too. So, it isn't the codomain only that holds a cache.

The coercion model is of course a permanent object. It references a TripleDict to store the coercion maps. The coercion map references the codomain. Hence, if the domain is strongly referenced from somewhere, then the coerce map and thus the codomain can not be garbage collected.

Can we perhaps make it so that the cache in the coercion model only keeps a WEAK reference to the coerce map? I worry about performance, though, since getting the referenced object from a weak reference is a bit costly.

Do we need to worry about premature collection of coerce map? If I understand correctly, the cache in the coercion model is mainly for performance, as the map is cached as an attribute of the codomain anyway. And the codomain of any map is a parent. Hence, the cache of the coercion model actually is redundant.

So, perhaps a better idea is to completely get rid of the coercion model cache, as the coercion model can use the codomain's cache. I'll try that.

comment:16 Changed 4 years ago by SimonKing

No, the coercion model's cache is needed. It is relevant for pushouts. There, we have no codomain, as it first needs to be constructed.

comment:17 Changed 4 years ago by SimonKing

Perhaps my diagnosis was wrong: It is not relevant for pushouts, but we want to cache the ABSENCE of a coercion from a parent to, say, int. So, if the codomain happens to be a parent then we can use its cache; otherwise there is no problem to use the coercion model's cache since int and friends will never be garbage collected anyway.

So, better not use weak references to coercion maps...

comment:18 Changed 4 years ago by SimonKing

It seems that #14058 is relevant -- and perhaps it fixes our problem (except that it has no branch).

comment:19 Changed 4 years ago by SimonKing

I am afraid that #14058 (which now has a branch) does not suffice to fix the problem: GF(31) still can't be collected after creating its algebraic closure. However, the coercion model is not mentioned any longer. Instead, the reference goes via a weak value dictionary, which seems to be the polynomial ring cache.

My guess:

  • There is some polynomial ring P over the base ring GF(31) that was constructed when creating the algebraic closure of GF(31).
  • There is only a weak reference to P from the polynomial ring cache (it is weak value dictionary). However, for a different reason there is a strong reference to P.
  • Hence, the entry for P in the weak value dictionary can not be freed. By consequence, the reference to the KEY of this entry, which is a STRONG reference to GF(31), can not be freed.
  • Thus, GF(31) can not be deallocated.

The question is: How is there a strong reference to P? Perhaps there is a strong reference from GF(31) to P? Maybe via caching the algebraic closure, which references an element of P? This would be enough to keep P alive.

comment:20 Changed 4 years ago by jmantysalo

  • Cc jmantysalo added

comment:21 Changed 4 years ago by SimonKing

I tried to find a backref chain as follows:

sage: import objgraph, gc, __main__
sage: K = GF(31)
sage: A = K.algebraic_closure()
sage: nK = id(K)
sage: del A,K
sage: L = [c for c in gc.get_objects() if id(c) == nK]
sage: objgraph.find_backref_chain(L[0], lambda x: x in __main__.__dict__.values(), extra_ignore=(id(L),))

but it crashes with a segfault. Any idea why it fails?

comment:22 Changed 4 years ago by SimonKing

And this one

sage: import objgraph, gc
sage: cm = sage.structure.element.get_coercion_model()
sage: K = GF(31)
sage: A = K.algebraic_closure()
sage: nK = id(K)
sage: del A,K
sage: gc.collect()
0
sage: L = [c for c in gc.get_objects() if id(c) == nK]
sage: objgraph.find_backref_chain(L[0], lambda x: x is cm, extra_ignore=(id(L),))

doesn't finish after several minutes.

comment:23 follow-up: Changed 4 years ago by nbruin

I get pretty good plots from

objgraph.show_backrefs(L,filename='plot.png',max_depth=5)

It seems a global link exists here:

sage: [a for a in sage.rings.polynomial.polynomial_ring_constructor.__dict__['_cache'].keys() if id(a[0]) == id(L[0])]
[(Finite Field of size 31, ('x',), False, None),
 (Finite Field of size 31, ('x',), False, 'FLINT')]

comment:24 in reply to: ↑ 23 ; follow-up: Changed 4 years ago by SimonKing

Replying to nbruin:

I get pretty good plots from

objgraph.show_backrefs(L,filename='plot.png',max_depth=5)

Ahm, L?? That's a list.

It seems a global link exists here:

sage: [a for a in sage.rings.polynomial.polynomial_ring_constructor.__dict__['_cache'].keys() if id(a[0]) == id(L[0])]
[(Finite Field of size 31, ('x',), False, None),
 (Finite Field of size 31, ('x',), False, 'FLINT')]

Of course. As long as P=GF(31)['x'] lives, its base ring will live, too. The question is why the polynomial ring can't be collected. So, perhaps it would be better to try and find a chain for the polynomial ring instead.

Changed 4 years ago by SimonKing

Backref graph for a polynomial ring over finite field

comment:25 Changed 4 years ago by SimonKing

attachment:test_P_14058.png is interesting. It seems that the reference chain to the polynomial ring is via TRACEBACKS! The question then arises: Why are the tracebacks not garbage collected?

comment:26 in reply to: ↑ 24 ; follow-up: Changed 4 years ago by nbruin

Replying to SimonKing:

Of course. As long as P=GF(31)['x'] lives, its base ring will live, too. The question is why the polynomial ring can't be collected. So, perhaps it would be better to try and find a chain for the polynomial ring instead.

OK, the finite field caches under algebraic_closure in its dict a CachedMethodCaller that references a PseudoConwayLattice object that in its __dict__ has a ring that is the polynomial ring. That's a reference chain from the finite field to the polynomial ring, preventing the polynomial ring from being deallocated.

comment:27 Changed 4 years ago by SimonKing

Oops. I see that the chain goes via objgraph. So, I guess the new attachment is worthless.

comment:28 in reply to: ↑ 26 Changed 4 years ago by SimonKing

Replying to nbruin:

OK, the finite field caches under algebraic_closure in its dict a CachedMethodCaller that references a PseudoConwayLattice object that in its __dict__ has a ring that is the polynomial ring. That's a reference chain from the finite field to the polynomial ring, preventing the polynomial ring from being deallocated.

And, by comment:19, it prevents the base ring from collection, because it is not simply a cyclic reference, but a strong reference from a weak value dictionary to one of its keys.

comment:29 follow-up: Changed 4 years ago by SimonKing

Why does the CachedMethodCaller reference a PseudoConwayLattice? Shouldn't it "only" reference the return value?

comment:30 in reply to: ↑ 29 Changed 4 years ago by SimonKing

Replying to SimonKing:

Why does the CachedMethodCaller reference a PseudoConwayLattice? Shouldn't it "only" reference the return value?

I see. The algebraic closure itself references the PseudoConwayLattice.

Hm. At some point we should use a weak reference. Maybe right here.

comment:31 Changed 4 years ago by SimonKing

The lattice that is stored in the algebraic closure is either passed as an argument to the init method, or is constructed during init:

    def __init__(self, base_ring, name, category=None, lattice=None, use_database=True):
        if not (is_FiniteField(base_ring) and base_ring.is_prime_field()):
            raise NotImplementedError('algebraic closures of finite fields are only implemented for prime fields')
        from sage.rings.finite_rings.conway_polynomials import PseudoConwayLattice
        p = base_ring.characteristic()
        if lattice is None:
            lattice = PseudoConwayLattice(p, use_database)
        elif not isinstance(lattice, PseudoConwayLattice) or lattice.p != p:
            raise TypeError('lattice must be a pseudo-Conway lattice with characteristic %s' % p)
        self._pseudo_conway_lattice = lattice
        AlgebraicClosureFiniteField_generic.__init__(self, base_ring, name, category)

That's a dilemma. If "lattice" is not passed as an argument, it is no problem to weakly reference it, as it can be reconstructed, should it be garbage collected. But otherwise? Hm.

comment:32 Changed 4 years ago by SimonKing

I think I know a potential solution.

The problem, by the above analysis: The polynomial ring cache is a weak value dictionary. Generally, a strong reference chain from key to value will prevent garbage collection of the key-value pair.

I suggest to remove the "global" polynomial ring cache. Instead, I suggest that the polynomial ring constructor uses a weak value dictionary that is stored as an attribute of the base ring (e.g., in self.__cached_methods, which is available for all parents).

The weakly referenced values are polynomial rings. The keys are the list of variable names and information on term order and implementation---so, strong references to them shouldn't be problematic.

In that model, a strong reference chain from the base ring to the polynomial ring would NOT prevent garbage collection, since in the worst case it is a reference cycle (base ring <-> polynomial ring).

comment:33 Changed 4 years ago by SimonKing

Hooray! With that change, I get

sage: import objgraph, gc
sage: K = GF(31)
sage: A = K.algebraic_closure()
sage: n = id(K)
sage: del A,K
sage: gc.collect()
186
sage: L = [c for c in gc.get_objects() if id(c) == n]
sage: L
[]

comment:34 Changed 4 years ago by SimonKing

  • Branch set to u/SimonKing/memleak_for_integer_mod_rel_14058

comment:35 Changed 4 years ago by SimonKing

  • Authors set to Simon King
  • Commit set to 3ada19d1acf58d1d085c1908b3058de8377901a4
  • Dependencies changed from #18897 to #18897, #14058
  • Status changed from new to needs_review

Last 10 new commits:

6d34077Simplify code for deallocation of binary trees
80ce876Further simplification
ac68d9bRemove strong references to parents used in binary operations in the coercion model.
e153d08#14058: Add doctest
b9141eeMerge branch 'ticket/14058' into develop
90ed181Trivial fix for a coercion doctest
64b572crefcount libsingular rings used in plural
9793dbcMake one test more stable
219fbf4Merge branch 't/14058/weakly_reference_binary_operation_codomains' into t/18905/memleak_for_integer_mod_rel_14058
3ada19dReplace the global polynomial ring cache by a cache in the base ring

comment:36 Changed 4 years ago by SimonKing

To my slight surprise, replacing the global polynomial ring cache by a local cache did not only solve the issue tracked here, but it did not introduce a new problem: With the attached branch, all tests should pass.

I have reviewed most part of #14058, but I think someone should have a look at my additions (review patch) there and finish the review.

comment:37 follow-up: Changed 4 years ago by vdelecroix

  • Status changed from needs_review to needs_work

From the patchbot

sage -t --long src/sage/structure/coerce.pyx
**********************************************************************
File "src/sage/structure/coerce.pyx", line 1307, in sage.structure.coerce.CoercionModel_cache_maps.coercion_maps
Failed example:
    print N2-N0
Expected:
    0
Got:
    -1

Is it what we should get?

comment:38 in reply to: ↑ 37 ; follow-up: Changed 4 years ago by SimonKing

Replying to vdelecroix:

From the patchbot ... Is it what we should get?

Note that the same error appears at #14058, even though the commits from here are not part of #14058, if I see that correctly.

Could someone verify if it is really the case that tests pass with "develop", but fail with the branch from here (merged in "develop" of course)? I currently do not have the bandwidth.

comment:39 in reply to: ↑ 38 ; follow-up: Changed 4 years ago by vdelecroix

Replying to SimonKing:

Replying to vdelecroix: Could someone verify if it is really the case that tests pass with "develop", but fail with the branch from here (merged in "develop" of course)? I currently do not have the bandwidth.

It does fail, see #14058 comment 62.

comment:40 in reply to: ↑ 39 ; follow-up: Changed 4 years ago by SimonKing

Replying to vdelecroix:

Replying to SimonKing:

Replying to vdelecroix: Could someone verify if it is really the case that tests pass with "develop", but fail with the branch from here (merged in "develop" of course)? I currently do not have the bandwidth.

It does fail, see #14058 comment 62.

Please be clearer. It fails in what setting? Does it only fail with #14058? Does it also fail with develop? Does it also fail with the branch from here?

If it fails both here and at #14058, but not with develop, then I reckon both branches trigger a memory leak that was introduced elsewhere.

comment:41 in reply to: ↑ 40 Changed 4 years ago by SimonKing

Replying to SimonKing:

Please be clearer. It fails in what setting? Does it only fail with #14058? Does it also fail with develop? Does it also fail with the branch from here?

Sorry for the noise. I just notice that #14058 is a dependency for the ticket here. So, we should focus on #14058.

comment:42 Changed 4 years ago by SimonKing

bump...

comment:43 follow-up: Changed 4 years ago by slabbe

I think the description of the ticket should be updated because I don't see those instances with multiple of one hundred anymore on sage-6.9.rc0. (Exactly what leaks are fixed in this ticket?)

Is the following line needed?

+from sage.structure.parent import Parent

Other than that, I verified that the branches indeed fixes the problem mentioned at comment 33. When the two thing above are fixed, to me, it will be a positive review.

comment:44 in reply to: ↑ 43 Changed 4 years ago by slabbe

Replying to slabbe:

because I don't see those instances with multiple of one hundred anymore on sage-6.9.rc0.

Sorry, I am retracting here as I do see them on sage-6.9.rc0:

sage: L = [1,1,0,1,0,1,2,1,1,0,0,1,2,0,0,1,1,0,2,0,0,0,0,0,1]
sage: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:sage: def test_mod_100(L, dim):
:....:     import gc
:....:     from collections import Counter
:....:     gc.collect()
:....:     pre={id(c) for c in gc.get_objects()}
:....:     m = matrix(dim, L)
:....:     for p in range(2,102):
:....:         m.change_ring(GF(nth_prime(p))).eigenvalues()
:....:     gc.collect()
:....:     post=Counter(type(o) for o in gc.get_objects() if id(o) not in pre)
:....:     return [(k,v) for (k,v) in post.iteritems() if v%100==0]
:--
sage: test_mod_100(L, 5)
[(<class 'sage.rings.algebraic_closure_finite_field.AlgebraicClosureFiniteField_pseudo_conway_with_category'>, 100),
 (<class 'sage.rings.homset.RingHomset_generic_with_category'>, 100),
 (<class 'sage.rings.finite_rings.conway_polynomials.PseudoConwayLattice'>, 100),
 (<type 'sage.rings.finite_rings.integer_mod.Int_to_IntegerMod'>, 200),
 (<type 'sage.rings.finite_rings.integer_mod.NativeIntStruct'>, 100),
 (<class 'sage.rings.finite_rings.finite_field_prime_modn.FiniteField_prime_modn_with_category'>, 100),
 (<type 'sage.categories.map.FormalCompositeMap'>, 200),
 (<type 'sage.rings.polynomial.polynomial_element.PolynomialBaseringInjection'>, 200),
 (<class 'sage.rings.ideal.Ideal_pid'>, 100),
 (<class 'sage.rings.homset.RingHomset_quo_ring_with_category'>, 100)]

and they do disappear with this ticket on top on sage-6.9.rc0:

sage: test_mod_100(L, 5)
[]

That being said, I do see some stuff on the second, third, fourth execution of test(L, 5):

sage: test(L, 5)
[(<type 'tuple'>, 5198),
 (<class 'sage.rings.algebraic_closure_finite_field.AlgebraicClosureFiniteField_pseudo_conway_with_category.element_class'>,  101),
 (<type 'sage.rings.finite_rings.element_givaro.FiniteField_givaroElement'>,  194),
 (<type 'sage.rings.finite_rings.element_pari_ffelt.FiniteFieldElement_pari_ffelt'>,  1104),
 (<type 'dict'>, 102)]

Is this problematic?

Last edited 4 years ago by slabbe (previous) (diff)

comment:45 Changed 3 years ago by chapoton

  • Branch changed from u/SimonKing/memleak_for_integer_mod_rel_14058 to public/18905
  • Commit changed from 3ada19d1acf58d1d085c1908b3058de8377901a4 to e952388409f97f3e40b9e4d56ab3e42b6cb13b88
  • Dependencies #18897, #14058 deleted
  • Milestone changed from sage-6.8 to sage-8.0

New commits:

e952388Merge branch 'u/SimonKing/memleak_for_integer_mod_rel_14058' in 8.0.b8
Note: See TracTickets for help on using tickets.