Sage: Ticket #12215: Memleak in UniqueRepresentation, @cached_method
https://trac.sagemath.org/ticket/12215
<p>
The documentation says that <a class="missing wiki">UniqueRepresentation?</a> uses weak refs, but this was switched over to the @cached_method decorator. The latter does currently use strong references, so unused unique parents stay in memory forever:
</p>
<pre class="wiki">import sage.structure.unique_representation
len(sage.structure.unique_representation.UniqueRepresentation.__classcall__.cache)
for i in range(2,1000):
ring = ZZ.quotient(ZZ(i))
vectorspace = ring^2
import gc
gc.collect()
len(sage.structure.unique_representation.UniqueRepresentation.__classcall__.cache)
</pre><p>
Related tickets:
</p>
<ul><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a> (needs review, introducing weak references for caching homsets), and
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a> (needs review, using weak references for caching coerce maps).
</li><li><a class="closed ticket" href="https://trac.sagemath.org/ticket/5970" title="defect: Weak references in Polynomial Ring cache (closed: duplicate)">#5970</a> (the polynomial rings cache use strong references, which may now be a duplicate, as I introduce the weak cache in <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>)
</li></ul><p>
Further notes:
</p>
<ul><li>not everything in Python can be weakref'ed, for example <code></code>None<code></code> cannot.
</li><li>some results that are expensive to compute should not just be cached by a weak reference. Perhaps there is place for a permanent cache, or maybe some minimal age before garbage collecting it.
</li></ul><p>
<span class="underline">Apply</span>
</p>
<ul><li><a class="attachment" href="https://trac.sagemath.org/attachment/ticket/12215/trac12215_weak_cached_function_combined.patch" title="Attachment 'trac12215_weak_cached_function_combined.patch' in Ticket #12215">trac12215_weak_cached_function_combined.patch</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/12215/trac12215_weak_cached_function_combined.patch" title="Download"></a>
</li><li><a class="attachment" href="https://trac.sagemath.org/attachment/ticket/12215/trac12215_safe_callback.patch" title="Attachment 'trac12215_safe_callback.patch' in Ticket #12215">trac12215_safe_callback.patch</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/12215/trac12215_safe_callback.patch" title="Download"></a>
</li></ul>en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/12215
Trac 1.1.6vbraunWed, 21 Dec 2011 20:04:42 GMTcc set
https://trac.sagemath.org/ticket/12215#comment:1
https://trac.sagemath.org/ticket/12215#comment:1
<ul>
<li><strong>cc</strong>
<em>SimonKing</em> added
</li>
</ul>
TicketSimonKingWed, 21 Dec 2011 20:31:58 GMTdescription changed
https://trac.sagemath.org/ticket/12215#comment:2
https://trac.sagemath.org/ticket/12215#comment:2
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=2">diff</a>)
</li>
</ul>
TicketSimonKingWed, 21 Dec 2011 21:51:34 GMT
https://trac.sagemath.org/ticket/12215#comment:3
https://trac.sagemath.org/ticket/12215#comment:3
<p>
See my comment at <a class="closed ticket" href="https://trac.sagemath.org/ticket/5970" title="defect: Weak references in Polynomial Ring cache (closed: duplicate)">#5970</a>: It seems that having a weak version of cached_function (which is used to decorate <code>UniqueRepresentation.__classcall__</code> is the missing bit (in addition to <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a> and a two-line change in the polynomial ring constructor) for fixing the issues at <a class="closed ticket" href="https://trac.sagemath.org/ticket/5970" title="defect: Weak references in Polynomial Ring cache (closed: duplicate)">#5970</a>.
</p>
<p>
I think this should be done on top of <a class="closed ticket" href="https://trac.sagemath.org/ticket/11115" title="enhancement: Rewrite cached_method in Cython (closed: fixed)">#11115</a>, which rewrites cached methods and already has a positive review.
</p>
TicketSimonKingWed, 21 Dec 2011 22:08:38 GMTdependencies set
https://trac.sagemath.org/ticket/12215#comment:4
https://trac.sagemath.org/ticket/12215#comment:4
<ul>
<li><strong>dependencies</strong>
set to <em>#11115</em>
</li>
</ul>
TicketSimonKingWed, 21 Dec 2011 22:41:12 GMT
https://trac.sagemath.org/ticket/12215#comment:5
https://trac.sagemath.org/ticket/12215#comment:5
<p>
Here is a patch. It isn't tested yet.
</p>
TicketSimonKingWed, 21 Dec 2011 22:53:40 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:6
https://trac.sagemath.org/ticket/12215#comment:6
<ul>
<li><strong>dependencies</strong>
changed from <em>#11115</em> to <em>#11115 #11900</em>
</li>
</ul>
<p>
... and I immediately updated the patch: Join categories were not using unique representation but cached_function (by <a class="closed ticket" href="https://trac.sagemath.org/ticket/11900" title="defect: Serious regression caused by #9138 (closed: fixed)">#11900</a>). So, that had to change.
</p>
TicketSimonKingWed, 21 Dec 2011 23:03:49 GMT
https://trac.sagemath.org/ticket/12215#comment:7
https://trac.sagemath.org/ticket/12215#comment:7
<p>
Sorry, it was impossible to use weak_cached_function on the join function in sage.categories.category, since it may return a list (not weakly referenceable). Hence, I had to work around. With the attached patch (applied on top of <a class="closed ticket" href="https://trac.sagemath.org/ticket/11900" title="defect: Serious regression caused by #9138 (closed: fixed)">#11900</a> and its dependencies), sage at least starts...
</p>
TicketSimonKingWed, 21 Dec 2011 23:16:48 GMT
https://trac.sagemath.org/ticket/12215#comment:8
https://trac.sagemath.org/ticket/12215#comment:8
<p>
It turns out that all the patches can still not fix the problem. We also have to deal with <code>sage.structure.factory.UniqueFactory</code>.
</p>
<p>
I suggest to add an option to <code>UniqueFactory</code>, that decides whether a strong or a weak cache is used. And I suggest to do this <em>here</em>, because I don't want to create yet another ticket.
</p>
<p>
The applications of <code>UniqueFactory</code> should mainly be in cases where weak references work. Therefore I suggest to use the weak cache by default - I am curious how many doc tests will fail...
</p>
<p>
Coercion sucks.
</p>
TicketSimonKingWed, 21 Dec 2011 23:35:07 GMT
https://trac.sagemath.org/ticket/12215#comment:9
https://trac.sagemath.org/ticket/12215#comment:9
<p>
It turns out that <code>UniqueFactory</code> already was somehow using weak references, but in an improper way. The new patch version replaces that by <code>WeakValueDictionary</code>.
</p>
<p>
It doesn't solve the problem, though.
</p>
TicketSimonKingThu, 22 Dec 2011 08:27:50 GMT
https://trac.sagemath.org/ticket/12215#comment:10
https://trac.sagemath.org/ticket/12215#comment:10
<p>
I have slightly updated my patch, so that there is no conflict with <a class="closed ticket" href="https://trac.sagemath.org/ticket/11935" title="enhancement: Make parent/element classes independent of base rings (closed: fixed)">#11935</a>.
</p>
TicketSimonKingThu, 22 Dec 2011 08:36:54 GMT
https://trac.sagemath.org/ticket/12215#comment:11
https://trac.sagemath.org/ticket/12215#comment:11
<p>
There is yet another location where it makes sense to use @weak_cached_function: For the cache of dynamic classes!
</p>
<p>
Namely, dynamic classes are frequently used in the category framework, they have a strong cache, and the parent/element classes keep a pointer to the category they belong to. So, that's preventing categories from being garbage collected.
</p>
<p>
I think that my patches from here, <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, and <a class="closed ticket" href="https://trac.sagemath.org/ticket/11935" title="enhancement: Make parent/element classes independent of base rings (closed: fixed)">#11935</a> (which reduces the number of dynamic classes created) might actually be enough to fix the problem. When I run
</p>
<pre class="wiki">sage: for p in primes(2,1000000):
....: R = GF(p)['x','y','z']
....: print get_memory_usage()
</pre><p>
then one initially still sees an increased memory usage. But after a while it seems to stabilise.
</p>
TicketSimonKingThu, 22 Dec 2011 18:18:14 GMTstatus, description changed
https://trac.sagemath.org/ticket/12215#comment:12
https://trac.sagemath.org/ticket/12215#comment:12
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=12">diff</a>)
</li>
</ul>
<p>
I have updated the patch. It documents the changes, and at least the tests in sage/misc/cachefunc.pyx, in sage/categories/..., in sage/rings/... and in sage/structure/unique_representation.py pass.
</p>
<p>
Hence, needs review!
</p>
TicketSimonKingThu, 22 Dec 2011 18:18:30 GMTauthor set
https://trac.sagemath.org/ticket/12215#comment:13
https://trac.sagemath.org/ticket/12215#comment:13
<ul>
<li><strong>author</strong>
set to <em>Simon King</em>
</li>
</ul>
TicketSimonKingThu, 22 Dec 2011 18:20:21 GMTdescription changed
https://trac.sagemath.org/ticket/12215#comment:14
https://trac.sagemath.org/ticket/12215#comment:14
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=14">diff</a>)
</li>
</ul>
TicketSimonKingFri, 23 Dec 2011 22:29:40 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:15
https://trac.sagemath.org/ticket/12215#comment:15
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>segfaults for elliptic curves</em>
</li>
</ul>
<p>
While the tests in sage/categories, sage/rings and sage/structure/unique_representation.py pass, I get some segfaults for the elliptic curve tests. Thus, needs work.
</p>
TicketSimonKingFri, 23 Dec 2011 22:46:22 GMT
https://trac.sagemath.org/ticket/12215#comment:16
https://trac.sagemath.org/ticket/12215#comment:16
<p>
I did <code>sage -t --verbose "devel/sage-main/sage/schemes/elliptic_curves/ell_point.py"</code>, and it did <em>not</em> reveal a segfault while running the tests. The test process itself crashed:
</p>
<pre class="wiki">830 tests in 54 items.
830 passed and 0 failed.
Test passed.
The doctested process was killed by signal 11
[23.8 s]
----------------------------------------------------------------------
The following tests failed:
sage -t --verbose "devel/sage-main/sage/schemes/elliptic_curves/ell_point.py" # Killed/crashed
</pre><p>
Strange.
</p>
TicketSimonKingSat, 24 Dec 2011 23:45:34 GMT
https://trac.sagemath.org/ticket/12215#comment:17
https://trac.sagemath.org/ticket/12215#comment:17
<p>
I think I found the problem.
</p>
<p>
Some doctest of the form
</p>
<pre class="wiki">sage: K.residue_field()
<expected answer>
</pre><p>
segfaults. But when the result is assigned to a variable, like this
</p>
<pre class="wiki">sage: RF = K.residue_field(); RF
<expected answer>
</pre><p>
then everything works.
</p>
<p>
Is it perhaps the case that garbage collection of the residue field (that was enabled by my patch) happens between the creation and the computation of the string representation of the object?
</p>
<p>
But that is strange. There are variables <code>_</code> and <code>__</code>, which are supposed to provide strong references to the last two results - hence, there should be no garbage collection.
</p>
TicketSimonKingSat, 24 Dec 2011 23:53:35 GMT
https://trac.sagemath.org/ticket/12215#comment:18
https://trac.sagemath.org/ticket/12215#comment:18
<p>
<code>sage.structure.factory.UniqueFactory</code> did use weak references before. But it did so - I think - improperly, namely without using <code>weakref.WeakValueDictionary</code>. The new patch version changes that.
</p>
<p>
It isn't ready for review, yet, because of the segfaults.
</p>
TicketSimonKingTue, 27 Dec 2011 10:16:25 GMT
https://trac.sagemath.org/ticket/12215#comment:19
https://trac.sagemath.org/ticket/12215#comment:19
<p>
Some old code is <em>not</em> using the cache: There was some coerce map created in sage/rings/residue_field.pyx, whose parent was not created by <code>Hom(domain,codomain)</code>, but directly by <code>RingHomset(domain,codomain)</code>.
</p>
<p>
Changing it fixed at least one segfault. I wish all segfaults would go away so easily...
</p>
TicketSimonKingTue, 27 Dec 2011 17:44:58 GMT
https://trac.sagemath.org/ticket/12215#comment:20
https://trac.sagemath.org/ticket/12215#comment:20
<p>
Fortunately, I now have a short example that triggers a memory access error when leaving Sage:
</p>
<pre class="wiki">sage: E = EllipticCurve('15a1')
sage: K.<t>=NumberField(x^2+2*x+10)
sage: EK=E.base_extend(K)
sage: EK.torsion_subgroup()
Torsion Subgroup isomorphic to Z/4 + Z/4 associated to the Elliptic Curve defined by y^2 + x*y + y = x^3 + x^2 + (-10)*x + (-10) over Number Field in t with defining polynomial x^2 + 2*x + 10
sage: quit
Exiting Sage (CPU time 0m1.98s, Wall time 0m52.03s).
local/bin/sage-sage: Zeile 303: 30045 Speicherzugriffsfehler sage-ipython "$@" -i
</pre><p>
However, I wonder how I can trigger the error without leaving Sage, and how I can trace what is going on.
</p>
TicketSimonKingTue, 27 Dec 2011 17:53:12 GMT
https://trac.sagemath.org/ticket/12215#comment:21
https://trac.sagemath.org/ticket/12215#comment:21
<p>
Actually <code>EK._torsion_bound(number_of_places=20)</code> is enough to trigger the memory access error.
</p>
TicketvbraunTue, 27 Dec 2011 18:47:04 GMT
https://trac.sagemath.org/ticket/12215#comment:22
https://trac.sagemath.org/ticket/12215#comment:22
<p>
Here is the stack:
</p>
<pre class="wiki">Program terminated with signal 11, Segmentation fault.
#0 cgetg (y=22, x=<optimized out>) at ../src/kernel/none/level1.h:114
114 ../src/kernel/none/level1.h: No such file or directory.
in ../src/kernel/none/level1.h
Traceback (most recent call last):
File "/usr/share/gdb/auto-load/usr/lib64/libstdc++.so.6.0.16-gdb.py", line 59, in <module>
from libstdcxx.v6.printers import register_libstdcxx_printers
File "/usr/lib64/../share/gcc-4.6.2/python/libstdcxx/v6/printers.py", line 19, in <module>
import itertools
ImportError: No module named itertools
Missing separate debuginfos, use: debuginfo-install atlas-3.8.4-1.fc16.x86_64 expat-2.0.1-11.fc15.x86_64 fontconfig-2.8.0-4.fc16.x86_64 keyutils-libs-1.5.2-1.fc16.x86_64 krb5-libs-1.9.2-4.fc16.x86_64 libcom_err-1.41.14-2.fc15.x86_64 libselinux-2.1.6-5.fc16.x86_64 ncurses-libs-5.9-2.20110716.fc16.x86_64 openssl-1.0.0e-1.fc16.x86_64
(gdb) bt
#0 cgetg (y=22, x=<optimized out>) at ../src/kernel/none/level1.h:114
#1 convi (x=0x288b2a8, l=0x7fff6a2f8a38) at ../src/kernel/gmp/mp.c:1288
#2 0x00007f11fb1637ec in itostr_sign (x=<optimized out>, sx=1, len=0x7fff6a2f8b48) at ../src/language/es.c:500
#3 0x00007f11fb167b4f in str_absint (x=0x288b2a8, S=0x7fff6a2f8cb0) at ../src/language/es.c:1778
#4 bruti_intern (g=0x288b2a8, T=<optimized out>, S=0x7fff6a2f8cb0, addsign=1) at ../src/language/es.c:2557
#5 0x00007f11fb168453 in bruti_intern (g=0x288b2d8, T=0x7f11fb4b27a0, S=0x7fff6a2f8cb0, addsign=<optimized out>)
at ../src/language/es.c:2730
#6 0x00007f11fb1679ae in GENtostr_fun (out=0x7f11fb16a7b0 <bruti>, T=0x7f11fb4b27a0, x=0x288b2d8)
at ../src/language/es.c:1645
#7 GENtostr (x=0x288b2d8) at ../src/language/es.c:1651
#8 0x00007f11f5ae5c44 in gcmp_sage (y=0x583d1b8, x=<optimized out>) at sage/libs/pari/misc.h:60
#9 __pyx_f_4sage_4libs_4pari_3gen_3gen__cmp_c_impl (__pyx_v_left=<optimized out>, __pyx_v_right=<optimized out>)
at sage/libs/pari/gen.c:8513
#10 0x00007f11f8663227 in __pyx_f_4sage_9structure_7element_7Element__richcmp_c_impl (__pyx_v_left=0x5780e10,
__pyx_v_right=<optimized out>, __pyx_v_op=2) at sage/structure/element.c:7775
#11 0x00007f11f86875ec in __pyx_f_4sage_9structure_7element_7Element__richcmp (__pyx_v_left=0x5780e10,
__pyx_v_right=0x5863f70, __pyx_v_op=2) at sage/structure/element.c:7498
#12 0x00007f11f5ae045b in __pyx_pf_4sage_4libs_4pari_3gen_3gen_44__richcmp__ (__pyx_v_left=<optimized out>,
__pyx_v_right=<optimized out>, __pyx_v_op=<optimized out>) at sage/libs/pari/gen.c:8475
#13 0x00007f1208b32e6a in try_rich_compare (v=0x5780e10, w=0x5863f70, op=2) at Objects/object.c:619
#14 0x00007f1208b3518d in try_rich_compare_bool (op=<optimized out>, w=<optimized out>, v=<optimized out>)
at Objects/object.c:647
#15 try_rich_to_3way_compare (w=0x5863f70, v=0x5780e10) at Objects/object.c:681
#16 do_cmp (w=0x5863f70, v=0x5780e10) at Objects/object.c:834
#17 PyObject_Compare (v=0x5780e10, w=0x5863f70) at Objects/object.c:863
#18 0x00007f1208af5ae5 in PyObject_Cmp (o1=<optimized out>, o2=<optimized out>, result=0x7fff6a2f8f0c)
at Objects/abstract.c:41
#19 0x00007f1208b879d4 in builtin_cmp (self=<optimized out>, args=<optimized out>) at Python/bltinmodule.c:422
#20 0x00007f1208b917fd in call_function (oparg=<optimized out>, pp_stack=0x7fff6a2f9000) at Python/ceval.c:3706
#21 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2389
#22 0x00007f1208b934d9 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=<optimized out>,
args=<optimized out>, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:2968
#23 0x00007f1208b1f7f6 in function_call (func=0x1fec9b0, arg=0x5864518, kw=0x0) at Objects/funcobject.c:524
#24 0x00007f1208af97a3 in PyObject_Call (func=0x1fec9b0, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#25 0x00007f1208b0667f in instancemethod_call (func=0x1fec9b0, arg=0x5864518, kw=0x0)
at Objects/classobject.c:2579
#26 0x00007f1208af97a3 in PyObject_Call (func=0x579d0f0, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#27 0x00007f1208b545c6 in half_compare (self=<optimized out>, other=<optimized out>) at Objects/typeobject.c:5253
#28 0x00007f1208b547a5 in _PyObject_SlotCompare (self=0x5713af0, other=0x5866af0) at Objects/typeobject.c:5278
#29 0x00007f1208b35260 in do_cmp (w=0x5866af0, v=0x5713af0) at Objects/object.c:817
#30 PyObject_Compare (v=0x5713af0, w=0x5866af0) at Objects/object.c:863
#31 0x00007f1208af5ae5 in PyObject_Cmp (o1=<optimized out>, o2=<optimized out>, result=0x7fff6a2f955c)
at Objects/abstract.c:41
#32 0x00007f1208b879d4 in builtin_cmp (self=<optimized out>, args=<optimized out>) at Python/bltinmodule.c:422
#33 0x00007f1208af97a3 in PyObject_Call (func=0x7f120903e2d8, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#34 0x00007f11e5b541fc in __pyx_pf_4sage_5rings_13residue_field_20ResidueField_generic_8__cmp__ (
__pyx_self=<optimized out>, __pyx_args=<optimized out>, __pyx_kwds=<optimized out>)
at sage/rings/residue_field.c:7317
#35 0x00007f1208af97a3 in PyObject_Call (func=0x22e7200, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
---Type <return> to continue, or q <return> to quit---
#36 0x00007f1208b0667f in instancemethod_call (func=0x22e7200, arg=0x586d320, kw=0x0)
at Objects/classobject.c:2579
#37 0x00007f1208af97a3 in PyObject_Call (func=0x4e32aa0, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#38 0x00007f1208b545c6 in half_compare (self=<optimized out>, other=<optimized out>) at Objects/typeobject.c:5253
#39 0x00007f1208b547a5 in _PyObject_SlotCompare (self=0x57f2410, other=0x5906e90) at Objects/typeobject.c:5278
#40 0x00007f1208b35260 in do_cmp (w=0x5906e90, v=0x57f2410) at Objects/object.c:817
#41 PyObject_Compare (v=0x57f2410, w=0x5906e90) at Objects/object.c:863
#42 0x00007f1208af5ae5 in PyObject_Cmp (o1=<optimized out>, o2=<optimized out>, result=0x7fff6a2f99bc)
at Objects/abstract.c:41
#43 0x00007f1208b879d4 in builtin_cmp (self=<optimized out>, args=<optimized out>) at Python/bltinmodule.c:422
#44 0x00007f1208b917fd in call_function (oparg=<optimized out>, pp_stack=0x7fff6a2f9ab0) at Python/ceval.c:3706
#45 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2389
#46 0x00007f1208b92593 in fast_function (nk=<optimized out>, na=2, n=<optimized out>, pp_stack=0x7fff6a2f9c10,
func=0x3b0a410) at Python/ceval.c:3792
#47 call_function (oparg=<optimized out>, pp_stack=0x7fff6a2f9c10) at Python/ceval.c:3727
#48 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2389
#49 0x00007f1208b934d9 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=<optimized out>,
args=<optimized out>, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:2968
#50 0x00007f1208b1f7f6 in function_call (func=0x3af28c0, arg=0x588d7a0, kw=0x0) at Objects/funcobject.c:524
#51 0x00007f1208af97a3 in PyObject_Call (func=0x3af28c0, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#52 0x00007f1208b0667f in instancemethod_call (func=0x3af28c0, arg=0x588d7a0, kw=0x0)
at Objects/classobject.c:2579
#53 0x00007f1208af97a3 in PyObject_Call (func=0x4e29be0, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#54 0x00007f1208b545c6 in half_compare (self=<optimized out>, other=<optimized out>) at Objects/typeobject.c:5253
#55 0x00007f1208b547a5 in _PyObject_SlotCompare (self=0x579f908, other=0x58c5528) at Objects/typeobject.c:5278
#56 0x00007f1208b34dad in PyObject_RichCompare (v=0x579f908, w=0x58c5528, op=2) at Objects/object.c:967
#57 0x00007f1208b3505f in PyObject_RichCompareBool (v=<optimized out>, w=<optimized out>, op=<optimized out>)
at Objects/object.c:1001
#58 0x00007f1208b49264 in tuplerichcompare (op=2, w=0x5898a70, v=0x578bf38) at Objects/tupleobject.c:546
#59 tuplerichcompare (v=0x578bf38, w=0x5898a70, op=2) at Objects/tupleobject.c:517
#60 0x00007f1208b34d71 in PyObject_RichCompare (v=0x578bf38, w=0x5898a70, op=2) at Objects/object.c:958
#61 0x00007f1208b3505f in PyObject_RichCompareBool (v=<optimized out>, w=<optimized out>, op=<optimized out>)
at Objects/object.c:1001
#62 0x00007f1208b49264 in tuplerichcompare (op=2, w=0x5898ab8, v=0x579c368) at Objects/tupleobject.c:546
#63 tuplerichcompare (v=0x579c368, w=0x5898ab8, op=2) at Objects/tupleobject.c:517
#64 0x00007f1208b34d71 in PyObject_RichCompare (v=0x579c368, w=0x5898ab8, op=2) at Objects/object.c:958
#65 0x00007f1208b3505f in PyObject_RichCompareBool (v=<optimized out>, w=<optimized out>, op=<optimized out>)
at Objects/object.c:1001
#66 0x00007f1208b2f305 in lookdict (mp=0x14c51d0, key=<optimized out>, hash=-1399715627429533172)
at Objects/dictobject.c:351
#67 0x00007f1208b3087c in PyDict_DelItem (op=0x14c51d0, key=0x5898ab8) at Objects/dictobject.c:742
#68 0x00007f1208b8e924 in PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:1555
#69 0x00007f1208b934d9 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=<optimized out>,
args=<optimized out>, argcount=1, kws=0x0, kwcount=0, defs=0x1458be8, defcount=1, closure=0x0)
at Python/ceval.c:2968
#70 0x00007f1208b1f7f6 in function_call (func=0x1464320, arg=0x7f1208fdf510, kw=0x0) at Objects/funcobject.c:524
#71 0x00007f1208af97a3 in PyObject_Call (func=0x1464320, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2492
#72 0x00007f1208afa1e0 in PyObject_CallFunctionObjArgs (callable=0x1464320) at Objects/abstract.c:2723
#73 0x00007f1208bc4146 in handle_weakrefs (old=0x7f1208e52b40, unreachable=0x7fff6a2fa700)
---Type <return> to continue, or q <return> to quit---
at Modules/gcmodule.c:607
#74 collect (generation=2) at Modules/gcmodule.c:859
#75 0x00007f1208bc4b04 in PyGC_Collect () at Modules/gcmodule.c:1292
#76 0x00007f1208bb6d73 in Py_Finalize () at Python/pythonrun.c:424
#77 0x00007f1208bb5c38 in Py_Exit (sts=0) at Python/pythonrun.c:1714
#78 0x00007f1208bb5d2f in handle_system_exit () at Python/pythonrun.c:1116
#79 0x00007f1208bb5fc5 in handle_system_exit () at Python/pythonrun.c:1078
#80 PyErr_PrintEx (set_sys_last_vars=1) at Python/pythonrun.c:1126
#81 0x00007f1208bb643e in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=1,
flags=0x7fff6a2fa9e0) at Python/pythonrun.c:935
#82 0x00007f1208bc35a3 in Py_Main (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:599
#83 0x00007f1207e7569d in __libc_start_main (main=0x400620 <main>, argc=3, ubp_av=0x7fff6a2fab08,
init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff6a2faaf8)
at libc-start.c:226
#84 0x0000000000400651 in _start ()
</pre><p>
For the record, I enabled coredumps and then ran <code>gdb --core core.4522 local/bin/python</code>
</p>
TicketSimonKingTue, 27 Dec 2011 20:07:41 GMT
https://trac.sagemath.org/ticket/12215#comment:23
https://trac.sagemath.org/ticket/12215#comment:23
<p>
Thank you! How does one enable coredumps?
</p>
TicketSimonKingTue, 27 Dec 2011 20:22:52 GMT
https://trac.sagemath.org/ticket/12215#comment:24
https://trac.sagemath.org/ticket/12215#comment:24
<p>
If I understand correctly, the coredump says that it occurs while doing <code>c = cmp(self.p, x.p)</code>, where x and self are residue fields.
</p>
TicketSimonKingTue, 27 Dec 2011 20:25:55 GMT
https://trac.sagemath.org/ticket/12215#comment:25
https://trac.sagemath.org/ticket/12215#comment:25
<p>
Yep, I just inserted a print statement before and after the "cmp" line. When leaving sage, the first line was printed, and the segfault happened before printing the second line. Hence, the problem occurs when comparing fractional ideals.
</p>
TicketvbraunTue, 27 Dec 2011 20:37:28 GMT
https://trac.sagemath.org/ticket/12215#comment:26
https://trac.sagemath.org/ticket/12215#comment:26
<p>
To enable coredumps (at least with bash):
</p>
<pre class="wiki">ulimit -c unlimited
</pre>
TicketSimonKingTue, 27 Dec 2011 22:35:15 GMT
https://trac.sagemath.org/ticket/12215#comment:27
https://trac.sagemath.org/ticket/12215#comment:27
<p>
Aha! A comparison of two sage.libs.pari.gen.gen happens <em>after</em> _unsafe_deallocate_pari_stack is called, which closes pari. That is, of course, bad.
</p>
<p>
Only I wonder how the order can be changed. Alternatively, it could be tested before comparison whether pari is still alive. But that would result in a slow-down.
</p>
TicketSimonKingTue, 27 Dec 2011 22:38:15 GMT
https://trac.sagemath.org/ticket/12215#comment:28
https://trac.sagemath.org/ticket/12215#comment:28
<p>
I guess one must make sure that there is a strong reference to the (unique?) pari instance until all sage.libs.pari.gen.gen are deallocated.
</p>
TicketSimonKingTue, 27 Dec 2011 23:18:44 GMT
https://trac.sagemath.org/ticket/12215#comment:29
https://trac.sagemath.org/ticket/12215#comment:29
<p>
<code>pari._unsafe_deallocate_pari_stack</code> is called in sage.all.quit_sage. It does not help to move it to the end of quit_sage. I wonder why it is not put into a proper <code>__del__</code> method of <code>PariInstance</code>? Is it really needed to be in quit_sage??
</p>
TicketSimonKingTue, 27 Dec 2011 23:22:53 GMT
https://trac.sagemath.org/ticket/12215#comment:30
https://trac.sagemath.org/ticket/12215#comment:30
<p>
Yessss! When removing <code>_unsafe_deallocate_pari_stack</code> from quit_sage and renaming it into a <code>__dealloc__</code> method, then the segfault vanishes!
</p>
TicketSimonKingTue, 27 Dec 2011 23:36:02 GMT
https://trac.sagemath.org/ticket/12215#comment:31
https://trac.sagemath.org/ticket/12215#comment:31
<p>
Too bad. It fixes the segfault of <code>sage -t sage/rings/number_field/number_field_ideal.py</code>, but it doesn't help for <code>sage -t sage/schemes/elliptic_curves/heegner.py</code>.
</p>
<p>
Why is it always the elliptic curves code that causes trouble for my patches?
</p>
TicketSimonKingTue, 27 Dec 2011 23:39:24 GMT
https://trac.sagemath.org/ticket/12215#comment:32
https://trac.sagemath.org/ticket/12215#comment:32
<p>
I have attached a second patch, that fixes two or three segfaults - which isn't enough.
</p>
TicketvbraunWed, 28 Dec 2011 00:02:32 GMTcc changed
https://trac.sagemath.org/ticket/12215#comment:33
https://trac.sagemath.org/ticket/12215#comment:33
<ul>
<li><strong>cc</strong>
<em>jdemeyer</em> added
</li>
</ul>
<p>
In Python one must not use <span class="underline">dealloc</span>() to free C resources, at least not unless you are absolutely certain that the Python object does not participate in circular references.
</p>
<p>
Does it help to do an explicit <code>gc.collect()</code> at the end of <code>quit_sage</code> and only then deallocate Pari? If not we might have to give up clearing the Pari stack...
</p>
TicketSimonKingWed, 28 Dec 2011 01:26:43 GMT
https://trac.sagemath.org/ticket/12215#comment:34
https://trac.sagemath.org/ticket/12215#comment:34
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:33" title="Comment 33">vbraun</a>:
</p>
<blockquote class="citation">
<p>
In Python one must not use <span class="underline">dealloc</span>() to free C resources, at least not unless you are absolutely certain that the Python object does not participate in circular references.
</p>
</blockquote>
<p>
Do you mean <code>__del__</code>? If I remember correctly, <code>__dealloc__</code> is Cython, has nothing to do with the ability of the garbage collector to deal with circular references, and it is what one <em>must</em> have if there are C-resources to free after deleting all Python stuff. So, from all what I know, using <code>__dealloc__</code> (not <code>__del__</code>!) is a clean solution.
</p>
<blockquote class="citation">
<p>
Does it help to do an explicit <code>gc.collect()</code> at the end of <code>quit_sage</code> and only then deallocate Pari? If not we might have to give up clearing the Pari stack...
</p>
</blockquote>
<p>
I didn't try.
</p>
TicketvbraunWed, 28 Dec 2011 10:47:34 GMT
https://trac.sagemath.org/ticket/12215#comment:35
https://trac.sagemath.org/ticket/12215#comment:35
<p>
Yes, you are right: <code>__dealloc__</code> is ok, <code>__del__</code> is not.
</p>
<p>
But the problem seems to be that we finalize Pari before finalizing all Pari elements. Ideally, elements keep their parent alive because they hold a reference but I think GENs are often used in an ad-hoc way in Sage. So moving the Pari finalizer to <code>__dealloc__</code> just makes it run later, but still gives no guarantees about finalizer ordering.
</p>
TicketSimonKingWed, 28 Dec 2011 10:59:50 GMT
https://trac.sagemath.org/ticket/12215#comment:36
https://trac.sagemath.org/ticket/12215#comment:36
<p>
Pari elements have no parent, if I am not mistaken. Adding a parent means: Create an overhead, namely an additional pointer as part of all Pari elements. I am not sure if the number theorists would like that - one might ask on sage-nt.
</p>
TicketvbraunWed, 28 Dec 2011 12:06:18 GMT
https://trac.sagemath.org/ticket/12215#comment:37
https://trac.sagemath.org/ticket/12215#comment:37
<p>
There is <code>sage.rings.pari_ring</code> which implements parents and elements. But when Pari is used in the Sage library its usually directly via its C API.
</p>
<p>
Looking at Python's C API, it seems that <code>Py_AtExit()</code> is what we want: A callback for a cleanup function that is run after Python is finalized. In fact anything from <code>quit_sage()</code> that just finalizes a C library should probably be moved there. See <a class="ext-link" href="http://docs.python.org/c-api/sys.html"><span class="icon"></span>http://docs.python.org/c-api/sys.html</a>
</p>
TicketjdemeyerWed, 28 Dec 2011 21:05:26 GMT
https://trac.sagemath.org/ticket/12215#comment:38
https://trac.sagemath.org/ticket/12215#comment:38
<p>
Not sure why I was added to "cc". But the newly added doctest in <a class="missing attachment">trac12215_segfault_fixes.patch</a> looks bad because there really should be only <strong>one</strong> running <code>PariInstance</code>, since global variables are used for the PARI stack (this is the fault of PARI, not of Sage).
</p>
TicketSimonKingWed, 28 Dec 2011 23:12:06 GMT
https://trac.sagemath.org/ticket/12215#comment:39
https://trac.sagemath.org/ticket/12215#comment:39
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:38" title="Comment 38">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
But the newly added doctest in <a class="missing attachment">trac12215_segfault_fixes.patch</a> looks bad because there really should be only <strong>one</strong> running <code>PariInstance</code>
</p>
</blockquote>
<p>
How else could one test that <code>__dealloc__</code> works?
</p>
TicketjdemeyerWed, 28 Dec 2011 23:20:18 GMT
https://trac.sagemath.org/ticket/12215#comment:40
https://trac.sagemath.org/ticket/12215#comment:40
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:39" title="Comment 39">SimonKing</a>:
</p>
<blockquote class="citation">
<p>
How else could one test that <code>__dealloc__</code> works?
</p>
</blockquote>
<p>
By Sage not crashing upon exit. I don't see any other way here.
</p>
TicketSimonKingWed, 28 Dec 2011 23:26:05 GMT
https://trac.sagemath.org/ticket/12215#comment:41
https://trac.sagemath.org/ticket/12215#comment:41
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:40" title="Comment 40">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:39" title="Comment 39">SimonKing</a>:
</p>
<blockquote class="citation">
<p>
How else could one test that <code>__dealloc__</code> works?
</p>
</blockquote>
<p>
By Sage not crashing upon exit. I don't see any other way here.
</p>
</blockquote>
<p>
OK. I just thought Sage has a 100% doctest policy.
</p>
TicketSimonKingWed, 11 Jan 2012 17:43:30 GMTwork_issues changed
https://trac.sagemath.org/ticket/12215#comment:42
https://trac.sagemath.org/ticket/12215#comment:42
<ul>
<li><strong>work_issues</strong>
changed from <em>segfaults for elliptic curves</em> to <em>fix it...</em>
</li>
</ul>
<p>
With sage-5.0.prealpha0 plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/11780" title="defect: Creating a polynomial ring over a number field results in a non-unique ... (closed: fixed)">#11780</a> plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a> plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a> plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/12290" title="defect: Fix the hash of matrix spaces and improve its performance (closed: fixed)">#12290</a>, all tests pass. But if one adds the two patches from here, one gets
</p>
<pre class="wiki"> sage -t -force_lib devel/sage/sage/combinat/combinatorial_algebra.py # 4 doctests failed
sage -t -force_lib devel/sage/sage/combinat/partition.py # 3 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/kschur.py # 17 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/sfa.py # 284 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/macdonald.py # 107 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/hall_littlewood.py # 61 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/llt.py # 50 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/monomial.py # 16 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/orthotriang.py # 25 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/elementary.py # 9 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/homogeneous.py # 9 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/dual.py # 87 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/schur.py # 13 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/ns_macdonald.py # 2 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/powersum.py # 17 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/classical.py # 9 doctests failed
sage -t -force_lib devel/sage/sage/combinat/sf/jack.py # 35 doctests failed
sage -t -force_lib devel/sage/sage/combinat/species/product_species.py # 1 doctests failed
sage -t -force_lib devel/sage/sage/combinat/species/composition_species.py # 2 doctests failed
sage -t -force_lib devel/sage/sage/combinat/species/functorial_composition_species.py # 3 doctests failed
sage -t -force_lib devel/sage/sage/combinat/species/generating_series.py # 44 doctests failed
sage -t -force_lib devel/sage/sage/combinat/species/library.py # 4 doctests failed
sage -t -force_lib devel/sage/sage/combinat/species/species.py # 2 doctests failed
sage -t -force_lib devel/sage/sage/libs/pari/gen.pyx # Killed/crashed
</pre><p>
Hopefully most of these errors have a common root.
</p>
TicketSimonKingThu, 12 Jan 2012 11:31:27 GMT
https://trac.sagemath.org/ticket/12215#comment:43
https://trac.sagemath.org/ticket/12215#comment:43
<p>
It seems that a good deal of the errors comes from a method <code>sage.combinat.sf.sf.SymmetricFunctions.register_isomorphism</code>: It registers a coercion, but this is only possible when no coercion has been established for that object before.
</p>
<p>
What should one do: Catch the error and 'not<em> registering the coercion? Or wipe the registered coercions, by calling <code>sage.structure.parent.Parent.unset_coercions_used</code>?
</em></p>
TicketSimonKingThu, 12 Jan 2012 11:43:44 GMT
https://trac.sagemath.org/ticket/12215#comment:44
https://trac.sagemath.org/ticket/12215#comment:44
<p>
I have to slightly modify my preceding statement. The error is raised not if there has been established a coercion for that object before, but if there has been a coercion registered between the <em>two</em> objects before. Anyway, the problem remains the same.
</p>
TicketSimonKingThu, 12 Jan 2012 13:30:56 GMT
https://trac.sagemath.org/ticket/12215#comment:45
https://trac.sagemath.org/ticket/12215#comment:45
<p>
Here is a very short example triggering the error:
</p>
<pre class="wiki">sage: P = JackPolynomialsP(QQ,1)
sage: P([2,1])^2
</pre><p>
Hopefully this is short enough for debugging - I find it quite mysterious, so far.
</p>
TicketSimonKingThu, 12 Jan 2012 13:36:01 GMTcc changed
https://trac.sagemath.org/ticket/12215#comment:46
https://trac.sagemath.org/ticket/12215#comment:46
<ul>
<li><strong>cc</strong>
<em>mhansen</em> added
</li>
</ul>
<p>
Something else is interesting: The error changes when repeating it.
</p>
<pre class="wiki">sage: P = JackPolynomialsP(QQ,1)
sage: p = P([2,1])
sage: p^2
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (56, 0))
ERROR: Internal Python error in the inspect module.
Below is the traceback from this internal error.
Traceback (most recent call last):
...
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/python2.7/site-packages/sage/combinat/sf/sf.pyc in register_isomorphism(self, morphism)
324 mathematically wrong, as above. Use with care!
325 """
--> 326 morphism.codomain().register_coercion(morphism)
327
328 _shorthands = set(['e', 'h', 'm', 'p', 's'])
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/python2.7/site-packages/sage/structure/parent.so in sage.structure.parent.Parent.register_coercion (sage/structure/parent.c:11955)()
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/python2.7/site-packages/sage/structure/parent.so in sage.structure.parent.Parent.register_coercion (sage/structure/parent.c:11889)()
AssertionError: coercion from Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis to Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis already registered or discovered
sage: p^2
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (56, 0))
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
...
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/python2.7/site-packages/sage/combinat/sf/sfa.pyc in _from_cache(self, element, cache_function, cache_dict, **subs_dict)
631 if sum(part) not in cache_dict:
632 cache_function(sum(part))
--> 633 for part2, c2 in cache_dict[sum(part)][part].iteritems():
634 c3 = c*c2
635 if hasattr(c3,'subs'): # c3 may be in the base ring
KeyError: [2, 1]
</pre><p>
Cc to the author of sage/combinat/sf/jack.py.
</p>
TicketSimonKingThu, 12 Jan 2012 13:58:32 GMT
https://trac.sagemath.org/ticket/12215#comment:47
https://trac.sagemath.org/ticket/12215#comment:47
<p>
I inserted some print statement into the <code>register_isomorphism</code> method of symmetric functions. I found that with <em>or</em> without the patch, the isomorphisms are registered both during initialisation of the <code>JackPolynomialsP</code> <em>and</em> before raising an element to a power for the first time:
</p>
<pre class="wiki">sage: P = JackPolynomialsP(QQ,1)
registering Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Power symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Power symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Power symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Power symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Power symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Elementary symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Power symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Power symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Schur symmetric functions as basis
registering Symmetric Function Algebra over Rational Field, Power symmetric functions as basis TO Symmetric Function Algebra over Rational Field, Monomial symmetric functions as basis
sage: p = P([2,1])
sage: p^2
registering Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Elementary symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Homogeneous symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Schur symmetric functions as basis
registering Symmetric Function Algebra over Integer Ring, Power symmetric functions as basis TO Symmetric Function Algebra over Integer Ring, Monomial symmetric functions as basis
JackP[2, 2, 1, 1] + JackP[2, 2, 2] + JackP[3, 1, 1, 1] + 2*JackP[3, 2, 1] + JackP[3, 3] + JackP[4, 1, 1] + JackP[4, 2]
sage: p^2
JackP[2, 2, 1, 1] + JackP[2, 2, 2] + JackP[3, 1, 1, 1] + 2*JackP[3, 2, 1] + JackP[3, 3] + JackP[4, 1, 1] + JackP[4, 2]
</pre><p>
This gives rise to some questions:
</p>
<ul><li>Why are the symmetric functions registering the isomorphisms <em>twice</em>, even without my patch?
</li><li>Why is there no error without my patch? There should be, since double-registration of a coercion is illegal!
</li></ul><p>
I guess, the best solution would be to address the first question: Registering the same thing twice is a waste or resources anyway.
</p>
TicketSimonKingThu, 12 Jan 2012 14:00:01 GMT
https://trac.sagemath.org/ticket/12215#comment:48
https://trac.sagemath.org/ticket/12215#comment:48
<p>
Sorry, I was mistaken! It is not two times the same! The first time it is over the rational field, the second time over the integer ring. So, forget my previous questions.
</p>
TicketSimonKingThu, 12 Jan 2012 14:19:49 GMT
https://trac.sagemath.org/ticket/12215#comment:49
https://trac.sagemath.org/ticket/12215#comment:49
<p>
Now I understand the problem:
</p>
<p>
<code>SymmetricFunctions</code> is a subclass of <code>UniqueRepresentation</code>. By my patch, <code>UniqueRepresentation</code> is using weak references. Apparently <code>SymmetricFunctions</code> therefore can be garbage collected, but - and now comes the strange point - the coercion system still recalls that a coercion to them has already been registered.
</p>
<p>
Anyway, with my patch, <code>SymmetricFunctions(ZZ)</code> and <a class="missing wiki">SymmetricFunctions?</a>(QQ)` are created repeatedly, and that's bad.
</p>
TicketSimonKingThu, 12 Jan 2012 15:14:30 GMTstatus changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:50
https://trac.sagemath.org/ticket/12215#comment:50
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>work_issues</strong>
<em>fix it...</em> deleted
</li>
</ul>
<p>
I updated the second patch, which should solve the problem!!
</p>
<p>
First of all: The segfault in the tests of sage/libs/pari/gen.pyx was due to my test for the new dealloc method. Following Jeroen's advice, I removed it and stated in the docs that Sage not crashing at exit is an indirect doctest.
</p>
<p>
Then, all failures in sage/combinat could be fixed by using a <em>strong</em> cache for <code>SymmetricFunctions(...)</code>. So, I simply overrode the <code>__classcall__</code> method inherited from <code>UniqueRepresentation</code>.
</p>
<p>
I just tested that with the new patch all tests in sage/combinat and sage/libs/pari/gen.pyx pass. The others passed even with the old patch version, so that I am confident that they will pass as well (of course, one must try!).
</p>
TicketSimonKingThu, 12 Jan 2012 16:33:18 GMT
https://trac.sagemath.org/ticket/12215#comment:51
https://trac.sagemath.org/ticket/12215#comment:51
<p>
FWIW, <code>make ptest</code> succeeded.
</p>
TicketSimonKingThu, 19 Jan 2012 10:25:19 GMT
https://trac.sagemath.org/ticket/12215#comment:52
https://trac.sagemath.org/ticket/12215#comment:52
<p>
As I said in my previous post, the tests pass with this patch. The tests also pass with the patch from <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>. However, there are three segfaults that occur when <em>both</em> patches are applied. I have difficulties to trace it down.
</p>
TicketSimonKingThu, 19 Jan 2012 10:54:10 GMTcc changed
https://trac.sagemath.org/ticket/12215#comment:53
https://trac.sagemath.org/ticket/12215#comment:53
<ul>
<li><strong>cc</strong>
<em>vbraun</em> added
</li>
</ul>
<p>
Cc to Volker, because I expect he has enough knowledge to give me some advice on how I could trace down the following segfault.
</p>
<p>
With <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> and the patch from here, <code>sage -t -verbose -force_lib "devel/sage/doc/en/bordeaux_2008/half_integral.rst"</code> segfaults. By inspection of the core file, I found that the segfault occurs during deallocation of a functor.
</p>
<p>
For debugging, I added a <code>__dealloc__</code> method to <code>sage.categories.functor.Functor</code> that writes the type and the address of self and of the two cdef attributes <code>__domain</code> and <code>__codomain</code> to some file. The same is done during initialisation of the functor.
</p>
<p>
And the last lines of the resulting file (before the segfault) are:
</p>
<pre class="wiki">Dealloc Functor <type 'sage.structure.coerce_actions.LeftModuleAction'> at 71023056
Domain <class 'sage.categories.groupoid.Groupoid'> at 75636560
Codom. <class 'sage.categories.commutative_rings.CommutativeRings'> at 15429144
Dealloc Functor <type 'sage.structure.coerce_actions.LeftModuleAction'> at 71023056
Domain <type 'NoneType'> at 140661532564960
Codom. <type 'NoneType'> at 140661532564960
</pre><p>
In other words, the functor is deallocated twice, which is a legitimate reason to segfault.
</p>
<p>
How can I find out why Sage tries to deallocate it twice?
</p>
TicketvbraunFri, 20 Jan 2012 03:25:31 GMT
https://trac.sagemath.org/ticket/12215#comment:54
https://trac.sagemath.org/ticket/12215#comment:54
<p>
Is it actually being finalized twice? To me, it seems that just the malloc bin was reused for a second <code>LeftModuleAction</code> instance. In particular, why would domain and codomain be different in the second destructor call.
</p>
TicketSimonKingFri, 20 Jan 2012 06:30:27 GMT
https://trac.sagemath.org/ticket/12215#comment:55
https://trac.sagemath.org/ticket/12215#comment:55
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:54" title="Comment 54">vbraun</a>:
</p>
<blockquote class="citation">
<p>
In particular, why would domain and codomain be different in the second destructor call.
</p>
</blockquote>
<p>
Because domain and codomain were deleted the first time. The second time, they already are <code>NoneType</code>.
</p>
TicketSimonKingFri, 20 Jan 2012 06:32:37 GMT
https://trac.sagemath.org/ticket/12215#comment:56
https://trac.sagemath.org/ticket/12215#comment:56
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:54" title="Comment 54">vbraun</a>:
</p>
<blockquote class="citation">
<p>
Is it actually being finalized twice? To me, it seems that just the malloc bin was reused for a second <code>LeftModuleAction</code> instance.
</p>
</blockquote>
<p>
And I do believe it is the same instance. Namely, if what you say was right, then we should see a call to "init" between the two deallocations (I made both init and dealloc write to the same log file). But the two deallocations followed directly: No initialisation and no other deallocation in between.
</p>
TicketSimonKingFri, 20 Jan 2012 10:37:05 GMT
https://trac.sagemath.org/ticket/12215#comment:57
https://trac.sagemath.org/ticket/12215#comment:57
<p>
No progress on my side. For my project, it probably means that I have to pick between two evils: Either live with the memleak that would be fixed in <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, or live with the memleak that would be fixed here. Bad.
</p>
TicketSimonKingFri, 20 Jan 2012 12:57:29 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:58
https://trac.sagemath.org/ticket/12215#comment:58
<ul>
<li><strong>milestone</strong>
changed from <em>sage-5.0</em> to <em>sage-4.8</em>
</li>
</ul>
<p>
Now that's weird:
</p>
<p>
When I define
</p>
<pre class="wiki"> def __dealloc__(self):
if self.__domain is not None:
Py_INCREF(self.__domain)
if self.__codomain is not None:
Py_INCREF(self.__codomain)
</pre><p>
for sage.categories.functor.Functor, then the segfault disappears.
</p>
<p>
Can this be a solution? It looks weird.
</p>
TicketSimonKingFri, 20 Jan 2012 14:01:46 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:59
https://trac.sagemath.org/ticket/12215#comment:59
<ul>
<li><strong>milestone</strong>
changed from <em>sage-4.8</em> to <em>sage-5.0</em>
</li>
</ul>
TicketSimonKingFri, 20 Jan 2012 14:27:23 GMTdescription changed
https://trac.sagemath.org/ticket/12215#comment:60
https://trac.sagemath.org/ticket/12215#comment:60
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=60">diff</a>)
</li>
</ul>
<p>
I have updated the second patch, which was about fixing segfaults anyway.
</p>
<p>
As I already stated: I find it weird that the problem is solved by <em>incrementing</em> the reference count of the domain and codomain of an action when the action is deallocated. But it works, i.e., the doctests that used to segfault with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> and the old version of the patches run fine with the new patch version.
</p>
<p>
I need an expert opinion, though, and the full test suite is also to be run.
</p>
<p>
Concerning memleaks, here is the example from the ticket description.
</p>
<p>
With <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> and the patches from here:
</p>
<pre class="wiki">sage: import sage.structure.unique_representation
sage: len(sage.structure.unique_representation.UniqueRepresentation.__classcall__.cache)
135
sage:
sage: for i in range(2,1000):
....: ring = ZZ.quotient(ZZ(i))
....: vectorspace = ring^2
....:
sage: import gc
sage: gc.collect()
16641
sage: len(sage.structure.unique_representation.UniqueRepresentation.__classcall__.cache)
227
</pre><p>
With <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> only:
</p>
<pre class="wiki">sage: import sage.structure.unique_representation
sage: len(sage.structure.unique_representation.UniqueRepresentation.__classcall__.cache)
151
sage:
sage: for i in range(2,1000):
....: ring = ZZ.quotient(ZZ(i))
....: vectorspace = ring^2
....:
sage: import gc
sage: gc.collect()
3805
sage: len(sage.structure.unique_representation.UniqueRepresentation.__classcall__.cache)
5142
</pre><p>
So, it is a clear progress, and IIRC the patches comprise tests against at least one memory leak that is fixed. Needs review!
</p>
<p>
Apply trac12215_weak_cached_function.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingFri, 20 Jan 2012 17:46:33 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:61
https://trac.sagemath.org/ticket/12215#comment:61
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>Fix two tests</em>
</li>
</ul>
<p>
With sage-5.0.prealpha0 plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/11780" title="defect: Creating a polynomial ring over a number field results in a non-unique ... (closed: fixed)">#11780</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11290" title="enhancement: Implementation of non-commutative k-Schur functions in the nilCoxeter ... (closed: fixed)">#11290</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> and the patches from here, make ptest results in
</p>
<pre class="wiki"> sage -t -force_lib devel/sage/sage/combinat/sf/sf.py # 1 doctests failed
sage -t -force_lib devel/sage/sage/categories/category.py # 1 doctests failed
</pre><p>
So, it needs work (because all tests pass when the patches from here are not applied), but it should hopefully be easy to fix.
</p>
TicketvbraunFri, 20 Jan 2012 20:46:27 GMT
https://trac.sagemath.org/ticket/12215#comment:62
https://trac.sagemath.org/ticket/12215#comment:62
<p>
I tried the following in <code>cdef class Action</code>:
</p>
<div class="wiki-code"><div class="code"><pre> <span class="k">def</span> <span class="nf">__cinit__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">print</span> <span class="s">'Action __cinit__ '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="nb">id</span><span class="p">(</span><span class="bp">self</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">__dealloc__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">print</span> <span class="s">'Action __dealloc__ '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="nb">id</span><span class="p">(</span><span class="bp">self</span><span class="p">))</span>
</pre></div></div><p>
then I do get occasionally reused id (=memory address in CPython), for example
</p>
<pre class="wiki"> Action __cinit__ 105376976
Action __dealloc__ 105376976
Action __cinit__ 105376976
Action __dealloc__ 105376976
</pre><p>
But I don't see any double finalizers without the object being constructed in-between. I also don't get any segfault in <code>bordeaux_2008/half_integral.rst</code>.
</p>
TicketvbraunFri, 20 Jan 2012 20:51:26 GMT
https://trac.sagemath.org/ticket/12215#comment:63
https://trac.sagemath.org/ticket/12215#comment:63
<p>
For the record, I have these patches applied on top of sage-4.8.rc0:
</p>
<pre class="wiki">12221_debug.patch
trac_12247_var_construction.patch
9138_flat.patch
trac11900_category_speedup_combined.patch
trac11900_only_fix_singleton_hash.patch
trac11900_doctest.patch
11115_flat.patch
trac_11115_docfix.patch
trac12215_weak_cached_function.patch
trac12215_segfault_fixes.patch
</pre><p>
removed the <code>Py_INCREF(self.__domain)</code> and <code>Py_INCREF(self.__codomain)</code> bandaid. Still no segfault.
</p>
TicketSimonKingFri, 20 Jan 2012 22:29:12 GMT
https://trac.sagemath.org/ticket/12215#comment:64
https://trac.sagemath.org/ticket/12215#comment:64
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:63" title="Comment 63">vbraun</a>:
</p>
<blockquote class="citation">
<p>
For the record, I have these patches applied on top of sage-4.8.rc0:
</p>
<pre class="wiki">12221_debug.patch
trac_12247_var_construction.patch
9138_flat.patch
trac11900_category_speedup_combined.patch
trac11900_only_fix_singleton_hash.patch
trac11900_doctest.patch
11115_flat.patch
trac_11115_docfix.patch
trac12215_weak_cached_function.patch
trac12215_segfault_fixes.patch
</pre><p>
removed the <code>Py_INCREF(self.__domain)</code> and <code>Py_INCREF(self.__codomain)</code> bandaid. Still no segfault.
</p>
</blockquote>
<p>
Sure. As I stated in some post above, the segfault only results when applying both <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> (hence, its dependency <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a> as well) <em>and</em> the (old) patches from here.
</p>
<p>
If you only have the (old or new) patches from here or only have <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>+<a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> then there is no segfault.
</p>
TicketvbraunSat, 21 Jan 2012 00:52:19 GMT
https://trac.sagemath.org/ticket/12215#comment:65
https://trac.sagemath.org/ticket/12215#comment:65
<p>
I ran all doctests and there are a few crashes in <code>functor.so</code> elsewhere. I didn't have to apply any additional patches. It dies with
</p>
<pre class="wiki">Action __cinit__ 84546128
Action __dealloc__ 84546128
Action __cinit__ 84546128
Action __dealloc__ 84546128
Action __cinit__ 84628736
Action __cinit__ 84546128
Action __dealloc__ 84546128
Action __dealloc__ 84546128
/home/vbraun/opt/sage-4.8.rc0/local/lib/libcsage.so(print_backtrace+0x31)[0x7fcc0db1adf6]
/home/vbraun/opt/sage-4.8.rc0/local/lib/libcsage.so(sigdie+0x14)[0x7fcc0db1ae28]
/home/vbraun/opt/sage-4.8.rc0/local/lib/libcsage.so(sage_signal_handler+0x20c)[0x7fcc0db1aa76]
</pre><p>
It seems that its just memory corruption that manifests itself by freeing the object twice. But the error is presumably elsewhere. Also the gdb stack trace is completely corrupted.
</p>
TicketvbraunSat, 21 Jan 2012 01:39:48 GMT
https://trac.sagemath.org/ticket/12215#comment:66
https://trac.sagemath.org/ticket/12215#comment:66
<p>
Here is the stack trace:
</p>
<pre class="wiki">#0 0x00007ffaadb88511 in __pyx_tp_dealloc_4sage_10categories_7functor_Functor (o=0x63ed250) at sage/categories/functor.c:2845
#1 0x00007ffaad970cc8 in __pyx_tp_dealloc_4sage_10categories_6action_Action (o=0x63ed250) at sage/categories/action.c:5943
#2 0x00007ffaad5485a0 in __pyx_tp_dealloc_4sage_9structure_14coerce_actions_ModuleAction (o=0x63ed250) at sage/structure/coerce_actions.c:7505
#3 0x00007ffabbcf8f0c in type_call (type=<optimized out>, args=0x63e09e0, kwds=0x0) at Objects/typeobject.c:748
#4 0x00007ffabbca27a3 in PyObject_Call (func=0x7ffaad754ec0, arg=<optimized out>, kw=<optimized out>) at Objects/abstract.c:2492
#5 0x00007ffaad53ffbb in __pyx_pf_4sage_9structure_14coerce_actions_1detect_element_action (__pyx_self=0x0, __pyx_args=0x63fbb40, __pyx_kwds=0x0)
at sage/structure/coerce_actions.c:4616
#6 0x00007ffabbca27a3 in PyObject_Call (func=0x2683dd0, arg=<optimized out>, kw=<optimized out>) at Objects/abstract.c:2492
#7 0x00007ffaaeb0ea32 in __pyx_f_4sage_9structure_6parent_6Parent_discover_action (__pyx_v_self=0x644ab00, __pyx_v_S=0x6448770,
__pyx_v_op=0x7ffab525aea8, __pyx_v_self_on_left=1) at sage/structure/parent.c:16618
#8 0x00007ffaaed48057 in __pyx_f_4sage_9structure_10parent_old_6Parent_get_action_c_impl (__pyx_v_self=0x644ab00, __pyx_v_S=0x6448770,
__pyx_v_op=0x7ffab525aea8, __pyx_v_self_on_left=1) at sage/structure/parent_old.c:3312
#9 0x00007ffaaed47ea2 in __pyx_pf_4sage_9structure_10parent_old_6Parent_4get_action_impl (__pyx_v_self=0x644ab00, __pyx_args=0x63fb910,
__pyx_kwds=0x0) at sage/structure/parent_old.c:3258
#10 0x00007ffabbca27a3 in PyObject_Call (func=0x636a5a8, arg=<optimized out>, kw=<optimized out>) at Objects/abstract.c:2492
#11 0x00007ffaaed46ee7 in __pyx_f_4sage_9structure_10parent_old_6Parent_get_action_c (__pyx_v_self=0x644ab00, __pyx_v_S=0x6448770,
__pyx_v_op=0x7ffab525aea8, __pyx_v_self_on_left=1, __pyx_skip_dispatch=0) at sage/structure/parent_old.c:2935
#12 0x00007ffaaed4f19d in __pyx_f_4sage_9structure_10parent_old_6Parent__get_action_ (__pyx_v_self=0x644ab00, __pyx_v_other=0x6448770,
__pyx_v_op=0x7ffab525aea8, __pyx_v_self_on_left=1, __pyx_skip_dispatch=0) at sage/structure/parent_old.c:6228
#13 0x00007ffaaeb0b17c in __pyx_f_4sage_9structure_6parent_6Parent_get_action (__pyx_v_self=0x644ab00, __pyx_v_S=0x6448770, __pyx_skip_dispatch=0,
__pyx_optional_args=0x7fff38b8e2f0) at sage/structure/parent.c:15635
#14 0x00007ffaae1fa2e6 in __pyx_f_4sage_9structure_6coerce_24CoercionModel_cache_maps_discover_action (__pyx_v_self=0x26286d0, __pyx_v_R=0x644ab00,
__pyx_v_S=0x6448770, __pyx_v_op=0x7ffab525aea8, __pyx_skip_dispatch=0) at sage/structure/coerce.c:12473
#15 0x00007ffaae1f6564 in __pyx_f_4sage_9structure_6coerce_24CoercionModel_cache_maps_get_action (__pyx_v_self=0x26286d0, __pyx_v_R=0x644ab00,
__pyx_v_S=0x6448770, __pyx_v_op=0x7ffab525aea8, __pyx_skip_dispatch=0) at sage/structure/coerce.c:11424
#16 0x00007ffaae1e64e2 in __pyx_f_4sage_9structure_6coerce_24CoercionModel_cache_maps_bin_op (__pyx_v_self=0x26286d0, __pyx_v_x=0x6354b48,
__pyx_v_y=0x63e36b0, __pyx_v_op=0x7ffab525aea8, __pyx_skip_dispatch=0) at sage/structure/coerce.c:6583
#17 0x00007ffaae448f03 in __pyx_pf_4sage_9structure_7element_6Vector_1__mul__ (__pyx_v_left=0x6354b48, __pyx_v_right=0x63e36b0)
at sage/structure/element.c:16130
#18 0x00007ffabbc9dc5f in binary_op1 (v=0x6354b48, w=0x63e36b0, op_slot=16) at Objects/abstract.c:917
#19 0x00007ffabbca0cc8 in PyNumber_Multiply (v=0x6354b48, w=0x63e36b0) at Objects/abstract.c:1188
#20 0x00007ffa9be33b68 in __pyx_f_4sage_5rings_13residue_field_12ReductionMap__call_ (__pyx_v_self=0x63f10e8, __pyx_v_x=0x6405108,
__pyx_skip_dispatch=0) at sage/rings/residue_field.c:8140
</pre><p>
within <code>coercion_model.bin_op()</code> (frame 17) there are calls to Python methods (<code>PyObject_Call</code>), and in there the garbage collector is free to run. I suspect that this is what is happening somewhere...
</p>
TicketSimonKingSat, 21 Jan 2012 09:47:49 GMT
https://trac.sagemath.org/ticket/12215#comment:67
https://trac.sagemath.org/ticket/12215#comment:67
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:65" title="Comment 65">vbraun</a>:
</p>
<blockquote class="citation">
<p>
I ran all doctests and there are a few crashes in <code>functor.so</code> elsewhere. I didn't have to apply any additional patches.
</p>
</blockquote>
<p>
What exactly do you mean? Do you have the <em>old</em> patches from here applied (i.e., without the new <code>__dealloc__</code> method), or does the segfault even occur with the <em>new</em> patches?
</p>
<p>
Is it normal that both you and me see segfaults, and it seems to be analogous problems (namely double deallocation), but we see it in different examples and with different patches (namely, even with the <em>old</em> patches from here, all tests pass for me)?
</p>
<blockquote class="citation">
<p>
It dies with
...
It seems that its just memory corruption that manifests itself by freeing the object twice.
</p>
</blockquote>
<p>
So, you can confirm that it is the same object.
</p>
<blockquote class="citation">
<p>
But the error is presumably elsewhere. Also the gdb stack trace is completely corrupted.
</p>
</blockquote>
<p>
That sounds like one should write a complete log of all python code executed - according to your suggestion that the error somewhere occurs during a Python method.
</p>
TicketSimonKingSun, 22 Jan 2012 14:46:31 GMT
https://trac.sagemath.org/ticket/12215#comment:68
https://trac.sagemath.org/ticket/12215#comment:68
<p>
Here is some more info on the segfault.
</p>
<p>
Setting: I have sage-5.0.prealpha0 plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/11780" title="defect: Creating a polynomial ring over a number field results in a non-unique ... (closed: fixed)">#11780</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11290" title="enhancement: Implementation of non-commutative k-Schur functions in the nilCoxeter ... (closed: fixed)">#11290</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> and the patches from here, removing the <code>__dealloc__</code> method introduced by the last patch.
</p>
<p>
The segfault is triggered by doing
</p>
<pre class="wiki">sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 3, 10)
[]
sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 5, 10)
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/libcsage.so(print_backtrace+0x31)[0x7fe047add9c6]
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/libcsage.so(sigdie+0x14)[0x7fe047add9f8]
/home/simon/SAGE/sage-5.0.prealpha0/local/lib/libcsage.so(sage_signal_handler+0x20c)[0x7fe047add646]
/lib64/libpthread.so.0(+0xfd00)[0x7fe04cd80d00]
...
</pre><p>
When I revert the lines, that's to say, if I do
</p>
<pre class="wiki">sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 5, 10)
[q - 2*q^3 - 2*q^5 + 4*q^7 - q^9 + O(q^10)]
sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 3, 10)
[]
sage: quit
Exiting Sage (CPU time 0m2.02s, Wall time 0m20.49s).
**********************************************************************
Oops, Sage crashed. We do our best to make it stable, but...
A crash report was automatically generated with the following information:
- A verbatim copy of the crash traceback.
- A copy of your input history during this session.
- Data on your current Sage configuration.
It was left in the file named:
'/home/simon/.sage/ipython/Sage_crash_report.txt'
If you can email this file to the developers, the information in it will help
them in understanding and correcting the problem.
You can mail it to: sage-support at sage-support@googlegroups.com
with the subject 'Sage Crash Report'.
If you want to do it now, the following command will work (under Unix):
mail -s 'Sage Crash Report' sage-support@googlegroups.com < /home/simon/.sage/ipython/Sage_crash_report.txt
To ensure accurate tracking of this issue, please file a report about it at:
http://trac.sagemath.org/sage_trac
Press enter to exit:
</pre><p>
I was tracing all python commands for the first variant of the segfault. The last few lines of the log are as follows:
</p>
<pre class="wiki">sage.categories.pushout:__call__:2125 if self.p == other.p:
sage.categories.pushout:__call__:2126 from sage.all import Infinity
sage.categories.pushout:__call__:2127 if self.prec == other.prec:
sage.categories.pushout:__call__:2128 extras = self.extras.copy()
sage.categories.pushout:__call__:3102 except CoercionException:
sage.categories.pushout:__call__:3104 except (TypeError, ValueError, AttributeError, NotImplementedError), ex:
sage.categories.pushout:__call__:3108 raise CoercionException(ex)
weakref:__call__:49 self = selfref()
weakref:__call__:50 if self is not None:
weakref:__call__:51 del self.data[wr.key]
sage.rings.power_series_ring:__call__:556 s = "Power Series Ring in %s over %s"%(self.variable_name(), self.base_ring())
sage.rings.power_series_ring:__call__:557 if self.is_sparse():
sage.rings.power_series_ring:__call__:562 return self.__is_sparse
sage.rings.power_series_ring:__call__:559 return s
</pre><p>
So, indeed it seems that the problem has something to do with weak references. There is an item of a weak value dictionary deleted right before segfaulting.
</p>
<p>
To do: Find out what item of what dictionary is deleted, why it is deleted, and how deletion can be prevented.
</p>
TicketSimonKingSun, 22 Jan 2012 15:20:47 GMT
https://trac.sagemath.org/ticket/12215#comment:69
https://trac.sagemath.org/ticket/12215#comment:69
<p>
I was also tracing the deletion of items of weak value dictionaries: I was writing the key to a log file whenever an item was deleted.
</p>
<p>
Already when starting sage, we see that the same key (and presumably the same value as well) is deleted repeatedly:
</p>
<pre class="wiki">...
((<class 'sage.categories.category.JoinCategory'>, (Category of semirings, Category of infinite enumerated sets)), ())
((<class 'sage.categories.groupoid.Groupoid'>, Integer Ring), ())
((<class 'sage.categories.groupoid.Groupoid'>, Rational Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Rational Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Rational Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Complex Lazy Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Rational Field), ())
</pre><p>
When issuing the first line of the crashing example and repeating it, we see something like
</p>
<pre class="wiki">...
((<class 'sage.categories.groupoid.Groupoid'>, Complex Lazy Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Complex Lazy Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Cyclotomic Field of order 4 and degree 2), ())
((<class 'sage.matrix.matrix_space.MatrixSpace'>, Rational Field, 0, 0, False), ())
((<class 'sage.matrix.matrix_space.MatrixSpace'>, Rational Field, 10, 0, False), ())
((5, 0, 'prealpha0'), (Rational Field, 0, False, None))
((<class 'sage.matrix.matrix_space.MatrixSpace'>, Rational Field, 0, 0, False), ())
((<class 'sage.matrix.matrix_space.MatrixSpace'>, Rational Field, 10, 0, False), ())
</pre><p>
And at crashing, one has
</p>
<pre class="wiki">((<class 'sage.matrix.matrix_space.MatrixSpace'>, Ring of integers modulo 46337, 4, 10, False), ())
((<class 'sage.categories.vector_spaces.VectorSpaces'>, Ring of integers modulo 46337), ())
((<class 'sage.matrix.matrix_space.MatrixSpace'>, Integer Ring, 4, 10, False), ())
((<class 'sage.matrix.matrix_space.MatrixSpace'>, Rational Field, 4, 10, False), ())
((<class 'sage.categories.groupoid.Groupoid'>, Power Series Ring in q over Rational Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Power Series Ring in q over Rational Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Power Series Ring in q over Rational Field), ())
((<class 'sage.categories.groupoid.Groupoid'>, Power Series Ring in q over Integer Ring), ())
</pre><p>
Conclusion:
</p>
<p>
The occurring keys indicate that the deletions occur in <code>UniqueRepresentation</code>. While using weak references for <code>UniqueRepresentation</code> fixes memory leaks, it seems that far too often stuff is removed that would actually still be needed. Certainly it is bad for speed, and it seems that it is also responsible for the segmentation faults.
</p>
<p>
I am not sure how that problem should best be addressed.
</p>
TicketSimonKingFri, 09 Mar 2012 12:33:59 GMTwork_issues changed
https://trac.sagemath.org/ticket/12215#comment:70
https://trac.sagemath.org/ticket/12215#comment:70
<ul>
<li><strong>work_issues</strong>
changed from <em>Fix two tests</em> to <em>Fix a coercion problem in sage.combinat.sf.sf</em>
</li>
</ul>
<p>
I think I have not properly stated that with the latest patches applied to sage-5.0.prealpha0, the segfault is gone. However, at least when I also have a couple of other tickets (<a class="closed ticket" href="https://trac.sagemath.org/ticket/11780" title="defect: Creating a polynomial ring over a number field results in a non-unique ... (closed: fixed)">#11780</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12290" title="defect: Fix the hash of matrix spaces and improve its performance (closed: fixed)">#12290</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>. <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12357" title="defect: Make groupoids garbage collectable (closed: duplicate)">#12357</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12351" title="defect: AttributeError raised by method __eq__ of poset element (closed: fixed)">#12351</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/7797" title="enhancement: Full interface to letterplace from singular (closed: fixed)">#7797</a>), I get one coercion error in sage.combinat.sf.sf.
</p>
<p>
To be precise, I do not get that error when I only have all the other patches. So, it really seems to be caused by the patches from here. Trying to track it down...
</p>
TicketSimonKingFri, 09 Mar 2012 12:59:57 GMT
https://trac.sagemath.org/ticket/12215#comment:71
https://trac.sagemath.org/ticket/12215#comment:71
<p>
That's odd. The failing test is from the <code>__call__</code> method in sage.combinat.sf.sf. When I execute things in the command line, I get the following:
</p>
<pre class="wiki">sage: Sym = SymmetricFunctions(QQ[x])
sage: p = Sym.p(); s = Sym.s()
sage: P = p[1].parent()
sage: S = s[1].parent()
sage: P.coerce_map_from(S)
Generic morphism:
From: Symmetric Function Algebra over Univariate Polynomial Ring in x over Rational Field, Schur symmetric functions as basis
To: Symmetric Function Algebra over Univariate Polynomial Ring in x over Rational Field, Power symmetric functions as basis
sage: S.coerce_map_from(P)
Generic morphism:
From: Symmetric Function Algebra over Univariate Polynomial Ring in x over Rational Field, Power symmetric functions as basis
To: Symmetric Function Algebra over Univariate Polynomial Ring in x over Rational Field, Schur symmetric functions as basis
</pre><p>
However, when the same is executed as a doctest, then <em>there is no coercion map</em> between S and P. Could it be that some other doctest is messing with the coercion maps, and my patch (perhaps in combination with <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>) reveals it?
</p>
TicketSimonKingFri, 09 Mar 2012 15:30:18 GMTstatus, dependencies changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:72
https://trac.sagemath.org/ticket/12215#comment:72
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900</em> to <em>#11115 #11900 #12645</em>
</li>
<li><strong>work_issues</strong>
<em>Fix a coercion problem in sage.combinat.sf.sf</em> deleted
</li>
</ul>
<p>
That's even odder. With <a class="closed ticket" href="https://trac.sagemath.org/ticket/11780" title="defect: Creating a polynomial ring over a number field results in a non-unique ... (closed: fixed)">#11780</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12290" title="defect: Fix the hash of matrix spaces and improve its performance (closed: fixed)">#12290</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>. <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12357" title="defect: Make groupoids garbage collectable (closed: duplicate)">#12357</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12351" title="defect: AttributeError raised by method __eq__ of poset element (closed: fixed)">#12351</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/7797" title="enhancement: Full interface to letterplace from singular (closed: fixed)">#7797</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/12645" title="defect: Fix rst markup for sage/combinat/sf/sf.py (and add to manual) and ... (closed: fixed)">#12645</a> (so, adding <a class="closed ticket" href="https://trac.sagemath.org/ticket/12645" title="defect: Fix rst markup for sage/combinat/sf/sf.py (and add to manual) and ... (closed: fixed)">#12645</a>, which only changes the rst markup in sage/combinat/sf/sf.py), all tests in sage/combinat pass.
</p>
<p>
Anyway. Since the second patch is in conflict with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12645" title="defect: Fix rst markup for sage/combinat/sf/sf.py (and add to manual) and ... (closed: fixed)">#12645</a> anyway, I am rebasing it. Since the doctest error has vanished, I put it back to "needs review", even though I wish I knew what was the reason for the temporary problem.
</p>
TicketSimonKingSat, 10 Mar 2012 10:08:52 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:73
https://trac.sagemath.org/ticket/12215#comment:73
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>coercion in symmetric function algebras</em>
</li>
</ul>
<p>
Bad. Meanwhile I work on top of sage-5.0.beta7. This time, it is the first patch that creates a coercion error in sage/combinat/sf/sf.py. Needs work.
</p>
TicketSimonKingSun, 15 Apr 2012 15:20:49 GMT
https://trac.sagemath.org/ticket/12215#comment:74
https://trac.sagemath.org/ticket/12215#comment:74
<p>
Even worse: After applying related tickets (<a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12357" title="defect: Make groupoids garbage collectable (closed: duplicate)">#12357</a>) to sage-5.0.beta13, 16 out of 18 hunks fail to apply. So, I need to find out where the problem comes from.
</p>
TicketSimonKingSun, 15 Apr 2012 15:25:50 GMTdependencies, work_issues changed
https://trac.sagemath.org/ticket/12215#comment:75
https://trac.sagemath.org/ticket/12215#comment:75
<ul>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645</em> to <em>#11115 #11900 #12645 #11599</em>
</li>
<li><strong>work_issues</strong>
changed from <em>coercion in symmetric function algebras</em> to <em>Rebase wrt #11599. Coercion in symmetric function algebras</em>
</li>
</ul>
<p>
It comes from <a class="closed ticket" href="https://trac.sagemath.org/ticket/11599" title="enhancement: Wrap fan morphism in toric morphism (closed: fixed)">#11599</a>, which fixes the same docstring misformattings that I fix in my patch as well...
</p>
TicketSimonKingSun, 15 Apr 2012 18:15:22 GMT
https://trac.sagemath.org/ticket/12215#comment:76
https://trac.sagemath.org/ticket/12215#comment:76
<p>
Arrgh. With <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11935" title="enhancement: Make parent/element classes independent of base rings (closed: fixed)">#11935</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12357" title="defect: Make groupoids garbage collectable (closed: duplicate)">#12357</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/7797" title="enhancement: Full interface to letterplace from singular (closed: fixed)">#7797</a> on top of sage-5.0.beta13, all tests pass. But adding the (rebased) patch from here, I get failures in
</p>
<pre class="wiki"> sage -t -force_lib "devel/sage/sage/structure/coerce_dict.pyx"
sage -t -force_lib "devel/sage/sage/combinat/sf/macdonald.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/llt.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/jack.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/kschur.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/hall_littlewood.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/sfa.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/multiplicative.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/schur.py"
sage -t -force_lib "devel/sage/sage/combinat/species/library.py"
sage -t -force_lib "devel/sage/sage/combinat/combinatorial_algebra.py"
sage -t -force_lib "devel/sage/sage/categories/homset.py"
</pre><p>
That's not good.
</p>
TicketSimonKingSun, 15 Apr 2012 18:17:02 GMT
https://trac.sagemath.org/ticket/12215#comment:77
https://trac.sagemath.org/ticket/12215#comment:77
<p>
Oops, I had only the first of the two patches from here applied. Nevertheless, it doesn't look good.
</p>
TicketSimonKingSun, 15 Apr 2012 21:42:55 GMTwork_issues changed
https://trac.sagemath.org/ticket/12215#comment:78
https://trac.sagemath.org/ticket/12215#comment:78
<ul>
<li><strong>work_issues</strong>
changed from <em>Rebase wrt #11599. Coercion in symmetric function algebras</em> to <em>Coercion in symmetric function algebras</em>
</li>
</ul>
<p>
I have rebased the first patch relative to <a class="closed ticket" href="https://trac.sagemath.org/ticket/11599" title="enhancement: Wrap fan morphism in toric morphism (closed: fixed)">#11599</a>.
</p>
<p>
With both patches, one "only" has errors in
</p>
<pre class="wiki"> sage -t -force_lib "devel/sage/sage/structure/coerce_dict.pyx"
sage -t -force_lib "devel/sage/sage/combinat/sf/sf.py"
sage -t -force_lib "devel/sage/sage/categories/homset.py"
</pre><p>
So, it still needs work, but it is less bad than I thought...
</p>
<p>
Apply trac12215_weak_cached_function.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingMon, 16 Apr 2012 17:33:07 GMT
https://trac.sagemath.org/ticket/12215#comment:79
https://trac.sagemath.org/ticket/12215#comment:79
<p>
A bit more detail: The tests in coerce_dict.pyx and homset.py fail even if only the first patch is applied. But the tests in sf.py pass if only the first patch is applied.
</p>
TicketSimonKingTue, 17 Apr 2012 10:02:01 GMTwork_issues changed
https://trac.sagemath.org/ticket/12215#comment:80
https://trac.sagemath.org/ticket/12215#comment:80
<ul>
<li><strong>work_issues</strong>
changed from <em>Coercion in symmetric function algebras</em> to <em>Keep the fix from #12313. Coercion in symmetric function algebras</em>
</li>
</ul>
<p>
I tested whether the problem comes from the combination of this ticket with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12357" title="defect: Make groupoids garbage collectable (closed: duplicate)">#12357</a>. But it turns out that the following test
</p>
<pre class="wiki"> sage: K = GF(1<<55,'t')
sage: for i in range(50):
... a = K.random_element()
... E = EllipticCurve(j=a)
... P = E.random_point()
... Q = 2*P
sage: import gc
sage: n = gc.collect()
sage: from sage.schemes.elliptic_curves.ell_finite_field import EllipticCurve_finite_field
sage: LE = [x for x in gc.get_objects() if isinstance(x, EllipticCurve_finite_field)]
sage: len(LE) # indirect doctest
1
</pre><p>
still fails. The test has been introduced in <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>. And of course it is not acceptable that <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> makes a memory leak disappear, but <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> makes it show up again.
</p>
TicketSimonKingTue, 17 Apr 2012 10:16:05 GMT
https://trac.sagemath.org/ticket/12215#comment:81
https://trac.sagemath.org/ticket/12215#comment:81
<p>
I think I located the problem. By some patch, I had introduced a weak dictionary in sage.structure.factory. But somehow I managed to remove the corresponding hunk from the patch. Now, I need to find out where that has happened...
</p>
TicketSimonKingTue, 17 Apr 2012 10:22:19 GMT
https://trac.sagemath.org/ticket/12215#comment:82
https://trac.sagemath.org/ticket/12215#comment:82
<p>
Aha! It turns out that I introduced the <code>WeakValueDictionary</code> in the first patch from here, but somehow I managed to delete it. Now the leak remains fixed, the patch is updated.
</p>
<p>
Apply trac12215_weak_cached_function.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingTue, 17 Apr 2012 10:25:39 GMT
https://trac.sagemath.org/ticket/12215#comment:83
https://trac.sagemath.org/ticket/12215#comment:83
<p>
With the updated version of the first patch (applied on top of <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/11935" title="enhancement: Make parent/element classes independent of base rings (closed: fixed)">#11935</a>), the tests in sage/structure/coerce_dict and sage/categories/homset pass.
</p>
<p>
There remains the problem with symmetric functions, but this is due to the second patch...
</p>
TicketSimonKingTue, 17 Apr 2012 10:34:22 GMTowner deleted
https://trac.sagemath.org/ticket/12215#comment:84
https://trac.sagemath.org/ticket/12215#comment:84
<ul>
<li><strong>owner</strong>
changed from <em>rlm</em> to <em>(none)</em>
</li>
</ul>
<p>
What exactly is the problem?
</p>
<p>
It is
</p>
<pre class="wiki"> sage: S = SymmetricFunctions(ZZ)
sage: S.inject_shorthands()
doctest:...: RuntimeWarning: redefining global value `e`
doctest:...: RuntimeWarning: redefining global value `m`
sage: s[1] + e[2] * p[1,1] + 2*h[3] + m[2,1]
s[1] - 2*s[1, 1, 1] + s[1, 1, 1, 1] + s[2, 1] + 2*s[2, 1, 1] + s[2, 2] + 2*s[3] + s[3, 1]
</pre><p>
The last line fails with an error when doctesting, but works fine when doing the same in an interactive session.
</p>
TicketSimonKingTue, 17 Apr 2012 10:59:53 GMT
https://trac.sagemath.org/ticket/12215#comment:85
https://trac.sagemath.org/ticket/12215#comment:85
<p>
The failure is really strange. If one does
</p>
<pre class="wiki">sage: S = SymmetricFunctions(ZZ)
sage: S.inject_shorthands()
sage: e.has_coerce_map_from(m)
</pre><p>
on the command line, then one gets the answer "True". Doing the same in a <em>separate</em> doctest, one still gets "True". But doing the same in line 384 of sage.combinat.sf.sf.py, one gets "False". So, there seems to be a nasty diffcult-to-debug side effect, which apparently was introduced by the second patch.
</p>
TicketSimonKingTue, 17 Apr 2012 11:08:52 GMT
https://trac.sagemath.org/ticket/12215#comment:86
https://trac.sagemath.org/ticket/12215#comment:86
<p>
The error disappears if one does not override that <code>__classcall__</code> method of symmetric function algebras. However, by the first ticket, it uses a weak cache, which results in many errors elsewhere...
</p>
<p>
But if I recall correctly, there has been a recent ticket dealing with coercion for symmetric functions. Perhaps a miracle occurs and the strongly cached custom <code>__classcall__</code> can be cancelled (count the words that start with "c"...)?
</p>
TicketSimonKingTue, 17 Apr 2012 12:45:50 GMT
https://trac.sagemath.org/ticket/12215#comment:87
https://trac.sagemath.org/ticket/12215#comment:87
<p>
How unfortunate. If I remove the custom (strongly cached) <code>__classcall__</code> of symmetric function algebras, I get
</p>
<pre class="wiki">The following tests failed:
sage -t -force_lib "devel/sage/sage/combinat/sf/macdonald.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/llt.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/jack.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/kschur.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/hall_littlewood.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/classical.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/sfa.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/elementary.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/multiplicative.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/schur.py"
sage -t -force_lib "devel/sage/sage/combinat/sf/homogeneous.py"
sage -t -force_lib "devel/sage/sage/combinat/species/library.py"
sage -t -force_lib "devel/sage/sage/combinat/combinatorial_algebra.py"
</pre><p>
But if one has a custom strong cache for symmetric function algebras, then one has the single failure in
</p>
<pre class="wiki"> sage -t -force_lib "devel/sage/sage/combinat/sf/sf.py"
</pre>
TicketSimonKingTue, 17 Apr 2012 14:16:41 GMT
https://trac.sagemath.org/ticket/12215#comment:88
https://trac.sagemath.org/ticket/12215#comment:88
<p>
Hooray! It turns out that one can fix the failing doctest in sf.py by not only providing a strongly cached <code>sf.SymmetricFunctions.__classcall__</code>, but by additionally providing a strongly cached <code>sfa.SymmetricFunctionAlgebra_generic.__classcall__</code>.
</p>
<p>
It is a bit unfortunate that a strong cache creeps back in, but apparently the assumption of strong caching is extensively used in sage.combinat.sf. Having weakly cached unique representation everywhere except in sage.combinat.sf is at least something...
</p>
<p>
I am now running the full testsuite with the new modification.
</p>
TicketSimonKingTue, 17 Apr 2012 15:52:55 GMTstatus changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:89
https://trac.sagemath.org/ticket/12215#comment:89
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>work_issues</strong>
<em>Keep the fix from #12313. Coercion in symmetric function algebras</em> deleted
</li>
</ul>
<p>
Now the second patch has been updated as well. As I have announced, the second patch is now not only introducing a strong custom cache for <code>SymmetricFunctions</code>, but also for the different representations, i.e., for <code>SymmetricFunctionAlgebra_generic</code>. It is certainly not ideal that symmetric functions need to be strongly cached, but I think that there may be a different ticket to resolve this special case.
</p>
<p>
Anyway, with sage-5.0.beta13 plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11935" title="enhancement: Make parent/element classes independent of base rings (closed: fixed)">#11935</a> and the two patches from here, all doctests pass. I am confident that they would also pass when one only has 5.0.beta13 plus the two patches from here.
</p>
<p>
Needs review, then!
</p>
TicketSimonKingMon, 30 Apr 2012 13:25:56 GMTstatus, dependencies changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:90
https://trac.sagemath.org/ticket/12215#comment:90
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645 #11599</em> to <em>#11115 #11900 #12645 #11599 #12215</em>
</li>
<li><strong>work_issues</strong>
set to <em>rebase wrt #12808</em>
</li>
</ul>
<p>
There is a conflict with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12808" title="enhancement: Optimize ClassCallMetaClass using Cython (closed: fixed)">#12808</a>, which has a positive review. Hence, we need to rebase it!
</p>
TicketnthieryTue, 01 May 2012 21:22:55 GMT
https://trac.sagemath.org/ticket/12215#comment:91
https://trac.sagemath.org/ticket/12215#comment:91
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:90" title="Comment 90">SimonKing</a>:
</p>
<blockquote class="citation">
<p>
There is a conflict with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12808" title="enhancement: Optimize ClassCallMetaClass using Cython (closed: fixed)">#12808</a>, which has a positive review. Hence, we need to rebase it!
</p>
</blockquote>
<p>
(Trivial) rebase done in the updated patch (at the end of the day, I needed it urgently, so I just took over the task).
</p>
TicketSimonKingWed, 02 May 2012 08:57:05 GMTdependencies changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:92
https://trac.sagemath.org/ticket/12215#comment:92
<ul>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645 #11599 #12215</em> to <em>#11115 #11900 #12645 #11599 #12808</em>
</li>
<li><strong>work_issues</strong>
<em>rebase wrt #12808</em> deleted
</li>
</ul>
<p>
Thank you, Nicolas! I planned to do the change today.
</p>
TicketSimonKingWed, 02 May 2012 08:57:31 GMTstatus changed
https://trac.sagemath.org/ticket/12215#comment:93
https://trac.sagemath.org/ticket/12215#comment:93
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
... and I guess it needs review, again?
</p>
TicketSimonKingMon, 16 Jul 2012 21:26:46 GMTstatus, dependencies changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:94
https://trac.sagemath.org/ticket/12215#comment:94
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645 #11599 #12808</em> to <em>#11115 #11900 #12645 #11599 #12808 #7980</em>
</li>
<li><strong>work_issues</strong>
set to <em>Rebase wrt #7980</em>
</li>
</ul>
<p>
Anne mentioned on combinat-devel that the patch needs to be rebased because of <a class="closed ticket" href="https://trac.sagemath.org/ticket/7980" title="enhancement: Implement generic support for parents with (multiple) realizations (closed: fixed)">#7980</a>.
</p>
TicketSimonKingMon, 23 Jul 2012 12:16:24 GMTstatus, description changed
https://trac.sagemath.org/ticket/12215#comment:95
https://trac.sagemath.org/ticket/12215#comment:95
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=95">diff</a>)
</li>
</ul>
<p>
The patches are now rebased rel <a class="closed ticket" href="https://trac.sagemath.org/ticket/7980" title="enhancement: Implement generic support for parents with (multiple) realizations (closed: fixed)">#7980</a>. I have not been able to replace my first patch, because trac believes that it isn't my patch but Nicolas'...
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingMon, 23 Jul 2012 12:16:34 GMTwork_issues deleted
https://trac.sagemath.org/ticket/12215#comment:96
https://trac.sagemath.org/ticket/12215#comment:96
<ul>
<li><strong>work_issues</strong>
<em>Rebase wrt #7980</em> deleted
</li>
</ul>
TicketSimonKingMon, 23 Jul 2012 12:51:45 GMT
https://trac.sagemath.org/ticket/12215#comment:97
https://trac.sagemath.org/ticket/12215#comment:97
<p>
I don't know how, but I managed to mess up the patch. Now it should work.
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingMon, 23 Jul 2012 14:55:55 GMT
https://trac.sagemath.org/ticket/12215#comment:98
https://trac.sagemath.org/ticket/12215#comment:98
<p>
When applying
</p>
<pre class="wiki">trac12969_fix_coercion_cache.patch
trac12215_weak_cached_function-sk.patch
trac12215_segfault_fixes.patch
trac_715_combined.patch
trac_11521_homset_weakcache_combined.patch
trac_12313-mono_dict-combined-random-sk.patch
</pre><p>
on top of sage-5.2.rc0, all tests on bsd.math pass.
</p>
TicketSimonKingWed, 15 Aug 2012 15:22:17 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:99
https://trac.sagemath.org/ticket/12215#comment:99
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>Rebase rel sage-5.3.beta2</em>
</li>
</ul>
<p>
Alas. The second patch does not apply to sage-5.3.beta2. Needs work.
</p>
TicketSimonKingWed, 15 Aug 2012 15:26:04 GMT
https://trac.sagemath.org/ticket/12215#comment:100
https://trac.sagemath.org/ticket/12215#comment:100
<p>
Apparently it is a very severe conflict with sage/combinat/sf/sf.py. Bad, apparently that file has totally changed. I can't even recognise where the patch was supposed to be applied.
</p>
TicketSimonKingWed, 15 Aug 2012 15:32:14 GMTstatus changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:101
https://trac.sagemath.org/ticket/12215#comment:101
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>work_issues</strong>
<em>Rebase rel sage-5.3.beta2</em> deleted
</li>
</ul>
<p>
Perhaps the changes to sage/combinat/sf/sf.py are not needed? I have replaced the second patch by one that does not touch sf.py (in particular, it does not introduce a strong cache to symmetric function algebras). If that won't work, I can still put the necessary changes into a third patch...
</p>
<p>
For the moment:
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingWed, 15 Aug 2012 16:14:20 GMT
https://trac.sagemath.org/ticket/12215#comment:102
https://trac.sagemath.org/ticket/12215#comment:102
<p>
Arrgh. The first patch needs two little changes in the documentation. There were two lines in a doc string that started with an indentation, which was a typo.
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketnbruinWed, 15 Aug 2012 19:27:42 GMT
https://trac.sagemath.org/ticket/12215#comment:103
https://trac.sagemath.org/ticket/12215#comment:103
<p>
Hi Simon,
</p>
<p>
I've read the patches to see if there is anything weird or questionable. It looks good for the most part. Just some things you might be able to comment on:
</p>
<p>
<strong>main patch:</strong>
</p>
<p>
sage/categories/category.py
</p>
<p>
You make some changes to _join, which has the note
</p>
<blockquote>
<p>
It is used for getting a temporary speed-up at trac ticket <a class="closed ticket" href="https://trac.sagemath.org/ticket/11900" title="defect: Serious regression caused by #9138 (closed: fixed)">#11900</a>.
But it is supposed to be replaced by a better solution at trac
ticket <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a>.
</p>
</blockquote>
<p>
and <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a> was merged in 5.1! Is this function still used?
</p>
<p>
Otherwise: Looks good!
</p>
<p>
<strong>sefault_fixes:</strong>
</p>
<p>
sage/combinat/sf/sf.py
</p>
<p>
There is still a change to that file, namely, you add
"from sage.misc.cachefunc import cached_function"
</p>
<p>
which is an odd thing to do in one patch by itself. Perhaps it's a change you failed to revert?
</p>
<p>
INCREF:
</p>
<p>
scary stuff ... I don't like black magic in code, but you seem to know what
you're doing...
</p>
<p>
<strong>Conclusion:</strong>
</p>
<p>
I'd be close to giving a positive review. I haven't run test suits (yet), but you've done a lot of that and the bot should be doing that anyway. The changes themselves look fairly straightforward.
</p>
TicketSimonKingFri, 17 Aug 2012 05:05:56 GMT
https://trac.sagemath.org/ticket/12215#comment:104
https://trac.sagemath.org/ticket/12215#comment:104
<p>
Hi Nils,
</p>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:103" title="Comment 103">nbruin</a>:
</p>
<blockquote class="citation">
<p>
<strong>main patch:</strong>
</p>
<p>
sage/categories/category.py
</p>
<p>
You make some changes to _join, which has the note
</p>
<blockquote>
<p>
It is used for getting a temporary speed-up at trac ticket <a class="closed ticket" href="https://trac.sagemath.org/ticket/11900" title="defect: Serious regression caused by #9138 (closed: fixed)">#11900</a>.
But it is supposed to be replaced by a better solution at trac
ticket <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a>.
</p>
</blockquote>
<p>
and <a class="closed ticket" href="https://trac.sagemath.org/ticket/11943" title="enhancement: The category graph should comply with Python's method resolution order (closed: fixed)">#11943</a> was merged in 5.1! Is this function still used?
</p>
</blockquote>
<p>
Thank you for spotting it. The plan was to replace it, but apparently it wasn't done. So, I removed the note.
</p>
<blockquote class="citation">
<p>
<strong>sefault_fixes:</strong>
</p>
<p>
sage/combinat/sf/sf.py
</p>
<p>
There is still a change to that file, namely, you add
"from sage.misc.cachefunc import cached_function"
</p>
<p>
which is an odd thing to do in one patch by itself. Perhaps it's a change you failed to revert?
</p>
</blockquote>
<p>
I have removed the import. I also removed the <em>strongly</em> cached classcall from sage/combinat/sf/sfa.py: it turns out that a strong cache for symmetric function algebras is not needed any more, after the recent overhaul.
</p>
<blockquote class="citation">
<p>
INCREF:
</p>
<p>
scary stuff ...
</p>
</blockquote>
<p>
I know. I am now testing whether it it still needed.
</p>
<p>
Anyway, it should now be ready for review. I think, to be on the safe side, we should use <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a> as a dependency, which meanwhile has a positive review (thank you!).
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketSimonKingFri, 17 Aug 2012 05:06:21 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:105
https://trac.sagemath.org/ticket/12215#comment:105
<ul>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645 #11599 #12808 #7980</em> to <em>#11115 #11900 #12645 #11599 #12808 #7980 #11521</em>
</li>
</ul>
TicketSimonKingFri, 17 Aug 2012 05:33:34 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:106
https://trac.sagemath.org/ticket/12215#comment:106
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>Do not INCREF when deallocating</em>
</li>
</ul>
<p>
Great! The scary INCREF apparently is not needed!! All tests in sage/schemes and sage/sf pass. The following two examples previously resulted in a segfault, but now they don't:
</p>
<pre class="wiki">sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 3, 10)
[]
sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 5, 10)
[q - 2*q^3 - 2*q^5 + 4*q^7 - q^9 + O(q^10)]
sage: quit
Exiting Sage (CPU time 0m0.36s, Wall time 0m32.82s).
</pre><pre class="wiki">sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 5, 10)
[q - 2*q^3 - 2*q^5 + 4*q^7 - q^9 + O(q^10)]
sage: half_integral_weight_modform_basis(DirichletGroup(16,QQ).1, 3, 10)
[]
sage: quit
Exiting Sage (CPU time 0m0.29s, Wall time 0m4.68s).
</pre><p>
So, I will (a bit later today) update the segfault patch, fortunately removing the odd INCREF.
</p>
TicketSimonKingFri, 17 Aug 2012 06:24:03 GMTstatus changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:107
https://trac.sagemath.org/ticket/12215#comment:107
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>work_issues</strong>
<em>Do not INCREF when deallocating</em> deleted
</li>
</ul>
<p>
The INCREF is gone!
</p>
<p>
Note that in the commit message of the old patch version, I mention a segfault that occurs in combination with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>. But I think - if the segfault still occurs - it should be dealt with there.
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketnbruinFri, 17 Aug 2012 08:14:24 GMT
https://trac.sagemath.org/ticket/12215#comment:108
https://trac.sagemath.org/ticket/12215#comment:108
<p>
Are you sure you produced these patches against a clean 5.3beta1? Your <a class="missing attachment">attachment:trac12215_segfault_fixes.patch</a> still has a (trivial) hunk for <code>sage/combinat/sf/sfa.py</code> which fails to apply for me. If that were the only problem you could just remove that hunk, but I'm also getting quite some doctest failures along the lines of <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:42" title="Comment 42">comment:42</a> (but the pari test is fine now!)
</p>
<p>
I confirm that I'm not seeing any of the problems that necessitated the INCREF hack, so you're good for that. I'm doubtful that the combinatorics stuff is fixed, though ...
</p>
TicketSimonKingFri, 17 Aug 2012 10:13:44 GMT
https://trac.sagemath.org/ticket/12215#comment:109
https://trac.sagemath.org/ticket/12215#comment:109
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:108" title="Comment 108">nbruin</a>:
</p>
<blockquote class="citation">
<p>
Are you sure you produced these patches against a clean 5.3beta1?
</p>
</blockquote>
<p>
I am sure that I produced these patches against 5.3.beta2 (not beta1). I don't know if Jeroen did some last minute changes to beta2; I downloaded it two days ago.
</p>
TicketSimonKingFri, 17 Aug 2012 10:28:14 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:110
https://trac.sagemath.org/ticket/12215#comment:110
<ul>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645 #11599 #12808 #7980 #11521</em> to <em>#11115 #11900 #12645 #11599 #12808 #7980 #11521 #5457</em>
</li>
</ul>
<p>
I guess <a class="closed ticket" href="https://trac.sagemath.org/ticket/5457" title="enhancement: Refactor symmetric functions and k-bounded subspace (closed: fixed)">#5457</a> needs to be added as a dependency. It rewrites the symmetric function code, and was merged in beta2.
</p>
TicketSimonKingFri, 17 Aug 2012 17:31:00 GMT
https://trac.sagemath.org/ticket/12215#comment:111
https://trac.sagemath.org/ticket/12215#comment:111
<p>
The patchbot reports:
</p>
<pre class="wiki">sage -t -force_lib devel/sage-12215/sage/structure/coerce.pyx
**********************************************************************
File "/opt/patchbot-5.3.beta2/devel/sage-12215/sage/structure/coerce.pyx", line 179:
sage: cm.get_stats()
Expected:
((0, 1.0, 4), (0, 0.25, 1))
Got:
((0, 1.0, 4), (0, 0.0, 0))
</pre><p>
Since I do not get that error, it seems that the distribution of data into the buckets is machine dependent. Hence, I suggest to mark that test as random.
</p>
TicketSimonKingFri, 17 Aug 2012 17:50:47 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:112
https://trac.sagemath.org/ticket/12215#comment:112
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>Fix one doc test</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:111" title="Comment 111">SimonKing</a>:
</p>
<blockquote class="citation">
<p>
Since I do not get that error,
</p>
</blockquote>
<p>
Wrong! I <em>do</em> get the same error. Hence, needs work.
</p>
TicketSimonKingFri, 17 Aug 2012 17:54:22 GMTstatus changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:113
https://trac.sagemath.org/ticket/12215#comment:113
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>work_issues</strong>
<em>Fix one doc test</em> deleted
</li>
</ul>
<p>
I fixed the doc test (by modifying the second patch).
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketnbruinFri, 17 Aug 2012 19:04:37 GMT
https://trac.sagemath.org/ticket/12215#comment:114
https://trac.sagemath.org/ticket/12215#comment:114
<p>
Good! indeed, with the new dependency things apply cleanly.
</p>
<pre class="wiki">sage/structure/coerce.pyx
</pre><p>
Agreed. Things get placed in buckets based on address. I'd expect that to depend on not just the machine but even on the run!
</p>
<pre class="wiki">sage/combinat/integer_vectors_mod_permgroup.py
</pre><p>
That one times out for me too. With <code>--verbose</code> I get that all 271 tests pass but then it hangs on
</p>
<pre class="wiki">A workspace appears to have been corrupted... automatically rebuilding (this is harmless).
</pre><p>
which explains a timeout. That might just been my setup, but since one of the bots experienced a similar problem, perhaps this needs attention.
</p>
<p>
I've seen some startup time failures on <code>sage.math</code>, but the machine is rather loaded. I don't think any of the <a class="closed ticket" href="https://trac.sagemath.org/ticket/5457" title="enhancement: Refactor symmetric functions and k-bounded subspace (closed: fixed)">#5457</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> patches are individually responsible for slowdowns.
</p>
<p>
So at this point, it's just technical bits -- nothing intrinsic about the code proposed here.
</p>
TicketSimonKingSat, 18 Aug 2012 06:15:42 GMT
https://trac.sagemath.org/ticket/12215#comment:115
https://trac.sagemath.org/ticket/12215#comment:115
<p>
When I last changed the second patch, I modified the expected result of some cm.get_stats() test. However, I forgot to mark the result "random". You seem to agree that it is indeed random (depending on memory addresses). Hence, I just changed the second patch accordingly.
</p>
<p>
Apply trac12215_weak_cached_function-sk.patch trac12215_segfault_fixes.patch
</p>
TicketnbruinTue, 21 Aug 2012 02:07:25 GMT
https://trac.sagemath.org/ticket/12215#comment:116
https://trac.sagemath.org/ticket/12215#comment:116
<p>
Bot's only complaint:
</p>
<pre class="wiki">sage -t -force_lib devel/sage-12215/sage/sat/boolean_polynomials.py # Killed/crashed
</pre><p>
I cannot confirm this error on a clean 5.3b2 patched up to <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>. It pretty much has to be some assert failing in polybori, because I think that's the only component called from there.
</p>
TicketSimonKingTue, 21 Aug 2012 13:57:10 GMT
https://trac.sagemath.org/ticket/12215#comment:117
https://trac.sagemath.org/ticket/12215#comment:117
<p>
The log says:
</p>
<pre class="wiki">sage -t -force_lib devel/sage-12215/sage/sat/boolean_polynomials.py
The doctested process was killed by signal 6
</pre><p>
Has this anything to do with my patch? What is signal 6?
</p>
TicketnbruinFri, 24 Aug 2012 23:15:24 GMTstatus changed; reviewer set
https://trac.sagemath.org/ticket/12215#comment:118
https://trac.sagemath.org/ticket/12215#comment:118
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
<li><strong>reviewer</strong>
set to <em>Nils Bruin</em>
</li>
</ul>
TicketjdemeyerMon, 27 Aug 2012 11:08:50 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:119
https://trac.sagemath.org/ticket/12215#comment:119
<ul>
<li><strong>milestone</strong>
changed from <em>sage-5.3</em> to <em>sage-5.4</em>
</li>
</ul>
TicketjdemeyerMon, 27 Aug 2012 11:17:47 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:120
https://trac.sagemath.org/ticket/12215#comment:120
<ul>
<li><strong>dependencies</strong>
changed from <em>#11115 #11900 #12645 #11599 #12808 #7980 #11521 #5457</em> to <em>#11521</em>
</li>
</ul>
TicketjdemeyerMon, 10 Sep 2012 06:31:57 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:121
https://trac.sagemath.org/ticket/12215#comment:121
<ul>
<li><strong>milestone</strong>
changed from <em>sage-5.4</em> to <em>sage-pending</em>
</li>
</ul>
TicketSimonKingMon, 17 Sep 2012 09:38:49 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:122
https://trac.sagemath.org/ticket/12215#comment:122
<ul>
<li><strong>dependencies</strong>
changed from <em>#11521</em> to <em>#13447</em>
</li>
</ul>
<p>
There were problems with <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>+<a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a> on OS X. It is solved by <a class="needs_work ticket" href="https://trac.sagemath.org/ticket/13447" title="defect: Make libsingular multivariate polynomial rings collectable (needs_work)">#13447</a>. Hence, moving the dependency...
</p>
TicketjdemeyerSat, 29 Sep 2012 07:14:04 GMT
https://trac.sagemath.org/ticket/12215#comment:123
https://trac.sagemath.org/ticket/12215#comment:123
<p>
Are the dependencies of this ticket correct, does it really depend on <a class="needs_work ticket" href="https://trac.sagemath.org/ticket/13447" title="defect: Make libsingular multivariate polynomial rings collectable (needs_work)">#13447</a>?
</p>
TicketnbruinSat, 29 Sep 2012 09:12:59 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:124
https://trac.sagemath.org/ticket/12215#comment:124
<ul>
<li><strong>dependencies</strong>
changed from <em>#13447</em> to <em>#11521</em>
</li>
</ul>
<p>
No patches were changed, only the dependencies. One of the patches on <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a> ensures that the problem that <a class="needs_work ticket" href="https://trac.sagemath.org/ticket/13447" title="defect: Make libsingular multivariate polynomial rings collectable (needs_work)">#13447</a> solves doesn't occur, so this ticket should be safe while only depending on <a class="closed ticket" href="https://trac.sagemath.org/ticket/715" title="defect: Parents probably not reclaimed due to too much caching (closed: fixed)">#715</a>, <a class="closed ticket" href="https://trac.sagemath.org/ticket/11521" title="defect: Use weak references to cache homsets (closed: fixed)">#11521</a>.
</p>
TicketjdemeyerWed, 03 Oct 2012 14:48:42 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:125
https://trac.sagemath.org/ticket/12215#comment:125
<ul>
<li><strong>milestone</strong>
changed from <em>sage-pending</em> to <em>sage-5.5</em>
</li>
</ul>
TicketjdemeyerTue, 23 Oct 2012 14:55:29 GMT
https://trac.sagemath.org/ticket/12215#comment:126
https://trac.sagemath.org/ticket/12215#comment:126
<p>
When applying this on top of sage-5.5.beta0 (not released), I get doctest failures in <code>sage/crypto/mq</code>. I'll investigate further and report back.
</p>
TicketnbruinFri, 26 Oct 2012 18:03:47 GMT
https://trac.sagemath.org/ticket/12215#comment:127
https://trac.sagemath.org/ticket/12215#comment:127
<p>
I <strong>[edit:] cannot quite</strong> confirm that
</p>
<pre class="wiki">sage -t "devel/sage-main/sage/crypto/mq/mpolynomialsystem.py"
</pre><p>
on sage-5.5.beta0+<a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> on sage.math does not succeed. In fact, when I run it, I usually get "Connection to sage.math.washington.edu closed by remote host.". A screen session doesn't help either, since that gets killed too.
</p>
<p>
It seems to hang somewhere around line 971 (that's the last I see from <code>sage -t --verbose</code>), which is in the doctesting of "coefficient_matrix". Running that doctest by itself doesn't cause any problems.
</p>
<p>
Oops, I had one run now where my connection wasn't closed! I got a " TIMED OUT! PROCESS KILLED!" this time.
</p>
<p>
And in fact, the "connection closed" thing seems to happen quite a bit, so I don't think I have confirmation that there's really a bug. It may be that sage.math is just flaky.
</p>
TicketjdemeyerTue, 30 Oct 2012 08:18:04 GMT
https://trac.sagemath.org/ticket/12215#comment:128
https://trac.sagemath.org/ticket/12215#comment:128
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:127" title="Comment 127">nbruin</a>:
</p>
<blockquote class="citation">
<p>
And in fact, the "connection closed" thing seems to happen quite a bit, so I don't think I have confirmation that there's really a bug. It may be that sage.math is just flaky.
</p>
</blockquote>
<p>
Maybe your ssh program is flaky?
</p>
TicketjdemeyerTue, 30 Oct 2012 16:37:42 GMT
https://trac.sagemath.org/ticket/12215#comment:129
https://trac.sagemath.org/ticket/12215#comment:129
<p>
Rebased patch because it applied with fuzz.
</p>
TicketjdemeyerTue, 30 Oct 2012 16:41:25 GMT
https://trac.sagemath.org/ticket/12215#comment:130
https://trac.sagemath.org/ticket/12215#comment:130
<p>
Irrelevant remark: you might replace <code>trac ticket 12215</code> in documentation by <code>:trac:`12215`</code>.
</p>
TicketnbruinTue, 30 Oct 2012 19:22:17 GMT
https://trac.sagemath.org/ticket/12215#comment:131
https://trac.sagemath.org/ticket/12215#comment:131
<p>
OK, the "connection lost" problem was resolved by <code>rm -rf ~/.sage</code>. I don't know what was screwed up, but something there was making *any* sage session very prone to terminating the whole connection.
</p>
<p>
It seems that line 971 in <code>mpolynomialsystem.py</code> is indeed a problematic doctest. It seems to hang. When I run the doctest in <code>gdb</code> I can interrupt and get a traceback (first bit):
</p>
<pre class="wiki">#0 0x00007fb5259748aa in PyObject_Free (p=0x57d0300) at Objects/obmalloc.c:969
#1 0x00007fb525985dcc in PyTuple_ClearFreeList () at Objects/tupleobject.c:916
#2 0x00007fb525a0d5cb in collect (generation=2) at Modules/gcmodule.c:792
#3 0x00007fb525a0d87e in _PyObject_GC_Malloc (basicsize=<value optimized out>) at Modules/gcmodule.c:996
#4 0x00007fb525a0d92d in _PyObject_GC_New (tp=0x7fb525c801a0) at Modules/gcmodule.c:1467
#5 0x00007fb52595e64c in PyList_New (size=0) at Objects/listobject.c:142
#6 0x00007fb51c6c47e5 in __pyx_pw_4sage_9structure_11coerce_dict_10TripleDict_1__init__ (__pyx_v_self=0xf7abc50,
__pyx_args=<value optimized out>, __pyx_kwds=<value optimized out>) at sage/structure/coerce_dict.c:1440
#7 0x00007fb5259885a8 in type_call (type=<value optimized out>, args=0xf714d0, kwds=0x0) at Objects/typeobject.c:735
#8 0x00007fb52592c308 in PyObject_Call (func=0x7fb51c8d1620, arg=0xf714d0, kw=0x0) at Objects/abstract.c:2529
#9 0x00007fb51cf2d550 in __pyx_f_4sage_9structure_6parent_6Parent_init_coerce (__pyx_v_self=0x4acacf0,
__pyx_optional_args=<value optimized out>) at sage/structure/parent.c:5757
#10 0x00007fb51d17176b in __pyx_f_4sage_9structure_10parent_old_6Parent_init_coerce (__pyx_v_self=0x57d0300,
__pyx_optional_args=<value optimized out>) at sage/structure/parent_old.c:1638
</pre><p>
so it seems <code>TripleDict</code> is implicated. I've tried it a couple of times and the tracebacks are not completely identical all the time, but the <code>collect (generation=2)</code> always seems to be there. So either the system gets stuck in a loop in which it is spending a large percentage of the time in <code>collect</code> or somehow <code>collect</code> itself gets caught in an infinite loop. I guess instrumenting <a class="missing wiki">TripleDict?</a> to see what it's chewing on is most likely to resolve this one (anything below seems python library).
</p>
TicketnbruinWed, 31 Oct 2012 23:45:04 GMT
https://trac.sagemath.org/ticket/12215#comment:132
https://trac.sagemath.org/ticket/12215#comment:132
<p>
Of course this is another heisenbug: If I take the doctesting python file that gets produced, <code>~/.sage/tmp/mpolynomialsystem_*.py</code> and run that through python via
</p>
<pre class="wiki"> $ ./sage -sh
$ python ~/.sage/tmp/mpolynomialsystem_*.py
</pre><p>
the doctest succeeds, which is weird, because that is exactly what <code>sage-doctest</code> is supposed to run too.
</p>
TicketnbruinThu, 01 Nov 2012 17:30:56 GMT
https://trac.sagemath.org/ticket/12215#comment:133
https://trac.sagemath.org/ticket/12215#comment:133
<p>
FWIW, with <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> doctests pass, so perhaps we should just merge them together?
</p>
TicketjdemeyerThu, 01 Nov 2012 17:36:21 GMTdependencies changed
https://trac.sagemath.org/ticket/12215#comment:134
https://trac.sagemath.org/ticket/12215#comment:134
<ul>
<li><strong>dependencies</strong>
changed from <em>#11521</em> to <em>#11521, merge with #12313</em>
</li>
</ul>
TicketjdemeyerMon, 05 Nov 2012 14:50:19 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:135
https://trac.sagemath.org/ticket/12215#comment:135
<ul>
<li><strong>milestone</strong>
changed from <em>sage-5.5</em> to <em>sage-5.6</em>
</li>
</ul>
TicketSimonKingThu, 22 Nov 2012 13:10:38 GMTstatus, dependencies changed; work_issues set
https://trac.sagemath.org/ticket/12215#comment:136
https://trac.sagemath.org/ticket/12215#comment:136
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_work</em>
</li>
<li><strong>dependencies</strong>
changed from <em>#11521, merge with #12313</em> to <em>#11521, #13741, merge with #12313</em>
</li>
<li><strong>work_issues</strong>
set to <em>rebase rel #13741</em>
</li>
</ul>
<p>
This had a positive review, however I think one can not realistically expect it will soon go in: It depends on other tickets, that have a tendency to uncover nasty problems.
</p>
<p>
Hence, I suggest to cut it into smaller pieces - one of them being the Pari deallocation, that is now the new dependency <a class="closed ticket" href="https://trac.sagemath.org/ticket/13741" title="defect: Proper deallocation of the (unique) pari instance (closed: fixed)">#13741</a>. The second patch thus needs to be rebased.
</p>
TicketnthierySat, 29 Dec 2012 12:14:11 GMTstatus changed; work_issues deleted
https://trac.sagemath.org/ticket/12215#comment:137
https://trac.sagemath.org/ticket/12215#comment:137
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>work_issues</strong>
<em>rebase rel #13741</em> deleted
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:136" title="Comment 136">SimonKing</a>:
</p>
<blockquote class="citation">
<p>
Hence, I suggest to cut it into smaller pieces - one of them being the Pari deallocation, that is now the new dependency <a class="closed ticket" href="https://trac.sagemath.org/ticket/13741" title="defect: Proper deallocation of the (unique) pari instance (closed: fixed)">#13741</a>. The second patch thus needs to be rebased.
</p>
</blockquote>
<p>
I just did that rebase for the sage-combinat queue (by removing the two relevant hunks), uploaded the updated patch here and tentatively set this ticket back to needs review.
</p>
TicketjpfloriFri, 04 Jan 2013 10:26:50 GMTcc changed
https://trac.sagemath.org/ticket/12215#comment:138
https://trac.sagemath.org/ticket/12215#comment:138
<ul>
<li><strong>cc</strong>
<em>jpflori</em> added
</li>
</ul>
TicketSimonKingThu, 17 Jan 2013 13:17:05 GMTattachment set
https://trac.sagemath.org/ticket/12215
https://trac.sagemath.org/ticket/12215
<ul>
<li><strong>attachment</strong>
set to <em>trac12215_weak_cached_function_combined.patch</em>
</li>
</ul>
<p>
Implement a weak version of cached_function, and use it for <code>UniqueRepresentation</code>. Properly use <code>WeakValueDictionary</code> in <code>UniqueFactory</code>. Combined patch
</p>
TicketSimonKingThu, 17 Jan 2013 13:18:28 GMTdescription changed
https://trac.sagemath.org/ticket/12215#comment:139
https://trac.sagemath.org/ticket/12215#comment:139
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=139">diff</a>)
</li>
</ul>
<p>
I have combined the two patches into one. I haven't tested it yet, but will do in Sage's debug version.
</p>
<p>
Apply trac12215_weak_cached_function_combined.patch
</p>
TicketSimonKingThu, 17 Jan 2013 21:12:42 GMT
https://trac.sagemath.org/ticket/12215#comment:140
https://trac.sagemath.org/ticket/12215#comment:140
<p>
FWIW: In sage-5.6.rc0 built with SAGE_DEBUG=yes (see <a class="closed ticket" href="https://trac.sagemath.org/ticket/13864" title="task: Configure Python with pydebug when SAGE_DEBUG is set (closed: fixed)">#13864</a>) plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> plus <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, all doctests pass on my x86_64 <code>openSuse</code> 12.1 laptop with <code>MALLOC_CHECK_=3</code>.
</p>
TicketnbruinFri, 18 Jan 2013 18:17:21 GMTstatus changed
https://trac.sagemath.org/ticket/12215#comment:141
https://trac.sagemath.org/ticket/12215#comment:141
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:131" title="Comment 131">nbruin</a>:
</p>
<blockquote class="citation">
<p>
It seems that line 971 in <code>mpolynomialsystem.py</code> is indeed a problematic doctest. It seems to hang.
</p>
</blockquote>
<p>
That behaviour is entirely consistent with a double free (and hence a circular freelist) that we solved in <a class="closed ticket" href="https://trac.sagemath.org/ticket/13896" title="defect: Fix cython's gc_track and gc_untrack (closed: fixed)">#13896</a>. So, back to positive review from me!
</p>
TicketjdemeyerSat, 19 Jan 2013 09:30:54 GMTdependencies, milestone changed
https://trac.sagemath.org/ticket/12215#comment:142
https://trac.sagemath.org/ticket/12215#comment:142
<ul>
<li><strong>dependencies</strong>
changed from <em>#11521, #13741, merge with #12313</em> to <em>merge with #12313 and #13378</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage-5.6</em> to <em>sage-5.7</em>
</li>
</ul>
TicketjdemeyerMon, 21 Jan 2013 21:12:01 GMTdependencies, milestone changed
https://trac.sagemath.org/ticket/12215#comment:143
https://trac.sagemath.org/ticket/12215#comment:143
<ul>
<li><strong>dependencies</strong>
changed from <em>merge with #12313 and #13378</em> to <em>merge with #12313</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage-5.7</em> to <em>sage-pending</em>
</li>
</ul>
TicketnbruinMon, 21 Jan 2013 21:23:09 GMTdependencies deleted
https://trac.sagemath.org/ticket/12215#comment:144
https://trac.sagemath.org/ticket/12215#comment:144
<ul>
<li><strong>dependencies</strong>
<em>merge with #12313</em> deleted
</li>
</ul>
<p>
We only introduced a codependence on <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> because of comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:133" title="Comment 133">133</a>. In view of comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:141" title="Comment 141">141</a> I suspect the source of the problem noted in comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:131" title="Comment 131">131</a> was actually solved, rather than hidden by merging tickets together.
</p>
<p>
Hence, I propose that this ticket is considered without dependencies and be considered for merging in sage-5.7 anyway.
</p>
TicketjdemeyerMon, 21 Jan 2013 22:23:14 GMTmilestone changed
https://trac.sagemath.org/ticket/12215#comment:145
https://trac.sagemath.org/ticket/12215#comment:145
<ul>
<li><strong>milestone</strong>
changed from <em>sage-pending</em> to <em>sage-5.7</em>
</li>
</ul>
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:144" title="Comment 144">nbruin</a>:
</p>
<blockquote class="citation">
<p>
We only introduced a codependence on <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> because of comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:133" title="Comment 133">133</a>. In view of comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:141" title="Comment 141">141</a> I suspect the source of the problem noted in comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:131" title="Comment 131">131</a> was actually solved, rather than hidden by merging tickets together.
</p>
<p>
Hence, I propose that this ticket is considered without dependencies and be considered for merging in sage-5.7 anyway.
</p>
</blockquote>
<p>
Just to have a second opinion: Simon, do you agree?
</p>
TicketSimonKingTue, 22 Jan 2013 07:01:56 GMT
https://trac.sagemath.org/ticket/12215#comment:146
https://trac.sagemath.org/ticket/12215#comment:146
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:145" title="Comment 145">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:144" title="Comment 144">nbruin</a>:
</p>
<blockquote class="citation">
<p>
We only introduced a codependence on <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> because of comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:133" title="Comment 133">133</a>. In view of comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:141" title="Comment 141">141</a> I suspect the source of the problem noted in comment <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:131" title="Comment 131">131</a> was actually solved, rather than hidden by merging tickets together.
</p>
<p>
Hence, I propose that this ticket is considered without dependencies and be considered for merging in sage-5.7 anyway.
</p>
</blockquote>
<p>
Just to have a second opinion: Simon, do you agree?
</p>
</blockquote>
<p>
Yes. Note also <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:52" title="Comment 52">comment:52</a>: It used to be the case that both <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> were fine <em>separately</em>, but problems occurred when they were used <em>together</em>. But in later patch versions or Sage versions, it was observed that having them together makes the Heisenbug magically disappear - and the suggestion to merge them together was based on this observation.
</p>
<p>
Now it very much seems that the Cython upgrade was enough to fix the crashes. We should of course verify that no problems occur when <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> is merged without <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>, but I think there is no reason to merge <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> and <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> together.
</p>
TicketjdemeyerTue, 22 Jan 2013 11:02:30 GMTstatus changed
https://trac.sagemath.org/ticket/12215#comment:147
https://trac.sagemath.org/ticket/12215#comment:147
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
With <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a>+<a class="closed ticket" href="https://trac.sagemath.org/ticket/13378" title="defect: Do not cache the non-existence of coerce/convert map too often, and do ... (closed: fixed)">#13378</a> but without <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>:
</p>
<pre class="wiki">sage -t -force_lib devel/sage/sage/schemes/elliptic_curves/heegner.py
------------------------------------------------------------------------
/release/merger/sage-5.7.beta0/local/lib/libcsage.so(print_backtrace+0x2b)[0x2b9501a4093d]
/release/merger/sage-5.7.beta0/local/lib/libcsage.so(sigdie+0x34)[0x2b9501a40ae4]
/release/merger/sage-5.7.beta0/local/lib/libcsage.so(sage_signal_handler+0x15b)[0x2b9501a40317]
/lib/libpthread.so.0[0x2b94ffa697d0]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/coerce_dict.so[0x2b9508ca14b6]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/coerce_dict.so[0x2b9508ca1ee6]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_CallFunctionObjArgs+0x161)[0x2b94ff6be631]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_ClearWeakRefs+0x44a)[0x2b94ff728b4a]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/category_object.so[0x2b9508656a38]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff719e01]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff716681]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff716681]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6fd66b]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff719e5c]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/categories/functor.so[0x2b95099ab54e]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6ee447]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6ee447]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/coerce_dict.so[0x2b9508c9a967]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff79d13d]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(_PyObject_GC_Malloc+0xee)[0x2b94ff79d89e]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(_PyObject_GC_New+0xd)[0x2b94ff79d94d]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyDict_New+0xcd)[0x2b94ff6fcffd]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/libs/pari/gen.so[0x2b950b87c3e9]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/libs/pari/gen.so[0x2b950b8df1da]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/libs/pari/gen.so[0x2b950b870eaa]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6ecb3e]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6f156d]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6f1b0b]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff7185a8]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/libs/pari/gen.so[0x2b950b873c3e]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/rings/polynomial/polynomial_rational_flint.so[0x2b9514ecad42]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/rings/polynomial/polynomial_rational_flint.so[0x2b9514ecf271]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff7185a8]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x37bf)[0x2b94ff76103f]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6e99b9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/coerce_maps.so[0x2b950fe80b58]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/parent.so[0x2b9508434031]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/rings/number_field/number_field_element.so[0x2b95183388a5]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff7177dc]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x2b94ff75cb26]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6d8b53]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/rings/number_field/number_field_element_quadratic.so[0x2b95185aa867]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff7185a8]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1299)[0x2b94ff75eb19]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6e99b9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/coerce_maps.so[0x2b950fe80b58]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/parent.so[0x2b9508434031]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x2b94ff75cb26]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff75a238]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5de2)[0x2b94ff763662]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6e99b9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/rings/residue_field.so[0x2b951cd4a7b5]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/factory.so[0x2b951431ab1d]
/release/merger/sage-5.7.beta0/local/lib/python/site-packages/sage/structure/factory.so[0x2b9514317940]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1299)[0x2b94ff75eb19]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x69c5)[0x2b94ff764245]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x2b94ff765472]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x714e)[0x2b94ff7649ce]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6e99b9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1299)[0x2b94ff75eb19]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6e99b9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1299)[0x2b94ff75eb19]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6e99b9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0[0x2b94ff6cc8bf]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b94ff6bc308]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1299)[0x2b94ff75eb19]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ae4)[0x2b94ff763364]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x852)[0x2b94ff765352]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x2b94ff765472]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyRun_FileExFlags+0xc1)[0x2b94ff7891f1]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0x1f9)[0x2b94ff7894c9]
/release/merger/sage-5.7.beta0/local/lib/libpython2.7.so.1.0(Py_Main+0xb15)[0x2b94ff79c115]
/lib/libc.so.6(__libc_start_main+0xf4)[0x2b950031e1f4]
python[0x400679]
</pre>
TicketSimonKingTue, 22 Jan 2013 11:24:18 GMT
https://trac.sagemath.org/ticket/12215#comment:148
https://trac.sagemath.org/ticket/12215#comment:148
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12215#comment:147" title="Comment 147">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
With <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a>+<a class="closed ticket" href="https://trac.sagemath.org/ticket/13378" title="defect: Do not cache the non-existence of coerce/convert map too often, and do ... (closed: fixed)">#13378</a> but without <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>:
</p>
<pre class="wiki">sage -t -force_lib devel/sage/sage/schemes/elliptic_curves/heegner.py
</pre></blockquote>
<p>
Ouch. Well, I hope I can reproduce it in Sage-5.6.rc0 debug version.
</p>
TicketSimonKingTue, 22 Jan 2013 11:55:25 GMT
https://trac.sagemath.org/ticket/12215#comment:149
https://trac.sagemath.org/ticket/12215#comment:149
<p>
Fortunately I can confirm it (at least with MALLOC_CHECK_=3). I'm running it now under gdb.
</p>
TicketSimonKingTue, 22 Jan 2013 12:00:08 GMT
https://trac.sagemath.org/ticket/12215#comment:150
https://trac.sagemath.org/ticket/12215#comment:150
<p>
What I get is:
</p>
<pre class="wiki">Program received signal SIGSEGV, Segmentation fault.
0x00007fffedeb9f4c in __pyx_pf_4sage_9structure_11coerce_dict_16TripleDictEraser_2__call__ (__pyx_v_self=0x73982d0, __pyx_v_r=0x7fffea2aaf00) at sage/structure/coerce_dict.c:1107
1107 __pyx_t_10 = PyList_GET_ITEM(__pyx_t_1, (__pyx_v_h % PyList_GET_SIZE(__pyx_t_4)));
(gdb) bt
#0 0x00007fffedeb9f4c in __pyx_pf_4sage_9structure_11coerce_dict_16TripleDictEraser_2__call__ (__pyx_v_self=0x73982d0, __pyx_v_r=0x7fffea2aaf00) at sage/structure/coerce_dict.c:1107
#1 0x00007fffedeb9592 in __pyx_pw_4sage_9structure_11coerce_dict_16TripleDictEraser_3__call__ (__pyx_v_self=0x73982d0, __pyx_args=0x75bfd10, __pyx_kwds=0x0) at sage/structure/coerce_dict.c:966
#2 0x00007ffff79be33e in PyObject_Call (func=0x73982d0, arg=0x75bfd10, kw=0x0) at Objects/abstract.c:2529
#3 0x00007ffff79bf059 in PyObject_CallFunctionObjArgs (callable=0x73982d0) at Objects/abstract.c:2760
#4 0x00007ffff7a64194 in handle_callback (ref=0x7fffea2aaf00, callback=0x73982d0) at Objects/weakrefobject.c:881
#5 0x00007ffff7a645e9 in PyObject_ClearWeakRefs (object=0x90c07b0) at Objects/weakrefobject.c:965
#6 0x00007fffee53af5b in __pyx_tp_dealloc_4sage_9structure_15category_object_CategoryObject (o=0x90c07b0) at sage/structure/category_object.c:8990
#7 0x00007fffee7e6fe0 in __pyx_tp_dealloc_4sage_9structure_6parent_Parent (o=0x90c07b0) at sage/structure/parent.c:21519
#8 0x00007fffeea2aa7f in __pyx_tp_dealloc_4sage_9structure_10parent_old_Parent (o=0x90c07b0) at sage/structure/parent_old.c:7261
#9 0x00007fffeec3bca8 in __pyx_tp_dealloc_4sage_9structure_11parent_base_ParentWithBase (o=0x90c07b0) at sage/structure/parent_base.c:1876
#10 0x00007fffead58cbc in __pyx_tp_dealloc_4sage_9structure_11parent_gens_ParentWithGens (o=0x90c07b0) at sage/structure/parent_gens.c:5865
#11 0x00007ffff7a4ce4c in subtype_dealloc (self=0x90c07b0) at Objects/typeobject.c:1014
#12 0x00007ffff7a27be4 in _Py_Dealloc (op=0x90c07b0) at Objects/object.c:2243
#13 0x00007ffff7a480b0 in tupledealloc (op=0x845ab50) at Objects/tupleobject.c:220
#14 0x00007ffff7a27be4 in _Py_Dealloc (op=0x845ab50) at Objects/object.c:2243
#15 0x00007ffff7a480b0 in tupledealloc (op=0x7fffea2a0760) at Objects/tupleobject.c:220
#16 0x00007ffff7a27be4 in _Py_Dealloc (op=0x7fffea2a0760) at Objects/object.c:2243
#17 0x00007ffff7a18e8e in dict_dealloc (mp=0x9052b70) at Objects/dictobject.c:985
#18 0x00007ffff7a27be4 in _Py_Dealloc (op=0x9052b70) at Objects/object.c:2243
#19 0x00007ffff7a4cd73 in subtype_dealloc (self=0x85045a0) at Objects/typeobject.c:999
#20 0x00007ffff7a27be4 in _Py_Dealloc (op=0x85045a0) at Objects/object.c:2243
#21 0x00007fffed102313 in __pyx_tp_dealloc_4sage_10categories_7functor_Functor (o=0x7985950) at sage/categories/functor.c:3209
#22 0x00007fffecee64a1 in __pyx_tp_dealloc_4sage_10categories_6action_Action (o=0x7985950) at sage/categories/action.c:6461
#23 0x00007fffbcd614ea in __pyx_tp_dealloc_4sage_6matrix_6action_MatrixMulAction (o=0x7985950) at sage/matrix/action.c:4724
#24 0x00007ffff7a27be4 in _Py_Dealloc (op=0x7985950) at Objects/object.c:2243
#25 0x00007ffff7a02c8e in list_dealloc (op=0x759fa38) at Objects/listobject.c:309
#26 0x00007ffff7a27be4 in _Py_Dealloc (op=0x759fa38) at Objects/object.c:2243
#27 0x00007ffff7a02c8e in list_dealloc (op=0x7964858) at Objects/listobject.c:309
#28 0x00007ffff7a27be4 in _Py_Dealloc (op=0x7964858) at Objects/object.c:2243
#29 0x00007fffedecec13 in __pyx_tp_clear_4sage_9structure_11coerce_dict_TripleDict (o=0x3cbd8d0) at sage/structure/coerce_dict.c:5921
#30 0x00007ffff7b1378b in delete_garbage (collectable=0x7fffffff30f0, old=0x7ffff7dc1540 <generations+96>) at Modules/gcmodule.c:769
#31 0x00007ffff7b13d04 in collect (generation=2) at Modules/gcmodule.c:930
#32 0x00007ffff7b13f06 in collect_generations () at Modules/gcmodule.c:996
#33 0x00007ffff7b14bcc in _PyObject_GC_Malloc (basicsize=264) at Modules/gcmodule.c:1457
#34 0x00007ffff7b14c04 in _PyObject_GC_New (tp=0x7ffff7d9c5a0 <PyDict_Type>) at Modules/gcmodule.c:1467
#35 0x00007ffff7a16cc7 in PyDict_New () at Objects/dictobject.c:277
#36 0x00007fffeaa83860 in __pyx_f_4sage_4libs_4pari_3gen_12PariInstance_new_ref (__pyx_v_self=0xcceae0, __pyx_v_g=0xa968360, __pyx_v_parent=0xa8d8748) at sage/libs/pari/gen.c:49228
#37 0x00007fffea9fb417 in __pyx_pf_4sage_4libs_4pari_3gen_3gen_80__getitem__ (__pyx_v_self=0xa8d8748, __pyx_v_n=0x61f7f0) at sage/libs/pari/gen.c:8638
#38 0x00007fffea9f63b7 in __pyx_pw_4sage_4libs_4pari_3gen_3gen_81__getitem__ (__pyx_v_self=0xa8d8748, __pyx_v_n=0x61f7f0) at sage/libs/pari/gen.c:7643
#39 0x00007fffeaa9a688 in __pyx_sq_item_4sage_4libs_4pari_3gen_gen (o=0xa8d8748, i=1) at sage/libs/pari/gen.c:55757
#40 0x00007ffff79bcdd7 in PySequence_GetItem (s=0xa8d8748, i=1) at Objects/abstract.c:1989
#41 0x00007ffff7a01934 in iter_iternext (iterator=0xa99c300) at Objects/iterobject.c:58
#42 0x00007ffff7a04abe in listextend (self=0xa7db060, b=0xa8d8748) at Objects/listobject.c:872
#43 0x00007ffff7a08ad9 in list_init (self=0xa7db060, args=0xa7a3920, kw=0x0) at Objects/listobject.c:2458
#44 0x00007ffff7a4c1ad in type_call (type=0x7ffff7d9a3c0 <PyList_Type>, args=0xa7a3920, kwds=0x0) at Objects/typeobject.c:737
#45 0x00007ffff79be33e in PyObject_Call (func=0x7ffff7d9a3c0 <PyList_Type>, arg=0xa7a3920, kw=0x0) at Objects/abstract.c:2529
#46 0x00007fffea9eb6f1 in __pyx_pf_4sage_4libs_4pari_3gen_3gen_12list (__pyx_v_self=0xa8d87d0) at sage/libs/pari/gen.c:4507
#47 0x00007fffea9eb4a0 in __pyx_pw_4sage_4libs_4pari_3gen_3gen_13list (__pyx_v_self=0xa8d87d0, unused=0x0) at sage/libs/pari/gen.c:4455
#48 0x00007ffff7a21156 in PyCFunction_Call (func=0xa85b858, arg=0x7ffff7f90060, kw=0x0) at Objects/methodobject.c:90
#49 0x00007ffff79be33e in PyObject_Call (func=0xa85b858, arg=0x7ffff7f90060, kw=0x0) at Objects/abstract.c:2529
#50 0x00007fffe14ff0aa in __pyx_pf_4sage_5rings_10polynomial_25polynomial_rational_flint_25Polynomial_rational_flint_6__init__ (__pyx_v_self=0xa781258, __pyx_v_parent=0x135f730, __pyx_v_x=0xa8d87d0,
__pyx_v_check=0x7ffff7d89ec0 <_Py_TrueStruct>, __pyx_v_is_gen=0x7ffff7d89e80 <_Py_ZeroStruct>, __pyx_v_construct=0x7ffff7d89e80 <_Py_ZeroStruct>)
at sage/rings/polynomial/polynomial_rational_flint.cpp:5760
#51 0x00007fffe14fc966 in __pyx_pw_4sage_5rings_10polynomial_25polynomial_rational_flint_25Polynomial_rational_flint_7__init__ (__pyx_v_self=0xa781258, __pyx_args=0xa6f97d0, __pyx_kwds=0xa83c050)
at sage/rings/polynomial/polynomial_rational_flint.cpp:5165
#52 0x00007ffff7a4c1ad in type_call (type=0x7fffe174bc20 <__pyx_type_4sage_5rings_10polynomial_25polynomial_rational_flint_Polynomial_rational_flint>, args=0xa6f97d0, kwds=0xa83c050)
at Objects/typeobject.c:737
#53 0x00007ffff79be33e in PyObject_Call (func=0x7fffe174bc20 <__pyx_type_4sage_5rings_10polynomial_25polynomial_rational_flint_Polynomial_rational_flint>, arg=0xa6f97d0, kw=0xa83c050)
at Objects/abstract.c:2529
#54 0x00007ffff7ac8282 in ext_do_call (func=0x7fffe174bc20 <__pyx_type_4sage_5rings_10polynomial_25polynomial_rational_flint_Polynomial_rational_flint>, pp_stack=0x7fffffff3a98, flags=2, na=4, nk=1)
at Python/ceval.c:4334
#55 0x00007ffff7ac1a8b in PyEval_EvalFrameEx (f=0xa663520, throwflag=0) at Python/ceval.c:2705
#56 0x00007ffff7ac420b in PyEval_EvalCodeEx (co=0x7fffe303bb40, globals=0x10a78b0, locals=0x0, args=0xa8acf10, argcount=2, kws=0x0, kwcount=0, defs=0x7fffe17556e8, defcount=4, closure=0x0)
at Python/ceval.c:3253
#57 0x00007ffff79fd447 in function_call (func=0x7fffdfc76ae0, arg=0xa8acee8, kw=0x0) at Objects/funcobject.c:526
#58 0x00007ffff79be33e in PyObject_Call (func=0x7fffdfc76ae0, arg=0xa8acee8, kw=0x0) at Objects/abstract.c:2529
#59 0x00007ffff79da359 in instancemethod_call (func=0x7fffdfc76ae0, arg=0xa8acee8, kw=0x0) at Objects/classobject.c:2578
#60 0x00007ffff79be33e in PyObject_Call (func=0x7ffff0cdf060, arg=0x7fffea521990, kw=0x0) at Objects/abstract.c:2529
#61 0x00007fffe6888d3b in __pyx_f_4sage_9structure_11coerce_maps_24DefaultConvertMap_unique__call_ (__pyx_v_self=0x7fffbcf7f3f0, __pyx_v_x=0xa8d87d0, __pyx_skip_dispatch=0)
at sage/structure/coerce_maps.c:3485
#62 0x00007fffee7acda1 in __pyx_pf_4sage_9structure_6parent_6Parent_28__call__ (__pyx_v_self=0x135f730, __pyx_v_x=0xa8d87d0, __pyx_v_args=0x7ffff7f90060, __pyx_v_kwds=0xa751480)
at sage/structure/parent.c:7415
#63 0x00007fffee7ac0a4 in __pyx_pw_4sage_9structure_6parent_6Parent_29__call__ (__pyx_v_self=0x135f730, __pyx_args=0xa80fc30, __pyx_kwds=0x0) at sage/structure/parent.c:7096
#64 0x00007ffff79be33e in PyObject_Call (func=0x135f730, arg=0xa80fc30, kw=0x0) at Objects/abstract.c:2529
#65 0x00007fffddd720a3 in __pyx_pf_4sage_5rings_12number_field_20number_field_element_18NumberFieldElement_2__init__ (__pyx_v_self=0xa860400, __pyx_v_parent=0x331b730, __pyx_v_f=0xa8d87d0)
at sage/rings/number_field/number_field_element.cpp:6090
#66 0x00007fffddd6d545 in __pyx_pw_4sage_5rings_12number_field_20number_field_element_18NumberFieldElement_3__init__ (__pyx_v_self=0xa860400, __pyx_args=0xa85eab0, __pyx_kwds=0x0)
at sage/rings/number_field/number_field_element.cpp:5340
#67 0x00007ffff7a595f6 in wrap_init (self=0xa860400, args=0xa85eab0,
wrapped=0x7fffddd6d316 <__pyx_pw_4sage_5rings_12number_field_20number_field_element_18NumberFieldElement_3__init__(PyObject*, PyObject*, PyObject*)>, kwds=0x0) at Objects/typeobject.c:4719
#68 0x00007ffff79e2145 in wrapper_call (wp=0xa99c220, args=0xa85eab0, kwds=0x0) at Objects/descrobject.c:998
#69 0x00007ffff79be33e in PyObject_Call (func=0xa99c220, arg=0xa85eab0, kw=0x0) at Objects/abstract.c:2529
#70 0x00007ffff7ac6404 in PyEval_CallObjectWithKeywords (func=0xa99c220, arg=0xa85eab0, kw=0x0) at Python/ceval.c:3890
#71 0x00007ffff79e1194 in wrapperdescr_call (descr=0x7fffde6d9ae0, args=0xa85eab0, kwds=0x0) at Objects/descrobject.c:306
#72 0x00007ffff79be33e in PyObject_Call (func=0x7fffde6d9ae0, arg=0xa881360, kw=0x0) at Objects/abstract.c:2529
#73 0x00007fffddaddacf in __pyx_pf_4sage_5rings_12number_field_30number_field_element_quadratic_28NumberFieldElement_quadratic___init__ (__pyx_v_self=0xa860400, __pyx_v_parent=0x331b730,
__pyx_v_f=0xa8d87d0) at sage/rings/number_field/number_field_element_quadratic.cpp:3893
#74 0x00007fffddadbbdb in __pyx_pw_4sage_5rings_12number_field_30number_field_element_quadratic_28NumberFieldElement_quadratic_1__init__ (__pyx_v_self=0xa860400, __pyx_args=0xa85ca38, __pyx_kwds=0x0)
at sage/rings/number_field/number_field_element_quadratic.cpp:3386
#75 0x00007ffff7a4c1ad in type_call (type=0x7fffddd15020 <__pyx_type_4sage_5rings_12number_field_30number_field_element_quadratic_NumberFieldElement_quadratic>, args=0xa85ca38, kwds=0x0)
at Objects/typeobject.c:737
#76 0x00007ffff79be33e in PyObject_Call (func=0x7fffddd15020 <__pyx_type_4sage_5rings_12number_field_30number_field_element_quadratic_NumberFieldElement_quadratic>, arg=0xa85ca38, kw=0x0)
at Objects/abstract.c:2529
#77 0x00007ffff7ac7bc3 in do_call (func=0x7fffddd15020 <__pyx_type_4sage_5rings_12number_field_30number_field_element_quadratic_NumberFieldElement_quadratic>, pp_stack=0x7fffffff4a10, na=2, nk=0)
at Python/ceval.c:4239
#78 0x00007ffff7ac6efc in call_function (pp_stack=0x7fffffff4a10, oparg=2) at Python/ceval.c:4044
#79 0x00007ffff7ac17f9 in PyEval_EvalFrameEx (f=0x33b1e00, throwflag=0) at Python/ceval.c:2666
#80 0x00007ffff7ac71f4 in fast_function (func=0x7fffddd47450, pp_stack=0x7fffffff4d90, n=2, na=2, nk=0) at Python/ceval.c:4107
#81 0x00007ffff7ac6ee0 in call_function (pp_stack=0x7fffffff4d90, oparg=1) at Python/ceval.c:4042
#82 0x00007ffff7ac17f9 in PyEval_EvalFrameEx (f=0x37a3270, throwflag=0) at Python/ceval.c:2666
#83 0x00007ffff7ac420b in PyEval_EvalCodeEx (co=0x7fffde989ca0, globals=0x1108b20, locals=0x0, args=0xa850f10, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3253
#84 0x00007ffff79fd447 in function_call (func=0x7fffddd43108, arg=0xa850ee8, kw=0x0) at Objects/funcobject.c:526
#85 0x00007ffff79be33e in PyObject_Call (func=0x7fffddd43108, arg=0xa850ee8, kw=0x0) at Objects/abstract.c:2529
#86 0x00007ffff79da359 in instancemethod_call (func=0x7fffddd43108, arg=0xa850ee8, kw=0x0) at Objects/classobject.c:2578
#87 0x00007ffff79be33e in PyObject_Call (func=0x7fffc2ecc360, arg=0xa70d1b0, kw=0x0) at Objects/abstract.c:2529
#88 0x00007fffe6888d3b in __pyx_f_4sage_9structure_11coerce_maps_24DefaultConvertMap_unique__call_ (__pyx_v_self=0x7fffbcd4b780, __pyx_v_x=0xa8d87d0, __pyx_skip_dispatch=0)
at sage/structure/coerce_maps.c:3485
#89 0x00007fffee7acda1 in __pyx_pf_4sage_9structure_6parent_6Parent_28__call__ (__pyx_v_self=0x331b730, __pyx_v_x=0xa8d87d0, __pyx_v_args=0x7ffff7f90060, __pyx_v_kwds=0xa9d32c0)
at sage/structure/parent.c:7415
#90 0x00007fffee7ac0a4 in __pyx_pw_4sage_9structure_6parent_6Parent_29__call__ (__pyx_v_self=0x331b730, __pyx_args=0xa7a3a70, __pyx_kwds=0x0) at sage/structure/parent.c:7096
#91 0x00007ffff79be33e in PyObject_Call (func=0x331b730, arg=0xa7a3a70, kw=0x0) at Objects/abstract.c:2529
#92 0x00007ffff7ac6404 in PyEval_CallObjectWithKeywords (func=0x331b730, arg=0xa7a3a70, kw=0x0) at Python/ceval.c:3890
#93 0x00007ffff7ab2344 in builtin_map (self=0x0, args=0xa8dde70) at Python/bltinmodule.c:1038
#94 0x00007ffff7a210f2 in PyCFunction_Call (func=0x7ffff7f52060, arg=0xa8dde70, kw=0x0) at Objects/methodobject.c:81
#95 0x00007ffff7ac6cdc in call_function (pp_stack=0x7fffffff5900, oparg=2) at Python/ceval.c:4021
#96 0x00007ffff7ac17f9 in PyEval_EvalFrameEx (f=0x3944d10, throwflag=0) at Python/ceval.c:2666
...
</pre><p>
I see a couple of familiar names in the backtrace...
</p>
TicketSimonKingTue, 22 Jan 2013 12:07:27 GMTattachment set
https://trac.sagemath.org/ticket/12215
https://trac.sagemath.org/ticket/12215
<ul>
<li><strong>attachment</strong>
set to <em>sage_crash_WgD9iG.log</em>
</li>
</ul>
<p>
Crash log
</p>
TicketSimonKingTue, 22 Jan 2013 12:19:53 GMT
https://trac.sagemath.org/ticket/12215#comment:151
https://trac.sagemath.org/ticket/12215#comment:151
<p>
Thanks to Volker's enhanced backtraces, running the test in verbose mode and without gdb yields <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/12215/sage_crash_WgD9iG.log" title="Attachment 'sage_crash_WgD9iG.log' in Ticket #12215">this backtrace</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/12215/sage_crash_WgD9iG.log" title="Download"></a>, and the crash occurs here (line 6467 of heegner.py)
</p>
<pre class="wiki"> sage: E = EllipticCurve('681b')
sage: I = E.heegner_index(-8); I
</pre><p>
Unfortunately, running this in an interactive session works just fine.
</p>
TicketSimonKingTue, 22 Jan 2013 12:32:34 GMT
https://trac.sagemath.org/ticket/12215#comment:152
https://trac.sagemath.org/ticket/12215#comment:152
<p>
Got it, I think.
</p>
<p>
The crash happens in the last line of the following snippet
</p>
<pre class="wiki"> __pyx_t_1 = __pyx_v_self->D->buckets;
__Pyx_INCREF(__pyx_t_1);
__pyx_t_4 = __pyx_v_self->D->buckets;
__Pyx_INCREF(__pyx_t_4);
__pyx_t_10 = PyList_GET_ITEM(__pyx_t_1, (__pyx_v_h % PyList_GET_SIZE(__pyx_t_4)));
</pre><p>
and according to the crash log, we have
</p>
<pre class="wiki"> __pyx_t_1 = 0x7f3db87dcb00 <_Py_NoneStruct>
</pre><p>
Hence, again, we have the problem that some attributes have already become invalid. I think this was fixed by <a class="ext-link" href="http://trac.sagemath.org/sage_trac/attachment/ticket/12313/trac_12313-safe_callback.patch"><span class="icon"></span>the second patch</a> from <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>.
</p>
<p>
Suggestion: In order to keep things modular, the part of the second <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> patch that applies to <code>TripleDict</code> shall be moved here, so that <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> remains independent of <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a>. And then, the second patch of <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> should be replaced by something that only takes care of the new <code>MonoDict</code>.
</p>
<p>
Rationale: <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> has a problem with a time regression, while <a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a> should (hopefully) be fine after installing the fix.
</p>
TicketSimonKingTue, 22 Jan 2013 12:40:43 GMTattachment set
https://trac.sagemath.org/ticket/12215
https://trac.sagemath.org/ticket/12215
<ul>
<li><strong>attachment</strong>
set to <em>trac12215_safe_callback.patch</em>
</li>
</ul>
<p>
Safer callback in <code>TripleDictEraser</code>
</p>
TicketSimonKingTue, 22 Jan 2013 12:44:17 GMTstatus, description changed
https://trac.sagemath.org/ticket/12215#comment:153
https://trac.sagemath.org/ticket/12215#comment:153
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/12215?action=diff&version=153">diff</a>)
</li>
</ul>
<p>
The patch's up, and it fixes the crash in heegner.py (tested in sage-5.6.rc0 debug version with MALLOC_CHECK_=3)
</p>
<p>
Apply trac12215_weak_cached_function_combined.patch trac12215_safe_callback.patch
</p>
TicketnbruinTue, 22 Jan 2013 18:07:03 GMTstatus changed
https://trac.sagemath.org/ticket/12215#comment:154
https://trac.sagemath.org/ticket/12215#comment:154
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
</ul>
<p>
Yes, this solves the problem here as well, so positive review.
</p>
<p>
It looks like the analysis on <a class="ext-link" href="http://trac.sagemath.org/sage_trac/ticket/12313#comment:266"><span class="icon"></span>#12313:226</a> and the patch that followed from it was based on this ticket. I probably pulled the non-raw patches for <a class="closed ticket" href="https://trac.sagemath.org/ticket/12313" title="defect: Fix yet another memory leak caused by caching of coercion data (closed: fixed)">#12313</a> when I tested ... Should we factor out a utility from the Patchbot to pull and apply patches given a ticket number?
</p>
<p>
Happy to see this work did find some use after all. Again, I believe that in the future, when <code>TripleDictEraser</code> holds a weakref to its dictionary, this won't be necessary anymore, because the weakref will be broken before attributes on the dictionary get erased.
</p>
<p>
That enhanced traceback (including cython code!) is extremely cool. A big thanks to Volker for making that happen. With that traceback, you only only have to stare at the traceback to diagnose this problem.
</p>
TicketSimonKingTue, 22 Jan 2013 19:31:50 GMT
https://trac.sagemath.org/ticket/12215#comment:155
https://trac.sagemath.org/ticket/12215#comment:155
<p>
Just for the record: All tests pass on my <code>openSuse</code> laptop in the debug version of sage-5.6.rc0+<a class="closed ticket" href="https://trac.sagemath.org/ticket/13878" title="defect: Fix failing assertion in linbox/matrix/permutation-matrix.h (closed: fixed)">#13878</a>+<a class="closed ticket" href="https://trac.sagemath.org/ticket/13378" title="defect: Do not cache the non-existence of coerce/convert map too often, and do ... (closed: fixed)">#13378</a>+<a class="closed ticket" href="https://trac.sagemath.org/ticket/12215" title="defect: Memleak in UniqueRepresentation, @cached_method (closed: fixed)">#12215</a>, with MALLOC_CHECK_=3.
</p>
TicketjdemeyerFri, 25 Jan 2013 13:07:03 GMTstatus changed; resolution, merged set
https://trac.sagemath.org/ticket/12215#comment:156
https://trac.sagemath.org/ticket/12215#comment:156
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
<li><strong>merged</strong>
set to <em>sage-5.7.beta1</em>
</li>
</ul>
Ticket