Ticket #715: trac_715-reviewer-take2.patch
File trac_715-reviewer-take2.patch, 19.3 KB (added by , 9 years ago) |
---|
-
sage/categories/action.pyx
# HG changeset patch # User Jean-Pierre Flori <jean-pierre.flor@ssi.gouv.fr> # Date 1334221874 -7200 # Node ID 3d76af023be692183fcaa41b18875402a36a2f75 # Parent 75769cbc372eb71c07067d54c3bf4c8e57cb2c04 #715: Reviewer patch diff --git a/sage/categories/action.pyx b/sage/categories/action.pyx
a b 1 1 r""" 2 Group, ring, etc. actions on objects. 2 Group, ring, etc. actions on objects. 3 3 4 The terminology and notation used is suggestive of groups 5 acting on sets, but this framework can be used for modules, 6 algebras, etc. 4 The terminology and notation used is suggestive of groups acting on sets, 5 but this framework can be used for modules, algebras, etc. 7 6 8 A group action $G \times S \rightarrow S$ is a functor from $G$ to Sets. 7 A group action $G \times S \rightarrow S$ is a functor from $G$ to Sets. 9 8 10 AUTHORS: 11 -- Robert Bradshaw: initial version 9 .. WARNING:: 10 11 An :class:`Action` object only keeps a weak reference to the underlying set 12 which is acted upon. This decision was made in :trac:`715` in order to 13 allow garbage collection within the coercion framework (this is where 14 actions are mainly used) and avoid memory leaks. 15 16 :: 17 18 sage: from sage.categories.action import Action 19 sage: class P: pass 20 sage: A = Action(P(),P()) 21 sage: import gc 22 sage: _ = gc.collect() 23 sage: A 24 Traceback (most recent call last): 25 ... 26 RuntimeError: This action acted on a set that became garbage collected 27 28 To avoid garbage collection of the underlying set, it is sufficient to 29 create a strong reference to it before the action is created. 30 31 :: 32 33 sage: _ = gc.collect() 34 sage: from sage.categories.action import Action 35 sage: class P: pass 36 sage: q = P() 37 sage: A = Action(P(),q) 38 sage: gc.collect() 39 0 40 sage: A 41 Left action by <__main__.P instance at ...> on <__main__.P instance at ...> 42 43 AUTHOR: 44 45 - Robert Bradshaw: initial version 12 46 """ 13 47 14 48 #***************************************************************************** … … 130 164 sage: A.left_domain() is R 131 165 True 132 166 133 By trac ticket #715, there is only a weak reference to the underlying134 set.Hence, the underlying set may be garbage collected, even when the167 By :trac:`715`, there is only a weak reference to the underlying set. 168 Hence, the underlying set may be garbage collected, even when the 135 169 action is still alive. This may result in a runtime error, as follows:: 136 170 137 171 sage: from sage.categories.action import Action … … 143 177 Left action by <__main__.P instance at ...> on <__main__.P instance at ...> 144 178 sage: del q 145 179 sage: import gc 146 sage: n= gc.collect()180 sage: _ = gc.collect() 147 181 sage: A 148 182 Traceback (most recent call last): 149 183 ... -
sage/matrix/action.pyx
diff --git a/sage/matrix/action.pyx b/sage/matrix/action.pyx
a b 1 1 """ 2 These are the actions used by the coercion model for matrix and vector multiplications. 2 These are the actions used by the coercion model for matrix and vector 3 multiplications. 3 4 4 AUTHORS: 5 -- Robert Bradshaw (2007-09): Initial version. 5 .. WARNING:: 6 7 The class :class:`MatrixMulAction` and its descendants extends the class 8 :class:`Action`. As a cosnequence objects from these classes only keep weak 9 references to the underlying sets which are acted upon. This decision was 10 made in :trac:`715` in order to allow garbage collection within the coercion 11 framework, where actions are mainly used, and avoid memory leaks. 12 13 To ensure that the underlying set of such an object does not get garbage 14 collected, it is sufficient to explicitely create a strong reference to it 15 before creating the action. 16 17 :: 18 19 sage: MSQ = MatrixSpace(QQ, 2) 20 sage: MSZ = MatrixSpace(ZZ['x'], 2) 21 sage: A = MSQ.get_action(MSZ) 22 sage: A 23 Left action by Full MatrixSpace of 2 by 2 dense matrices over Rational Field on Full MatrixSpace of 2 by 2 dense matrices over Univariate Polynomial Ring in x over Integer Ring 24 sage: import gc 25 sage: _ = gc.collect() 26 sage: A 27 Left action by Full MatrixSpace of 2 by 2 dense matrices over Rational Field on Full MatrixSpace of 2 by 2 dense matrices over Univariate Polynomial Ring in x over Integer Ring 28 29 .. NOTE:: 30 31 The :func:`MatrixSpace` function caches the objects it creates. Therefore, 32 the underlying set ``MSZ`` in the above example will not be garbage 33 collected, even if it is not strongly ref'ed. Nonetheless, there is no 34 guarantee that the set that is acted upon will always be cached in such a 35 way, so that following the above example is good practice. 36 37 AUTHOR: 38 39 - Robert Bradshaw (2007-09): Initial version. 6 40 """ 7 41 8 42 #***************************************************************************** … … 40 74 """ 41 75 EXAMPLES: 42 76 43 By trac ticket #715, there only is a weak reference on the underlying set,44 so that it can be garbage collected if only the action itself is explicitly45 referred to. Hence, we first assign the involved matrix spaces to a46 variable::77 By :trac:`715`, there only is a weak reference on the underlying set, 78 so that it can be garbage collected if only the action itself is 79 explicitly referred to. Hence, we first assign the involved matrix 80 spaces to a variable:: 47 81 48 82 sage: MSQ = MatrixSpace(QQ, 2) 49 83 sage: MSZ = MatrixSpace(ZZ['x'], 2) … … 55 89 Full MatrixSpace of 2 by 2 dense matrices over Univariate Polynomial Ring in x over Integer Ring 56 90 sage: A.codomain() 57 91 Full MatrixSpace of 2 by 2 dense matrices over Univariate Polynomial Ring in x over Rational Field 92 93 .. NOTE:: 94 95 The :func:`MatrixSpace` function caches the object it creates. 96 Therefore, the underlying set ``MSZ`` in the above example will not 97 be garbage collected, even if it is not strongly ref'ed. 98 Nonetheless, there is no guarantee that the set that is acted upon 99 will always be cached in such a way, so that following the above 100 example is good practice. 101 58 102 """ 59 103 return self.underlying_set() 60 104 … … 64 108 """ 65 109 EXAMPLES: 66 110 67 By trac ticket #715, there only is a weak reference on the underlying set,68 so that it can be garbage collected if only the action itself is explicitly69 referred to. Hence, we first assign the involved matrix spaces to a70 variable::111 By :trac:`715`, there only is a weak reference on the underlying set, 112 so that it can be garbage collected if only the action itself is 113 explicitly referred to. Hence, we first assign the involved matrix 114 spaces to a variable:: 71 115 72 116 sage: R.<x> = ZZ[] 73 117 sage: MSR = MatrixSpace(R, 3, 3) … … 81 125 [ 0 x] 82 126 [2*x 3*x] 83 127 [4*x 5*x] 128 129 .. NOTE:: 130 131 The :func:`MatrixSpace` function caches the object it creates. 132 Therefore, the underlying set ``MSZ`` in the above example will not 133 be garbage collected, even if it is not strongly ref'ed. 134 Nonetheless, there is no guarantee that the set that is acted upon 135 will always be cached in such a way, so that following the above 136 example is good practice. 137 84 138 """ 85 139 if not is_MatrixSpace(S): 86 140 raise TypeError, "Not a matrix space: %s" % S … … 90 144 """ 91 145 EXAMPLES: 92 146 93 By trac ticket #715, there only is a weak reference on the underlying set,94 so that it can be garbage collected if only the action itself is explicitly95 referred to. Hence, we first assign the involved matrix spaces to a96 variable::147 By :trac:`715`, there only is a weak reference on the underlying set, 148 so that it can be garbage collected if only the action itself is 149 explicitly referred to. Hence, we first assign the involved matrix 150 spaces to a variable:: 97 151 98 152 sage: from sage.matrix.action import MatrixMatrixAction 99 153 sage: R.<x> = ZZ[] … … 103 157 Left action by Full MatrixSpace of 3 by 3 dense matrices over Univariate Polynomial Ring in x over Integer Ring on Full MatrixSpace of 3 by 2 dense matrices over Rational Field 104 158 sage: A.codomain() 105 159 Full MatrixSpace of 3 by 2 dense matrices over Univariate Polynomial Ring in x over Rational Field 160 161 .. NOTE:: 162 163 The :func:`MatrixSpace` function caches the object it creates. 164 Therefore, the underlying set ``MSZ`` in the above example will not 165 be garbage collected, even if it is not strongly ref'ed. 166 Nonetheless, there is no guarantee that the set that is acted upon 167 will always be cached in such a way, so that following the above 168 example is good practice. 169 106 170 """ 107 171 if self.G.ncols() != self.underlying_set().nrows(): 108 172 raise TypeError, "incompatible dimensions %s, %s" % (self.G.ncols(), self.underlying_set().nrows()) -
sage/structure/coerce_dict.pyx
diff --git a/sage/structure/coerce_dict.pyx b/sage/structure/coerce_dict.pyx
a b 22 22 23 23 cdef class TripleDictEraser: 24 24 """ 25 Erases items from a :class:`TripleDict` when a weak reference becomes invalid. 25 Erases items from a :class:`TripleDict` when a weak reference becomes 26 invalid. 26 27 27 This is of internal use only. Instances of this class will be passed as a callback28 function when creating a weak reference.28 This is of internal use only. Instances of this class will be passed as a 29 callback function when creating a weak reference. 29 30 30 31 EXAMPLES:: 31 32 … … 46 47 47 48 AUTHOR: 48 49 49 Simon King (2012-01)50 - Simon King (2012-01) 50 51 """ 51 52 52 53 def __init__(self, D): … … 64 65 65 66 """ 66 67 self.D = D 68 67 69 def __call__(self, r): 68 70 """ 69 71 INPUT: 70 72 71 73 A weak reference with key. 72 74 73 When this is called with a weak reference ``r``, then each item containing ``r``74 is removed from the associated :class:`TripleDict`. Normally, this only happens75 when a weak reference becomes invalid.75 When this is called with a weak reference ``r``, then each item 76 containing ``r`` is removed from the associated :class:`TripleDict`. 77 Normally, this only happens when a weak reference becomes invalid. 76 78 77 79 EXAMPLES:: 78 80 … … 90 92 sage: n = gc.collect() 91 93 sage: len(T) # indirect doctest 92 94 0 93 94 95 """ 95 96 # r is a (weak) reference (typically to a parent), 96 97 # and it knows the stored key of the unique triple r() had been part of. … … 115 116 """ 116 117 This is a hashtable specifically designed for (read) speed in 117 118 the coercion model. 118 119 119 120 It differs from a python dict in the following important ways: 120 121 121 122 - All keys must be sequence of exactly three elements. All sequence 122 123 types (tuple, list, etc.) map to the same item. 123 124 - Comparison is done using the 'is' rather than '==' operator. 124 125 125 126 There are special cdef set/get methods for faster access. 126 127 It is bare-bones in the sense that not all dictionary methods are 127 128 implemented. 128 129 129 130 It is implemented as a list of lists (hereafter called buckets). The bucket 130 is chosen according to a very simple hash based on the object pointer .131 and each bucket is of the form [id(k1), id(k2), id(k3), value, id(k1), id(k2),132 id(k 3), value, ...], on which a linear search is performed.133 131 is chosen according to a very simple hash based on the object pointer, 132 and each bucket is of the form [id(k1), id(k2), id(k3), value, id(k1), 133 id(k2), id(k3), value, ...], on which a linear search is performed. 134 134 135 To spread objects evenly, the size should ideally be a prime, and certainly 135 136 not divisible by 2. 136 137 137 138 138 EXAMPLES:: 139 139 140 140 sage: from sage.structure.coerce_dict import TripleDict 141 141 sage: L = TripleDict(31) 142 142 sage: a = 'a'; b = 'b'; c = 'c' … … 183 183 Traceback (most recent call last): 184 184 ... 185 185 KeyError: 'a' 186 186 187 187 The following illustrates why even sizes are bad:: 188 188 189 189 sage: L = TripleDict(4, L) … … 195 195 Note that this kind of dictionary is also used for caching actions 196 196 and coerce maps. In previous versions of Sage, the cache was by 197 197 strong references and resulted in a memory leak in the following 198 example. However, this leak was fixed by trac ticket #715, using199 weak references::198 example. However, this leak was fixed by trac ticket :trac:`715`, 199 using weak references:: 200 200 201 201 sage: K = GF(1<<55,'t') 202 202 sage: for i in range(50): … … 211 211 sage: len(LE) # indirect doctest 212 212 1 213 213 214 AUTHOR ::214 AUTHORS: 215 215 216 216 - Robert Bradshaw, 2007-08 217 217 218 - Simon King, 2012-01 218 219 """ 219 220 220 221 def __init__(self, size, data=None, threshold=0): 221 222 """ 222 Create a special dict using triples for keys. 223 223 Create a special dict using triples for keys. 224 224 225 EXAMPLES:: 225 226 226 227 sage: from sage.structure.coerce_dict import TripleDict … … 238 239 if data is not None: 239 240 for (k1,k2,k3), v in data.iteritems(): 240 241 self.set(k1,k2,k3, v) 241 242 242 243 def __len__(self): 243 244 """ 244 245 The number of items in self. 245 246 246 247 EXAMPLES:: 247 248 248 249 sage: from sage.structure.coerce_dict import TripleDict … … 256 257 3 257 258 """ 258 259 return self._size 259 260 260 261 def stats(self): 261 262 """ 262 The distribution of items in buckets. 263 264 OUTPUT: :263 The distribution of items in buckets. 264 265 OUTPUT: 265 266 266 267 - (min, avg, max) 267 268 268 269 EXAMPLES:: 269 270 270 271 sage: from sage.structure.coerce_dict import TripleDict … … 277 278 sage: for i in range(100): L[i,i,i] = None 278 279 sage: L.stats() # random 279 280 (0, 0.03325573661456601, 1) 280 281 281 282 sage: L = TripleDict(1) 282 283 sage: for i in range(100): L[i,i,i] = None 283 284 sage: L.stats() … … 293 294 else: 294 295 min = 0 295 296 return min, 1.0*size/len(self.buckets), max 296 297 297 298 def bucket_lens(self): 298 299 """ 299 The distribution of items in buckets. 300 300 The distribution of items in buckets. 301 301 302 OUTPUT: 302 303 A list of how many items are in each bucket. 304 303 304 A list of how many items are in each bucket. 305 305 306 EXAMPLES:: 306 307 307 308 sage: from sage.structure.coerce_dict import TripleDict … … 311 312 [3, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4, 3, 3, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 4, 3, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4] 312 313 sage: sum(L.bucket_lens()) 313 314 100 314 315 315 316 sage: L = TripleDict(1) 316 317 sage: for i in range(100): L[i,i,i] = None 317 318 sage: L.bucket_lens() 318 319 [100] 319 320 """ 320 321 return [len(self.buckets[i])/4 for i from 0 <= i < len(self.buckets)] 321 322 322 323 def _get_buckets(self): 323 324 """ 324 The actual buckets of self, for debugging. 325 326 EXAMPLE ::325 The actual buckets of self, for debugging. 326 327 EXAMPLES:: 327 328 328 329 sage: from sage.structure.coerce_dict import TripleDict 329 330 sage: L = TripleDict(3) … … 332 333 [[0, 0, 0, None], [], []] 333 334 """ 334 335 return self.buckets 335 336 336 337 def __getitem__(self, k): 337 338 """ 338 339 EXAMPLES:: … … 350 351 except (TypeError,ValueError): 351 352 raise KeyError, k 352 353 return self.get(k1, k2, k3) 353 354 354 355 cdef get(self, object k1, object k2, object k3): 355 356 cdef Py_ssize_t h = (<Py_ssize_t><void *>k1 + 13*<Py_ssize_t><void *>k2 ^ 503*<Py_ssize_t><void *>k3) 356 357 #if h < 0: h = -h # shouldn't "%" result in a positive number anyway? … … 367 368 if <Py_ssize_t>tmp == <Py_ssize_t><void *>k3: 368 369 return <object>PyList_GET_ITEM(bucket, i+3) 369 370 raise KeyError, (k1, k2, k3) 370 371 371 372 def __setitem__(self, k, value): 372 373 """ 373 374 EXAMPLES:: … … 385 386 except (TypeError,ValueError): 386 387 raise KeyError, k 387 388 self.set(k1, k2, k3, value) 388 389 389 390 cdef set(self, object k1, object k2, object k3, value): 390 391 if self.threshold and self._size > len(self.buckets) * self.threshold: 391 392 self.resize() 392 393 cdef Py_ssize_t h1 = <Py_ssize_t><void *>k1 393 394 cdef Py_ssize_t h2 = <Py_ssize_t><void *>k2 394 395 cdef Py_ssize_t h3 = <Py_ssize_t><void *>k3 395 396 396 397 cdef Py_ssize_t h = (h1 + 13*h2 ^ 503*h3) 397 398 #if h < 0: h = -h # shouldn't "%" return a positive number anyway? 398 399 cdef Py_ssize_t i … … 429 430 except TypeError: 430 431 PyList_Append(_refcache.setdefault((h1, h2, h3), []),k3) 431 432 self._size += 1 432 433 433 434 def __delitem__(self, k): 434 435 """ 435 436 EXAMPLES:: … … 459 460 self._size -= 1 460 461 return 461 462 raise KeyError, k 462 463 463 464 def resize(self, int buckets=0): 464 465 """ 465 Changes the number of buckets of self, while preserving the contents. 466 467 If the number of buckets is 0 or not given, it resizes self to the 468 smallest prime that is at least twice as large as self. 469 466 Changes the number of buckets of self, while preserving the contents. 467 468 If the number of buckets is 0 or not given, it resizes self to the 469 smallest prime that is at least twice as large as self. 470 470 471 EXAMPLES:: 471 472 472 473 sage: from sage.structure.coerce_dict import TripleDict … … 508 509 [((1, 2, 3), None)] 509 510 """ 510 511 return TripleDictIter(self) 511 512 512 513 def __reduce__(self): 513 514 """ 514 Note that we don't expect equality as this class concerns itself with 515 object identity rather than object equality. 515 Note that we don't expect equality as this class concerns itself with 516 object identity rather than object equality. 516 517 517 518 EXAMPLES:: 518 519 … … 525 526 [((1, 2, 3), True)] 526 527 """ 527 528 return TripleDict, (len(self.buckets), dict(self.iteritems()), self.threshold) 528 529 529 530 530 cdef class TripleDictIter: 531 531 def __init__(self, pairs): … … 540 540 """ 541 541 self.pairs = pairs 542 542 self.buckets = iter(self.pairs.buckets) 543 543 544 def __iter__(self): 544 545 """ 545 546 EXAMPLES:: … … 551 552 ((1, 2, 3), None) 552 553 """ 553 554 return self 555 554 556 def __next__(self): 555 557 """ 556 558 EXAMPLES:: … … 578 580 return self.next() 579 581 580 582 581 582 583 cdef long next_odd_prime(long n): 583 584 if n % 2 == 0: 584 585 n -= 1