Ticket #12215: trac12215_weak_cached_function_combined.patch

File trac12215_weak_cached_function_combined.patch, 19.2 KB (added by SimonKing, 7 years ago)

Implement a weak version of cached_function, and use it for UniqueRepresentation. Properly use WeakValueDictionary in UniqueFactory. Combined patch

  • sage/categories/action.pyx

    # HG changeset patch
    # User Simon King <simon.king@uni-jena.de>
    # Date 1324770536 -3600
    # Node ID 6e2bba30bfd186e77fe2332c4f12dba827471dda
    # Parent  b1071622078a3c8cf1e17b87cbd97510fcd1d1d1
    #12215: Introduce weak_cached_function, and use it in UniqueRepresentation.
    
    diff --git a/sage/categories/action.pyx b/sage/categories/action.pyx
    a b  
    8686        self.US = ref(S)
    8787        self._is_left = is_left
    8888        self.op = op
    89        
     89
    9090    def _apply_functor(self, x):
    9191        return self(x)
    9292       
  • sage/categories/category.py

    diff --git a/sage/categories/category.py b/sage/categories/category.py
    a b  
    106106from sage.structure.unique_representation import UniqueRepresentation
    107107from sage.structure.dynamic_class import dynamic_class_internal
    108108
    109 @cached_function
     109from weakref import WeakValueDictionary
     110_join_cache = WeakValueDictionary()
     111
    110112def _join(categories, as_list):
    111113    """
    112114    This is an auxiliary function for :meth:`Category.join`
    113115
    114     NOTE:
    115 
    116     It is used for getting a temporary speed-up at trac ticket #11900.
    117     But it is supposed to be replaced by a better solution at trac
    118     ticket #11943.
    119 
    120116    INPUT:
    121117
    122118    - ``categories``: A tuple (no list) of categories.
     
    135131        from objects import Objects
    136132        return Objects()
    137133
     134    if not as_list:
     135        try:
     136            return _join_cache[categories]
     137        except KeyError:
     138            pass
     139
    138140    # Ensure associativity by flattening JoinCategory's
    139141    # Invariant: the super categories of a JoinCategory are not JoinCategories themselves
    140142    categories = sum( (tuple(category._super_categories) if isinstance(category, JoinCategory) else (category,)
     
    149151    if as_list:
    150152        return list(result)
    151153    if len(result) == 1:
    152         return result[0]
     154        out = _join_cache[categories] = result[0]
    153155    else:
    154         return JoinCategory(result)
     156        out = _join_cache[categories] = JoinCategory(result)
     157    return out
    155158
    156159
    157160class Category(UniqueRepresentation, SageObject):
     
    264267        ...               return "D"
    265268        ...
    266269
    267     Categories should always have uniq representation. We check
    268     this before proceeding::
     270    Categories should always have unique representation; by trac ticket
     271    #12215, this means that it will be kept in cache, but only if there
     272    is still some strong reference to it.
     273   
     274    We check this before proceeding::
    269275
     276        sage: import gc
     277        sage: idAs = id(As())
     278        sage: _ = gc.collect()
     279        sage: n == id(As())
     280        False
     281        sage: a = As()
    270282        sage: id(As()) == id(As())
    271283        True
    272284        sage: As().parent_class == As().parent_class
  • sage/combinat/sf/sfa.py

    diff --git a/sage/combinat/sf/sfa.py b/sage/combinat/sf/sfa.py
    a b  
    774774        sage: s(m([2,1]))
    775775        -2*s[1, 1, 1] + s[2, 1]
    776776    """
    777 
    778777    def __init__(self, Sym, basis_name = None, prefix = None):
    779778        r"""
    780779        Initializes the symmetric function algebra.
  • sage/misc/cachefunc.pyx

    diff --git a/sage/misc/cachefunc.pyx b/sage/misc/cachefunc.pyx
    a b  
    99  methods to instances).
    1010- Tom Boothby (added DiskCachedFunction).
    1111- Simon King (improved performance, more doctests, cython version,
    12   added CachedMethodCallerNoArgs).
     12  added CachedMethodCallerNoArgs. Weak cached function).
    1313
    1414EXAMPLES:
    1515
     
    2929    sage: test_pfunc(5) is test_pfunc(5)
    3030    True
    3131
     32In some cases, one would only want to keep the result in cache as long
     33as there is any other reference to the result. By :trac:`12215`, this is
     34enabled for :class:`~sage.structure.unique_representation.UniqueRepresentation`,
     35which is used to create unique parents: If an algebraic structure, such
     36as a finite field, is only temporarily used, then it will not stay in
     37cache forever. That behaviour is implemented using ``weak_cached_function``,
     38that behaves the same as ``cached_function``, except that it uses a
     39``WeakValueDictionary`` for storing the results.
     40::
     41
     42    sage: from sage.misc.cachefunc import weak_cached_function
     43    sage: class A: pass
     44    sage: @weak_cached_function
     45    ... def f():
     46    ...    print "doing a computation"
     47    ...    return A()
     48    ...
     49    sage: a = f()
     50    doing a computation
     51
     52The result is cached::
     53   
     54    sage: b = f()
     55    sage: a is b
     56    True
     57
     58However, if there are no strong references left, the result
     59may be garbage collected, and thus a new computation would
     60take place::
     61
     62    sage: del a
     63    sage: del b
     64    sage: import gc
     65    sage: n = gc.collect()
     66    sage: a = f()
     67    doing a computation
     68
    3269Unfortunately, cython functions do not allow arbitrary
    3370decorators. However, one can wrap a Cython function and
    34 turn it into a cached function, by trac ticket #11115.
     71turn it into a cached function, by :trac:`11115`.
    3572We need to provide the name that the wrapped method or
    3673function should have, since otherwise the name of the
    3774original function would be used::
     
    71108    sage: O.wrapped_method(5) is O.wrapped_method(5)
    72109    True
    73110
    74 By trac ticket #11115, even if a parent does not allow attribute
     111In some cases, one would only want to keep the result in cache as long
     112as there is any other reference to the result. By :trac:`12215`, this is
     113enabled for :class:`~sage.structure.unique_representation.UniqueRepresentation`,
     114which is used to create unique parents: If an algebraic structure, such
     115as a finite field, is only temporarily used, then it will not stay in
     116cache forever. That behaviour is implemented using ``weak_cached_function``,
     117that behaves the same as ``cached_function``, except that it uses a
     118``WeakValueDictionary`` for storing the results.
     119::
     120
     121    sage: from sage.misc.cachefunc import weak_cached_function
     122    sage: class A: pass
     123    sage: @weak_cached_function
     124    ... def f():
     125    ...    print "doing a computation"
     126    ...    return A()
     127    ...
     128    sage: a = f()
     129    doing a computation
     130
     131The result is cached::
     132   
     133    sage: b = f()
     134    sage: a is b
     135    True
     136
     137However, if there are no strong references left, the result
     138may be garbage collected, and thus a new computation would
     139take place::
     140
     141    sage: del a
     142    sage: del b
     143    sage: import gc
     144    sage: n = gc.collect()
     145    sage: a = f()
     146    doing a computation
     147
     148By :trac:`11115`, even if a parent does not allow attribute
    75149assignment, it can inherit a cached method from the parent class of a
    76150category (previously, the cache would have been broken)::
    77151
     
    272346import os
    273347from sage.misc.sageinspect import sage_getfile, sage_getsourcelines, sage_getargspec
    274348
     349from weakref import WeakValueDictionary
     350
    275351def _cached_function_unpickle(module,name):
    276352    """
    277353    Unpickling of cached functions.
     
    716792
    717793cached_function = CachedFunction
    718794
     795cdef class WeakCachedFunction(CachedFunction):
     796    """
     797    A version of :class:`CachedFunction` using weak references on the values.
     798   
     799    If f is a function, do either ``g = weak_cached_function(f)`` to make
     800    a cached version of f, or put ``@weak_cached_function`` right before
     801    the definition of f (i.e., use Python decorators, but then
     802    the optional argument ``name`` can not be used)::
     803
     804        @weak_cached_function
     805        def f(...):
     806            ...
     807
     808    EXAMPLES::
     809
     810        sage: from sage.misc.cachefunc import weak_cached_function
     811        sage: class A: pass
     812        sage: @weak_cached_function
     813        ... def f():
     814        ...    print "doing a computation"
     815        ...    return A()
     816        ...
     817        sage: a = f()
     818        doing a computation
     819
     820    The result is cached::
     821
     822        sage: b = f()
     823        sage: a is b
     824        True
     825
     826    However, if there are no strong references left, the result
     827    may be garbage collected, and thus a new computation would
     828    take place::
     829
     830        sage: del a
     831        sage: del b
     832        sage: import gc
     833        sage: n = gc.collect()
     834        sage: a = f()
     835        doing a computation
     836
     837    """
     838    def __init__(self, f, classmethod=False, name=None):
     839        """
     840        The inputs to the function must be hashable.
     841        The outputs to the function must be weakly referenceable.
     842
     843        TESTS::
     844
     845            sage: from sage.misc.cachefunc import weak_cached_function
     846            sage: class A: pass
     847            sage: @weak_cached_function
     848            ... def f():
     849            ...    return A()
     850            ...
     851            sage: f
     852            Cached version of <function f at ...>
     853
     854        We demonstrate that pickling works, provided the uncached function
     855        is available::
     856
     857            sage: import __main__
     858            sage: __main__.f = f
     859            sage: loads(dumps(f))
     860            Cached version of <function f at ...>
     861            sage: f.cache
     862            <WeakValueDictionary at ...>
     863
     864        """
     865        self._common_init(f, ArgumentFixer(f,classmethod=classmethod), name=name)
     866        self.cache = WeakValueDictionary()
     867    def __call__(self, *args, **kwds):
     868        """
     869        Return value from cache or call the wrapped function,
     870        caching the output.
     871
     872        TESTS::
     873
     874            sage: from sage.misc.cachefunc import weak_cached_function
     875            sage: class A: pass
     876            sage: @weak_cached_function
     877            ... def f():
     878            ...    print "doing a computation"
     879            ...    return A()
     880            ...
     881            sage: a = f()    # indirect doctest
     882            doing a computation
     883
     884        The result is cached::
     885
     886            sage: b = f()
     887            sage: a is b
     888            True
     889
     890        However, if there are no strong references left, the result
     891        may be garbage collected, and thus a new computation would
     892        take place::
     893
     894            sage: del a
     895            sage: del b
     896            sage: import gc
     897            sage: n = gc.collect()
     898            sage: a = f()
     899            doing a computation
     900
     901        """
     902        # We shortcut a common case of no arguments
     903        if args or kwds:
     904            k = self._fix_to_pos(*args, **kwds)
     905        else:
     906            if self._default_key is not None:
     907                k = self._default_key
     908            else:
     909                k = self._default_key = self._fix_to_pos()
     910
     911        try:
     912            return self.cache[k]
     913        except KeyError:
     914            w = self.f(*args, **kwds)
     915            self.cache[k] = w
     916            return w
     917    def is_in_cache(self, *args, **kwds):
     918        """
     919        Checks if the argument list is in the cache.
     920
     921        EXAMPLES::
     922
     923            sage: from sage.misc.cachefunc import weak_cached_function
     924            sage: class A:
     925            ...     def __init__(self, x):
     926            ...         self.x = x
     927            ...
     928            sage: @weak_cached_function
     929            ... def f(n):
     930            ...    return A(n)
     931            ...
     932            sage: a = f(5)
     933
     934        The key 5 is in the cache, as long as there is a strong
     935        reference to the corresponding value::
     936
     937            sage: f.is_in_cache(5)
     938            True
     939
     940        However, if there are no strong references left, the cached
     941        item is removed from cache after garbage collection::
     942
     943            sage: del a
     944            sage: import gc
     945            sage: n = gc.collect()
     946            sage: f.is_in_cache(5)
     947            False
     948
     949        """
     950        return self._fix_to_pos(*args, **kwds) in self.cache
     951
     952    def set_cache(self, value, *args, **kwds):
     953        """
     954        Set the value for those args and keyword args
     955        Mind the unintuitive syntax (value first).
     956        Any idea on how to improve that welcome!
     957
     958        It is required that the given value is weak
     959        referenceable. The item will be removed from
     960        cache if the value is garbage collected.
     961
     962        EXAMPLES::
     963
     964            sage: from sage.misc.cachefunc import weak_cached_function
     965            sage: @weak_cached_function
     966            ... def f(n):
     967            ...     raise RuntimeError
     968            ...
     969            sage: f.set_cache(ZZ, 5)
     970            sage: f(5)
     971            Integer Ring
     972
     973         """
     974        self.cache[self._fix_to_pos(*args, **kwds)] = value
     975
     976
     977weak_cached_function = WeakCachedFunction
     978
    719979class CachedMethodPickle(object):
    720980    """
    721981    This class helps to unpickle cached methods.
  • sage/rings/polynomial/polynomial_ring_constructor.py

    diff --git a/sage/rings/polynomial/polynomial_ring_constructor.py b/sage/rings/polynomial/polynomial_ring_constructor.py
    a b  
    4242from sage.rings.integer import Integer
    4343from sage.rings.finite_rings.integer_mod_ring import is_IntegerModRing
    4444
    45 from sage.misc.cachefunc import cached_function
     45from sage.misc.cachefunc import weak_cached_function
    4646
    4747from sage.categories.fields import Fields
    4848_Fields = Fields()
     
    572572_IntegralDomains = categories.integral_domains.IntegralDomains()
    573573_Rings = category = categories.rings.Rings()
    574574
    575 @cached_function
     575@weak_cached_function
    576576def polynomial_default_category(base_ring,multivariate):
    577577    """
    578578    Choose an appropriate category for a polynomial ring.
  • sage/rings/residue_field.pyx

    diff --git a/sage/rings/residue_field.pyx b/sage/rings/residue_field.pyx
    a b  
    774774        self._PBinv = PBinv
    775775        self._to_order = to_order # used for lift
    776776        self._PB = PB # used for lift
    777         from sage.categories.homset import Hom
    778777        from sage.categories.all import SetsWithPartialMaps
    779778        self._repr_type_str = "Partially defined reduction"
    780779        Map.__init__(self, Hom(K, F, SetsWithPartialMaps()))
     
    938937        abar
    939938        sage: (1+abar)^179
    940939        24*abar + 12
    941         sage: k.coerce_map_from(OK)
     940        sage: phi = k.coerce_map_from(OK); phi
    942941        Ring morphism:
    943942          From: Maximal Order in Number Field in a with defining polynomial x^3 - 7
    944943          To:   Residue field in abar of Fractional ideal (2*a^2 + 3*a - 10)
    945944
     945    By trac ticket #12215, phi is in fact contained in the homset with
     946    domain ``OK`` and codomain ``k``::
     947
     948        sage: phi in Hom(OK,k)
     949        True
     950
    946951        #sage: R.<t> = GF(19)[]; P = R.ideal(t^2 + 5)
    947952        #sage: k.<a> = R.residue_field(P)
    948953        #sage: f = k.coerce_map_from(R); f
     
    986991        self._PBinv = PBinv
    987992        self._PB = PB # used for lift
    988993        self._to_order = to_order # used for lift
    989         from sage.rings.homset import RingHomset
    990994        self._repr_type_str = "Reduction"
    991         RingHomomorphism.__init__(self, RingHomset(K, F))
     995        RingHomomorphism.__init__(self, Hom(K,F))
    992996
    993997    cpdef Element _call_(self, x):
    994998        """
  • sage/structure/coerce.pyx

    diff --git a/sage/structure/coerce.pyx b/sage/structure/coerce.pyx
    a b  
    176176            sage: A = cm.get_action(ZZ, NumberField(x^2-2, 'a'), operator.mul)
    177177            sage: f, g = cm.coercion_maps(QQ, int)
    178178            sage: f, g = cm.coercion_maps(ZZ, int)
    179             sage: cm.get_stats()
    180             ((0, 1.0, 4), (0, 0.25, 1))
     179            sage: cm.get_stats()  # random
     180            ((0, 1.0, 4), (0, 0.0, 0))
    181181           
    182182        .. note::
    183183
  • sage/structure/dynamic_class.py

    diff --git a/sage/structure/dynamic_class.py b/sage/structure/dynamic_class.py
    a b  
    116116#                  http://www.gnu.org/licenses/
    117117#*****************************************************************************
    118118
    119 from sage.misc.cachefunc import cached_function
     119from sage.misc.cachefunc import weak_cached_function
    120120from sage.structure.unique_representation import ClasscallMetaclass
    121121
    122122def dynamic_class(name, bases, cls = None, reduction = None, doccls=None):
     
    256256    #    assert(cls is None or issubtype(type(cls), type) or type(cls) is classobj)
    257257    return dynamic_class_internal(name, bases, cls, reduction, doccls)
    258258
    259 @cached_function
     259@weak_cached_function
    260260def dynamic_class_internal(name, bases, cls = None, reduction = None, doccls = None):
    261261    r"""
    262262    See sage.structure.dynamic_class.dynamic_class? for indirect doctests.
  • sage/structure/factory.pyx

    diff --git a/sage/structure/factory.pyx b/sage/structure/factory.pyx
    a b  
    7979            Rational Field
    8080        """
    8181        self._name = name
    82         self._cache = {}
     82        self._cache = weakref.WeakValueDictionary()
    8383       
    8484    def __reduce__(self):
    8585        """
     
    164164            False
    165165        """
    166166        try:
    167             obj = self._cache[version, key]()
    168             if obj is not None:
    169                 return obj
     167            return self._cache[version, key]
    170168        except KeyError:
    171169            pass
    172170        obj = self.create_object(version, key, **extra_args)
    173         self._cache[version, key] = weakref.ref(obj)
     171        self._cache[version, key] = obj
    174172        try:
    175173            other_keys = self.other_keys(key, obj)
    176174            for key in other_keys:
    177175                try:
    178                     new_obj = self._cache[version, key]()
    179                     if new_obj is not None:
    180                         obj = new_obj
    181                         break
     176                    obj = self._cache[version, key]
     177                    break
    182178                except KeyError:
    183179                    pass
    184180            for key in other_keys:
    185                 self._cache[version, key] = weakref.ref(obj)
     181                self._cache[version, key] = obj
    186182            obj._factory_data = self, version, key, extra_args
    187183            if obj.__class__.__reduce__.__objclass__ is object:
    188184                # replace the generic object __reduce__ to use this one
     
    190186        except AttributeError:
    191187            pass
    192188        return obj
    193        
     189
    194190    cpdef get_version(self, sage_version):
    195191        """
    196192        This is provided to allow more or less granular control over
  • sage/structure/unique_representation.py

    diff --git a/sage/structure/unique_representation.py b/sage/structure/unique_representation.py
    a b  
    2222#                  http://www.gnu.org/licenses/
    2323#******************************************************************************
    2424
    25 from sage.misc.cachefunc import cached_function
     25from sage.misc.cachefunc import weak_cached_function
    2626from sage.misc.classcall_metaclass import ClasscallMetaclass, typecall
    2727
    2828class UniqueRepresentation:
     
    9797        ...
    9898
    9999    Two coexisting instances of MyClass created with the same
    100     argument data are guaranteed to share the same identity::
     100    argument data are guaranteed to share the same identity. Since
     101    trac ticket #12215, this is only the case if there is some
     102    strong reference to the returned instance, since otherwise
     103    it may be garbage collected::
    101104
    102105        sage: x = MyClass(1)
    103106        sage: y = MyClass(1)
    104         sage: x is y
     107        sage: x is y               # There is a strong reference
    105108        True
    106109        sage: z = MyClass(2)
    107110        sage: x is z
     
    446449
    447450    _included_private_doc_ = ["__classcall__"]
    448451
    449     @cached_function # automatically a staticmethod
     452    @weak_cached_function # automatically a staticmethod
    450453    def __classcall__(cls, *args, **options):
    451454        """
    452455        Constructs a new object of this class or reuse an existing one.