Ticket #11490: trac11490-coercion_tutorial.patch

File trac11490-coercion_tutorial.patch, 68.7 KB (added by SimonKing, 9 years ago)

Add a thematic tutorial on categories and coercion

  • new file doc/en/thematic_tutorials/coercion_and_categories.rst

    # HG changeset patch
    # User Simon King <simon.king@uni-jena.de>
    # Date 1360822669 -3600
    # Node ID a49fed5ce9ff92015b5c69fe1aa4e3cc32e5603b
    # Parent  d6bf47d4c28e02974b86cb01cc5ef94edb97afbb
    #11490: Add a thematic tutorial on categories and coercion
    
    diff --git a/doc/en/thematic_tutorials/coercion_and_categories.rst b/doc/en/thematic_tutorials/coercion_and_categories.rst
    new file mode 100644
    - +  
     1.. -*- coding: utf-8 -*-
     2
     3=================================================
     4How to implement new algebraic structures in Sage
     5=================================================
     6--------------------------------------
     7Sage's category and coercion framework
     8--------------------------------------
     9
     10.. MODULEAUTHOR::
     11    Simon King,
     12    Friedrich\--Schiller\--Universität Jena,
     13    <simon.king@uni-jena.de>
     14    © 2011/2013
     15
     16.. toctree::
     17   :maxdepth: 2
     18
     19.. linkall
     20
     21
     22
     23The aim of this tutorial is to explain how one can benefit from Sage's
     24category framework and coercion model when implementing new algebraic
     25structures. It is based on a worksheet created in 2011.
     26
     27We illustrate the concepts of Sage's category framework and coercion model by
     28means of a detailed example, namely a toy implementation of fraction fields.
     29The code is developed step by step, so that the reader can focus on one detail
     30in each part of this tutorial. The complete code can be found in the appendix.
     31
     32Outline
     33=======
     34
     35- Use existing base classes
     36
     37  For using Sage's coercion system, it is essential to work with sub\--classes
     38  of :class:`sage.structure.parent.Parent` or
     39  :class:`sage.structure.element.Element`, respectively. They provide default
     40  implementations of many "magical" double-underscore Python methods, which
     41  must not be overridden. Instead, the actual implementation should be in
     42  *single underscore* methods, such as ``_add_`` or ``_mul_``.
     43
     44- Turn your parent structure into an object of a category
     45
     46  Declare the category during initialisation\---Your parent structure will
     47  inherit further useful methods and consistency tests.
     48
     49- Provide your parent structure with an element class
     50
     51  Assign to it an attribute called ``Element``\---The elements will inherit
     52  further useful methods from the category. In addition, some basic
     53  conversions will immediately work.
     54
     55- Implement further conversions
     56
     57  Never override a parent's ``__call__`` method! Provide the method
     58  ``_element_constructor_`` instead.
     59
     60- Declare coercions
     61
     62  If a conversion happens to be a morphism, you may consider to turn it into a
     63  coercion. It will then *implicitly* be used in arithmetic operations.
     64
     65- Advanced coercion:  Define construction functors for your parent structure
     66
     67  Sage will automatically create new parents for you when needed, by the
     68  so\--called :func:`sage.categories.pushout.pushout` construction.
     69
     70- Run the automatic test suites
     71
     72  Each method should be documented and provide a doc test (we are not giving
     73  examples here). In addition, any method defined for the objects or elements
     74  of a category should be supported by a test method, that is executed when
     75  running the test suite.
     76
     77Base classes
     78============
     79
     80In Sage, a "Parent" is an object of a category and contains elements.  Parents
     81should inherit from :class:`sage.structure.parent.Parent` and their elements
     82from :class:`sage.structure.element.Element`.
     83
     84Sage provides appropriate sub\--classes of
     85:class:`~sage.structure.parent.Parent` and
     86:class:`~sage.structure.element.Element` for a variety of more concrete
     87algebraic structures, such as groups, rings, or fields, and of their
     88elements. But some old stuff in Sage doesn't use it.  **Volunteers for
     89refactoring are welcome!**
     90
     91 
     92
     93The parent
     94----------
     95
     96Since we wish to implement a special kind of fields, namely fraction fields,
     97it makes sense to build on top of the base class
     98:class:`sage.rings.ring.Field` provided by Sage.  ::
     99
     100    sage: from sage.rings.ring import Field
     101
     102
     103This base class provides a lot more methods than a general parent::
     104
     105    sage: [p for p in dir(Field) if p not in dir(Parent)]
     106    ['__div__', '__fraction_field', '__ideal_monoid', '__iter__', '__pow__',
     107     '__rdiv__', '__rpow__', '__rxor__', '__xor__', '_an_element',
     108     '_an_element_c', '_an_element_impl', '_coerce_', '_coerce_c',
     109     '_coerce_impl', '_coerce_self', '_coerce_try', '_default_category',
     110     '_gens', '_gens_dict', '_has_coerce_map_from', '_ideal_class_',
     111     '_latex_names', '_list', '_one_element', '_pseudo_fraction_field',
     112     '_random_nonzero_element', '_richcmp', '_unit_ideal', '_zero_element',
     113     '_zero_ideal', 'algebraic_closure', 'base_extend', 'cardinality',
     114     'class_group', 'coerce_map_from_c', 'coerce_map_from_impl', 'content',
     115     'divides', 'extension', 'fraction_field', 'gcd', 'gen', 'gens',
     116     'get_action_c', 'get_action_impl', 'has_coerce_map_from_c',
     117     'has_coerce_map_from_impl', 'ideal', 'ideal_monoid', 'integral_closure',
     118     'is_atomic_repr', 'is_commutative', 'is_field', 'is_finite',
     119     'is_integral_domain', 'is_integrally_closed', 'is_noetherian',
     120     'is_prime_field', 'is_ring', 'is_subring', 'is_zero', 'krull_dimension',
     121     'list', 'ngens', 'one', 'one_element', 'order', 'prime_subfield',
     122     'principal_ideal', 'quo', 'quotient', 'quotient_ring', 'random_element',
     123     'unit_ideal', 'zero', 'zero_element', 'zero_ideal', 'zeta', 'zeta_order']
     124
     125The following is a very basic implementation of fraction fields, that needs to
     126be complemented later.
     127::
     128
     129    sage: from sage.structure.unique_representation import UniqueRepresentation
     130    sage: class MyFrac(UniqueRepresentation, Field):
     131    ...       def __init__(self, base):
     132    ...           if base not in IntegralDomains():
     133    ...               raise ValueError, "%s is no integral domain"%base
     134    ...           Field.__init__(self, base)
     135    ...       def _repr_(self):
     136    ...           return "NewFrac(%s)"%repr(self.base())
     137    ...       def base_ring(self):
     138    ...           return self.base().base_ring()
     139    ...       def characteristic(self):
     140    ...           return self.base().characteristic()
     141
     142.. end ouf output
     143
     144This basic implementation is formed by the following steps:
     145
     146- Any ring in Sage has a **base** and a **base ring**. The "usual" fraction
     147  field of a ring `R` has the base `R` and the base ring ``R.base_ring()``::
     148
     149      sage: Frac(QQ['x']).base(), Frac(QQ['x']).base_ring()
     150      (Univariate Polynomial Ring in x over Rational Field, Rational Field)
     151
     152
     153  Declaring the base is easy: We just pass it as an argument to the field
     154  constructor.
     155  ::
     156
     157      sage: Field(ZZ['x']).base()
     158      Univariate Polynomial Ring in x over Integer Ring
     159
     160  .. end of output
     161
     162  We are implementing a seperate method returning the base ring.
     163
     164- Python uses double\--underscore methods for arithemetic methods and string
     165  representations. Sage's base classes often have a default implementation,
     166  and it is requested to **implement SINGLE underscore methods _repr_, and
     167  similarly _add_, _mul_ etc.**
     168
     169- You are encouraged to **make your parent "unique"**. That's to say, parents
     170  should only evaluate equal if they are identical. Sage provides frameworks
     171  to create unique parents. We use here the most easy one: Inheriting from the
     172  class :class:`sage.structure.unique_representation.UniqueRepresentation` is
     173  enough. Making parents unique can be quite important for an efficient
     174  implementation, because the repeated creation of "the same" parent would
     175  take a lot of time.
     176
     177- Fraction fields are only defined for integral domains. Hence, we raise an
     178  error if the given ring does not belong to the category of integral
     179  domains. This is our first use case of categories.
     180
     181- Last, we add a method that returns the characteristic of the field. We don't
     182  go into details, but some automated tests that we study below implicitly
     183  rely on this method.
     184
     185We see that our basic implementation correctly refuses a ring that is not an
     186integral domain::
     187
     188    sage: MyFrac(ZZ['x'])
     189    NewFrac(Univariate Polynomial Ring in x over Integer Ring)
     190    sage: MyFrac(Integers(15))
     191    Traceback (most recent call last):
     192    ...
     193    ValueError: Ring of integers modulo 15 is no integral domain
     194
     195.. NOTE::
     196
     197    Inheritance from :class:`~sage.structure.unique_representation.UniqueRepresentation`
     198    automatically provides our class with pickling, preserving the unique
     199    parent condition. If we had defined the class in some external module or
     200    in an interactive session, pickling would work immediately.
     201
     202    However, for making the following example work in Sage's doctesting
     203    framework, we need to assign our class as an attribute of the ``__main__``
     204    module, so that the class can be looked up during unpickling.
     205
     206::
     207
     208    sage: import __main__
     209    sage: __main__.MyFrac = MyFrac
     210    sage: loads(dumps(MyFrac(ZZ))) is MyFrac(ZZ)
     211    True
     212
     213.. NOTE::
     214
     215    In the following sections, we will successively add or change details of
     216    ``MyFrac``. Rather than giving a full class definition in each step, we
     217    define new versions of ``MyFrac`` by inheriting from the previously
     218    defined version of ``MyFrac``. We believe this will help the reader to
     219    focus on the single detail that is relevant in each section.
     220
     221    The complete code can be found in the appendix.
     222
     223The elements
     224------------
     225
     226We use the base class :class:`sage.structure.element.FieldElement`. Note that
     227in the creation of field elements it is not tested that the given parent is a
     228field::
     229
     230    sage: from sage.structure.element import FieldElement
     231    sage: FieldElement(ZZ)
     232    Generic element of a structure
     233
     234Our toy implementation of fraction field elements is based on the following
     235considerations:
     236
     237- A fraction field element is defined by numerator and denominator, which both
     238  need to be elements of the base. There should be methods returning numerator
     239  resp. denominator.
     240
     241- The denominator must not be zero, and (provided that the base is an ordered
     242  ring) we can make it non-negative, without loss of generality. By default,
     243  the denominator is one.
     244
     245- The string representation is returned by the single\--underscore method
     246  ``_repr_``. In order to make our fraction field elements distinguishable
     247  from those already present in Sage, we use a different string representation.
     248
     249- Arithmetic is implemented in single\--underscore method ``_add_``, ``_mul_``,
     250  etc. **We do not override the default double underscore __add__, __mul__**,
     251  since otherwise, we could not use Sage's coercion model.
     252
     253- In the single underscore methods and in ``__cmp__``, we can assume that
     254  *both arguments belong to the same parent*. This is one benefit of the
     255  coercion model. Note that ``__cmp__`` should be provided, since otherwise
     256  comparison does not work in the way expected in Python::
     257
     258      sage: class Foo(sage.structure.element.Element):
     259      ...    def __init__(self, x, parent=None):
     260      ...        self.x = x
     261      ...    def _repr_(self):
     262      ...        return "<%s>"%self.x
     263      sage: a = Foo(1,parent=ZZ)
     264      sage: b = Foo(2,parent=ZZ)
     265      sage: cmp(a,b)
     266      Traceback (most recent call last):
     267      ...
     268      NotImplementedError: BUG: sort algorithm for elements of 'None' not implemented
     269
     270- When constructing new elements as the result of arithmetic operations, we do
     271  not directly name our class, but we use ``self.__class__``. Later, this will
     272  come in handy.
     273
     274This gives rise to the following code::
     275
     276    sage: class MyElement(FieldElement):
     277    ...       def __init__(self, n,d=None, parent=None):
     278    ...           if parent is None:
     279    ...               raise ValueError, "The parent must be provided"
     280    ...           B = parent.base()
     281    ...           if d is None:
     282    ...               d = B.one_element()
     283    ...           if n not in B or d not in B:
     284    ...               raise ValueError, "Numerator and denominator must be elements of %s"%B
     285    ...           # Numerator and denominator should not just be "in" B,
     286    ...           # but should be defined as elements of B
     287    ...           d = B(d)
     288    ...           n = B(n)
     289    ...           if d==0:
     290    ...               raise ZeroDivisionError, "The denominator must not be zero"
     291    ...           if d<0:
     292    ...               self.n = -n
     293    ...               self.d = -d
     294    ...           else:
     295    ...               self.n = n
     296    ...               self.d = d
     297    ...           FieldElement.__init__(self,parent)
     298    ...       def numerator(self):
     299    ...           return self.n
     300    ...       def denominator(self):
     301    ...           return self.d
     302    ...       def _repr_(self):
     303    ...           return "(%s):(%s)"%(self.n,self.d)
     304    ...       def __cmp__(self, other):
     305    ...           return cmp(self.n*other.denominator(), other.numerator()*self.d)
     306    ...       def _add_(self, other):
     307    ...           C = self.__class__
     308    ...           D = self.d*other.denominator()
     309    ...           return C(self.n*other.denominator()+self.d*other.numerator(),D, self.parent())
     310    ...       def _sub_(self, other):
     311    ...           C = self.__class__
     312    ...           D = self.d*other.denominator()
     313    ...           return C(self.n*other.denominator()-self.d*other.numerator(),D, self.parent())
     314    ...       def _mul_(self, other):
     315    ...           C = self.__class__
     316    ...           return C(self.n*other.numerator(), self.d*other.denominator(), self.parent())
     317    ...       def _div_(self, other):
     318    ...           C = self.__class__
     319    ...           return C(self.n*other.denominator(), self.d*other.numerator(), self.parent())
     320
     321.. end of output
     322
     323
     324Features and limitations of the basic implementation
     325^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     326
     327Thanks to the single underscore methods, some basic arithmetics works, **if**
     328we stay inside a single parent structure::
     329
     330    sage: P = MyFrac(ZZ)
     331    sage: a = MyElement(3,4,P)
     332    sage: b = MyElement(1,2,P)
     333    sage: a+b, a-b, a*b, a/b
     334    ((10):(8), (2):(8), (3):(8), (6):(4))
     335    sage: a-b == MyElement(1,4,P)
     336    True
     337
     338.. end of output
     339
     340We didn't implement exponentiation\---but it just works::
     341
     342    sage: a^3
     343    (27):(64)
     344
     345.. end of output
     346
     347There is a default implementation of element tests. We can already do
     348::
     349
     350    sage: a in P
     351    True
     352
     353.. end of output
     354
     355since `a` is defined as an element of `P`. However, we can not verify yet that
     356the integers are contained in the fraction field of the ring of integers. It
     357does not even give a wrong answer, but results in an error::
     358
     359    sage: 1 in P
     360    Traceback (most recent call last):
     361    ...
     362    NotImplementedError
     363
     364.. end of output
     365
     366We will take care of this later.
     367
     368Categories in Sage
     369==================
     370
     371Sometimes the base classes do not reflect the mathematics: The set of `m\times
     372n` matrices over a field forms, in general, not more than a vector
     373space. Hence, this set (called :class:`~sage.matrix.matrix_space.MatrixSpace`)
     374is not implemented on top of :class:`sage.rings.ring.Ring`.  However, if
     375`m=n`, then the matrix space is an algebra, thus, is a ring.
     376
     377From the point of view of Python base classes, both cases are the same::
     378
     379    sage: MS1 = MatrixSpace(QQ,2,3)
     380    sage: isinstance(MS1, Ring)
     381    False
     382    sage: MS2 = MatrixSpace(QQ,2)
     383    sage: isinstance(MS2, Ring)
     384    False
     385
     386.. end of output
     387
     388Sage's category framework can differentiate the two cases::
     389
     390    sage: Rings()
     391    Category of rings
     392    sage: MS1 in Rings()
     393    False
     394    sage: MS2 in Rings()
     395    True
     396
     397.. end of output
     398
     399Surprisingly, ``MS2`` has *more* methods than ``MS1``, even though their classes
     400coincide::
     401
     402    sage: import inspect
     403    sage: len([s for s in dir(MS1) if inspect.ismethod(getattr(MS1,s,None))])
     404    53
     405    sage: len([s for s in dir(MS2) if inspect.ismethod(getattr(MS2,s,None))])
     406    75
     407    sage: MS1.__class__ is MS2.__class__
     408    True
     409
     410.. end of output
     411
     412Below, we will explain how this can be taken advantage of.
     413
     414It is no surprise that our parent `P` defined above knows that it belongs to
     415the category of fields, as it is derived from the base class of fields.
     416
     417::
     418
     419    sage: P.category()
     420    Category of fields
     421
     422.. end of output
     423
     424However, we could choose a smaller category, namely the category of quotient fields.
     425
     426Why should one choose a category?
     427---------------------------------
     428
     429One can provide **default methods** *for all objects* of a category, and
     430*for all elements* of such objects. Hence, the category framework is a way
     431to inherit useful stuff that is not present in the base classes.  These
     432default methods do not rely on implementation details, but on mathematical
     433concepts.
     434
     435In addition, the categories define **test suites** for their objects and
     436elements\---see the last section. Hence, one also gets basic sanity tests for
     437free.
     438
     439
     440How does the  *category framework* work?
     441^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     442
     443Abstract base classes for the objects ("parent_class") and the elements of
     444objects ("element_class") are provided by attributes of the category. During
     445initialisation of a parent, the class of the parent is *dynamically changed*
     446into a sub\--class of the category's parent class. Likewise, sub\--classes of
     447the category's element class are available for the creation of elements of the
     448parent, as explained below.
     449
     450A dynamic change of classes does not work in Cython. Nevertheless, method
     451inheritance still works, by virtue of a ``__getattr__`` method.
     452
     453.. NOTE::
     454
     455    It is strongly recommended to use the category framework both in Python
     456    and in Cython.
     457
     458Let us see whether there is any gain in chosing the category of quotient
     459fields instead of the category of fields::
     460
     461    sage: QuotientFields().parent_class, QuotientFields().element_class
     462    (<class 'sage.categories.quotient_fields.QuotientFields.parent_class'>,
     463     <class 'sage.categories.quotient_fields.QuotientFields.element_class'>)
     464    sage: [p for p in dir(QuotientFields().parent_class) if p not in dir(Fields().parent_class)]
     465    []
     466    sage: [p for p in dir(QuotientFields().element_class) if p not in dir(Fields().element_class)]
     467    ['_derivative', 'denominator', 'derivative', 'factor',
     468     'numerator', 'partial_fraction_decomposition']
     469
     470.. end of output
     471
     472So, there is no immediate gain for our fraction fields, but additional methods
     473become available to our fraction field elements. Note that some of these
     474methods are place-holders: There is no default implementation, but it is
     475*required* (respectively is *optional*) to implement these methods::
     476
     477    sage: QuotientFields().element_class.denominator
     478    <abstract method denominator at ...>
     479    sage: from sage.misc.abstract_method import abstract_methods_of_class
     480    sage: abstract_methods_of_class(QuotientFields().element_class)['optional']
     481    ['_add_', '_mul_']
     482    sage: abstract_methods_of_class(QuotientFields().element_class)['required']
     483    ['__nonzero__', 'denominator', 'numerator']
     484
     485Hence, when implementing elements of a quotient field, it is *required* to
     486implement methods returning the denominator and the numerator, and a method
     487that tells whether the element is nonzero, and in addition, it is *optional*
     488(but certainly recommended) to provide some arithmetic methods. If one forgets
     489to implement the required methods, the test suites of the category framework
     490will complain\---see below.
     491
     492
     493Implementing the category framework for the parent
     494--------------------------------------------------
     495
     496We simply need to declare the correct category by an optional argument of the
     497field constructor, where we provide the possibility to override the default
     498category::
     499
     500    sage: from sage.categories.quotient_fields import QuotientFields
     501    sage: class MyFrac(MyFrac):
     502    ...       def __init__(self, base, category=None):
     503    ...           if base not in IntegralDomains():
     504    ...               raise ValueError, "%s is no integral domain"%base
     505    ...           Field.__init__(self, base, category=category or QuotientFields())
     506
     507When constructing instances of ``MyFrac``, their class is dynamically changed
     508into a new class called ``MyFrac_with_category``. It is a common sub\--class of
     509``MyFrac`` and of the category's parent class::
     510
     511    sage: P = MyFrac(ZZ)
     512    sage: type(P)
     513    <class '__main__.MyFrac_with_category'>
     514    sage: isinstance(P,MyFrac)
     515    True
     516    sage: isinstance(P,QuotientFields().parent_class)
     517    True
     518
     519The fraction field `P` inherits additional methods. For example, the base
     520class :class:`~sage.rings.fields.Field` does not have a method ``sum``. But
     521`P` inherits such method from the category of commutative additive
     522monoids\---see
     523:meth:`~sage.categories.commutative_additive_monoids.CommutativeAdditiveMonoids.ParentMethods.sum`::
     524
     525    sage: P.sum.__module__
     526    'sage.categories.commutative_additive_monoids'
     527
     528.. end of output
     529
     530We have seen above that we can add elements. Nevertheless, the ``sum`` method
     531does not work, yet::
     532
     533    sage: a = MyElement(3,4,P)
     534    sage: b = MyElement(1,2,P)
     535    sage: c = MyElement(-1,2,P)
     536    sage: P.sum([a,b,c])
     537    Traceback (most recent call last):
     538    ...
     539    NotImplementedError
     540
     541.. end of output
     542
     543The reason is that the ``sum`` method starts with the return value of
     544``P.zero_element()``, which defaults to ``P(0)``\---but the conversion of
     545integers into ``P`` is not implemented, yet.
     546
     547Implementing the category framework for the elements
     548----------------------------------------------------
     549
     550Similar to what we have seen for parents, a new class is dynamically created
     551that combines the element class of the parent's category with the class that
     552we have implemented above. However, the category framework is implemented in a
     553different way for elements than for parents:
     554
     555- We provide the parent `P` (or its class) with an attribute called
     556  "``Element``", whose value is a class.
     557- The parent *automatically* obtains an attribute ``P.element_class``, that
     558  subclasses both ``P.Element`` and ``P.category().element_class``.
     559 
     560Hence, for providing our fraction fields with their own element classes, **we
     561just need to add a single line to our class**::
     562
     563    sage: class MyFrac(MyFrac):
     564    ...       Element = MyElement
     565
     566
     567.. end of output
     568
     569This little change provides several benefits:
     570
     571- We can now create elements by simply calling the parent::
     572
     573      sage: P = MyFrac(ZZ)
     574      sage: P(1), P(2,3)
     575      ((1):(1), (2):(3))
     576
     577- There is a method ``zero_element`` returning the expected result::
     578
     579      sage: P.zero_element()
     580      (0):(1)
     581
     582- The ``sum`` method mentioned above suddenly works::
     583
     584      sage: a = MyElement(9,4,P)
     585      sage: b = MyElement(1,2,P)
     586      sage: c = MyElement(-1,2,P)
     587      sage: P.sum([a,b,c])
     588      (36):(16)
     589
     590.. end of output
     591
     592What did happen behind the scenes to make this work?
     593^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     594
     595We provided ``P.Element``, and thus obtain ``P.element_class``, which is a
     596*lazy attribute*.  It provides a *dynamic* class, which is a sub\--class of
     597both ``MyElement`` defined above and of ``P.category().element_class``::
     598
     599    sage: P.__class__.element_class
     600    <sage.misc.lazy_attribute.lazy_attribute object at ...>
     601    sage: P.element_class
     602    <class '__main__.MyFrac_with_category.element_class'>
     603    sage: type(P.element_class)
     604    <class 'sage.structure.dynamic_class.DynamicMetaclass'>
     605    sage: issubclass(P.element_class, MyElement)
     606    True
     607    sage: issubclass(P.element_class,P.category().element_class)
     608    True
     609
     610.. end of output
     611
     612The *default* ``__call__`` method of `P` passes the given arguments to
     613``P.element_class``, adding the argument ``parent=P``. This is why we are now
     614able to create elements by calling the parent.
     615
     616In particular, these elements are instances of that new dynamic class::
     617
     618    sage: type(P(2,3))
     619    <class '__main__.MyFrac_with_category.element_class'>
     620
     621.. end of output
     622
     623.. NOTE::
     624
     625    *All* elements of `P` should use the element class. In order to make sure
     626    that this also holds for the result of arithmetic operations, we created
     627    them as instances of ``self.__class__`` in the arithmetic methods of
     628    ``MyElement``.
     629
     630``P.zero_element()`` defaults to returning ``P(0)`` and thus returns an
     631instance of ``P.element_class``. Since ``P.sum([...])`` starts the summation with
     632``P.zero_element()`` and the class of the sum only depends on the first
     633summand, by our implementation, we have::
     634
     635    sage: type(a)
     636    <class '__main__.MyElement'>
     637    sage: isinstance(a,P.element_class)
     638    False
     639    sage: type(P.sum([a,b,c]))
     640    <class '__main__.MyFrac_with_category.element_class'>
     641
     642.. end of output
     643
     644The method ``factor`` provided by ``P.category().element_class`` (see above)
     645simply works::
     646
     647    sage: a; a.factor(); P(6,4).factor()
     648    (9):(4)
     649    2^-2 * 3^2
     650    2^-1 * 3
     651
     652.. end of output
     653
     654But that's surprising: The element `a` is just an instance of ``MyElement``,
     655but not of ``P.element_class``, and its class does not know about the factor
     656method.  In fact, this is due to a ``__getattr__`` method defined for
     657:class:`sage.structure.element.Element`.
     658::
     659
     660    sage: hasattr(type(a), 'factor')
     661    False
     662    sage: hasattr(P.element_class, 'factor')
     663    True
     664    sage: hasattr(a, 'factor')
     665    True
     666
     667.. end of output
     668
     669A first note on performance
     670---------------------------
     671
     672The category framework is sometimes blamed for speed regressions, as in
     673:trac:`9138` and :trac:`11900`. But if the category framework is *used
     674properly*, then it is fast. For illustration, we determine the time needed to
     675access an attribute inherited from the element class. First, we consider an
     676element that uses the class that we implemented above, but does not use the
     677category framework properly::
     678
     679    sage: type(a)
     680    <class '__main__.MyElement'>
     681    sage: timeit('a.factor',number=1000)     # random
     682    1000 loops, best of 3: 2 us per loop
     683
     684.. end of output
     685
     686Now, we consider an element that is equal to `a`, but uses the category
     687framework properly::
     688
     689    sage: a2 = P(9,4)
     690    sage: a2 == a
     691    True
     692    sage: type(a2)
     693    <class '__main__.MyFrac_with_category.element_class'>
     694    sage: timeit('a2.factor',number=1000)    # random
     695    1000 loops, best of 3: 365 ns per loop
     696
     697.. end of output
     698
     699So,  *don't be afraid of using categories!*
     700
     701
     702Coercion\---the basics
     703======================
     704
     705Theoretical background
     706----------------------
     707
     708Coercion is not just *type conversion*
     709^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     710
     711"Coercion" in the C programming language means "automatic type
     712conversion". However, in Sage, coercion is involved if one wants to be able to
     713do arithmetic, comparisons, etc. between elements of distinct parents. Hence,
     714**coercion is not about a change of types, but about a change of parents.**
     715
     716As an illustration, we show that elements of the same type may very well belong
     717to rather different parents::
     718
     719    sage: P1 = QQ['v,w']; P2 = ZZ['w,v']
     720    sage: type(P1.gen()) == type(P2.gen())
     721    True
     722    sage: P1 == P2
     723    False
     724
     725.. end of output
     726
     727`P_2` naturally is a sub\--ring of `P_1`. So, it makes sense to be able to add
     728elements of the two rings\---the result should then live in `P_1`, and indeed
     729it does::
     730
     731    sage: (P1.gen()+P2.gen()).parent() is P1
     732    True
     733
     734.. end of output
     735
     736It would be rather inconvenient if one needed to *manually* convert an element
     737of `P_2` into `P_1` before adding. The coercion system does that conversion
     738automatically.
     739
     740Not every conversion is a coercion
     741^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     742
     743A coercion happens implicitly, without being explicitly requested by the
     744user. Hence, coercion must be based on mathematical rigour. In our example,
     745any element of `P_2` can be naturally interpreted as an element of `P_1`. We
     746thus have::
     747
     748    sage: P1.has_coerce_map_from(P2)
     749    True
     750    sage: P1.coerce_map_from(P2)
     751    Call morphism:
     752      From: Multivariate Polynomial Ring in w, v over Integer Ring
     753      To:   Multivariate Polynomial Ring in v, w over Rational Field
     754
     755While there is a conversion from `P_1` to `P_2` (namely restricted to
     756polynomials with integral coefficients), this conversion is not a coercion::
     757
     758    sage: P2.convert_map_from(P1)
     759    Call morphism:
     760      From: Multivariate Polynomial Ring in v, w over Rational Field
     761      To:   Multivariate Polynomial Ring in w, v over Integer Ring
     762    sage: P2.has_coerce_map_from(P1)
     763    False
     764    sage: P2.coerce_map_from(P1) is None
     765    True
     766
     767.. end of output
     768
     769The four axioms requested for coercions
     770.......................................
     771
     7721. A coercion is a morphism in an appropriate category.
     773
     774   This first axiom has two implications:
     775
     776   A. A coercion is defined on all elements of a parent.
     777
     778      A polynomial of degree zero over the integers can be interpreted as an
     779      integer\---but the attempt to convert a polynomial of non-zero degree
     780      would result in an error::
     781
     782          sage: ZZ(P2.one())
     783          1
     784          sage: ZZ(P2.gen(1))
     785          Traceback (most recent call last):
     786          ...
     787          TypeError: not a constant polynomial
     788
     789      Hence, we only have a *partial* map. This is fine for a *conversion*,
     790      but a partial map does not qualify as a *coercion*.
     791
     792   B. Coercions are structure preserving.
     793
     794      Any real number can be converted to an integer, namely by
     795      rounding. However, such a conversion is not useful in arithmetic
     796      operations, since the underlying algebraic structure is not preserved::
     797
     798          sage: int(1.6)+int(2.7) == int(1.6+2.7)
     799          False
     800
     801      .. end of output
     802
     803      The structure that is to be preserved depends on the category of the
     804      involved parents. For example, the coercion from the integers into the
     805      rational field is a homomorphism of euclidean domains::
     806
     807          sage: QQ.coerce_map_from(ZZ).category_for()
     808          Category of euclidean domains
     809
     810      .. end of output
     811
     8122. There is at most one coercion from one parent to another
     813
     814   In addition, if there is a *coercion* from `P_2` to `P_1`, then a
     815   *conversion* from `P_2` to `P_1` is defined for all elements of `P_2` and
     816   coincides with the coercion.
     817   ::
     818
     819       sage: P1.coerce_map_from(P2) is P1.convert_map_from(P2)
     820       True
     821
     822   .. end of output
     823
     8243. Coercions can be composed
     825
     826   If there is a coercion `\varphi: P_1 \to P_2` and another coercion `\psi:
     827   P_2 \to P_3`, then the composition of `\varphi` followed by `\psi` must
     828   yield the unique coercion from `P_1` to `P_3`.
     829
     8304. The identity is a coercion
     831
     832   Together with the two preceding axioms, it follows: If there are coercions
     833   from `P_1` to `P_2` and from `P_2` to `P_1`, then they are mutually
     834   inverse.
     835
     836
     837Implementing a conversion
     838-------------------------
     839
     840We have seen above that some conversions into our fraction fields became
     841available after providing the attribute ``Element``.  However, we can not
     842convert elements of a fraction field into elements of another fraction field,
     843yet::
     844
     845    sage: P(2/3)
     846    Traceback (most recent call last):
     847    ...
     848    ValueError: Numerator and denominator must be elements of Integer Ring
     849
     850.. end of output
     851
     852For implementing a conversion, **the default __call__ method should (almost)
     853never be overridden.** Instead, **we implement the method
     854_element_constructor_**, that should return an instance of the parent's
     855element class.  Some old parent classes violate that rule\---please help to
     856refactor them!
     857::
     858
     859    sage: class MyFrac(MyFrac):
     860    ...       def _element_constructor_(self, *args,**kwds):
     861    ...           if len(args)!=1:
     862    ...               return self.element_class(*args,parent=self,**kwds)
     863    ...           x = args[0]
     864    ...           try:
     865    ...               P = x.parent()
     866    ...           except AttributeError:
     867    ...               return self.element_class(x,parent=self,**kwds)
     868    ...           if P in QuotientFields() and P != self.base():
     869    ...               return self.element_class(x.numerator(),x.denominator(),parent=self,**kwds)
     870    ...           return self.element_class(x,parent=self,**kwds)
     871
     872
     873.. end of output
     874
     875In addition to the conversion from the base ring and from pairs of base ring
     876elements, we now also have a conversion from the rationals to our fraction
     877field of `\ZZ`:
     878
     879
     880::
     881
     882    sage: P = MyFrac(ZZ)
     883    sage: P(2); P(2,3); P(3/4)
     884    (2):(1)
     885    (2):(3)
     886    (3):(4)
     887
     888.. end of output
     889
     890Recall that above, the test `1 \in P` failed with an error. We try again and
     891find that the error has disappeared. This is because we are now able to
     892convert the integer `1` into `P`. But the containment test still yields a
     893wrong answer::
     894
     895    sage: 1 in P
     896    False
     897
     898.. end of output
     899
     900The technical reason: We have a conversion `P(1)` of `1` into `P`, but this is
     901not known as a coercion\---yet!
     902::
     903
     904    sage: P.has_coerce_map_from(ZZ), P.has_coerce_map_from(QQ)
     905    (False, False)
     906
     907.. end of output
     908
     909Establishing a coercion
     910-----------------------
     911
     912There are two main ways to make Sage use a particular conversion as a
     913coercion:
     914
     915- One can use :meth:`sage.structure.parent.Parent.register_coercion`, normally
     916  during initialisation of the parent (see documentation of the method).
     917- A more flexible way is to provide a method ``_coerce_map_from_`` for the
     918  parent.
     919
     920Let `P` and `R` be parents. If ``P._coerce_map_from_(R)`` returns ``False``
     921or ``None``, then there is no coercion from `R` to `P`. If it returns a map
     922with domain `R` and codomain `P`, then this map is used for coercion. If it
     923returns ``True``, then the conversion from `R` to `P` is used as coercion.
     924
     925Note that in the following implementation, we need a special case for the
     926rational field, since ``QQ.base()`` is not the ring of integers.
     927::
     928
     929    sage: class MyFrac(MyFrac):
     930    ...       def _coerce_map_from_(self, S):
     931    ...           if self.base().has_coerce_map_from(S):
     932    ...               return True
     933    ...           if S in QuotientFields():
     934    ...               if self.base().has_coerce_map_from(S.base()):
     935    ...                   return True
     936    ...               if hasattr(S,'ring_of_integers') and self.base().has_coerce_map_from(S.ring_of_integers()):
     937    ...                   return True
     938
     939
     940.. end of output
     941
     942By the method above, a parent coercing into the base ring will also coerce
     943into the fraction field, and a fraction field coerces into another fraction
     944field if there is a coercion of the corresponding base rings. Now, we have::
     945
     946    sage: P = MyFrac(QQ['x'])
     947    sage: P.has_coerce_map_from(ZZ['x']), P.has_coerce_map_from(Frac(ZZ['x'])), P.has_coerce_map_from(QQ)
     948    (True, True, True)
     949
     950.. end of output
     951
     952We can now use coercion from `\ZZ[x]` and from `\QQ` into `P` for arithmetic
     953operations between the two rings::
     954
     955    sage: 3/4+P(2)+ZZ['x'].gen(), (P(2)+ZZ['x'].gen()).parent() is P
     956    ((4*x + 11):(4), True)
     957
     958.. end of output
     959
     960Equality and element containment
     961^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     962
     963Recall that above, the test `1 \in P` gave a wrong answer. Let us repeat the
     964test now::
     965
     966    sage: 1 in P
     967    True
     968
     969.. end of output
     970
     971Why is that?
     972
     973The default element containment test `x \in P` is based on the interplay of
     974three building blocks: conversion, coercion, and equality test.
     975
     976#. Clearly, if the conversion `P(x)` raises an error, then `x` can not be seen as an element of `P`. On the other hand, a conversion `P(x)` can generally do very nasty things. So, the fact that `P(x)` works without error is necessary, but not sufficient for `x \in P`.
     977#. If `P` is the parent of `x`, then the conversion `P(x)` will not change `x` (at least, that's the default). Hence, we will have `x=P(x)`.
     978#. Sage uses coercion not only for arithmetic operations, but also for comparison: *If* there is a coercion from the parent of `x` to `P`, then the equality test ``x==P(x)`` reduces to ``P(x)==P(x)``. Otherwise, ``x==P(x)`` will evaluate as false.
     979
     980That leads to the following default implementation of element containment testing:
     981
     982.. NOTE::
     983
     984    `x \in P` holds if and only if the test ``x==P(x)`` does not
     985    raise an error and evaluates as true.
     986
     987If the user is not happy with that behaviour, the "magical" Python method
     988``__contains__`` can be overridden.
     989
     990Coercion\---the advanced parts
     991==============================
     992
     993So far, we are able to add integers and rational numbers to elements of our
     994new implementation of the fraction field of `\ZZ`.
     995
     996::
     997
     998    sage: P = MyFrac(ZZ)
     999
     1000
     1001.. end of output
     1002
     1003::
     1004
     1005    sage: 1/2+P(2,3)+1
     1006    (13):(6)
     1007
     1008.. end of output
     1009
     1010Surprisingly, we can even add a polynomial over the integers to an element of
     1011`P`, even though the *result lives in a new parent*, namely in a polynomial
     1012ring over `P`::
     1013
     1014    sage: P(1/2) + ZZ['x'].gen(), (P(1/2) + ZZ['x'].gen()).parent() is P['x']
     1015    ((1):(1)*x + (1):(2), True)
     1016
     1017.. end of output
     1018
     1019In the next, seemingly more easy example, there "obviously" is a coercion from
     1020the fraction field of `\ZZ` to the fraction field of `\ZZ[x]`.  However, Sage
     1021does not know enough about our new implementation of fraction fields. Hence,
     1022it does not recognise the coercion::
     1023
     1024    sage: Frac(ZZ['x']).has_coerce_map_from(P)
     1025    False
     1026
     1027.. end of output
     1028
     1029Two obvious questions arise:
     1030
     1031#. How / why has the new ring been constructed in the example above?
     1032#. How can we establish a coercion from `P`  to  `\mathrm{Frac}(\ZZ[x])`?
     1033
     1034The key to answering both question is the construction of parents from simpler
     1035pieces, that we are studying now. Note that we will answer the second question
     1036*not* by providing a coercion from `P`  to  `\mathrm{Frac}(\ZZ[x])`, but by
     1037teaching Sage to automatically construct `\mathrm{MyFrac}(\ZZ[x])` and coerce
     1038both `P` and `\mathrm{Frac}(\ZZ[x])` into it.
     1039
     1040If we are lucky, a parent can tell how it has been constructed::
     1041
     1042    sage: Poly,R = QQ['x'].construction()
     1043    sage: Poly,R
     1044    (Poly[x], Rational Field)
     1045    sage: Fract,R = QQ.construction()
     1046    sage: Fract,R
     1047    (FractionField, Integer Ring)
     1048
     1049In both cases, the first value returned by
     1050:meth:`~sage.structure.parent.Parent.construction` is a mathematical
     1051construction, called *construction functor*\---see
     1052:class:`~sage.categories.pushout.ConstructionFunctor`. The second return value
     1053is a simpler parent to which the construction functor is applied.
     1054
     1055Being functors, the same construction can be applied to different objects of a
     1056category::
     1057
     1058    sage: Poly(QQ) is QQ['x']
     1059    True
     1060    sage: Poly(ZZ) is ZZ['x']
     1061    True
     1062    sage: Poly(P) is P['x']
     1063    True
     1064    sage: Fract(QQ['x'])
     1065    Fraction Field of Univariate Polynomial Ring in x over Rational Field
     1066
     1067Let us see on which categories these construction functors are defined::
     1068
     1069    sage: Poly.domain()
     1070    Category of rings
     1071    sage: Poly.codomain()
     1072    Category of rings
     1073    sage: Fract.domain()
     1074    Category of integral domains
     1075    sage: Fract.codomain()
     1076    Category of fields
     1077
     1078In particular, the construction functors can be composed::
     1079
     1080    sage: Poly*Fract
     1081    Poly[x](FractionField(...))
     1082    sage: (Poly*Fract)(ZZ) is QQ[x]
     1083    True
     1084
     1085.. end of output
     1086
     1087In addition, it is assumed that we have a coercion from input to output of the
     1088construction functor::
     1089
     1090    sage: ((Poly*Fract)(ZZ))._coerce_map_from_(ZZ)
     1091    Composite map:
     1092      From: Integer Ring
     1093      To:   Univariate Polynomial Ring in x over Rational Field
     1094      Defn:   Natural morphism:
     1095              From: Integer Ring
     1096              To:   Rational Field
     1097            then
     1098              Polynomial base injection morphism:
     1099              From: Rational Field
     1100              To:   Univariate Polynomial Ring in x over Rational Field
     1101
     1102.. end of output
     1103
     1104Construction functors do not necessarily commute::
     1105
     1106    sage: (Fract*Poly)(ZZ)
     1107    Fraction Field of Univariate Polynomial Ring in x over Integer Ring
     1108
     1109.. end of output
     1110
     1111
     1112The pushout of construction functors
     1113------------------------------------
     1114
     1115We can now formulate our problem. We have parents `P_1`, `P_2` and `R`, and
     1116construction functors `F_1`, `F_2`, such that `P_1 = F_1(R)` and `P_2 =
     1117F_2(R)`. We want to find a new construction functor `F_3`, such that both
     1118`P_1` and `P_2` coerce into `P_3 = F_3(R)`.
     1119
     1120In analogy to a notion of category theory, `P_3` is called the
     1121:func:`~sage.categories.pushout.pushout` of `P_1` and `P_2`; and similarly
     1122`F_3` is called the pushout of `F_1` and `F_2`.
     1123::
     1124
     1125    sage: from sage.categories.pushout import pushout
     1126    sage: pushout(Fract(ZZ),Poly(ZZ))
     1127    Univariate Polynomial Ring in x over Rational Field
     1128
     1129.. end of output
     1130
     1131`F_1\circ F_2` and `F_2\circ F_1` are natural candidates for the pushout of
     1132`F_1` and `F_2`. However, the order of the functors must rely on a canonical
     1133choice. "Indecomposable" construction functors have a *rank*, and this allows
     1134to order them canonically:
     1135
     1136.. NOTE::
     1137
     1138    If ``F1.rank`` is smaller than ``F2.rank``, then the pushout is `F_2\circ
     1139    F_1` (hence, `F_1` is applied first).
     1140
     1141We have
     1142::
     1143
     1144    sage: Fract.rank, Poly.rank
     1145    (5, 9)
     1146
     1147.. end of output
     1148
     1149and thus the pushout is
     1150::
     1151
     1152    sage: Fract.pushout(Poly), Poly.pushout(Fract)
     1153    (Poly[x](FractionField(...)), Poly[x](FractionField(...)))
     1154
     1155.. end of output
     1156
     1157This is why the example above has worked.
     1158
     1159However, only "elementary" construction functors have a rank::
     1160
     1161    sage: (Fract*Poly).rank
     1162    Traceback (most recent call last):
     1163    ...
     1164    AttributeError: 'CompositeConstructionFunctor' object has no attribute 'rank'
     1165
     1166.. end of output
     1167
     1168Shuffling composite construction functors
     1169^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     1170
     1171If composed construction fuctors `...\circ F_2\circ F_1` and `...\circ
     1172G_2\circ G_1` are given, then Sage determines their pushout by *shuffling* the
     1173constituents:
     1174
     1175- If ``F1.rank < G1.rank`` then we apply `F_1` first, and continue with `...\circ F_3\circ F_2` and `...\circ G_2\circ G_1`.
     1176- If ``F1.rank > G1.rank`` then we apply `G_1` first, and continue with `...\circ F_2\circ F_1` and `...\circ G_3\circ G_2`.
     1177
     1178If ``F1.rank == G1.rank``, then the tie needs to be broken by other techniques
     1179(see below).
     1180
     1181As an illustration, we first get us some functors and then see how chains of
     1182functors are shuffled.
     1183::
     1184
     1185    sage: AlgClos, R = CC.construction(); AlgClos
     1186    AlgebraicClosureFunctor
     1187
     1188.. end of output
     1189
     1190::
     1191
     1192    sage: Compl, R = RR.construction(); Compl
     1193    Completion[+Infinity]
     1194
     1195.. end of output
     1196
     1197::
     1198
     1199    sage: Matr, R = (MatrixSpace(ZZ,3)).construction(); Matr
     1200    MatrixFunctor
     1201
     1202.. end of output
     1203
     1204::
     1205
     1206    sage: AlgClos.rank, Compl.rank, Fract.rank, Poly.rank, Matr.rank
     1207    (3, 4, 5, 9, 10)
     1208
     1209.. end of output
     1210
     1211When we apply ``Fract``, ``AlgClos``, ``Poly`` and ``Fract`` to the ring of
     1212integers, we obtain::
     1213
     1214    sage: (Fract*Poly*AlgClos*Fract)(ZZ)
     1215    Fraction Field of Univariate Polynomial Ring in x over Algebraic Field
     1216
     1217.. end of output
     1218
     1219When we apply ``Compl``, ``Matr`` and ``Poly`` to the ring of integers, we
     1220obtain::
     1221
     1222    sage: (Poly*Matr*Compl)(ZZ)
     1223    Univariate Polynomial Ring in x over Full MatrixSpace of 3 by 3 dense matrices over Real Field with 53 bits of precision
     1224
     1225.. end of output
     1226
     1227Applying the shuffling procedure yields
     1228::
     1229
     1230    sage: (Poly*Matr*Fract*Poly*AlgClos*Fract*Compl)(ZZ)
     1231    Univariate Polynomial Ring in x over Full MatrixSpace of 3 by 3 dense matrices over Fraction Field of Univariate Polynomial Ring in x over Complex Field with 53 bits of precision
     1232
     1233.. end of output
     1234
     1235and this is indeed equal to the pushout found by Sage::
     1236
     1237    sage: pushout((Fract*Poly*AlgClos*Fract)(ZZ), (Poly*Matr*Compl)(ZZ))
     1238    Univariate Polynomial Ring in x over Full MatrixSpace of 3 by 3 dense matrices over Fraction Field of Univariate Polynomial Ring in x over Complex Field with 53 bits of precision
     1239
     1240.. end of output
     1241
     1242Breaking the tie
     1243^^^^^^^^^^^^^^^^
     1244
     1245If ``F1.rank==G1.rank`` then Sage's pushout constructions offers two ways to
     1246proceed:
     1247
     1248#. Construction functors have a method :meth:`~sage.categories.pushout.ConstructionFunctor.merge` that either returns ``None`` or returns a construction functor\---see below. If either ``F1.merge(G1)`` or ``G1.merge(F1)`` returns a construction functor `H_1`, then we apply `H_1` and continue with `...\circ F_3\circ F_2` and `...\circ G_3\circ G_2`.
     1249#. Construction functors have a method :meth:`~sage.categories.pushout.ConstructionFunctor.commutes`. If either ``F1.commutes(G1)`` or ``G1.commutes(F1)`` returns ``True``, then we apply both `F_1` and `G_1` in any order, and continue with `...\circ F_3\circ F_2` and `...\circ G_3\circ G_2`.
     1250
     1251By default, ``F1.merge(G1)`` returns ``F1`` if ``F1==G1``, and returns
     1252``None`` otherwise. The ``commutes()`` method exists, but it seems that so far
     1253nobody has implemented two functors of the same rank that commute.
     1254
     1255Establishing a default implementation
     1256-------------------------------------
     1257
     1258The typical application of
     1259:meth:`~sage.categories.pushout.ConstructionFunctor.merge` is to provide a
     1260coercion between *different implementations* of the *same algebraic
     1261structure*.
     1262
     1263.. NOTE::
     1264
     1265    If ``F1(P)`` and ``F2(P)`` are different implementations of the same thing, then ``F1.merge(F2)(P)`` should return the default implementation.
     1266
     1267We want to boldly turn our toy implementation of fraction fields into the new
     1268default implementation. Hence:
     1269
     1270- Next, we implement a new version of the "usual" fraction field functor, having the same rank, but returning our new implementation.
     1271- We make our new implementation the default, by virtue of a merge method.
     1272
     1273.. WARNING::
     1274
     1275  - Do not override the default ``__call__`` method of :class:`~sage.categories.pushout.ConstructionFunctor`\---implement ``_apply_functor`` instead.
     1276  - Declare domain and codomain of the functor during initialisation.
     1277
     1278::
     1279
     1280    sage: from sage.categories.pushout import ConstructionFunctor
     1281    sage: class MyFracFunctor(ConstructionFunctor):
     1282    ...       rank = 5
     1283    ...       def __init__(self):
     1284    ...           ConstructionFunctor.__init__(self, IntegralDomains(), Fields())
     1285    ...       def _apply_functor(self, R):
     1286    ...           return MyFrac(R)
     1287    ...       def merge(self, other):
     1288    ...           if isinstance(other, (type(self), sage.categories.pushout.FractionField)):
     1289    ...               return self
     1290
     1291
     1292.. end of output
     1293
     1294::
     1295
     1296    sage: MyFracFunctor()
     1297    MyFracFunctor
     1298
     1299.. end of output
     1300
     1301We verify that our functor can really be used to construct our implementation of fraction fields, and that it can be merged with either itself or the usual fraction field constructor:
     1302
     1303
     1304::
     1305
     1306    sage: MyFracFunctor()(ZZ)
     1307    NewFrac(Integer Ring)
     1308
     1309.. end of output
     1310
     1311::
     1312
     1313    sage: MyFracFunctor().merge(MyFracFunctor())
     1314    MyFracFunctor
     1315
     1316.. end of output
     1317
     1318::
     1319
     1320    sage: MyFracFunctor().merge(Fract)
     1321    MyFracFunctor
     1322
     1323.. end of output
     1324
     1325There remains to let our new fraction fields know about the new construction functor:
     1326
     1327
     1328::
     1329
     1330    sage: class MyFrac(MyFrac):
     1331    ...       def construction(self):
     1332    ...           return MyFracFunctor(), self.base()
     1333
     1334
     1335.. end of output
     1336
     1337::
     1338
     1339    sage: MyFrac(ZZ['x']).construction()
     1340    (MyFracFunctor, Univariate Polynomial Ring in x over Integer Ring)
     1341
     1342.. end of output
     1343
     1344Due to merging, we have:
     1345
     1346
     1347::
     1348
     1349    sage: pushout(MyFrac(ZZ['x']), Frac(QQ['x']))
     1350    NewFrac(Univariate Polynomial Ring in x over Rational Field)
     1351
     1352.. end of output
     1353
     1354A second note on performance
     1355----------------------------
     1356
     1357Being able to do arithmetics involving elements of different parents, with the
     1358automatic creation of a pushout to contain the result, is certainly
     1359convenient\---but one should not rely on it, if speed matters. Simply the
     1360conversion of elements into different parents takes time. Moreover, by
     1361:trac:`14058`, the pushout may be subject to Python's cyclic garbage
     1362collection. Hence, if one does not keep a strong reference to it, the same
     1363parent may be created repeatedly, which is a waste of time. In the following
     1364example, we illustrate the slow\--down resulting from blindly relying on
     1365coercion::
     1366
     1367    sage: ZZxy = ZZ['x','y']
     1368    sage: a = ZZxy('x')
     1369    sage: b = 1/2
     1370    sage: timeit("c = a+b")    # random
     1371    10000 loops, best of 3: 172 us per loop
     1372    sage: QQxy = QQ['x','y']
     1373    sage: timeit("c2 = QQxy(a)+QQxy(b)") # random
     1374    10000 loops, best of 3: 168 us per loop
     1375    sage: a2 = QQxy(a)
     1376    sage: b2 = QQxy(b)
     1377    sage: timeit("c2 = a2+b2") # random
     1378    100000 loops, best of 3: 10.5 us per loop
     1379
     1380Hence, if one avoids the explicit or implicit conversion into the pushout, but
     1381works in the pushout right away, one can get a more than 10\--fold speed\--up.
     1382
     1383The test suites of the category framework
     1384=========================================
     1385
     1386The category framework does not only provide functionality but also a test
     1387framework. This section logically belongs to the section on categories, but
     1388without the bits that we have implemented in the section on coercion, our
     1389implementation of fraction fields would not have passed the tests yet.
     1390
     1391"Abstract" methods
     1392------------------
     1393
     1394We have already seen above that a category can require/suggest certain parent
     1395or element methods, that the user must/should implement. This is in order to
     1396smoothly blend with the methods that already exist in Sage.
     1397
     1398The methods that ought to be provided are called
     1399:func:`~sage.misc.abstract_method.abstract_method`. Let us see what methods
     1400are needed for quotient fields and their elements::
     1401
     1402    sage: from sage.misc.abstract_method import abstract_methods_of_class
     1403
     1404.. end of output
     1405
     1406::
     1407
     1408    sage: abstract_methods_of_class(QuotientFields().parent_class)['optional']
     1409    []
     1410    sage: abstract_methods_of_class(QuotientFields().parent_class)['required']
     1411    ['__contains__']
     1412
     1413.. end of output
     1414
     1415Hence, the only required method (that is actually required for all parents
     1416that belong to the category of sets) is an element containment test. That's
     1417fine, because the base class :class:`~sage.structure.parent.Parent` provides a
     1418default containment test.
     1419
     1420The elements have to provide more::
     1421
     1422    sage: abstract_methods_of_class(QuotientFields().element_class)['optional']
     1423    ['_add_', '_mul_']
     1424    sage: abstract_methods_of_class(QuotientFields().element_class)['required']
     1425    ['__nonzero__', 'denominator', 'numerator']
     1426
     1427.. end of output
     1428
     1429Hence, the elements must provide ``denominator()`` and ``numerator()``
     1430methods, and must be able to tell whether they are zero or not. The base class
     1431:class:`~sage.structure.element.Element` provides a default ``__nonzero__()``
     1432method. In addition, the elements may provide Sage's single underscore
     1433arithmetic methods (actually any ring element *should* provide them).
     1434
     1435The ``_test_...`` methods
     1436-------------------------
     1437
     1438If a parent or element method's name start with "_test_", it gives rise to a
     1439test in the automatic test suite. For example, it is tested
     1440
     1441- whether a parent `P` actually is an instance of the parent class of the category of `P`,
     1442- whether the user has implemented the required abstract methods,
     1443- whether some defining structural properties (e.g., commutativity) hold.
     1444
     1445For example, if one forgets to implement required methods, one obtains the
     1446following error::
     1447
     1448    sage: class Foo(Parent):
     1449    ...    Element = sage.structure.element.Element
     1450    ...    def __init__(self):
     1451    ...        Parent.__init__(self, category=QuotientFields())
     1452    sage: Bar = Foo()
     1453    sage: bar = Bar.element_class(Bar)
     1454    sage: bar._test_not_implemented_methods()
     1455    Traceback (most recent call last):
     1456    ...
     1457    AssertionError: Not implemented method: denominator
     1458
     1459Here are the tests that form the test suite of quotient fields::
     1460
     1461    sage: [t for t in dir(QuotientFields().parent_class) if t.startswith('_test_')]
     1462    ['_test_additive_associativity',
     1463     '_test_an_element',
     1464     '_test_associativity',
     1465     '_test_characteristic',
     1466     '_test_characteristic_fields',
     1467     '_test_distributivity',
     1468     '_test_elements',
     1469     '_test_elements_eq',
     1470     '_test_one',
     1471     '_test_prod',
     1472     '_test_some_elements',
     1473     '_test_zero']
     1474
     1475.. end of output
     1476
     1477We have implemented all abstract methods (or inherit them from base classes),
     1478we use the category framework, and we have implemented coercions. So, we are
     1479confident that the test suite runs without an error. In fact, it does!
     1480
     1481.. NOTE::
     1482
     1483    The following trick with the ``__main__`` module is only needed in
     1484    doctests, not in an interactive session or when defining the classes
     1485    externally.
     1486
     1487::
     1488
     1489    sage: __main__.MyFrac = MyFrac
     1490    sage: __main__.MyElement = MyElement
     1491    sage: P = MyFrac(ZZ['x'])
     1492    sage: TestSuite(P).run()
     1493
     1494.. end of output
     1495
     1496Let us see what tests are actually performed::
     1497
     1498    sage: TestSuite(P).run(verbose=True)
     1499    running ._test_additive_associativity() . . . pass
     1500    running ._test_an_element() . . . pass
     1501    running ._test_associativity() . . . pass
     1502    running ._test_category() . . . pass
     1503    running ._test_characteristic() . . . pass
     1504    running ._test_characteristic_fields() . . . pass
     1505    running ._test_distributivity() . . . pass
     1506    running ._test_elements() . . .
     1507      Running the test suite of self.an_element()
     1508      running ._test_category() . . . pass
     1509      running ._test_eq() . . . pass
     1510      running ._test_nonzero_equal() . . . pass
     1511      running ._test_not_implemented_methods() . . . pass
     1512      running ._test_pickling() . . . pass
     1513      pass
     1514    running ._test_elements_eq() . . . pass
     1515    running ._test_eq() . . . pass
     1516    running ._test_not_implemented_methods() . . . pass
     1517    running ._test_one() . . . pass
     1518    running ._test_pickling() . . . pass
     1519    running ._test_prod() . . . pass
     1520    running ._test_some_elements() . . . pass
     1521    running ._test_zero() . . . pass
     1522
     1523.. end of output
     1524
     1525Implementing a new category with additional tests
     1526-------------------------------------------------
     1527
     1528As one can see, tests are also performed on elements. There are methods that
     1529return one element or a list of some elements, relying on "typical" elements
     1530that can be found in most algebraic structures.
     1531::
     1532
     1533    sage: P.an_element(); P.some_elements()
     1534    (2):(1)
     1535    [(2):(1)]
     1536
     1537.. end of output
     1538
     1539Unfortunately, the list of elements that is returned by the default method is
     1540of length one, and that single element could also be a bit more interesting.
     1541The method an_element relies on a method ``_an_element_()``, so, we implement
     1542that. We also override the some_elements method.
     1543::
     1544
     1545    sage: class MyFrac(MyFrac):
     1546    ...       def _an_element_(self):
     1547    ...           a = self.base().an_element()
     1548    ...           b = self.base_ring().an_element()
     1549    ...           if (a+b)!=0:
     1550    ...               return self(a)**2/(self(a+b)**3)
     1551    ...           if b != 0:
     1552    ...               return self(a)/self(b)**2
     1553    ...           return self(a)**2*self(b)**3
     1554    ...       def some_elements(self):
     1555    ...           return [self.an_element(),self(self.base().an_element()),self(self.base_ring().an_element())]
     1556
     1557
     1558.. end of output
     1559
     1560::
     1561
     1562    sage: P = MyFrac(ZZ['x'])
     1563    sage: P.an_element(); P.some_elements()
     1564    (x^2):(x^3 + 3*x^2 + 3*x + 1)
     1565    [(x^2):(x^3 + 3*x^2 + 3*x + 1), (x):(1), (1):(1)]
     1566
     1567.. end of output
     1568
     1569Now, as we have more interesting elements, we may also add a test for the
     1570"factor" method. Recall that the method was inherited from the category, but
     1571it appears that it is not tested.
     1572
     1573Normally, a test for a method defined by a category should be provided by the
     1574same category. Hence, since ``factor`` is defined in the category of quotient
     1575fields, a test should be added there. But we won't change source code here and
     1576will instead create a sub\--category.
     1577
     1578Apparently, If `e` is an element of a quotient field, the product of the
     1579factors returned by ``e.factor()`` should be equal to `e`. For forming the
     1580product, we use the ``prod`` method, that, no surprise, is inherited from
     1581another category::
     1582
     1583    sage: P.prod.__module__
     1584    'sage.categories.monoids'
     1585
     1586.. end of output
     1587
     1588When we want to create a sub\--category, we need to provide a method
     1589:meth:`~sage.categories.category.Category.super_categories`, that returns a
     1590list of all immediate super categories (here: category of quotient fields).
     1591
     1592.. WARNING::
     1593
     1594    A sub\--category `S` of a category `C` is *not* implemented as a
     1595    sub\--class of ``C.__class__``! `S` becomes a sub\--category of `C` only
     1596    if ``S.super_categories()`` returns (a sub\--category of) `C`!
     1597
     1598The parent and element methods of a category are provided as methods of
     1599classes that are the attributes ``ParentMethods`` and ``Element Methods`` of
     1600the category, as follows::
     1601
     1602    sage: from sage.categories.category import Category
     1603    sage: class QuotientFieldsWithTest(Category): # do *not* inherit from QuotientFields, but ...
     1604    ...       def super_categories(self):
     1605    ...           return [QuotientFields()]       # ... declare QuotientFields as a super category!
     1606    ...       class ParentMethods:
     1607    ...           pass
     1608    ...       class ElementMethods:
     1609    ...           def _test_factorisation(self, **options):
     1610    ...               P = self.parent()
     1611    ...               assert self == P.prod([P(b)**e for b,e in self.factor()])
     1612
     1613
     1614.. end of output
     1615
     1616We provide an instance of our quotient field implementation with that new
     1617category. Note that categories have a default ``_repr_`` method, that guesses
     1618a good string representation from the name of the class:
     1619``QuotientFieldsWithTest`` becomes "quotient fields with test".
     1620
     1621.. NOTE::
     1622
     1623    The following trick with the ``__main__`` module is only needed in
     1624    doctests, not in an interactive session or when defining the classes
     1625    externally.
     1626
     1627::
     1628
     1629    sage: __main__.MyFrac = MyFrac
     1630    sage: __main__.MyElement = MyElement
     1631    sage: __main__.QuotientFieldsWithTest = QuotientFieldsWithTest
     1632    sage: P = MyFrac(ZZ['x'], category=QuotientFieldsWithTest())
     1633    sage: P.category()
     1634    Category of quotient fields with test
     1635
     1636.. end of output
     1637
     1638The new test is inherited from the category. Since ``an_element()`` is returning a
     1639complicated element, ``_test_factorisation`` is a serious test::
     1640
     1641    sage: P.an_element()._test_factorisation
     1642    <bound method MyFrac_with_category.element_class._test_factorisation of (x^2):(x^3 + 3*x^2 + 3*x + 1)>
     1643
     1644.. end of output
     1645
     1646::
     1647
     1648    sage: P.an_element().factor()
     1649    (x + 1)^-3 * x^2
     1650
     1651.. end of output
     1652
     1653Last, we observe that the new test has automatically become part of the test
     1654suite. We remark that the existing tests became more serious as well, since we
     1655made :meth:`sage.structure.parent.Parent.an_element` return something more
     1656interesting.
     1657::
     1658
     1659    sage: TestSuite(P).run(verbose=True)
     1660    running ._test_additive_associativity() . . . pass
     1661    running ._test_an_element() . . . pass
     1662    running ._test_associativity() . . . pass
     1663    running ._test_category() . . . pass
     1664    running ._test_characteristic() . . . pass
     1665    running ._test_characteristic_fields() . . . pass
     1666    running ._test_distributivity() . . . pass
     1667    running ._test_elements() . . .
     1668      Running the test suite of self.an_element()
     1669      running ._test_category() . . . pass
     1670      running ._test_eq() . . . pass
     1671      running ._test_factorisation() . . . pass
     1672      running ._test_nonzero_equal() . . . pass
     1673      running ._test_not_implemented_methods() . . . pass
     1674      running ._test_pickling() . . . pass
     1675      pass
     1676    running ._test_elements_eq() . . . pass
     1677    running ._test_eq() . . . pass
     1678    running ._test_not_implemented_methods() . . . pass
     1679    running ._test_one() . . . pass
     1680    running ._test_pickling() . . . pass
     1681    running ._test_prod() . . . pass
     1682    running ._test_some_elements() . . . pass
     1683    running ._test_zero() . . . pass
     1684
     1685.. end of output
     1686
     1687Appendix: The complete code
     1688===========================
     1689
     1690.. highlight:: python
     1691   :linenothreshold: 2
     1692
     1693::
     1694
     1695    # Importing base classes, ...
     1696    import sage
     1697    from sage.rings.ring import Field
     1698    from sage.structure.element import FieldElement
     1699    from sage.categories.category import Category
     1700    # ... the UniqueRepresentation tool,
     1701    from sage.structure.unique_representation import UniqueRepresentation
     1702    # ... some categories, and ...
     1703    from sage.categories.fields import Fields
     1704    from sage.categories.quotient_fields import QuotientFields
     1705    from sage.categories.integral_domains import IntegralDomains
     1706    # construction functors
     1707    from sage.categories.pushout import ConstructionFunctor
     1708
     1709    # Fraction field elements
     1710    class MyElement(FieldElement):
     1711        def __init__(self, n,d=None, parent=None):
     1712            if parent is None:
     1713                raise ValueError, "The parent must be provided"
     1714            B = parent.base()
     1715            if d is None:
     1716                # The default denominator is one
     1717                d = B.one_element()
     1718            # verify that both numerator and denominator belong to the base
     1719            if n not in B or d not in B:
     1720                raise ValueError, "Numerator and denominator must be elements of %s"%B
     1721            # Numerator and denominator should not just be "in" B,
     1722            # but should be defined as elements of B
     1723            d = B(d)
     1724            n = B(n)
     1725            # the denominator must not be zero
     1726            if d==0:
     1727                raise ZeroDivisionError, "The denominator must not be zero"
     1728            # normalize the denominator: WLOG, it shall be non-negative.
     1729            if d<0:
     1730                self.n = -n
     1731                self.d = -d
     1732            else:
     1733                self.n = n
     1734                self.d = d
     1735            FieldElement.__init__(self,parent)
     1736
     1737        # Methods required by the category of fraction fields:
     1738        def numerator(self):
     1739            return self.n
     1740        def denominator(self):
     1741            return self.d
     1742
     1743        # String representation (single underscore!)
     1744        def _repr_(self):
     1745            return "(%s):(%s)"%(self.n,self.d)
     1746
     1747        # Comparison: We can assume that both arguments are coerced
     1748        # into the same parent, which is a fraction field. Hence, we
     1749        # are allowed to use the denominator() and numerator() methods
     1750        # on the second argument.
     1751        def __cmp__(self, other):
     1752            return cmp(self.n*other.denominator(), other.numerator()*self.d)
     1753        # Arithmetic methods, single underscore. We can assume that both
     1754        # arguments are coerced into the same parent.
     1755        # We return instances of self.__class__, because self.__class__ will
     1756        # eventually be a sub-class of MyElement.
     1757        def _add_(self, other):
     1758            C = self.__class__
     1759            D = self.d*other.denominator()
     1760            return C(self.n*other.denominator()+self.d*other.numerator(),D, self.parent())
     1761        def _sub_(self, other):
     1762            C = self.__class__
     1763            D = self.d*other.denominator()
     1764            return C(self.n*other.denominator()-self.d*other.numerator(),D, self.parent())
     1765        def _mul_(self, other):
     1766            C = self.__class__
     1767            return C(self.n*other.numerator(), self.d*other.denominator(), self.parent())
     1768        def _div_(self, other):
     1769            C = self.__class__
     1770            return C(self.n*other.denominator(), self.d*other.numerator(), self.parent())
     1771
     1772    # Inheritance from UniqueRepresentation implements the unique parent
     1773    # behaviour. Moreover, it implements pickling (provided that Python
     1774    # succeeds to look up the class definition).
     1775    class MyFrac(UniqueRepresentation, Field):
     1776        # Implement the category framework for elements, which also
     1777        # makes some basic conversions work.
     1778        Element = MyElement
     1779
     1780        # Allow to pass to a different category, by an optional argument
     1781        def __init__(self, base, category=None):
     1782            # Fraction fields only exist for integral domains
     1783            if base not in IntegralDomains():
     1784                raise ValueError, "%s is no integral domain"%base
     1785            # Implement the category framework for the parent
     1786            Field.__init__(self, base, category=category or QuotientFields())
     1787
     1788        # Single-underscore method for string representation
     1789        def _repr_(self):
     1790            return "NewFrac(%s)"%repr(self.base())
     1791
     1792        # Two methods that are implicitly used in some tests
     1793        def base_ring(self):
     1794            return self.base().base_ring()
     1795        def characteristic(self):
     1796            return self.base().characteristic()
     1797
     1798        # Implement conversions. Do not override __call__!
     1799        def _element_constructor_(self, *args,**kwds):
     1800            if len(args)!=1:
     1801               return self.element_class(*args,parent=self,**kwds)
     1802            x = args[0]
     1803            try:
     1804                P = x.parent()
     1805            except AttributeError:
     1806                return self.element_class(x,parent=self,**kwds)
     1807            if P in QuotientFields() and P != self.base():
     1808                return self.element_class(x.numerator(),x.denominator(),parent=self,**kwds)
     1809            return self.element_class(x,parent=self,**kwds)
     1810
     1811        # Implement coercion from the base and from fraction fields
     1812        # over a ring that coerces into the base
     1813        def _coerce_map_from_(self, S):
     1814            if self.base().has_coerce_map_from(S):
     1815                return True
     1816            if S in QuotientFields():
     1817                if self.base().has_coerce_map_from(S.base()):
     1818                    return True
     1819                if hasattr(S,'ring_of_integers') and self.base().has_coerce_map_from(S.ring_of_integers()):
     1820                    return True
     1821        # Tell how this parent was constructed, in order to enable pushout constructions
     1822        def construction(self):
     1823            return MyFracFunctor(), self.base()
     1824
     1825        # return some elements of this parent
     1826        def _an_element_(self):
     1827            a = self.base().an_element()
     1828            b = self.base_ring().an_element()
     1829            if (a+b)!=0:
     1830                return self(a)**2/(self(a+b)**3)
     1831            if b != 0:
     1832                return self(a)/self(b)**2
     1833            return self(a)**2*self(b)**3
     1834        def some_elements(self):
     1835            return [self.an_element(),self(self.base().an_element()),self(self.base_ring().an_element())]
     1836
     1837
     1838    # A construction functor for our implementation of fraction fields
     1839    class MyFracFunctor(ConstructionFunctor):
     1840        # The rank is the same for Sage's original fraction field functor
     1841        rank = 5
     1842        def __init__(self):
     1843            # The fraction field construction is a functor
     1844            # from the category of integral domains into the category of
     1845            # fields
     1846            # NOTE: We could actually narrow the codomain and use the
     1847            # category QuotientFields()
     1848            ConstructionFunctor.__init__(self, IntegralDomains(), Fields())
     1849        # Applying the functor to an object. Do not override __call__!
     1850        def _apply_functor(self, R):
     1851            return MyFrac(R)
     1852        # Note: To apply the functor to morphisms, implement
     1853        #       _apply_functor_to_morphism
     1854
     1855        # Make sure that arithmetic involving elements of Frac(R) and
     1856        # MyFrac(R) works and yields elements of MyFrac(R)
     1857        def merge(self, other):
     1858            if isinstance(other, (type(self), sage.categories.pushout.FractionField)):
     1859                return self
     1860
     1861    # A quotient field category with additional tests.
     1862    # Notes:
     1863    # - Category inherits from UniqueRepresentation. Hence, there
     1864    #   is only one category for given arguments.
     1865    # - Since QuotientFieldsWithTest is a singleton (there is only
     1866    #   one instance of this class), we could inherit from
     1867    #   sage.categories.category_singleton.Category_singleton
     1868    #   rather than from sage.categories.category.Category
     1869    class QuotientFieldsWithTest(Category):
     1870        # Our category is a sub-category of the category of quotient fields,
     1871        # by means of the following method.
     1872        def super_categories(self):
     1873            return [QuotientFields()]
     1874
     1875        # Here, we could implement methods that are available for
     1876        # all objects in this category.
     1877        class ParentMethods:
     1878            pass
     1879
     1880        # Here, we add a new test that is available for all elements
     1881        # of any object in this category.
     1882        class ElementMethods:
     1883            def _test_factorisation(self, **options):
     1884                P = self.parent()
     1885                # The methods prod() and factor() are inherited from
     1886                # some other categories.
     1887                assert self == P.prod([P(b)**e for b,e in self.factor()])
     1888
     1889
     1890.. highlight:: python
     1891   :linenothreshold: 22222
  • doc/en/thematic_tutorials/index.rst

    diff --git a/doc/en/thematic_tutorials/index.rst b/doc/en/thematic_tutorials/index.rst
    a b  
    2121   lie
    2222   linear_programming
    2323   numtheory_rsa
     24   coercion_and_categories
    2425   tutorial-objects-and-classes
    2526
    2627Indices and tables
  • doc/en/thematic_tutorials/tutorial-objects-and-classes.rst

    diff --git a/doc/en/thematic_tutorials/tutorial-objects-and-classes.rst b/doc/en/thematic_tutorials/tutorial-objects-and-classes.rst
    a b  
    11.. _tutorial-objects-and-classes:
    22
    3 ================================================
    4 Tutorial: Objects and Classes in Python and Sage
    5 ================================================
     3======================================
     4Objects and Classes in Python and Sage
     5======================================
    66
    77.. MODULEAUTHOR:: Florent Hivert <florent.hivert@univ-rouen.fr>
    88