Forgetful Functor not working properly

[gh-mjungmath]

We have currently the following bug:

sage: from sage.categories.functor import ForgetfulFunctor
sage: F = ForgetfulFunctor(Rings(), Sets())
sage: F(ZZ)
Integer Ring

The result should rather be:

sage: Set(ZZ)
Set of elements of Integer Ring

[gh-mjungmath]

Digging into the source code, the forgetful functor F calls the following code from its class parent Functor:

    def _apply_functor(self, x):
        """
        Apply the functor to an object of ``self``'s domain.

        NOTE:

        Each subclass of :class:`Functor` should overload this method. By default,
        this method coerces into the codomain, without checking whether the
        argument belongs to the domain.

        TESTS::

            sage: from sage.categories.functor import Functor
            sage: F = Functor(FiniteFields(),Fields())
            sage: F._apply_functor(ZZ)
            Rational Field

        """
        return self.__codomain(x)

Here, __codomain should be the destination category. However, the call function of a category is treated as some kind of coercion/conversion:

    def __call__(self, x, *args, **opts):
        """
        Construct an object in this category from the data in ``x``,
        or throw ``TypeError`` or ``NotImplementedError``.

        If ``x`` is readily in ``self`` it is returned unchanged.
        Categories wishing to extend this minimal behavior should
        implement :meth:`._call_`.

        EXAMPLES::

            sage: Rings()(ZZ)
            Integer Ring
        """
        if x in self:
            return x
        return self._call_(x, *args, **opts)

as the following note in sage/categories/sets_cat.py indicates, too:

    def _call_(self, X, enumerated_set=False):
        r"""
        Construct an object in this category from the data ``X``.

        INPUT:

        - ``X`` -- an object to be converted into a set

        - ``enumerated_set`` -- if set to ``True`` and the input is either a
          Python tuple or a Python list then the output will be a finite
          enumerated set.

        EXAMPLES::

            sage: Sets()(ZZ)
            Integer Ring
            sage: Sets()([1, 2, 3])
            {1, 2, 3}

            sage: S = Sets()([1, 2, 3]); S.category()
            Category of finite sets
            sage: S = Sets()([1, 2, 3], enumerated_set=True); S.category()
            Category of facade finite enumerated sets

        .. NOTE::

           Using ``Sets()(A)`` used to implement some sort of forgetful functor
           into the ``Sets()`` category. This feature has been removed, because
           it was not consistent with the semantic of :meth:`Category.__call__`.
           Proper forgetful functors will eventually be implemented, with
           another syntax.
        """
        if enumerated_set and type(X) in (tuple, list, range):
            from sage.categories.enumerated_sets import EnumeratedSets
            return EnumeratedSets()(X)
        from sage.sets.set import Set
        return Set(X)

Proper forgetful functors will eventually be implemented, with another syntax. It seems, this hasn't been done so far.

[gh-mjungmath]

One could try the following: overload _apply_functor for forgetful functors and call _call_ directly. That would be a minimal solution, but I am not sure whether it is the most elegant.

[tscrim]

I don't think this is a bug. The code is performing as it should because ZZ is already in Sets(). It doesn't need to create a new object, much less an instance of a different class. I think if you want to change your object, you should call the explicit constructor rather than relying on a functor. Moreover, if it is not going to be the current behavior, then I think you're asking for a solution to an impossible problem.

[gh-mjungmath]

Replying to tscrim:

I don't think this is a bug. The code is performing as it should because ZZ is already in Sets(). It doesn't need to create a new object, much less an instance of a different class.

Then, the same would hold for manifolds in #31241. Differentiable manifolds are also in the category of topological manifolds (or at least should be). But we already agreed that it's a bug there. What is the difference then?

I think if you want to change your object, you should call the explicit constructor rather than relying on a functor. Moreover, if it is not going to be the current behavior, then I think you're asking for a solution to an impossible problem.

Well, the forgetful functor has the purpose to forget imposed structures, and see the object as an object in the super category only. This condition is currently not met:

sage: from sage.categories.functor import ForgetfulFunctor 
....: F = ForgetfulFunctor(Rings(), Sets()) 
....: F(ZZ).category()                                                    
Join of Category of euclidean domains and Category of infinite enumerated sets and Category of metric spaces
sage: F(ZZ)(1) + F(ZZ)(2)                                                       
3

Usual set operations are not even compatible:

sage: Set([1,2,3]).union(F(ZZ))                                                    
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
...
TypeError: X (=Integer Ring) must be a Set

Of course, different structures need different implementations.

For manifolds, the current stage of behavior is rather bad. Differentiable manifolds can be seen as topological manifolds, however each differentiable manifold comes with a differentiable structure which Sage keeps implicitly track of. This must be dropped after the forgetful functor had been applied. But with the above behavior, this is not possible because the original instance would be returned.

[mkoeppe]

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

Another example:

sage: R.<x> = QQ[]
sage: phi = R.hom([x^2]); phi
Ring endomorphism of Univariate Polynomial Ring in x over Rational Field
  Defn: x |--> x^2
sage: phi.category_for()
Join of Category of euclidean domains and Category of commutative algebras over (number fields and quotient fields and metric spaces) and Category of infinite sets
sage: F = ForgetfulFunctor(Rings(), Sets())
sage: F(phi)
Ring endomorphism of Univariate Polynomial Ring in x over Rational Field
  Defn: x |--> x^2

I hoped to compute the correct category of an image in #32121 (Replace MapCombinatorialClass, add methods Map.image, Map.pushforward)...

[tscrim]

This in and of itself is not a bug IMO as per comment:4. However, what is a bug IMO is that

sage: F(phi).parent()
Set of Homomorphisms from Univariate Polynomial Ring in x over Rational Field
 to Univariate Polynomial Ring in x over Rational Field

which leads to this

sage: F(phi).category_for()
Join of Category of euclidean domains
 and Category of commutative algebras over (number fields and quotient fields and metric spaces)
 and Category of infinite sets

For morphisms, where the category is part of the homset constructor, we can create the new homset parent and then convert the morphism into that parent. Mostly this just consists of rebuilding the same object but with a different parent. This is meaningful when we want to compose maps (as opposed to manipulating sets).