Opened 7 years ago

Closed 4 years ago

#15731 closed defect (duplicate)

Too early coercion causes weird behavior of comparison

Reported by: strenner Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: coercion Keywords:
Cc: nbruin, jakobkroeker Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

Let's say I want to create my own number objects and I want be able to compare my objects with regular number types.

class SpecialNumber():
    def __init__(self, number):
        self.number = number
    def __lt__(self, other):
        return self.number < other
    def __gt__(self, other):
        return self.number > other

When comparing an int and a SpecialNumber?, and SpecialNumber? is on the left, everything works well, because SpecialNumber?'s comparison operators are called. However, when the SpecialNumber? is on the right, it fail, i.e.:

sage: a = SpecialNumber(3)
sage: a > 2
True
sage: 2 < a
False

The exact same code in pure python would return True, because there is no __lt__ method to call on an int that works, so instead a > 2 is tried which returns True.

Whereas Sage, after not being able to call the __lt__ method of the Integer object, tries coercion right away without trying to evaluate a > 2. Since there is no coercion, it checks if the types of the two objects are the same or something like that, and since not, it returns false. I think the right behavior would be check first if a > 2 can be evaluated, and try to coerce only after that.

Change History (11)

comment:1 Changed 7 years ago by strenner

  • Priority changed from minor to major

comment:2 follow-up: Changed 7 years ago by nbruin

How about a<2, 2<a then? Do you want the same there? That would give an infinite recursion.

In python, the meaning of comparison is in the hands of the left hand side. In general, you shouldn't expect any consistency when applying comparisons between unrelated objects. If you want to cooperate with sage objects, you'll probably have to support coercion.

comment:3 in reply to: ↑ 2 ; follow-up: Changed 7 years ago by strenner

Replying to nbruin:

How about a<2, 2<a then? Do you want the same there? That would give an infinite recursion.

I am hoping that there exists an implementation that avoids infinite recursion.

In python, the meaning of comparison is in the hands of the left hand side.

This is not entirely true. From http://stackoverflow.com/questions/878943/why-return-notimplemented-instead-of-raising-notimplementederror (the same thing works for other comparison operators):

"NotImplemented signals to the runtime that it should ask someone else to satisfy the operation. In the expression a == b, if a.__eq__(b) returns NotImplemented, then Python tries b.__eq__(a). If b knows enough to return True or False, then the expression can succeed. If it doesn't, then the runtime will fall back to the built-in behavior (which is based on identity for == and !=)."

For instance, if Sage could override this built-in behavior and try coercion instead, that would save us from infinite recursion.

comment:4 in reply to: ↑ 3 Changed 7 years ago by nbruin

Replying to strenner:

This is not entirely true. From http://stackoverflow.com/questions/878943/why-return-notimplemented-instead-of-raising-notimplementederror (the same thing works for other comparison operators):

I'm not sure it's entirely true; see http://docs.python.org/2/reference/datamodel.html#object.__lt__ (the official documentation tends to be a little more authoritative than stackoverflow, and in the case of python, usually very clear too). Indeed, these methods can return to NotImplemented, but I think that leads python to try different variants, and never "swapped" (rather, it will try __ge__ after __lt__ has failed). The final showstopper is probably the older __cmp__ protocol that predates python's "rich" comparisons and is actually what is used most throughout sage. I don't see a "notImplemented" escape documented for that. Normal python semantics seem to never swap LHS and RHS for comparison operators, contrasting what python specifies for + with __add__ and __radd__

For instance, if Sage could override this built-in behavior and try coercion instead, that would save us from infinite recursion.

comment:5 follow-up: Changed 7 years ago by strenner

Well, I am not sure how it works either, but I did experiments in a python interpreter (with the SpecialNumber being defined as the original description), and I got this:

>>> a = SpecialNumber(3)
>>> 4 < a
False
>>> 2 < a
True

The only way I can explain this behavior is the SpecialNumber.__gt__ is called at some point.

comment:6 in reply to: ↑ 5 Changed 7 years ago by nbruin

Replying to strenner:

The only way I can explain this behavior is the SpecialNumber.__gt__ is called at some point.

Which you can confirm by inserting a print message in the relevant __gt__. It indeed seems that if, in evaluation of a<b, it happens that a.__lt__(b) returns NotImplemented, then the next thing tried is b.__gt__(a). On second reading (or perhaps just correct reading) this is as documented:

""" There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, lt() and gt() are each other’s reflection, le() and ge() are each other’s reflection, and eq() and ne() are their own reflection. """

So indeed, if sage were a little more liberal in returning NotImplemented your approach could work.

The problem is that parts of sage are quite intent on having all objects comparable, meaning that the sage implementations try quite hard to return True or False rather than NotImplemented. So changing this will be pretty hard.

comment:7 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:8 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:9 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:10 Changed 4 years ago by jakobkroeker

  • Cc nbruin jakobkroeker added

the sage implementations try quite hard to return True or False

IMHO this behaviour will pretty easy break most of the math... Thus for me it is worth to open a blocker ticket for that issue

comment:11 Changed 4 years ago by jdemeyer

  • Milestone changed from sage-6.4 to sage-duplicate/invalid/wontfix
  • Resolution set to duplicate
  • Status changed from new to closed

Duplicate of #21163.

Note: See TracTickets for help on using tickets.