Opened 6 years ago

Closed 17 months ago

#14305 closed defect (fixed)

Doctest: Immediate simplifications of symbolic powers

Reported by: zimmerma Owned by: burcin
Priority: critical Milestone: sage-8.1
Component: calculus Keywords:
Cc: mjo, burcin, rws Merged in:
Authors: Ralf Stephan Reviewers: Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: e5d5ba0 (Commits) Commit: e5d5ba0b7b43dd21852984d6f43c28ef334d5890
Dependencies: #23325 Stopgaps:

Description (last modified by kcrisman)

in Sage 5.7 we get:

sage: sqrt(x^2).simplify_radical()           
x

This is wrong (consider x=-1 for example). Even:

sage: assume(x<0)
sage: sqrt(x^2).simplify_radical()
x

Previously it was

sage: sqrt(x^2).simplify_radical()
abs(x)

Note: this invalidates a whole part of our book (in french) about Sage at http://sagebook.gforge.inria.fr/

Change History (57)

comment:1 Changed 6 years ago by kcrisman

  • Description modified (diff)

Regarding these three tickets (#14305, #14306, #14308 (thanks for the pointer, Paul!)) perhaps it would be good to get that book in the standard documentation under fr/ so that it can be part of doctests? Just a brainstorm, perhaps it is not a good idea for various reasons.

Last edited 6 years ago by kcrisman (previous) (diff)

comment:2 follow-up: Changed 6 years ago by kcrisman

This is the problem.

(%i1) radcan(sqrt(x^2));
(%o1)                               abs(x)
(%i2) domain:complex;
(%o2)                               complex
(%i3) radcan(sqrt(x^2));
(%o3)                                  x

Except this was not exposed before, apparently - it is the case in older Sage as well. This is due to #12780.

comment:3 Changed 6 years ago by zimmerma

Karl-Dieter, I guess you mean #14308 instead of #14307?

Yes of course, in fact some chapters are already included, see tests/french_book.

Other chapters are waiting for work or reviewers: #10983, #11745. Feel free to help!

Paul

comment:4 in reply to: ↑ 2 Changed 6 years ago by kcrisman

  • Cc mjo burcin added

This is the problem.

(%i1) radcan(sqrt(x^2));
(%o1)                               abs(x)
(%i2) domain:complex;
(%o2)                               complex
(%i3) radcan(sqrt(x^2));
(%o3)                                  x

By the way, I should point out that Maxima folks probably won't consider this a bug, for the same reasons as discussed in the past on other tickets, and expressed best by Richard Fateman here. But if we don't want this, we'll have to figure out how to deal with this.

Except this was not exposed before, apparently - it is the case in older Sage as well. This is due to #12780.

I've also posted a comment there, hopefully mjo or someone else will respond.

Last edited 6 years ago by kcrisman (previous) (diff)

comment:5 Changed 6 years ago by zimmerma

any news on this ticket?

Paul

comment:6 Changed 6 years ago by mjo

Jeroen asked a similar question a few months ago:

https://groups.google.com/forum/?fromgroups=#!topic/sage-support/jhCJujRtNA4/discussion

Maxima should notice that x is real (if you make the assumption), but it doesn't. The simplification of sqrt(x^2) to abs(x) relies on the "simplification domain" in Maxima being set to real -- in fact, if I remember correctly, that's the only thing the simplification domain does affect. To get the answer from sage <5.6, you'd need,

sage: maxima_lib.eval('domain: real;')
'real'
sage: sqrt(x^2).simplify()            
abs(x)

It sucks that you have to do that, and we should really add either a global method or a property of maxima_lib that lets you change it. That's one solution.

A better approach would be to notice when an expression contains only real variables, and to set the simplification domain appropriately. This could be done in either Maxima or Sage (pynac/ginac), but probably belongs in both. Symbols in sage have an associated domain, too:

sage: x = SR.symbol('x', domain='real')
sage: sqrt(x^2)
sqrt(x^2)

This never passes through Maxima, so arguably, Sage should simplify it if we can convince Maxima to do the same thing. But it's not at all easy to determine whether an expression is real; otherwise, I would have written it a long time ago! I mainly work over the reals myself and would love to have a solution that doesn't involve manually fooling with Maxima.

comment:7 follow-up: Changed 6 years ago by zimmerma

before #12780, the default domain was 'real', and sqrt(x^2).simplify_radical() was correctly simplified to abs(x).

After #12780, the default domain is that of Maxima (which is 'complex' in Sage 5.9), and sqrt(x^2).simplify_radical() is incorrectly simplified to x.

I believe the authors of #12780 should fix that issue.

Paul

comment:8 in reply to: ↑ 7 ; follow-up: Changed 6 years ago by mjo

Replying to zimmerma:

before #12780, the default domain was 'real', and sqrt(x^2).simplify_radical() was correctly simplified to abs(x).


This isn't true. Before #12780, the domain everywhere in Sage except the simplify_radical function was 'complex'. The simplify_radical function would secretly change the domain to 'real', call radcan(), and then switch the domain back to 'complex'.

There is no simplification of sqrt(x^2) that is equal to abs(x), unless you know that x is real. Everywhere else in Sage, x is complex. Before sage-5.7, you got what you want, but you want a wrong answer.

I realize there's no other nice way to get the answer you want (that sucks, I want it too, it's not my fault, etc.), but that's not a good justification for adding back wrong behavior.

comment:9 in reply to: ↑ 8 Changed 6 years ago by nbruin

Replying to mjo:

There is no simplification of sqrt(x^2) that is equal to abs(x), unless you know that x is real. Everywhere else in Sage, x is complex. Before sage-5.7, you got what you want, but you want a wrong answer.

I'm not sure you can dispose of the problem just by declaring it invalid. We do accept assume(x<0) and with that assumption in place the answer we get back really IS wrong (since the assumption implies that x is real). I know we're not going to fix/improve maxima's assumption system any time soon, but there are cases where this bug is going to be a "won't fix" rather than an "invalid".

comment:10 follow-up: Changed 6 years ago by mjo

We also accept assume('x is a very pretty boy'), so I'm not sure we should use that as our guide!

I'm not disagreeing that there should be an easy way to do this. Maxima should just simplify it when x is assumed real. That's probably the first bug I would target to get it fixed properly.

A second step would be to have ginac or pynac perform the same simplification on expressions of the simple form sqrt(x^2) where x is a single symbol and the domain of x is 'real'.

Finally (this is the most work), Maxima could be made to recognize when an expression y is real, and then, simplify() could turn sqrt(y^2) into abs(y). Determining whether an expression is real or not seems rather complicated, though.

comment:11 Changed 6 years ago by mjo

Please take a look at #14630 which provides an easy way to get this simplification.

comment:12 follow-up: Changed 6 years ago by kcrisman

Thanks, that looks like a good step to take. Am I correct in saying it doesn't fix the point Nils makes about assume(x<0) (that is, when one doesn't use #14630, since the typical user would think it wasn't relevant...)?

comment:13 in reply to: ↑ 12 Changed 6 years ago by mjo

Replying to kcrisman:

Thanks, that looks like a good step to take. Am I correct in saying it doesn't fix the point Nils makes about assume(x<0) (that is, when one doesn't use #14630, since the typical user would think it wasn't relevant...)?

Correct, #14630 just provides a way to get the simplification in the description, as an alternative to setting the Maxima domain manually.

comment:14 Changed 6 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:15 Changed 6 years ago by zimmerma

while trying to see how we could tell Maxima that x is real after assume(x>0), I noticed that the examples with assume(x,'real') in Sage are fake:

1) in misc/functional.py we have:

        sage: a, b, c = var("a, b, c")                                          
        sage: assume((a, 'real'), (b, 'real'), (c, 'real'))                     
        sage: z = a + b*I                                                       
        sage: bool(norm(z).simplify() == a^2 + b^2)                             
        True

but the same works without assume:

+--------------------------------------------------------------------+
| Sage Version 5.11, Release Date: 2013-08-13                        |
| Type "notebook()" for the browser-based notebook interface.        |
| Type "help()" for help.                                            |
+--------------------------------------------------------------------+
sage: a, b, c = var("a, b, c")
sage: z = a + b*I
sage: norm(z).simplify()
a^2 + b^2

2) in symbolic/expression.pyx we have

            sage: a,b = var('a,b')
            sage: assume((a, 'real'), (b, 'real'))
            sage: f = a + b*I
            sage: f.rectform()
            a + I*b

and again it works without assume:

+--------------------------------------------------------------------+
| Sage Version 5.11, Release Date: 2013-08-13                        |
| Type "notebook()" for the browser-based notebook interface.        |
| Type "help()" for help.                                            |
+--------------------------------------------------------------------+
sage: a,b = var('a,b')
sage: f = a + b*I
sage: f.rectform()
a + I*b

Paul

comment:16 Changed 6 years ago by kcrisman

Interesting. Maybe that is a function of having upgraded Maxima or something? I presume that this wasn't the case at some time, but presumptions can be presumptuous.

comment:17 Changed 6 years ago by zimmerma

maybe simplify and rectform secretly change the domain of variables to the real domain (as simplify_radical previously did)?

Paul

comment:18 Changed 6 years ago by kcrisman

No, that is not it - simplify certainly doesn't, and rectform is just

return self.maxima_methods().rectform()

Sorry, that would have been easier to deal with. I particularly don't like the rectform example working without assuming real.

comment:19 Changed 6 years ago by mjo

I remember the first example, from #12845. This is the full test:

sage: a, b, c = var("a, b, c")
sage: assume((a, 'real'), (b, 'real'), (c, 'real'))
sage: z = a + b*I
sage: bool(norm(z).simplify() == a^2 + b^2)
True
sage: norm(a + b).simplify()
a^2 + 2*a*b + b^2
sage: v = vector([a, b, c])
sage: bool(norm(v).simplify() == sqrt(a^2 + b^2 + c^2))
True
sage: forget()

The assumptions aren't needed for the first result, but they are for the last one (the norm of the vector).

The rectform() test is also mine, but the reasoning I've forgotten.

comment:20 follow-up: Changed 6 years ago by zimmerma

The assumptions aren't needed for the first result, but they are for the last one (the norm of the vector).

maybe then the first test could be removed, if unrelated to assume? Or at least the assume call moved after the first test?

Anyway the following is very annoying:

sage: (x*conjugate(x)).simplify()
x^2

Paul

comment:21 in reply to: ↑ 20 Changed 6 years ago by kcrisman

Anyway the following is very annoying:

sage: (x*conjugate(x)).simplify()
x^2

Wow, agreed. Unfortunately, Maxima needs to declare this.

sage: assume(x,'complex')
sage: (x*conjugate(x)).simplify()
x*conjugate(x)

I guess a possible workaround would be to have all SR variables declared complex in Maxima when first starting Maxima, and then thereafter... but it would be very hackish and slow.

comment:22 Changed 6 years ago by kcrisman

Turns out that it's possible to get all variables used so far in Maxima with values, and one could do a blanket declare on all of them. (Of course, one might not want to do this to all of them...) Fateman also has the following enigmatic response to me on the Maxima list.

I think the understanding of complex variables is at best spotty, and if you want to do "a LOT" of
things with them you better check out exactly what Maxima does, and perhaps even  make it work
with names that have no special properties.

Anyway, I really don't want to go back to assuming things in Maxima on every variable defined in Sage (or even every SR variable) but I think we might have to go that way. Maybe there is a flag we could set as an underscore attribute in the SR vars indicating whether it has been declared complex in Maxima yet, which would be dealt with appropriately for ones which aren't complex, or have been assumed real, etc. - i.e., the next time one went into Maxima, they would be in a queue that was then declared complex. Not sure how hawkish that would be, but hopefully at least not slowing down Pynac variables appreciably.

comment:23 Changed 6 years ago by zimmerma

a possible workaround would be to have all SR variables declared complex in Maxima

indeed, maybe we could declare each new variable as complex whenever var is called (without domain argument or with domain='complex'). This is by the way what the documentation of var says:

sage: var?
...
   By default, var returns a complex variable. To define real or
   positive variables we can specify the domain as:

Maybe we can add a call to assume in the function symbol of symbolic/ring.pyx?

Paul

comment:24 Changed 6 years ago by kcrisman

Unfortunately, that would

  • start up Maxima the first time, which takes quite a while
  • use Maxima each and every time we create a symbolic variable, which would take some time

I don't think people would be pleased with the slowdown this would cause. That's why I'm wondering whether there might be some other workaround along these lines but less draconian. Variables are complex by default, just not when we send them to Maxima.

comment:25 Changed 6 years ago by zimmerma

then we could record the list of created variables (with corresponding domains), and each time a simplify function from Maxima would be called, we could make the corresponding assumptions in Maxima, along the lines of what you suggest in comment 22.

Paul

comment:26 follow-up: Changed 6 years ago by burcin

The problem with the Maxima interface mixing up variable domains is #6862. This could be fixed with #8734 by changing the Maxima conversion (in the libmaxima interface) to work directly off the pynac expression tree (instead of going through strings / parsers) and set the corresponding flag in the temporary variables it creates. I'm not sure if anybody has time to implement this though.

comment:27 in reply to: ↑ 26 Changed 6 years ago by nbruin

Replying to burcin:

The problem with the Maxima interface mixing up variable domains is #6862. This could be fixed with #8734 by changing the Maxima conversion (in the libmaxima interface) to work directly off the pynac expression tree (instead of going through strings / parsers)

And that is largely available already via the functions sage.interfaces.maxima_lib.sr_to_max and sage.interfaces.maxima_lib.max_to_sr, which already get used for, for instance, limits and integrals. Currently it asks the strings-based translator (once) for how to convert, but the information is all there to convert variables more intelligently.

Of course, a formula translation routine can't go and purposely change the global state of maxima (you wouldn't know when to convert back!), but it could store required global settings somewhere else, so that they can be applied when the actual functionality is called upon and undone afterwards.

Implementing this will indeed be a tedious and thankless job. It would feel a lot more worthwhile if first almost all of calculus were moved to using sr_to_max. Then is gets easier to start letting maxima_lib have different conversion semantics than maxima.

comment:28 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:29 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:30 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:31 Changed 4 years ago by kcrisman

  • Cc rws added
  • Summary changed from bug in simplify_radical to Clarify assumptions and domains in Maxima

Changing the description, as this isn't really about simplify_radical/its successor any more. In fact, I'm not really sure what this one is about, but I feel like #14630 and #6862 don't cover everything mentioned here.

comment:32 in reply to: ↑ 10 Changed 21 months ago by rws

  • Dependencies set to pynac-0.7.9

Replying to mjo:

A second step would be to have ginac or pynac perform the same simplification on expressions of the simple form sqrt(x^2) where x is a single symbol and the domain of x is 'real'.

I believe this is fixed in Pynac master: https://github.com/pynac/pynac/commit/0a3979a4ed8bdb3ed1be4c6ee5d328a9b2c690b8

sage: sqrt(x^2)
sqrt(x^2)
sage: x = SR.symbol('x', domain='real')
sage: sqrt(x^2)
abs(x)
sage: forget()
sage: assume(x<0)
sage: sqrt(x^2)
-x
sage: sqrt(x^4)
x^2
sage: forget()
sage: x = SR.symbol('x', domain='real')
sage: sqrt(x^4)
x^2
sage: sqrt(sin(x)^2)
abs(sin(x))
sage: sqrt((x+1)^2)
abs(x + 1)
sage: (x^3)^(1/3)
(x^3)^(1/3)
sage: (x^4)^(1/4)
abs(x)
sage: (x^8)^(1/4)
x^2
sage: (x^-4)^(1/4)
1/abs(x)
sage: (x^-8)^(1/4)
x^(-2)
sage: forget()
sage: assume(x<0)
sage: sqrt((x-1)^2)
-x + 1

This is immediate expansion. You can see it's not restricted to the inner exponent 2, and it also works on arbitrary expressions not just symbols.

The ticket should add the above (or any additions) as doctests.

comment:33 Changed 21 months ago by rws

  • Dependencies changed from pynac-0.7.9 to #23325

comment:34 Changed 19 months ago by rws

  • Branch set to u/rws/clarify_assumptions_and_domains_in_maxima

comment:35 Changed 19 months ago by rws

  • Authors set to Ralf Stephan
  • Commit set to 6125bd6f28f36c29752c3bf7cfba0c206652ec65
  • Milestone changed from sage-6.4 to sage-8.1
  • Status changed from new to needs_review
  • Summary changed from Clarify assumptions and domains in Maxima to Doctest: Immediate simplifications of symbolic powers

New commits:

6125bd614305: Doctest: Immediate simplifications of symbolic powers

comment:36 Changed 19 months ago by zimmerma

a small typo: Immediate simplification are applied should be: Immediate simplifications are applied.

comment:37 Changed 19 months ago by git

  • Commit changed from 6125bd6f28f36c29752c3bf7cfba0c206652ec65 to e5d5ba0b7b43dd21852984d6f43c28ef334d5890

Branch pushed to git repo; I updated commit sha1. New commits:

e5d5ba014305: fix typo

comment:38 Changed 17 months ago by jdemeyer

  • Reviewers set to Jeroen Demeyer
  • Status changed from needs_review to positive_review

comment:39 Changed 17 months ago by rws

Thanks.

comment:40 Changed 17 months ago by jdemeyer

I wonder why (x^3)^(1/3) is not simplified for real x though...

comment:41 Changed 17 months ago by zimmerma

try x=-1...

comment:42 Changed 17 months ago by rws

sage: var(x, domain='real')
x
sage: (x^3)^(1/3)
(x^3)^(1/3)
sage: assume(x > 0)
sage: (x^3)^(1/3)
x

comment:43 Changed 17 months ago by jdemeyer

I don't get this. Why should it matter whether x < 0 or x > 0?

Why is it valid to say that 1^(1/3) = 1 but not (-1)^(1/3) = -1 for real numbers?

comment:45 follow-up: Changed 17 months ago by zimmerma

z1/n for z a complex number is defined as the principal branch, which is that having the least argument, see for example https://en.wikipedia.org/wiki/Nth_root.

Thus the square root of -1 is I, having argument pi/2, whereas -I has argument 3*pi/2.

The cube root of -1 is 1/2 + I*sqrt(3)/2, having argument pi/3.

comment:46 in reply to: ↑ 45 Changed 17 months ago by rws

Replying to zimmerma:

The cube root of -1 is 1/2 + I*sqrt(3)/2, having argument pi/3.

One or three cube roots of -1 exist?

comment:47 follow-ups: Changed 17 months ago by zimmerma

there are three complex numbers t such that t^3=z, but when you write z^(1/3), the computer algebra system must choose one of them, for example to evaluate numerically, and the usual choice is the principal branch.

I recommend reading the paper "Branch Cuts in Computer Algebra" by Adam Dingle and Richard Fateman.

comment:48 in reply to: ↑ 47 Changed 17 months ago by rws

Replying to zimmerma:

there are three complex numbers t such that t^3=z, but when you write z^(1/3), the computer algebra system must choose one of them, for example to evaluate numerically, and the usual choice is the principal branch.

I recommend reading the paper "Branch Cuts in Computer Algebra" by Adam Dingle and Richard Fateman.

I'm not evaluating numerically, t^3=z does not per se represent a function, just an expression. If your argument were true, I could not even solve for -1.

comment:49 follow-ups: Changed 17 months ago by rws

I mean why do I even argument, Jeroen is a better math, so I trust him when he objects.

comment:50 Changed 17 months ago by rws

Paul, maybe the problem is that you transfer the fact that RR is in CC to that "expressions with real variables" behave like "expressions with complex variables"?

comment:51 in reply to: ↑ 49 Changed 17 months ago by jdemeyer

Replying to rws:

Jeroen is a better math, so I trust him when he objects.

Not sure why you say that. I understand Paul's reasoning. It's hard to tell who is right. If only variables have a domain (as opposed to constants), then (-1)^(1/3) must indeed evaluate to a complex number because we can't guess whether the -1 represents the real or complex -1.

comment:52 in reply to: ↑ 47 Changed 17 months ago by rws

A complex function is not necessarily a complex-variable function, that's maybe the mistake I made.

comment:53 Changed 17 months ago by zimmerma

indeed we cannot know in advance whether -1 represents a real or complex number. However, another point is that the "simplication rule" should be consistent between the real and complex numbers: when a complex number z tends to the real number -1, you want that z^(1/3) tends to (-1)^(1/3), except maybe on a line (called the branch cut) going out from 0. The principal branch gives this consistency.

comment:54 Changed 17 months ago by mjo

This is messy largely because we have never decided on what sqrt or e.g. x^(1/3) is supposed to do, particularly when applied to a symbolic expression.

  • Is it a function from the real numbers to the real numbers (choose the positive root)?
  • Is it a function from the complex numbers to the complex numbers (choose the principal branch)?
  • Is the codomain of sqrt the same as a "domain" of its argument? For example, if x is a variable with domain=real, should sqrt(x) be an expression whose domain is "real"?
  • Do we want sqrt and friends to be functions at all? When solving equations, it'd be nice if we had access to both square roots.

etc.

If you ask five different people, you'll get five different answers. I'm wary of any "simplification" that might return an incorrect result to any of those people: simplification should be unambiguous and obviously correct. Any other more-powerful but less-straightforward operation should be called something else, like we've done with x.canonicalize_radical().

comment:55 in reply to: ↑ 49 Changed 17 months ago by rws

Replying to rws:

I mean why do I even argument, Jeroen is a better math, so I trust him when he objects.

I just reread that and yes I forgot to add "than me", opening unintended interpretations.

comment:56 Changed 17 months ago by kcrisman

If you ask five different people, you'll get five different answers. I'm wary of any "simplification" that might return an incorrect result to any of those people: simplification should be unambiguous and obviously correct. Any other more-powerful but less-straightforward operation should be called something else, like we've done with x.canonicalize_radical().

Correct, this whole thread is exactly the same thing. Basically, is sqrt(x) a symbol or a function? If you want long (and undoubtedly correct in at least one interpretation) answers to this, ask Fateman :-) with whom we had long discussions about it.

comment:57 Changed 17 months ago by vbraun

  • Branch changed from u/rws/clarify_assumptions_and_domains_in_maxima to e5d5ba0b7b43dd21852984d6f43c28ef334d5890
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.