Opened 8 years ago
Closed 4 years ago
#14305 closed defect (fixed)
Doctest: Immediate simplifications of symbolic powers
Reported by:  zimmerma  Owned by:  burcin 

Priority:  critical  Milestone:  sage8.1 
Component:  calculus  Keywords:  
Cc:  mjo, burcin, rws  Merged in:  
Authors:  Ralf Stephan  Reviewers:  Jeroen Demeyer 
Report Upstream:  N/A  Work issues:  
Branch:  e5d5ba0 (Commits, GitHub, GitLab)  Commit:  e5d5ba0b7b43dd21852984d6f43c28ef334d5890 
Dependencies:  #23325  Stopgaps: 
Description (last modified by )
in Sage 5.7 we get:
sage: sqrt(x^2).simplify_radical() x
This is wrong (consider x=1 for example). Even:
sage: assume(x<0) sage: sqrt(x^2).simplify_radical() x
Previously it was
sage: sqrt(x^2).simplify_radical() abs(x)
Note: this invalidates a whole part of our book (in french) about Sage at http://sagebook.gforge.inria.fr/
Change History (57)
comment:1 Changed 8 years ago by
 Description modified (diff)
comment:2 followup: ↓ 4 Changed 8 years ago by
This is the problem.
(%i1) radcan(sqrt(x^2)); (%o1) abs(x) (%i2) domain:complex; (%o2) complex (%i3) radcan(sqrt(x^2)); (%o3) x
Except this was not exposed before, apparently  it is the case in older Sage as well. This is due to #12780.
comment:3 Changed 8 years ago by
comment:4 in reply to: ↑ 2 Changed 8 years ago by
 Cc mjo burcin added
This is the problem.
(%i1) radcan(sqrt(x^2)); (%o1) abs(x) (%i2) domain:complex; (%o2) complex (%i3) radcan(sqrt(x^2)); (%o3) x
By the way, I should point out that Maxima folks probably won't consider this a bug, for the same reasons as discussed in the past on other tickets, and expressed best by Richard Fateman here. But if we don't want this, we'll have to figure out how to deal with this.
Except this was not exposed before, apparently  it is the case in older Sage as well. This is due to #12780.
I've also posted a comment there, hopefully mjo or someone else will respond.
comment:5 Changed 8 years ago by
any news on this ticket?
Paul
comment:6 Changed 8 years ago by
Jeroen asked a similar question a few months ago:
https://groups.google.com/forum/?fromgroups=#!topic/sagesupport/jhCJujRtNA4/discussion
Maxima should notice that x
is real (if you make the assumption), but it doesn't. The simplification of sqrt(x^2)
to abs(x)
relies on the "simplification domain" in Maxima being set to real
 in fact, if I remember correctly, that's the only thing the simplification domain does affect. To get the answer from sage <5.6, you'd need,
sage: maxima_lib.eval('domain: real;') 'real' sage: sqrt(x^2).simplify() abs(x)
It sucks that you have to do that, and we should really add either a global method or a property of maxima_lib
that lets you change it. That's one solution.
A better approach would be to notice when an expression contains only real variables, and to set the simplification domain appropriately. This could be done in either Maxima or Sage (pynac/ginac), but probably belongs in both. Symbols in sage have an associated domain, too:
sage: x = SR.symbol('x', domain='real') sage: sqrt(x^2) sqrt(x^2)
This never passes through Maxima, so arguably, Sage should simplify it if we can convince Maxima to do the same thing. But it's not at all easy to determine whether an expression is real; otherwise, I would have written it a long time ago! I mainly work over the reals myself and would love to have a solution that doesn't involve manually fooling with Maxima.
comment:7 followup: ↓ 8 Changed 8 years ago by
before #12780, the default domain was 'real', and sqrt(x^2).simplify_radical()
was correctly simplified to abs(x)
.
After #12780, the default domain is that of Maxima (which is 'complex' in Sage 5.9), and sqrt(x^2).simplify_radical()
is incorrectly simplified to x
.
I believe the authors of #12780 should fix that issue.
Paul
comment:8 in reply to: ↑ 7 ; followup: ↓ 9 Changed 8 years ago by
Replying to zimmerma:
before #12780, the default domain was 'real', and
sqrt(x^2).simplify_radical()
was correctly simplified toabs(x)
.
This isn't true. Before #12780, the domain everywhere in Sage except the simplify_radical
function was 'complex'. The simplify_radical
function would secretly change the domain to 'real', call radcan()
, and then switch the domain back to 'complex'.
There is no simplification of sqrt(x^2)
that is equal to abs(x)
, unless you know that x
is real. Everywhere else in Sage, x
is complex. Before sage5.7, you got what you want, but you want a wrong answer.
I realize there's no other nice way to get the answer you want (that sucks, I want it too, it's not my fault, etc.), but that's not a good justification for adding back wrong behavior.
comment:9 in reply to: ↑ 8 Changed 8 years ago by
Replying to mjo:
There is no simplification of
sqrt(x^2)
that is equal toabs(x)
, unless you know thatx
is real. Everywhere else in Sage,x
is complex. Before sage5.7, you got what you want, but you want a wrong answer.
I'm not sure you can dispose of the problem just by declaring it invalid. We do accept assume(x<0)
and with that assumption in place the answer we get back really IS wrong (since the assumption implies that x
is real). I know we're not going to fix/improve maxima's assumption system any time soon, but there are cases where this bug is going to be a "won't fix" rather than an "invalid".
comment:10 followup: ↓ 32 Changed 8 years ago by
We also accept assume('x is a very pretty boy')
, so I'm not sure we should use that as our guide!
I'm not disagreeing that there should be an easy way to do this. Maxima should just simplify it when x
is assumed real. That's probably the first bug I would target to get it fixed properly.
A second step would be to have ginac or pynac perform the same simplification on expressions of the simple form sqrt(x^2)
where x
is a single symbol and the domain of x
is 'real'.
Finally (this is the most work), Maxima could be made to recognize when an expression y
is real, and then, simplify()
could turn sqrt(y^2)
into abs(y)
. Determining whether an expression is real or not seems rather complicated, though.
comment:11 Changed 8 years ago by
Please take a look at #14630 which provides an easy way to get this simplification.
comment:12 followup: ↓ 13 Changed 8 years ago by
Thanks, that looks like a good step to take. Am I correct in saying it doesn't fix the point Nils makes about assume(x<0)
(that is, when one doesn't use #14630, since the typical user would think it wasn't relevant...)?
comment:13 in reply to: ↑ 12 Changed 8 years ago by
Replying to kcrisman:
Thanks, that looks like a good step to take. Am I correct in saying it doesn't fix the point Nils makes about
assume(x<0)
(that is, when one doesn't use #14630, since the typical user would think it wasn't relevant...)?
Correct, #14630 just provides a way to get the simplification in the description, as an alternative to setting the Maxima domain manually.
comment:14 Changed 8 years ago by
 Milestone changed from sage5.11 to sage5.12
comment:15 Changed 8 years ago by
while trying to see how we could tell Maxima that x
is real after assume(x>0)
, I noticed that the examples with assume(x,'real')
in Sage are fake:
1) in misc/functional.py
we have:
sage: a, b, c = var("a, b, c") sage: assume((a, 'real'), (b, 'real'), (c, 'real')) sage: z = a + b*I sage: bool(norm(z).simplify() == a^2 + b^2) True
but the same works without assume
:
++  Sage Version 5.11, Release Date: 20130813   Type "notebook()" for the browserbased notebook interface.   Type "help()" for help.  ++ sage: a, b, c = var("a, b, c") sage: z = a + b*I sage: norm(z).simplify() a^2 + b^2
2) in symbolic/expression.pyx
we have
sage: a,b = var('a,b') sage: assume((a, 'real'), (b, 'real')) sage: f = a + b*I sage: f.rectform() a + I*b
and again it works without assume
:
++  Sage Version 5.11, Release Date: 20130813   Type "notebook()" for the browserbased notebook interface.   Type "help()" for help.  ++ sage: a,b = var('a,b') sage: f = a + b*I sage: f.rectform() a + I*b
Paul
comment:16 Changed 8 years ago by
Interesting. Maybe that is a function of having upgraded Maxima or something? I presume that this wasn't the case at some time, but presumptions can be presumptuous.
comment:17 Changed 8 years ago by
maybe simplify
and rectform
secretly change the domain of variables to the real domain (as simplify_radical
previously did)?
Paul
comment:18 Changed 8 years ago by
No, that is not it  simplify
certainly doesn't, and rectform
is just
return self.maxima_methods().rectform()
Sorry, that would have been easier to deal with. I particularly don't like the rectform
example working without assuming real.
comment:19 Changed 8 years ago by
I remember the first example, from #12845. This is the full test:
sage: a, b, c = var("a, b, c") sage: assume((a, 'real'), (b, 'real'), (c, 'real')) sage: z = a + b*I sage: bool(norm(z).simplify() == a^2 + b^2) True sage: norm(a + b).simplify() a^2 + 2*a*b + b^2 sage: v = vector([a, b, c]) sage: bool(norm(v).simplify() == sqrt(a^2 + b^2 + c^2)) True sage: forget()
The assumptions aren't needed for the first result, but they are for the last one (the norm of the vector).
The rectform()
test is also mine, but the reasoning I've forgotten.
comment:20 followup: ↓ 21 Changed 8 years ago by
The assumptions aren't needed for the first result, but they are for the last one (the norm of the vector).
maybe then the first test could be removed, if unrelated to assume
?
Or at least the assume
call moved after the first test?
Anyway the following is very annoying:
sage: (x*conjugate(x)).simplify() x^2
Paul
comment:21 in reply to: ↑ 20 Changed 8 years ago by
Anyway the following is very annoying:
sage: (x*conjugate(x)).simplify() x^2
Wow, agreed. Unfortunately, Maxima needs to declare this.
sage: assume(x,'complex') sage: (x*conjugate(x)).simplify() x*conjugate(x)
I guess a possible workaround would be to have all SR variables declared complex
in Maxima when first starting Maxima, and then thereafter... but it would be very hackish and slow.
comment:22 Changed 8 years ago by
Turns out that it's possible to get all variables used so far in Maxima with values
, and one could do a blanket declare
on all of them. (Of course, one might not want to do this to all of them...) Fateman also has the following enigmatic response to me on the Maxima list.
I think the understanding of complex variables is at best spotty, and if you want to do "a LOT" of things with them you better check out exactly what Maxima does, and perhaps even make it work with names that have no special properties.
Anyway, I really don't want to go back to assuming things in Maxima on every variable defined in Sage (or even every SR variable) but I think we might have to go that way. Maybe there is a flag we could set as an underscore attribute in the SR vars indicating whether it has been declared complex in Maxima yet, which would be dealt with appropriately for ones which aren't complex, or have been assumed real, etc.  i.e., the next time one went into Maxima, they would be in a queue that was then declared complex. Not sure how hawkish that would be, but hopefully at least not slowing down Pynac variables appreciably.
comment:23 Changed 8 years ago by
a possible workaround would be to have all SR variables declared complex in Maxima
indeed, maybe we could declare each new variable as complex whenever var
is called (without domain
argument or with domain='complex'
).
This is by the way what the documentation of var
says:
sage: var? ... By default, var returns a complex variable. To define real or positive variables we can specify the domain as:
Maybe we can add a call to assume
in the function symbol
of symbolic/ring.pyx
?
Paul
comment:24 Changed 8 years ago by
Unfortunately, that would
 start up Maxima the first time, which takes quite a while
 use Maxima each and every time we create a symbolic variable, which would take some time
I don't think people would be pleased with the slowdown this would cause. That's why I'm wondering whether there might be some other workaround along these lines but less draconian. Variables are complex by default, just not when we send them to Maxima.
comment:25 Changed 8 years ago by
then we could record the list of created variables (with corresponding domains), and each time a simplify
function from Maxima would be called, we could make the corresponding assumptions in Maxima, along the lines of what you suggest in comment 22.
Paul
comment:26 followup: ↓ 27 Changed 8 years ago by
The problem with the Maxima interface mixing up variable domains is #6862. This could be fixed with #8734 by changing the Maxima conversion (in the libmaxima interface) to work directly off the pynac expression tree (instead of going through strings / parsers) and set the corresponding flag in the temporary variables it creates. I'm not sure if anybody has time to implement this though.
comment:27 in reply to: ↑ 26 Changed 8 years ago by
Replying to burcin:
The problem with the Maxima interface mixing up variable domains is #6862. This could be fixed with #8734 by changing the Maxima conversion (in the libmaxima interface) to work directly off the pynac expression tree (instead of going through strings / parsers)
And that is largely available already via the functions sage.interfaces.maxima_lib.sr_to_max
and sage.interfaces.maxima_lib.max_to_sr
, which already get used for, for instance, limits and integrals. Currently it asks the stringsbased translator (once) for how to convert, but the information is all there to convert variables more intelligently.
Of course, a formula translation routine can't go and purposely change the global state of maxima (you wouldn't know when to convert back!), but it could store required global settings somewhere else, so that they can be applied when the actual functionality is called upon and undone afterwards.
Implementing this will indeed be a tedious and thankless job. It would feel a lot more worthwhile if first almost all of calculus were moved to using sr_to_max
. Then is gets easier to start letting maxima_lib have different conversion semantics than maxima.
comment:28 Changed 7 years ago by
 Milestone changed from sage6.1 to sage6.2
comment:29 Changed 7 years ago by
 Milestone changed from sage6.2 to sage6.3
comment:30 Changed 7 years ago by
 Milestone changed from sage6.3 to sage6.4
comment:31 Changed 7 years ago by
 Cc rws added
 Summary changed from bug in simplify_radical to Clarify assumptions and domains in Maxima
comment:32 in reply to: ↑ 10 Changed 4 years ago by
 Dependencies set to pynac0.7.9
Replying to mjo:
A second step would be to have ginac or pynac perform the same simplification on expressions of the simple form
sqrt(x^2)
wherex
is a single symbol and the domain ofx
is 'real'.
I believe this is fixed in Pynac master: https://github.com/pynac/pynac/commit/0a3979a4ed8bdb3ed1be4c6ee5d328a9b2c690b8
sage: sqrt(x^2) sqrt(x^2) sage: x = SR.symbol('x', domain='real') sage: sqrt(x^2) abs(x) sage: forget() sage: assume(x<0) sage: sqrt(x^2) x sage: sqrt(x^4) x^2 sage: forget() sage: x = SR.symbol('x', domain='real') sage: sqrt(x^4) x^2 sage: sqrt(sin(x)^2) abs(sin(x)) sage: sqrt((x+1)^2) abs(x + 1) sage: (x^3)^(1/3) (x^3)^(1/3) sage: (x^4)^(1/4) abs(x) sage: (x^8)^(1/4) x^2 sage: (x^4)^(1/4) 1/abs(x) sage: (x^8)^(1/4) x^(2) sage: forget() sage: assume(x<0) sage: sqrt((x1)^2) x + 1
This is immediate expansion. You can see it's not restricted to the inner exponent 2
, and it also works on arbitrary expressions not just symbols.
The ticket should add the above (or any additions) as doctests.
comment:33 Changed 4 years ago by
 Dependencies changed from pynac0.7.9 to #23325
comment:34 Changed 4 years ago by
 Branch set to u/rws/clarify_assumptions_and_domains_in_maxima
comment:35 Changed 4 years ago by
 Commit set to 6125bd6f28f36c29752c3bf7cfba0c206652ec65
 Milestone changed from sage6.4 to sage8.1
 Status changed from new to needs_review
 Summary changed from Clarify assumptions and domains in Maxima to Doctest: Immediate simplifications of symbolic powers
New commits:
6125bd6  14305: Doctest: Immediate simplifications of symbolic powers

comment:36 Changed 4 years ago by
a small typo:
Immediate simplification are applied
should be:
Immediate simplifications are applied
.
comment:37 Changed 4 years ago by
 Commit changed from 6125bd6f28f36c29752c3bf7cfba0c206652ec65 to e5d5ba0b7b43dd21852984d6f43c28ef334d5890
Branch pushed to git repo; I updated commit sha1. New commits:
e5d5ba0  14305: fix typo

comment:38 Changed 4 years ago by
 Reviewers set to Jeroen Demeyer
 Status changed from needs_review to positive_review
comment:39 Changed 4 years ago by
Thanks.
comment:40 Changed 4 years ago by
I wonder why (x^3)^(1/3)
is not simplified for real x
though...
comment:41 Changed 4 years ago by
try x=1
...
comment:42 Changed 4 years ago by
sage: var(x, domain='real') x sage: (x^3)^(1/3) (x^3)^(1/3) sage: assume(x > 0) sage: (x^3)^(1/3) x
comment:43 Changed 4 years ago by
I don't get this. Why should it matter whether x < 0
or x > 0
?
Why is it valid to say that 1^(1/3) = 1
but not (1)^(1/3) = 1
for real numbers?
comment:44 Changed 4 years ago by
comment:45 followup: ↓ 46 Changed 4 years ago by
z^{1/n} for z a complex number is defined as the principal branch, which is that having the least argument, see for example https://en.wikipedia.org/wiki/Nth_root.
Thus the square root of 1 is I, having argument pi/2, whereas I has argument 3*pi/2.
The cube root of 1 is 1/2 + I*sqrt(3)/2, having argument pi/3.
comment:46 in reply to: ↑ 45 Changed 4 years ago by
Replying to zimmerma:
The cube root of 1 is 1/2 + I*sqrt(3)/2, having argument pi/3.
One or three cube roots of 1 exist?
comment:47 followups: ↓ 48 ↓ 52 Changed 4 years ago by
there are three complex numbers t
such that t^3=z
, but when you write z^(1/3)
, the computer algebra system must choose one of them, for example to evaluate numerically, and the usual choice is the principal branch.
I recommend reading the paper "Branch Cuts in Computer Algebra" by Adam Dingle and Richard Fateman.
comment:48 in reply to: ↑ 47 Changed 4 years ago by
Replying to zimmerma:
there are three complex numbers
t
such thatt^3=z
, but when you writez^(1/3)
, the computer algebra system must choose one of them, for example to evaluate numerically, and the usual choice is the principal branch.I recommend reading the paper "Branch Cuts in Computer Algebra" by Adam Dingle and Richard Fateman.
I'm not evaluating numerically, t^3=z
does not per se represent a function, just an expression. If your argument were true, I could not even solve for 1.
comment:49 followups: ↓ 51 ↓ 55 Changed 4 years ago by
I mean why do I even argument, Jeroen is a better math, so I trust him when he objects.
comment:50 Changed 4 years ago by
Paul, maybe the problem is that you transfer the fact that RR is in CC to that "expressions with real variables" behave like "expressions with complex variables"?
comment:51 in reply to: ↑ 49 Changed 4 years ago by
Replying to rws:
Jeroen is a better math, so I trust him when he objects.
Not sure why you say that. I understand Paul's reasoning. It's hard to tell who is right. If only variables have a domain (as opposed to constants), then (1)^(1/3)
must indeed evaluate to a complex number because we can't guess whether the 1
represents the real or complex 1
.
comment:52 in reply to: ↑ 47 Changed 4 years ago by
A complex function is not necessarily a complexvariable function, that's maybe the mistake I made.
comment:53 Changed 4 years ago by
indeed we cannot know in advance whether 1
represents a real or complex number.
However, another point is that the "simplication rule" should be consistent between the real and complex numbers: when a complex number z
tends to the real number 1
, you want that z^(1/3)
tends to (1)^(1/3)
, except maybe on a line (called the branch cut) going out from 0. The principal branch gives this consistency.
comment:54 Changed 4 years ago by
This is messy largely because we have never decided on what sqrt
or e.g. x^(1/3)
is supposed to do, particularly when applied to a symbolic expression.
 Is it a function from the real numbers to the real numbers (choose the positive root)?
 Is it a function from the complex numbers to the complex numbers (choose the principal branch)?
 Is the codomain of
sqrt
the same as a "domain" of its argument? For example, ifx
is a variable withdomain=real
, shouldsqrt(x)
be an expression whose domain is"real"
?  Do we want
sqrt
and friends to be functions at all? When solving equations, it'd be nice if we had access to both square roots.
etc.
If you ask five different people, you'll get five different answers. I'm wary of any "simplification" that might return an incorrect result to any of those people: simplification should be unambiguous and obviously correct. Any other morepowerful but lessstraightforward operation should be called something else, like we've done with x.canonicalize_radical()
.
comment:55 in reply to: ↑ 49 Changed 4 years ago by
Replying to rws:
I mean why do I even argument, Jeroen is a better math, so I trust him when he objects.
I just reread that and yes I forgot to add "than me", opening unintended interpretations.
comment:56 Changed 4 years ago by
If you ask five different people, you'll get five different answers. I'm wary of any "simplification" that might return an incorrect result to any of those people: simplification should be unambiguous and obviously correct. Any other morepowerful but lessstraightforward operation should be called something else, like we've done with x.canonicalize_radical().
Correct, this whole thread is exactly the same thing. Basically, is sqrt(x)
a symbol or a function? If you want long (and undoubtedly correct in at least one interpretation) answers to this, ask Fateman :) with whom we had long discussions about it.
comment:57 Changed 4 years ago by
 Branch changed from u/rws/clarify_assumptions_and_domains_in_maxima to e5d5ba0b7b43dd21852984d6f43c28ef334d5890
 Resolution set to fixed
 Status changed from positive_review to closed
Regarding these three tickets (#14305, #14306, #14308 (thanks for the pointer, Paul!)) perhaps it would be good to get that book in the standard documentation under fr/ so that it can be part of doctests? Just a brainstorm, perhaps it is not a good idea for various reasons.