Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

#12737 closed enhancement (fixed)

Remove simplify_radical() from simplify_full()

Reported by: mjo Owned by: mjo
Priority: major Milestone: sage-5.12
Component: symbolics Keywords:
Cc: jvkersch, navigium Merged in: sage-5.12.beta0
Authors: Michael Orlitzky Reviewers: Karl-Dieter Crisman, Beni Keller
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by kcrisman)

There are a number of tickets open due to the use of simplify_radical() in simplify_full().

Removing it would fix at least,

And maybe more. Here are some related tickets.

  • #11912 is related, though about simplify_radical specifically.
  • #11668 - abs(a+b)^2 == (a+b)^2 is now fixed by #12780
  • #14305 and #14630 are tangentially related

Apply sage-trac_12737-rebased.patch and trac_12737-de.patch

Attachments (4)

sage-trac_12737.patch (6.3 KB) - added by mjo 6 years ago.
Add the flag and update doctests.
sage-trac_12737-remove_radcan.patch (5.4 KB) - added by mjo 6 years ago.
Same patch with the functional.py doctest factored out
sage-trac_12737-rebased.patch (6.3 KB) - added by mjo 4 years ago.
Rebased patch against 5.11.beta3
trac_12737-de.patch (1.8 KB) - added by kcrisman 4 years ago.

Download all attachments as: .zip

Change History (36)

Changed 6 years ago by mjo

Add the flag and update doctests.

comment:1 Changed 6 years ago by mjo

  • Status changed from new to needs_review

comment:2 follow-up: Changed 6 years ago by nbruin

It's not entirely clear to me which simplifications are "safe" and which are not. Simplifying x^2/x to x is also not "safe" in the sense that equality does not hold for x=0. In many computer algebra systems, acceptable simplifications are not "safe" in this respect. By including a default unsafe=False I'm afraid you'll be raising expectations to a level that Sage does not attain (yet?).

comment:3 in reply to: ↑ 2 ; follow-up: Changed 6 years ago by mjo

Replying to nbruin:

It's not entirely clear to me which simplifications are "safe" and which are not. Simplifying x^2/x to x is also not "safe" in the sense that equality does not hold for x=0. In many computer algebra systems, acceptable simplifications are not "safe" in this respect. By including a default unsafe=False I'm afraid you'll be raising expectations to a level that Sage does not attain (yet?).

It's completely heuristic: the four I chose nobody seems to have a problem with. simplify_radical, on the other hand, wreaks havoc on trivial functions and has been the cause of numerous bug reports and mailing list discussions.

In simplify, after #12650, I say that a safe simplification is one "for which we are reasonably sure that the input will be considered equivalent to the output." That's probably the best we can do for now, and I'm reasonably sure that most people would consider x^2/x equivalent to x.

As it stands, I think full_simplify sounds safe so people will assume that anyway. This fix, while not perfect, at least improves things.

comment:4 in reply to: ↑ 3 ; follow-up: Changed 6 years ago by kcrisman

It's not entirely clear to me which simplifications are "safe" and which are not. Simplifying x^2/x to x is also not "safe" in the sense that equality does not hold for x=0. In many computer algebra systems, acceptable simplifications are not "safe" in this respect. By including a default unsafe=False I'm afraid you'll be raising expectations to a level that Sage does not attain (yet?).

It's completely heuristic: the four I chose nobody seems to have a problem with.

Only in that we couldn't find Trac tickets about them.

simplify_radical, on the other hand, wreaks havoc on trivial functions and has been the cause of numerous bug reports and mailing list discussions.

In simplify, after #12650, I say that a safe simplification is one "for which we are reasonably sure that the input will be considered equivalent to the output." That's probably the best we can do for now, and I'm reasonably sure that most people would consider x^2/x equivalent to x.

I'm reasonably sure that no mathematician would consider them equivalent unless you add "almost everywhere". Simplification simplifies, hence makes it not actually the same (at least potentially). This shouldn't be controversial.

As it stands, I think full_simplify sounds safe so people will assume that anyway. This fix, while not perfect, at least improves things.

Well, everything sounds safe unless you put in the word "unsafe".

I'm pretty confused as to the relation of this to #12650. See my comments there.

I'm sympathetic to doing something about radcan, but simplification is simplification, not identity.

Doesn't it make more sense to either document fully what can go wrong with simplify_full, or to add a flag simplify_radicals or something? I think that changing behavior in this case is probably the best compromise, but probably unsafe is sending the wrong message. There isn't anything unsafe about radcan, it's just a different point of view than ours - symbolic expressions versus functions. Spend some time on the Maxima list :)

comment:5 in reply to: ↑ 4 Changed 6 years ago by mjo

Replying to kcrisman:

It's not entirely clear to me which simplifications are "safe" and which are not. Simplifying x^2/x to x is also not "safe" in the sense that equality does not hold for x=0. In many computer algebra systems, acceptable simplifications are not "safe" in this respect. By including a default unsafe=False I'm afraid you'll be raising expectations to a level that Sage does not attain (yet?).

It's completely heuristic: the four I chose nobody seems to have a problem with.

Only in that we couldn't find Trac tickets about them.

simplify_radical, on the other hand, wreaks havoc on trivial functions and has been the cause of numerous bug reports and mailing list discussions.

In simplify, after #12650, I say that a safe simplification is one "for which we are reasonably sure that the input will be considered equivalent to the output." That's probably the best we can do for now, and I'm reasonably sure that most people would consider x^2/x equivalent to x.

I'm reasonably sure that no mathematician would consider them equivalent unless you add "almost everywhere". Simplification simplifies, hence makes it not actually the same (at least potentially). This shouldn't be controversial.

Before our discussion on another one of these tickets, I had assumed it was not controversial that simplification had to return an equivalent expression. So it is at least a little controversial.

As it stands, I think full_simplify sounds safe so people will assume that anyway. This fix, while not perfect, at least improves things.

Well, everything sounds safe unless you put in the word "unsafe".

I'm pretty confused as to the relation of this to #12650. See my comments there.

I'm sympathetic to doing something about radcan, but simplification is simplification, not identity.

Doesn't it make more sense to either document fully what can go wrong with simplify_full, or to add a flag simplify_radicals or something? I think that changing behavior in this case is probably the best compromise, but probably unsafe is sending the wrong message. There isn't anything unsafe about radcan, it's just a different point of view than ours - symbolic expressions versus functions. Spend some time on the Maxima list :)

All expressions in sage are callable like functions, so if you have both radicals and a variable in an expression, simplify_radical() is unsafe. I.e. you're probably gonna get the wrong answer. I think marking it as unsafe is more likely to catch someone's attention than asking him to read the documentation. I almost certainly used full_simplify() for years without doing so, since I thought I knew what it was doing and had used similar functions in e.g. Mathematica. I think the problem is that it's called "simplification," not that it exists.

Either way, f.full_simplify(simplify_radicals=True) is not much better than f.full_simplify().simplify_radical()... the keyword argument is the easy way out, but I feel like it should control more than one method call if we do it. And if we remove the unsafe simplifications from full_simplify(), that seems to make simplify() redundant, too. That's the dual situation to what we'd have after #12650, but at least we could add more stuff to full_simplify() in the future to change that.

comment:6 Changed 6 years ago by mjo

  • Status changed from needs_review to needs_info

There is not consensus that this is even a good idea, so I'll leave it alone until we can discuss it further.

comment:7 Changed 6 years ago by mjo

  • Authors set to Michael Orlitzky
  • Dependencies #12650 deleted
  • Description modified (diff)
  • Status changed from needs_info to needs_review
  • Summary changed from Add an `unsafe` argument to Expression.simplify_full() to Remove simplify_radical() from simplify_full()

This should be a more straight-forward proposal. I just removed simplify_radical() from simplify_full() and fixed a few doctests (details in the commit message).

One of the doctest fixes is duplicated in #12780; the doctest is wrong (missing assumptions) but we're masking that fact at the moment.

Changed 6 years ago by mjo

Same patch with the functional.py doctest factored out

comment:8 Changed 6 years ago by mjo

  • Dependencies set to #12845

I moved the common doctest to #12845 and removed it from this patch.

comment:9 Changed 5 years ago by kcrisman

  • Description modified (diff)

Lest it seem weird that I'm adding to the list in the description, let it be known that I haven't necessarily changed my mind here, but fairness dictates that I point this additional example out!

comment:10 Changed 5 years ago by kcrisman

  • Description modified (diff)

comment:11 Changed 5 years ago by kcrisman

  • Description modified (diff)

comment:12 Changed 5 years ago by mjo

  • Description modified (diff)

comment:13 follow-up: Changed 5 years ago by kcrisman

Burcin agrees here at SD that this should be done. I concur with him that probably we should solve #11912 (perhaps even deprecating the simplify name?) at the same time. What do you think?

comment:14 in reply to: ↑ 13 ; follow-up: Changed 5 years ago by mjo

Replying to kcrisman:

Burcin agrees here at SD that this should be done. I concur with him that probably we should solve #11912 (perhaps even deprecating the simplify name?) at the same time. What do you think?

Once it's gone from simplify_full(), personally I would deprecate simplify_radical() and rename it to radcan(). It's probably impossible to clearly explain what radcan does, but we can try to fix the docs and add many more examples.

comment:15 in reply to: ↑ 14 Changed 5 years ago by kcrisman

Burcin agrees here at SD that this should be done. I concur with him that probably we should solve #11912 (perhaps even deprecating the simplify name?) at the same time. What do you think?

Once it's gone from simplify_full(), personally I would deprecate simplify_radical() and rename it to radcan(). It's probably impossible to clearly explain what radcan does, but we can try to fix the docs and add many more examples.

Do you mind putting that on #11912, as the architect of this set of changes?

comment:16 follow-up: Changed 4 years ago by jvkersch

Here's another example of unexpected behavior introduced by using simplify_radical as a part of simplify_full: https://groups.google.com/forum/?fromgroups=#!topic/sage-devel/Pq9hyOTzrL8.

comment:17 Changed 4 years ago by jvkersch

  • Cc jvkersch added

comment:18 Changed 4 years ago by mjo

I think there's consensus to remove it now, but this will need rebasing. There are also a ton of related tickets which will all need to be coordinated so that the patches apply cleanly.

As soon as I get time I'll start by updating this patch.

Changed 4 years ago by mjo

Rebased patch against 5.11.beta3

comment:19 Changed 4 years ago by mjo

I just put up a new patch that should apply cleanly. There are some failing tests in doc/de/thematische_anleitungen/sage_gymnasium.rst, but I don't know what the surrounding context is and so I'm not sure how best to fix them. Will need some attention from the Germans.

comment:20 in reply to: ↑ 16 ; follow-up: Changed 4 years ago by kcrisman

Here's another example of unexpected behavior introduced by using simplify_radical as a part of simplify_full: https://groups.google.com/forum/?fromgroups=#!topic/sage-devel/Pq9hyOTzrL8.

In that same thread: Robert D. of Maxima reports

>   sage: u, v = var('u, v', domain='real') 
>   sage: sqrt(-1/(u^2+v^2-1)).simplify_radical()  # This will hang 

This is a bug in Maxima: 

(%i2) radcan (sqrt (-1 / (u^2 + v^2 - 1))), domain=complex; 

It waits apparently forever here -- but it is actually looping over 
*ALPHA (not sure why) which is a large prime number (8388593). 

(%i3) :lisp (setq *alpha 17) 
17 
(%i3) radcan(sqrt(-1/(u^2+v^2-1))), domain=complex; 
(%o3) 1/sqrt(-v^2-u^2+1) 

I've reported this as bug # 2605. 

E.g. Maxima 2605. We may want to open a ticket for this or something, maybe along with #11912.


I'll take a look at the failures in the new German tutorial at some point today, it's probably not a big deal.

comment:21 in reply to: ↑ 20 Changed 4 years ago by kcrisman

  • Cc navigium added
  • Dependencies #12845 deleted
  • Reviewers set to Karl-Dieter Crisman
  • Status changed from needs_review to needs_work

I'll take a look at the failures in the new German tutorial at some point today, it's probably not a big deal.

sage -t doc/de/thematische_anleitungen/sage_gymnasium.rst
**********************************************************************
File "doc/de/thematische_anleitungen/sage_gymnasium.rst", line 397, in doc.de.thematische_anleitungen.sage_gymnasium
Failed example:
    (log(8)/log(2)).simplify_full()
Expected:
    3
Got:
    log(8)/log(2)
**********************************************************************
File "doc/de/thematische_anleitungen/sage_gymnasium.rst", line 703, in doc.de.thematische_anleitungen.sage_gymnasium
Failed example:
    log(10^5).simplify_full()
Expected:
    5*log(5) + 5*log(2)
Got:
    log(100000)
**********************************************************************

Well, unfortunately the context is that the authors of the tutorial really really like using just one simplification command. In particular, they advertise simplify_full as providing log addition rules "for free". Which I don't really see as "simplification in the latter example! (Which is some of what you've been saying earlier.) However, it does mean it would be interesting to see whether it might be worth adding canonicalize_exp or canonicalize_log at #11912 as aliases...

Anyway, I can fix this, but I don't have time right now, because I would have to think about how best to preserve the authors' intentions. I assume it would be annoying to all of a sudden point out that there are multiple "simplification" options. I've cc:ed the author of #14550 to see if he has any suggestions.

@navigium - man könnte behaupten, dass log(10^5) = > 5*log(5) + 5*log(2) nicht eine Vereinfachung, sondern eine ... "Gezetzeseinsetzung" ist. Im jeden Fall sieht es nicht einfacher aus! Dennoch wollen wir die Einleitung nicht einfach so ohne weiteres zerlegen. Vorausgesetzt, dass simplify_radical von simplify_full verschwindet, wie wäre es am Besten (d. Hinsicht nach) simplify_radical im Gymnasialkontext zu erwähnen?

comment:22 follow-up: Changed 4 years ago by navigium

I'm at the moment unsure how to fix this. I guess the first example that fails could just be removed since it can be numerically evaluated. But I think the second example should remain in the tutorial and should be fixed. At the moment I don't have the time to look into it. I won't be around till mid August.

@kcrisman Ich habe es vor allem als Vereinfachung bezeichnet, da es die notwendige Umformung ist, welche man für das Lösen von Exponentialgleichungen benötigt. Aus diesem Grund halte ich es auch für wichtig, dass man erwähnt, wie man eine solche Umformung machen kann.

Man könnte es aber natürlich umbennen. Ich würde es als "Zerlegung durch Benutzen der Logarithmengesetze" oder ähnlich nennen.

Natürlich wäre es schon, wenn man simplify_radical nicht separat aufführen muss, um die Sache einfach zu halten. Wenn es aber aus simplify_full fliegt, müsste man es wohl erwähnen.

comment:23 in reply to: ↑ 22 Changed 4 years ago by kcrisman

@kcrisman Ich habe es vor allem als Vereinfachung bezeichnet, da es die notwendige Umformung ist, welche man für das Lösen von Exponentialgleichungen benötigt. Aus diesem Grund halte ich es auch für wichtig, dass man erwähnt, wie man eine solche Umformung machen kann.

Man könnte es aber natürlich umbennen. Ich würde es als "Zerlegung durch Benutzen der Logarithmengesetze" oder ähnlich nennen.

Eigentlich genügt das!

Natürlich wäre es schon, wenn man simplify_radical nicht separat aufführen muss, um die Sache einfach zu halten. Wenn es aber aus simplify_full fliegt, müsste man es wohl erwähnen.

Tja...

Michael, I'll try to keep this on the front burner for finding a good fix - navigium has a good phrase we can use, and I don't really see any other way to fix this, since it does happen to be very convenient that we can use radcan in some cases to do log expansion.

comment:24 Changed 4 years ago by mjo

It is regrettable that there's no other way to get log simplification of things like log(8)/log(2), but simplify_radical() will do the same thing even when the argument to the log function is a complex function of, say, z.

It'd be nice to have something that worked for constants at least, but avoided e.g. #12322.

comment:25 Changed 4 years ago by kcrisman

  • Description modified (diff)
  • Reviewers changed from Karl-Dieter Crisman to Karl-Dieter Crisman, Beni Keller
  • Status changed from needs_work to needs_review

I'd appreciate a quick review from navigium as to whether my German syntax is 100% correct, but this should take care of it, and conforms to what he suggested as a solution.

Patchbot, apply sage-trac_12737-rebased.patch and trac_12737-de.patch

Changed 4 years ago by kcrisman

comment:26 Changed 4 years ago by kcrisman

Patchbot, apply sage-trac_12737-rebased.patch and trac_12737-de.patch

comment:27 Changed 4 years ago by navigium

I think that's a great way to solve my document. Now the readers even get something extra because they are taught how to apply log rules both ways. Thanks @kcrisman for solving it.

comment:28 Changed 4 years ago by kcrisman

Great. Now someone (other than me) just needs to check that it passes tests and they can set it to positive review.

comment:29 Changed 4 years ago by jvkersch

  • Status changed from needs_review to positive_review

make ptestlong does not report any errors so I'm setting this to positive review.

comment:30 Changed 4 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:31 Changed 4 years ago by jdemeyer

  • Merged in set to sage-5.12.beta0
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:32 Changed 4 years ago by kcrisman

  • Description modified (diff)

Updates:

Removed

  • #3520 - inconsistency in simplify_radical

from description, as that was not about simplify_full to start with.

I posted updates on the ask.sagemath questions just now.

#11934 needs review.

#12322 needs to be reworded since we didn't use the "unsafe" parameter in simplify_full.

Note: See TracTickets for help on using tickets.