Opened 12 years ago

Closed 7 years ago

#3426 closed defect (duplicate)

bessel_K function is broken

Reported by: bober Owned by: gfurnish
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: calculus Keywords: bessel, bessel_K
Cc: burcin, kcrisman, benjaminfjones Merged in:
Authors: Reviewers: Karl-Dieter Crisman, Benjamin Jones
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description

Currently we have

sage: bessel_K(10 * I, 10)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

/home/bober/sage-3.0.2/devel/sage-bober/sage/functions/<ipython console> in <module>()

/home/bober/sage/local/lib/python2.5/site-packages/sage/functions/special.py in bessel_K(nu, z, algorithm, prec)
    586         from sage.libs.pari.all import pari
    587         RR,a = _setup(prec)
--> 588         b = RR(pari(nu).besselk(z))
    589         pari.set_real_precision(a)
    590         return b

/home/bober/sage-3.0.2/devel/sage-bober/sage/functions/real_mpfr.pyx in sage.rings.real_mpfr.RealField.__call__ (sage/rings/real_mpfr.c:3138)()

/home/bober/sage-3.0.2/devel/sage-bober/sage/functions/real_mpfr.pyx in sage.rings.real_mpfr.RealNumber._set (sage/rings/real_mpfr.c:5905)()

TypeError: Unable to convert x (='0.000000098241574381992468+0.E-161*I') to real number.
sage: bessel_K(10 * I, 10)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

/home/bober/sage-3.0.2/devel/sage-bober/sage/functions/<ipython console> in <module>()

/home/bober/sage/local/lib/python2.5/site-packages/sage/functions/special.py in bessel_K(nu, z, algorithm, prec)
    586         from sage.libs.pari.all import pari
    587         RR,a = _setup(prec)
--> 588         b = RR(pari(nu).besselk(z))
    589         pari.set_real_precision(a)
    590         return b

/home/bober/sage-3.0.2/devel/sage-bober/sage/functions/real_mpfr.pyx in sage.rings.real_mpfr.RealField.__call__ (sage/rings/real_mpfr.c:3138)()

/home/bober/sage-3.0.2/devel/sage-bober/sage/functions/real_mpfr.pyx in sage.rings.real_mpfr.RealNumber._set (sage/rings/real_mpfr.c:5905)()

TypeError: Unable to convert x (='0.000000098241574381992468+0.E-161*I') to real number.

In this case the result actually should be a real number, so we fix this by discarding the imaginary part of the result from pari. In other cases, however, the result is actually a complex number, and we shouldn't always be attempting to cast it to a real number (which the attached patch also fixes).

Attachments (2)

kbessel_fixes.patch (2.1 KB) - added by bober 12 years ago.
trac3426-fix-bessel-fns.patch (6.6 KB) - added by AlexGhitza 11 years ago.
apply instead of the previous patch

Download all attachments as: .zip

Change History (46)

Changed 12 years ago by bober

comment:1 Changed 12 years ago by mabshoff

  • Milestone changed from sage-3.0.3 to sage-3.0.4
  • Summary changed from bessel_K function is broken (with patch, needs review) to [with patch, needs review] bessel_K function is broken

comment:2 Changed 12 years ago by malb

  • Component changed from misc to calculus
  • Owner changed from bober to gfurnish

comment:3 Changed 12 years ago by craigcitro

  • Keywords editor_gfurnish added
  • Summary changed from [with patch, needs review] bessel_K function is broken to [with patch, under review] bessel_K function is broken

comment:4 Changed 12 years ago by bober

Regarding bessel_K being real for real argument and real or imaginary order, see, e.g. the appendix to H. Then, Maass cusp forms for large eigenvalues, Math. Comp. Volume 74, Number 249, pp. 363 - 381: "The K-Bessel function K_ir(x) is ... real for real arguments x and real or imaginary order ir."

comment:5 Changed 12 years ago by gregorybard

I can say that I agree that this is 100% correct.

comment:6 Changed 12 years ago by gfurnish

  • Summary changed from [with patch, under review] bessel_K function is broken to [with patch, with positive review] bessel_K function is broken

comment:7 Changed 12 years ago by gfurnish

  • Milestone changed from sage-3.0.4 to sage-3.0.3
  • Priority changed from minor to major

comment:8 Changed 12 years ago by mabshoff

  • Milestone changed from sage-3.0.3 to sage-3.0.4

comment:9 follow-up: Changed 11 years ago by rishi

I think a solution of the following type would be better.

try:
        from sage.libs.pari.all import pari
        RR,a = _setup(prec)
        b = RR(pari(nu).besselk(z))
        pari.set_real_precision

except TypeError:
        CC,a = _setup(prec)
        b = CC(pari(nu).besselk(z))
        pari.set_real_precision(a)

comment:10 in reply to: ↑ 9 Changed 11 years ago by rishi

Probably the correct code would be

try:
        from sage.libs.pari.all import pari
        RR,a = _setup(prec)
        b = RR(pari(nu).besselk(z))
        pari.set_real_precision

except TypeError:
        CC,a = _setup_CC(prec)
        b = CC(pari(nu).besselk(z))
        pari.set_real_precision(a)

comment:11 Changed 11 years ago by mabshoff

  • Summary changed from [with patch, with positive review] bessel_K function is broken to [with patch, needs work] bessel_K function is broken

Since Rishi commented on this it might be a good idea to discuss his comments.

Cheers,

Michael

comment:12 Changed 11 years ago by gfurnish

  • Summary changed from [with patch, needs work] bessel_K function is broken to [with patch, with positive review] bessel_K function is broken

The try code does not in my opinion work. The issue here is correcting numerical instabilities, which attempting to coerce into RR will not do. Positive review still.

comment:13 Changed 11 years ago by rishi

I am pretty confident that the try code works. Consider the following gp output. The current patch will try to coerce this to RR. If pari is right then this input after applying the patch will give a TypeError?.

? besselk(2,-1.121)
%1 = 1.234141459629829380224386595 - 0.5472316582663064541169798027*I

We probably eliminate the double evaluation of bessel function using pari which my current suggestion does.

comment:14 Changed 11 years ago by gfurnish

I don't understand what double evaluation of the bessel function you are talking about. Furthermore the entire point is that Pari can give wrong answers in some cases because of numerical cases. Therefore we want to manually correct the numerical noise. Trying to coerce into RR instead of clearing it completely misses the point.

comment:15 Changed 11 years ago by rishi

I refuse to believe such a large numerical instabilities. In fact maple gives the same answer as pari. I will investigate further and figure out. You cannot Willy nilly ignore the imaginary part.

comment:16 Changed 11 years ago by rishi

And Mathematica does the same as pari.

I suggested the try: except: statement because it does not require deep understanding of Bessel's K function and probably in the end we might end up with this solution.

comment:17 Changed 11 years ago by gfurnish

The try except statement does not significantly help the original complaint. Either the identity is true or we should give the patch a negative review and close it as invalid.

comment:18 Changed 11 years ago by gfurnish

besselk(nu,x,{flag = 0})

K-Bessel function of index nu (which can be complex) and argument x. Only real and positive arguments x are allowed in the present version 2.3.3. If flag is equal to 1, uses another implementation of this function which is faster when x >> 1.

The library syntax is kbessel(nu,x,prec) and kbessel2(nu,x,prec) respectively.

Therefore pari gives incorrect answers for your negative x. Perhaps this is another bug and we should merge this ticket and open another one for adding some error checking to besselk?

comment:19 Changed 11 years ago by rishi

The complaint is valid. In fact the current version is lot more broken that the patched version. The current version does not take any complex argument. The patched version only fails on negative real axis.

comment:20 Changed 11 years ago by gfurnish

Ok lets patch this then and open a new ticket for the negative real axis case.

comment:21 Changed 11 years ago by rishi

I think

if (real(nu) == 0 or imag(nu) == 0) and (imag(z) == 0)

should become

if (real(nu) == 0 or imag(nu) == 0) and (imag(z) == 0 and real(z) > 0)

comment:22 in reply to: ↑ description Changed 11 years ago by bober

  • Summary changed from [with patch, with positive review] bessel_K function is broken to [with patch, with mixed review] bessel_K function is broken

Following the philosophy that wrong answers are worse than errors, this patch should not go in as is. Probably the code in Rishi's most recent comment is all that is needed for a fix, as long as we're not missing something else. I can't fix this at exactly this moment, but the fix should be trivial. Anyway, even though I posted the original patch, I have to give this a -1.

comment:23 Changed 11 years ago by gfurnish

rishi's code does not prevent brokenness at all (in fact it is 100% equivalent to attempting to trying to return RR(answer, prec). The patch as is makes the answer "more correct," and then we can go back and write code (that makes use of this patch) to make it 100% correct. Alternatively, if someone wants to make a new patch that checks for real(z)>=0 in all cases and throws an error otherwise, I would give that a positive review. However, the modification of real(z)>0 is not sufficient to ensure correctness.

comment:24 Changed 11 years ago by mabshoff

  • Summary changed from [with patch, with mixed review] bessel_K function is broken to [with patch, needs work] bessel_K function is broken

This ticket has been sitting around for a while without any movement. Change the title so that the reports pick up this ticket correctly.

Cheers,

Michael

comment:25 Changed 11 years ago by AlexGhitza

  • Cc AlexGhitza added

comment:26 Changed 11 years ago by cremona

I note that Alex added himself to the CC for this. Whatever is done for this issue absolutely must take into account the work done for #4096, so at the least I suggest that the author of this patch looks at that one and reworks this. Anything relying on Sage/pari precision questions is likely to be useless otherwise.

comment:27 Changed 11 years ago by AlexGhitza

  • Cc AlexGhitza removed
  • Summary changed from [with patch, needs work] bessel_K function is broken to [with new patch, needs review] bessel_K function is broken

I looked up the definition and properties of the Bessel functions in several references (Section 7.2 of the "Bateman Manuscript Project", for instance).

I uploaded a brand new patch that implements the behavior described there, namely returning a real number if the result is theoretically known to be real, and a complex number otherwise. I added doctests that document this behavior, and checked all of them against Mathematica. I did this for all three Bessel functions that are implemented in special.py using Pari, namely J, K, and I. I also put in a workaround for a silly Pari buglet that complains about negative integer values of nu.

In the process I uncovered a couple of unrelated issues with special.py and Bessel functions, for which I'll open separate tickets.

The patch is made against 3.1.3.alpha2.

comment:28 Changed 11 years ago by ddrake

Uh oh:

sage: bessel_J(0,0)
---------------------------------------------------------------------------
PariError                                 Traceback (most recent call last)

/home/drake/.sage/temp/klee/32521/_home_drake__sage_init_sage_0.py in <module>()
----> 1 
      2 
      3 
      4 
      5 

/opt/sage-3.1.3.alpha2/local/lib/python2.5/site-packages/sage/functions/special.pyc in bessel_J(nu, z, algorithm, prec)
    570             z = C(z)
    571             K = C
--> 572         pari_bes = pari(nu).besselj(z, precision=prec)
    573         if K is R:
    574             return fudge*K(pari_bes.real())

/opt/sage-3.1.3.alpha2/local/lib/python2.5/site-packages/sage/libs/pari/gen.so in sage.libs.pari.gen._pari_trap (sage/libs/pari/gen.c:34414)()
   7865 
   7866 
-> 7867 
   7868 
   7869 

PariError:  (8)

Doing bessel_J(0, 0) in 3.1.2 works fine. I get similar errors with this patch for other Bessel functions, too.

comment:29 Changed 11 years ago by AlexGhitza

Thanks for catching this. It actually comes from a bug in Pari:

? besselj(0, 0)
%1 = 1.000000000000000000000000000
? besselj(0.E-19, 0)
  *** besselj: gpow: 0 to a non positive exponent.

I've reported it upstream, but I will post a patch with a workaround while we wait.

Changed 11 years ago by AlexGhitza

apply instead of the previous patch

comment:30 Changed 11 years ago by AlexGhitza

OK, I've replaced my patch with one that fixes the issue reported by Dan.

comment:31 Changed 11 years ago by ddrake

Now bessel_J(0,0) works but I'm seeing other problems. I'm concentrating here on the "K" functions since that's what this ticket is about. This is all with the current patch applied to 3.1.4.

  • bessel_K(0, -1) doesn't work at all; it just soaks up all the CPU and doesn't return. The correct answer is about 0.421024438240708 - 3.97746326050642*I.
  • bessel_K(-1*I - 1, 0) gives an uninformative Pari error; the function isn't defined there. Mathematica says "ComplexInfinity?"; can we give a better error message? Even just "function not defined at 0" would be fine.
  • bessel_K(-1, -1) says it can't convert -0.601907230197235-1.77549968921218*I to a real number, which I agree with. :) It looks like Pari has the right answer but we're trying to convert it to a real when we shouldn't be.
  • bessel_K(0, -1 - I) says "incorrect type", but Mathematica and Maple evaluate it just fine.

comment:32 Changed 11 years ago by AlexGhitza

  • Summary changed from [with new patch, needs review] bessel_K function is broken to [with patch, needs work] bessel_K function is broken

comment:33 Changed 11 years ago by rlm

See #4626, which at least fixes the bessel_J problem.

comment:34 Changed 10 years ago by AlexGhitza

  • Keywords editor_gfurnish removed
  • Report Upstream set to N/A
  • Summary changed from [with patch, needs work] bessel_K function is broken to bessel_K function is broken

This ticket is a huge mess :)

I now think that we should just use mpmath to evaluate Bessel functions, see

http://mpmath.googlecode.com/svn/trunk/doc/build/functions/bessel.html

For the examples that Dan gave:

sage: from mpmath import *
sage: mp.dps = 25; mp.pretty = True
sage: besselk(0, -1)
(0.4210244382407083333356274 - 3.97746326050642263725661j)
sage: besselk(-1*I - 1, 0)
+inf
sage: besselk(-1, -1)
(-0.60190723019723457473754 - 1.775499689212180946878577j)
sage: besselk(0, -1-I)
(-1.479697108749625193260947 + 2.588306443392007370808151j)

comment:35 Changed 10 years ago by rishi

From my quick experiments with the issues Bober was dealing with a year ago, I see that do not arise if we use mpmath, even when I set the precision to 5000.

I agree with Alex Ghitza.

I should say that the bessels functions for non integer indices have always bothered me. I believe computing will involve a log, and how do you consistently choose a branch.

comment:36 Changed 10 years ago by ddrake

I also agree with Alex -- both in that this ticket is a mess, and that we should just use mpmath. I'll see if I can work up a patch (although anyone else who wants to should feel free to do it).

comment:37 Changed 10 years ago by rishi

With some experiments, I saw that the branch of the log taken is the negative real axis. We should mention this in the documentation when it is implemented. I believe ddrake is working up a patch.

comment:38 Changed 10 years ago by burcin

  • Cc burcin added

comment:39 Changed 10 years ago by ddrake

Here are some more problems with bessel_K: http://groups.google.com/group/sage-support/browse_thread/thread/84e5cd4fb1381ad5

In resolving this ticket, we should make sure to have doctests related to those problems!

comment:40 Changed 9 years ago by kcrisman

  • Cc kcrisman added

comment:41 Changed 7 years ago by kcrisman

  • Cc benjaminfjones added

This would most likely be fixed by #4102.

comment:42 Changed 7 years ago by benjaminfjones

Yes, it will be fixed by #4102. I'll make a note to add a related doctest to that effect.

comment:43 Changed 7 years ago by kcrisman

  • Milestone changed from sage-5.7 to sage-duplicate/invalid/wontfix
  • Reviewers set to Karl-Dieter Crisman, Benjamin Jones
  • Status changed from needs_work to positive_review

Everything here is now in a doctest in #4102, including the stuff in the thread from three (!) years ago.

comment:44 Changed 7 years ago by jdemeyer

  • Resolution set to duplicate
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.