mpmath import problem warning?

[dsm]

A rather subtle bug just ate several hours of my (and Benjamin's) time, so for the sake of future sufferers, it's probably worth documenting and wondering if we can find a way to make sure this doesn't happen. The bug relates to trac #12455, which introduces symbolic Airy functions and uses mpmath for numerical evaluation.

The manifestation is unusual: without the patch, we have

sage: import mpmath
sage: from sage.libs.mpmath import utils as mpmath_utils
sage: mpmath_utils.call(mpmath.airyai, 3,4, prec=500)
0.0354943008052438353507905704856075851248207412675096681263890254113384424359941551501461978539207259766557679623911357156186928332753550261698746599899

but with the patch (after moving an import to avoid a circularity), we have

sage: import mpmath
sage: from sage.libs.mpmath import utils as mpmath_utils
sage: mpmath_utils.call(mpmath.airyai, 3,4, prec=500)
[...]
/Users/mcneil/sagedev/sage-5.1.beta0/local/lib/python2.7/site-packages/mpmath/libmp/libintmath.py in python_bitcount(n)
     88     if bc != 300:
     89         return bc
---> 90     bc = int(math.log(n, 2)) - 4
     91     return bc + bctable[n>>bc]
     92 

OverflowError: cannot convert float infinity to integer

even though we're not using any of the new functions, and it works fine for (say) prec=300 and lower!

This turns out to be because the patch created a new module airy.py in functions, and it was being imported in functions/all.py. mpmath determines its backend in libmp/backend.py by attempting to import sage.all, which fails (because it's not ready yet), and so mpmath decides it's not in Sage and uses the Python backend. This results in using math.log on a really big number, which leads to the OverflowError?. It's quite unlikely that a beginner would ever guess this happens -- because it *works* whenever the precision is low enough, so many of the standard tests one might try would be passed.

Would there be a good place to insert a verification that our mpmath knows it's in Sage?

[burcin]

I can't reproduce this with the current patches in #12455. Is there a different way to trigger the problem?

[eviatarbach]

I got this bug with the patch in #2516 with hypergeometric((10, 10), (50,), 2.), where I say it was due to an mpmath limitation before seeing this ticket. It seems indeed to be a circular import problem; I moved the import from mpmath import hyper into a function rather than have it at the top and it works.