Opened 3 years ago
Closed 3 years ago
#26360 closed enhancement (fixed)
Upgrade arb to 2.15.1
Reported by:  ghtimokau  Owned by:  

Priority:  major  Milestone:  sage8.4 
Component:  packages: standard  Keywords:  arb, upgrade 
Cc:  fredrik.johansson, embray, mmezzarobba, jdemeyer, fbissey  Merged in:  
Authors:  Timo Kaufmann  Reviewers:  Marc Mezzarobba 
Report Upstream:  N/A  Work issues:  
Branch:  30cc778 (Commits, GitHub, GitLab)  Commit:  30cc778d46579bd0c7537ed33e8d7a4f40fd5c31 
Dependencies:  Stopgaps: 
Description (last modified by )
Arb 2.15.1 is out and the upgrade breaks a lot (~200) doctests with changes that are not actually failures. Nearly all the changes are minor differences in the accuracy of the radius. As previously discussed in #25966, I think we should fix this once and for all. The tests are brittle and break with more or less every arb upgrade. All those "failures" distract from the actual failures. It makes arb upgrades painful and is a burden on distributions.
I suspect the solution here (lots of ...
) will be controversial. I don't like it very much myself. But I think it is better than the status quo. What we really want are different tolerances for the "mid" and the "radius". I don't know if that is possible in the doctesting framework without explicitly testing for .mid()
and .radius()
in different tests each time. We may even want to test the .accuracy()
instead of the radius.
Upstream tarball: https://github.com/fredrikjohansson/arb/archive/2.15.1.tar.gz
Change History (30)
comment:1 Changed 3 years ago by
comment:2 Changed 3 years ago by
 Cc fbissey added
comment:3 in reply to: ↑ description ; followups: ↓ 4 ↓ 7 Changed 3 years ago by
Replying to ghtimokau:
I suspect the solution here (lots of
...
) will be controversial. I don't like it very much myself.
I'm okay with it, but I'd like to see what Jeroen says since he insisted for exact doctests back then.
What we really want are different tolerances for the "mid" and the "radius".
Is that really necessary? If you use absolute tolerances, having the same tolerance for the center and the radius isn't too bad.
Something like what Fredrik suggests would be better, of course. Since arb provides a way to parse strings of the form [mid +/ rad]
, it is perhaps doable. (I'm not volunteering, though...)
comment:4 in reply to: ↑ 3 ; followup: ↓ 6 Changed 3 years ago by
Replying to mmezzarobba:
Replying to ghtimokau:
I suspect the solution here (lots of
...
) will be controversial. I don't like it very much myself.I'm okay with it, but I'd like to see what Jeroen says since he insisted for exact doctests back then.
Jeroen, do you still insist on exact doctests here?
comment:5 followup: ↓ 21 Changed 3 years ago by
What we really want are different tolerances for the "mid" and the "radius".
Is that really necessary? If you use absolute tolerances, having the same tolerance for the center and the radius isn't too bad.
That might also a (while not pretty) workable solution.
Something like what Fredrik suggests would be better, of course. Since arb provides a way to parse strings of the form
[mid +/ rad]
, it is perhaps doable. (I'm not volunteering, though...)
Is the doctesting framework even that flexible?
comment:6 in reply to: ↑ 4 Changed 3 years ago by
Replying to ghtimokau:
Jeroen, do you still insist on exact doctests here?
I don't recall why I originally insisted on that. So go ahead.
comment:7 in reply to: ↑ 3 Changed 3 years ago by
Replying to mmezzarobba:
Something like what Fredrik suggests would be better, of course. Since arb provides a way to parse strings of the form
[mid +/ rad]
, it is perhaps doable. (I'm not volunteering, though...)
I'm not convinced that we need to patch the doctest framework for that.
comment:8 Changed 3 years ago by
I can't really do much about the remaining failures, since I lack the subject knowledge to decide weather or not they are real errors:
sage t long src/sage/rings/complex_arb.pyx ********************************************************************** File "src/sage/rings/complex_arb.pyx", line 4403, in sage.rings.complex_arb.ComplexBall.elliptic_f Failed example: (CBF.pi()/2).elliptic_f(phi) Expected: [1.5092369540513 +/ ...e14] + [0.6251464152027 +/ ...e14]*I Got: nan + nan*I ********************************************************************** File "src/sage/rings/complex_arb.pyx", line 4409, in sage.rings.complex_arb.ComplexBall.elliptic_f Failed example: (CBF.pi()/2).elliptic_f(phi) Expected: [1.3393589639094 +/ ...e14] + [1.1104369690719 +/ ...e14]*I Got: nan + nan*I ********************************************************************** File "src/sage/rings/complex_arb.pyx", line 4441, in sage.rings.complex_arb.ComplexBall.elliptic_e_inc Failed example: (CBF.pi()/2).elliptic_e_inc(phi) Expected: [1.2838409578982 +/ ...e14] + [0.5317843366915 +/ ...e14]*I Got: nan + nan*I ********************************************************************** File "src/sage/rings/complex_arb.pyx", line 4447, in sage.rings.complex_arb.ComplexBall.elliptic_e_inc Failed example: (CBF.pi()/2).elliptic_e_inc(phi) Expected: [0.787564350925 +/ ...e13] + [0.686896129145 +/ ...e13]*I Got: nan + nan*I ********************************************************************** File "src/sage/rings/complex_arb.pyx", line 4481, in sage.rings.complex_arb.ComplexBall.elliptic_pi_inc Failed example: n.elliptic_pi_inc(CBF.pi()/2, m) Expected: [0.8934793755173 +/ ...e14] + [0.95707868710750 +/ ...e15]*I Got: nan + nan*I ********************************************************************** File "src/sage/rings/complex_arb.pyx", line 4488, in sage.rings.complex_arb.ComplexBall.elliptic_pi_inc Failed example: n.elliptic_pi_inc(CBF.pi()/2, m) Expected: [0.2969588746419 +/ ...e14] + [1.318879533274 +/ ...e13]*I Got: nan + nan*I ********************************************************************** 3 items had failures: 2 of 8 in sage.rings.complex_arb.ComplexBall.elliptic_e_inc 2 of 8 in sage.rings.complex_arb.ComplexBall.elliptic_f 2 of 10 in sage.rings.complex_arb.ComplexBall.elliptic_pi_inc [622 tests, 6 failures, 11.97 s]
sage t long src/sage/rings/real_arb.pyx ********************************************************************** File "src/sage/rings/real_arb.pyx", line 1305, in sage.rings.real_arb.RealBall.__init__ Failed example: RBF("2.3e10000000000000000000000 +/ ...e10000000000000000000000") Exception raised: Traceback (most recent call last): File "/home/timo/repos2/sage/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 659, in _run self.compile_and_execute(example, compiler, test.globs) File "/home/timo/repos2/sage/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 1070, in compile_and_execute exec(compiled, globs) File "<doctest sage.rings.real_arb.RealBall.__init__[35]>", line 1, in <module> RBF("2.3e10000000000000000000000 +/ ...e10000000000000000000000") File "sage/structure/parent.pyx", line 921, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9679) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 145, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4574) raise File "sage/structure/coerce_maps.pyx", line 140, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4462) return C._element_constructor(x) File "sage/rings/real_arb.pyx", line 512, in sage.rings.real_arb.RealBallField._element_constructor_ (build/cythonized/sage/rings/real_arb.c:5843) return self.element_class(self, mid, rad) File "sage/rings/real_arb.pyx", line 1376, in sage.rings.real_arb.RealBall.__init__ (build/cythonized/sage/rings/real_arb.c:13451) raise ValueError("unsupported string format") ValueError: unsupported string format ********************************************************************** 1 item had failures: 1 of 48 in sage.rings.real_arb.RealBall.__init__ [530 tests, 1 failure, 0.78 s]
comment:9 Changed 3 years ago by
That's a genuine regression in Arb 2.15. I have not tried to debug it, but quite likely the original code was buggy and only worked by accident  it works if you add CBF("+/ 1e10") to pi/2.
comment:10 Changed 3 years ago by
it works if you add CBF("+/ 1e10") to pi/2
So should we just do that in the doctests?
comment:11 Changed 3 years ago by
This commit should provide a bugfix: https://github.com/fredrikjohansson/arb/commit/eb60d7ac4ee85cd6f8353dd9cf52a10bfd5967c3
The other failure ("unsupported string format") is due to too greedy search & replace to insert ...
comment:12 Changed 3 years ago by
Do you plan to make a release with that fix included anytime soon?
comment:13 Changed 3 years ago by
comment:14 Changed 3 years ago by
Thank you! Could you expand on what you mean by "due to too greedy search & replace to insert"? As far as I can see it is caused by arb_set_str
returning false.
comment:15 Changed 3 years ago by
There is "..." in the input string which shouldn't have been inserted there in place of the numerical value.
comment:16 Changed 3 years ago by
 Commit changed from 00d3ccd2ee230d9252b9cdc3183adf9c7f9f45ad to 30cc778d46579bd0c7537ed33e8d7a4f40fd5c31
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
30cc778  Upgrade arb to 2.15.1

comment:17 Changed 3 years ago by
 Description modified (diff)
 Status changed from new to needs_review
 Summary changed from Upgrade arb to 2.15.0 to Upgrade arb to 2.15.1
Oh, stupid mistake. Thanks.
Doctests pass now.
comment:18 Changed 3 years ago by
I think rather than so many ...
, if there are changes we can make to the doctest parser to make these tests a little more natural that would be better, and probably not a big deal...
comment:19 Changed 3 years ago by
I wouldn't know how to do that.
comment:20 Changed 3 years ago by
I mean, I would, but I don't know exactly what it is you need in this case.
comment:21 in reply to: ↑ 5 ; followup: ↓ 22 Changed 3 years ago by
Replying to ghtimokau:
What we really want are different tolerances for the "mid" and the "radius".
Is that really necessary? If you use absolute tolerances, having the same tolerance for the center and the radius isn't too bad.
That might also a (while not pretty) workable solution.
Did you ever try this? I don't really see it as "pretty" or not "pretty". An # abs tol
will apply to both values, but we really only care about it in this case for the radius. Technically it would be applied to the mid too but it doesn't matter either way for the present purpose.
comment:22 in reply to: ↑ 21 Changed 3 years ago by
Replying to embray:
I mean, I would, but I don't know exactly what it is you need in this case.
We'd need to use proper comparison methods instead of just comparing the digits. Something like what Frederik said.
Replying to embray:
Replying to ghtimokau:
What we really want are different tolerances for the "mid" and the "radius".
Is that really necessary? If you use absolute tolerances, having the same tolerance for the center and the radius isn't too bad.
That might also a (while not pretty) workable solution.
Did you ever try this? I don't really see it as "pretty" or not "pretty". An
# abs tol
will apply to both values, but we really only care about it in this case for the radius. Technically it would be applied to the mid too but it doesn't matter either way for the present purpose.
No I haven't. It would require either a lot of manual adding of # abs tol
or a very clever sed script. It doesn't have high enough priority for me right now to spend the time doing that, but I'd be happy with the solution if somebody else would.
comment:23 Changed 3 years ago by
I'll give it a try and see. I'm pretty good at ridiculous regular expressions :)
comment:24 Changed 3 years ago by
I suggest merging this asis. The doctest improvement would be nice to have but it's not worth delaying the upgrade over it.
comment:25 Changed 3 years ago by
@fredrik.johansson was that a review? I'm guessing @embray has different priorities right now, would be nice to get this into the next release.
comment:26 Changed 3 years ago by
It's true I have different priorities. I'm pretty sad about this doctest issue but not enough to do anything about it right now so I don't think this should be held up over it. I'll open a ticket though.
comment:27 Changed 3 years ago by
See #26774 with an idea for a generalized solution.
comment:28 followup: ↓ 29 Changed 3 years ago by
#26774 would be really nice. In addition to arb I'm sure there are a lot of other areas where the doctests could be improved. That'll take a while however.
Lets get this merged for now.
comment:29 in reply to: ↑ 28 Changed 3 years ago by
 Reviewers set to Marc Mezzarobba
 Status changed from needs_review to positive_review
Replying to ghtimokau:
#26774 would be really nice. In addition to arb I'm sure there are a lot of other areas where the doctests could be improved. That'll take a while however.
Lets get this merged for now.
I agree.
comment:30 Changed 3 years ago by
 Branch changed from u/ghtimokau/arb2.15.0 to 30cc778d46579bd0c7537ed33e8d7a4f40fd5c31
 Resolution set to fixed
 Status changed from positive_review to closed
It would be nice to have some hook for doctests so that balls by default are compared in terms of overlapping and that the radii are the same within a factor 10, say.