Opened 11 years ago
Closed 9 years ago
#11334 closed task (fixed)
Update numpy to 1.7.0
| Reported by: | jason | Owned by: | tbd |
|---|---|---|---|
| Priority: | major | Milestone: | sage-5.10 |
| Component: | packages: standard | Keywords: | sd40.5 |
| Cc: | fbissey, kini, Snark | Merged in: | sage-5.10.beta0 |
| Authors: | François Bissey | Reviewers: | Steven Trogdon, Jeroen Demeyer |
| Report Upstream: | Fixed upstream, in a later stable release. | Work issues: | |
| Branch: | Commit: | ||
| Dependencies: | #12415, #13992 | Stopgaps: |
Status badges
Description (last modified by )
Numpy-1.7.0 is now available and fixes the coercion problem that prevented us to go to numpy-1.6.x.
New spkg: http://spkg-upload.googlecode.com/files/numpy-1.7.0.spkg
Apply:
Attachments (11)
Change History (138)
comment:1 Changed 11 years ago by
- Cc fbissey added
comment:2 Changed 11 years ago by
Thx. it looks like numpy has been bumped to 1.6 in Gentoo so I can do some quick compatibility tests. It looks like a flag for gfortran has been added in the ebuild to help with the compiler autodetection madness it may be worth having it sage side too.
Do you know if a scipy release is to follow, if not it would be best to revbump scipy to make sure it is rebuilt after numpy.
Other matter there has been noise about lapack in numpy do we want to disable it, make it optional or keep the status quo?
comment:3 Changed 11 years ago by
I don't remember seeing anything on the scipy lists about an imminent scipy release.
comment:4 Changed 11 years ago by
just in:
sage -t -long -force_lib "devel/sage/doc/en/numerical_sage/numpy.rst"
**********************************************************************
File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 97:
sage: 2.5*l
Expected:
array([ 0. , 2.5, 5. , 7.5, 10. , 12.5, 15. , 17.5, 20. , 22.5])
Got:
array([0.000000000000000, 2.50000000000000, 5.00000000000000,
7.50000000000000, 10.0000000000000, 12.5000000000000,
15.0000000000000, 17.5000000000000, 20.0000000000000,
22.5000000000000], dtype=object)
**********************************************************************
File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 289:
sage: xx
Expected:
array([[ 0. , 0.25, 0.5 , 0.75, 1. ],
[ 0. , 0.25, 0.5 , 0.75, 1. ],
[ 0. , 0.25, 0.5 , 0.75, 1. ],
[ 0. , 0.25, 0.5 , 0.75, 1. ],
[ 0. , 0.25, 0.5 , 0.75, 1. ]])
Got:
array([[0.000000000000000, 0.250000000000000, 0.500000000000000,
0.750000000000000, 1.00000000000000],
[0.000000000000000, 0.250000000000000, 0.500000000000000,
0.750000000000000, 1.00000000000000],
[0.000000000000000, 0.250000000000000, 0.500000000000000,
0.750000000000000, 1.00000000000000],
[0.000000000000000, 0.250000000000000, 0.500000000000000,
0.750000000000000, 1.00000000000000],
[0.000000000000000, 0.250000000000000, 0.500000000000000,
0.750000000000000, 1.00000000000000]], dtype=object)
**********************************************************************
File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 295:
sage: yy
Expected:
array([[ 0. , 0. , 0. , 0. , 0. ],
[ 0.25, 0.25, 0.25, 0.25, 0.25],
[ 0.5 , 0.5 , 0.5 , 0.5 , 0.5 ],
[ 0.75, 0.75, 0.75, 0.75, 0.75],
[ 1. , 1. , 1. , 1. , 1. ]])
Got:
array([[0.000000000000000, 0.000000000000000, 0.000000000000000,
0.000000000000000, 0.000000000000000],
[0.250000000000000, 0.250000000000000, 0.250000000000000,
0.250000000000000, 0.250000000000000],
[0.500000000000000, 0.500000000000000, 0.500000000000000,
0.500000000000000, 0.500000000000000],
[0.750000000000000, 0.750000000000000, 0.750000000000000,
0.750000000000000, 0.750000000000000],
[1.00000000000000, 1.00000000000000, 1.00000000000000,
1.00000000000000, 1.00000000000000]], dtype=object)
**********************************************************************
File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 301:
sage: f(xx,yy)
Expected:
array([[ 0. , 0.0625, 0.25 , 0.5625, 1. ],
[ 0.0625, 0.125 , 0.3125, 0.625 , 1.0625],
[ 0.25 , 0.3125, 0.5 , 0.8125, 1.25 ],
[ 0.5625, 0.625 , 0.8125, 1.125 , 1.5625],
[ 1. , 1.0625, 1.25 , 1.5625, 2. ]])
Got:
array([[0.000000000000000, 0.0625000000000000, 0.250000000000000,
0.562500000000000, 1.00000000000000],
[0.0625000000000000, 0.125000000000000, 0.312500000000000,
0.625000000000000, 1.06250000000000],
[0.250000000000000, 0.312500000000000, 0.500000000000000,
0.812500000000000, 1.25000000000000],
[0.562500000000000, 0.625000000000000, 0.812500000000000,
1.12500000000000, 1.56250000000000],
[1.00000000000000, 1.06250000000000, 1.25000000000000,
1.56250000000000, 2.00000000000000]], dtype=object)
**********************************************************************
2 items had failures:
1 of 55 in __main__.example_0
3 of 13 in __main__.example_1
***Test Failed*** 4 failures.
there may be more
comment:5 Changed 11 years ago by
- Description modified (diff)
I have cut a new spkg for 1.6.0, I decided to go for plain update for now.
comment:6 Changed 11 years ago by
seems my home install has problems. Probably stuff messed around between python 2.6 and 2.7 because I am working on #9958. Another install didn't report any problems. So I will check on a vanilla sage but I think we won't need to do any patching.
comment:7 Changed 11 years ago by
For some reason the other install didn't upgrade that's why I didn't see anything on the second machine. So right now it looks like we have the following to correct:
sage -t -long -force_lib devel/sage/doc/en/numerical_sage/numpy.rst #
4 doctests failed
sage -t -long -force_lib devel/sage/sage/symbolic/function.pyx # 1
doctests failed
sage -t -long -force_lib devel/sage/sage/plot/plot_field.py # 1
doctests failed
sage -t -long -force_lib devel/sage/sage/plot/matrix_plot.py # 1
doctests failed
sage -t -long -force_lib devel/sage/sage/matrix/matrix1.pyx # 1
doctests failed
sage -t -long -force_lib devel/sage/sage/numerical/optimize.py # 1
doctests failed
sage -t -long -force_lib devel/sage/sage/rings/integer.pyx # 1
doctests failed
sage -t -long -force_lib devel/sage/sage/functions/hyperbolic.py # 2
doctests failed
sage -t -long -force_lib devel/sage/sage/functions/trig.py # 3
doctests failed
sage -t -long -force_lib devel/sage/sage/functions/other.py # 2
doctests failed
The first one is already in the ticket, then we have
sage -t -long -force_lib "devel/sage/sage/symbolic/function.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function.pyx", line 596:
sage: csc(a)
Expected:
array([ inf, 1.18839511, 1.09975017, 7.0861674 , -1.32134871])
Got:
doctest:270: RuntimeWarning: divide by zero encountered in divide
array([ inf, 1.18839511, 1.09975017, 7.0861674 , -1.32134871])
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/plot/plot_field.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/plot/plot_field.py", line 216:
sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2)).show(aspect_ratio=1)
Expected nothing
Got:
doctest:622: RuntimeWarning: divide by zero encountered in divide
doctest:623: RuntimeWarning: invalid value encountered in multiply
doctest:624: RuntimeWarning: invalid value encountered in multiply
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/plot/matrix_plot.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/plot/matrix_plot.py", line 383:
sage: matrix_plot([[sin(x), cos(x)], [1, 0]])
Expected:
Traceback (most recent call last):
...
ValueError: can not convert entries to floating point numbers
Got:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_8[36]>", line 1, in <module>
matrix_plot([[sin(x), cos(x)], [Integer(1), Integer(0)]])###line 383:
sage: matrix_plot([[sin(x), cos(x)], [1, 0]])
File "/usr/lib/python2.7/site-packages/sage/misc/decorators.py", line 381, in wrapper
return func(*args, **kwds)
File "/usr/lib/python2.7/site-packages/sage/misc/decorators.py", line 432, in wrapper
return func(*args, **options)
File "/usr/lib/python2.7/site-packages/sage/plot/matrix_plot.py", line 423, in matrix_plot
raise TypeError, "mat must be a Matrix or a two dimensional array"
TypeError: mat must be a Matrix or a two dimensional array
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/matrix/matrix1.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/matrix/matrix1.pyx", line 468:
sage: sorted(numpy.typecodes.items())
Expected:
[('All', '?bhilqpBHILQPfdgFDGSUVO'), ('AllFloat', 'fdgFDG'), ('AllInteger', 'bBhHiIlLqQpP'), ('Character', 'c'), ('Complex', 'FDG'), ('Float', 'fdg'), ('Integer', 'bhilqp'), ('UnsignedInteger', 'BHILQP')]
Got:
[('All', '?bhilqpBHILQPefdgFDGSUVOMm'), ('AllFloat', 'efdgFDG'), ('AllInteger', 'bBhHiIlLqQpP'), ('Character', 'c'), ('Complex', 'FDG'), ('Datetime', 'Mm'), ('Float', 'efdg'), ('Integer', 'bhilqp'), ('UnsignedInteger', 'BHILQP')]
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/numerical/optimize.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/numerical/optimize.py", line 162:
sage: find_minimum_on_interval(f, 1, 5, tol=1e-3)
Expected:
(-3.28837136189098..., 3.42575079030572...)
Got:
(-3.28837136189, 3.4257507903057229)
sage -t -long -force_lib "devel/sage/sage/rings/integer.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/rings/integer.pyx", line 4738:
sage: numpy.array(2**40).dtype
Expected:
dtype('int64')
Got:
dtype('object')
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/functions/hyperbolic.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/hyperbolic.py", line 609:
sage: arcsech(a)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_23[4]>", line 1, in <module>
arcsech(a)###line 609:
sage: arcsech(a)
File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794)
File "/usr/lib/python2.7/site-packages/sage/functions/hyperbolic.py", line 612, in _eval_numpy_
return arccosh(1.0 / x)
ZeroDivisionError: float division by zero
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/hyperbolic.py", line 658:
sage: arccsch(a)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_26[4]>", line 1, in <module>
arccsch(a)###line 658:
sage: arccsch(a)
File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794)
File "/usr/lib/python2.7/site-packages/sage/functions/hyperbolic.py", line 661, in _eval_numpy_
return arcsinh(1.0 / x)
ZeroDivisionError: float division by zero
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/functions/trig.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/trig.py", line 827:
sage: atan2(a, b)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_34[16]>", line 1, in <module>
atan2(a, b)###line 827:
sage: atan2(a, b)
File "function.pyx", line 718, in sage.symbolic.function.GinacFunction.__call__ (sage/symbolic/function.cpp:6475)
File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794)
File "function.pyx", line 604, in sage.symbolic.function.Function._eval_numpy_ (sage/symbolic/function.cpp:5568)
NotImplementedError: The Function arctan2 does not support numpy arrays as arguments
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/trig.py", line 830:
sage: atan2(1,a)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_34[17]>", line 1, in <module>
atan2(Integer(1),a)###line 830:
sage: atan2(1,a)
File "function.pyx", line 718, in sage.symbolic.function.GinacFunction.__call__ (sage/symbolic/function.cpp:6475)
File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794)
File "function.pyx", line 604, in sage.symbolic.function.Function._eval_numpy_ (sage/symbolic/function.cpp:5568)
NotImplementedError: The Function arctan2 does not support numpy arrays as arguments
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/trig.py", line 833:
sage: atan2(a, 1)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_34[18]>", line 1, in <module>
atan2(a, Integer(1))###line 833:
sage: atan2(a, 1)
File "function.pyx", line 718, in sage.symbolic.function.GinacFunction.__call__ (sage/symbolic/function.cpp:6475)
File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794)
File "function.pyx", line 604, in sage.symbolic.function.Function._eval_numpy_ (sage/symbolic/function.cpp:5568)
NotImplementedError: The Function arctan2 does not support numpy arrays as arguments
**********************************************************************
sage -t -long -force_lib "devel/sage/sage/functions/other.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/other.py", line 211:
sage: ceil(a)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_5[20]>", line 1, in <module>
ceil(a)###line 211:
sage: ceil(a)
File "/usr/lib/python2.7/site-packages/sage/functions/other.py", line 251, in __call__
return numpy.ceil(x)
AttributeError: ceil
**********************************************************************
File "/usr/share/sage/devel/sage/sage/functions/other.py", line 366:
sage: floor(a)
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_9[15]>", line 1, in <module>
floor(a)###line 366:
sage: floor(a)
File "/usr/lib/python2.7/site-packages/sage/functions/other.py", line 405, in __call__
return numpy.floor(x)
AttributeError: floor
**********************************************************************
The last three are particularly concerning, could you check what's up with these in numpy?
comment:8 Changed 11 years ago by
This seems to have installed fine on OS X 10.4 PPC. I'm running doctests now (which will take a LONG time) but I did already see the matrix/matrix1.pyx one while testing something else.
comment:9 Changed 11 years ago by
That's good that it at least install. On gentoo there are interactions problem between numpy and scipy to get cblas/blas/lapack right in scipy. I should check if the resulting scipy in vanilla sage suffers from the same problem. http://bugs.gentoo.org/show_bug.cgi?id=371099
The last few failing tests are show stoppers as far as I am concerned.
comment:10 Changed 11 years ago by
Okay, I got pretty much the same failures on a long run. One weird thing... did we change find_minimum_on_interval away from SciPy recently? This is the full output I get on the (otherwise same as your failure) test for that.
sage: find_minimum_on_interval(f, 1, 5, tol=1e-3)
Expected:
(-3.28837136189098..., 3.42575079030572...)
Got:
(-3.28837136189, 3.4257507903057229)
GLPK Simplex Optimizer, v4.44
6 rows, 3 columns, 8 non-zeros
Preprocessing...
2 rows, 2 columns, 4 non-zeros
Scaling...
A: min|aij| = 2.400e+01 max|aij| = 5.000e+01 ratio = 2.083e+00
GM: min|aij| = 8.128e-01 max|aij| = 1.230e+00 ratio = 1.514e+00
EQ: min|aij| = 6.606e-01 max|aij| = 1.000e+00 ratio = 1.514e+00
Constructing initial basis...
Size of triangular part = 2
* 0: obj = -5.100000000e+01 infeas = 0.000e+00 (0)
* 1: obj = -5.225000000e+01 infeas = 0.000e+00 (0)
OPTIMAL SOLUTION FOUND
comment:11 Changed 10 years ago by
1.6.1 is now available. I'll have a look sometimes this week.
comment:12 Changed 10 years ago by
Looks like more of the same but maybe a little bit more. There are a few other things that may interfere on my setup and I had to solve that bit: http://bugs.gentoo.org/show_bug.cgi?id=371099 before doing the trial.
comment:13 Changed 10 years ago by
Once I had cleared a few blas problem that I had while solving the gentoo bug. I get exactly the same results. I also get the output from Karl in optimize. This is a verbose output of glpk which get called at some point. You only get to see the such verbose output if there was a failure in the test but not necessarily the test for which glpk is called funnily enough.
comment:14 Changed 10 years ago by
Should we try to move forward on this again? I will have a go at trying it with 4.8.alpha2. Any ideas about the current failures?
comment:15 Changed 10 years ago by
Ok this is still broken in 4.8.alpha2. Some food for thought
sage
----------------------------------------------------------------------
| Sage Version 4.8.alpha2, Release Date: 2011-11-19 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
**********************************************************************
* *
* Warning: this is a prerelease version, and it may be unstable. *
* *
**********************************************************************
sage: import numpy
sage: a = numpy.linspace(0,2,6)
sage: a
array([0.0, 0.4, 0.8, 1.2, 1.6, 2], dtype=object)
sage: ceil(a)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
/home/fbissey/<ipython console> in <module>()
/usr/lib64/python2.7/site-packages/sage/functions/other.py in __call__(self, x, maximum_bits)
250 elif type(x).__module__ == 'numpy':
251 import numpy
--> 252 return numpy.ceil(x)
253
254 x_original = x
AttributeError: ceil
with numpy-1.5.1 we just have
sage: a array([0.0, 0.4, 0.8, 1.2, 1.6, 2])
I suspect that for this one (functions/other.py) and functions/trig.py the pynac interface may need to be touched. All stuff that takes an numpy array and return a numpy array may have trouble.
comment:16 Changed 10 years ago by
Just to add a bit of info. Same python and ipython
ipython Python 2.7.2 (default, Oct 28 2011, 11:36:16) Type "copyright", "credits" or "license" for more information. IPython 0.10.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]: import numpy In [2]: a = numpy.linspace(0,2,6) In [3]: a Out[3]: array([ 0. , 0.4, 0.8, 1.2, 1.6, 2. ]) In [4]: ceil(a) --------------------------------------------------------------------------- NameError Traceback (most recent call last) /home/fbissey/<ipython console> in <module>() NameError: name 'ceil' is not defined In [5]: numpy.ceil(a) Out[5]: array([ 0., 1., 1., 2., 2., 2.])
So it works in ipython but we don't have the dtype=object attached to the array. Anyone knows what that means actually?
comment:17 Changed 10 years ago by
numpy-1.5.1
sage: numpy.dtype(a[1])
dtype('float64')
numpy-1.6.1
sage: numpy.dtype(a[1]) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/fbissey/<ipython console> in <module>() TypeError: data type not understood
comment:18 Changed 10 years ago by
sage: type(a[0]) <type 'float'> sage: type(a[0:5]) <type 'numpy.ndarray'> sage: type(a[2]) <type 'float'> sage: type(a[3]) <type 'float'> sage: type(a[4]) <type 'float'> sage: type(a[5]) <type 'sage.rings.integer.Integer'>
For some reason numpy.linspace return an array with sage objects rather than just floats. I am guessing it may be deeper than that. Numpy probably uses sage types rather than converting to numpy types. The types of a[0] to a[4] is strange, is there a sage type for floats? I mean I tried numpy.linspace(0,2.0,6) and I got <type 'sage.rings.real_mpfr.RealNumber?'> for a[0] to a[4] and <type 'sage.rings.real_mpfr.RealLiteral?'> for a[5].
Doing a cast numpy.array(a,dtype=float) returns something that will work.
comment:19 Changed 10 years ago by
- Description modified (diff)
- Summary changed from Update numpy to 1.6 to Update numpy to 1.6.1
comment:20 Changed 10 years ago by
I'll be starting to make patches to correct the doctests and document the new behavior in numpy 1.6.x. Some documentation may need updating.
comment:21 Changed 10 years ago by
Actually I am a bit confused by one of the tests:
``plot_slope_field`` takes a function of two variables xvar and yvar
(for instance, if the variables are `x` and `y`, take `f(x,y)`), and at
representative points `(x_i,y_i)` between xmin, xmax, and ymin, ymax
respectively, plots a line with slope `f(x_i,y_i)` (see below).
``plot_slope_field(f, (xvar, xmin, xmax), (yvar, ymin, ymax))``
EXAMPLES:
A logistic function modeling population growth::
sage: x,y = var('x y')
sage: capacity = 3 # thousand
sage: growth_rate = 0.7 # population increases by 70% per unit of time
sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2))
That function is independent of x, the next example after this depend on both x and y. I am suspecting the messages we get in this particular case is quite justified and it is surprising we didn't get any before. I am not sure what the correct formula should be.
comment:22 Changed 10 years ago by
I am guessing it should have been
growth_rate*(1-x/capacity)*y
y being the time and x the population in thousands. That satisfy a simple dimensional analysis.
comment:23 Changed 10 years ago by
No, wasn't it right before? That's the standard logistic population model, where y is the population. See http://en.wikipedia.org/wiki/Logistic_function#In_ecology:_modeling_population_growth, for example.
comment:24 Changed 10 years ago by
sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2)) }}}} > That function is independent of x, the next example after this depend on both x and y. I am suspecting the messages we get in This should be ok. The point is that the slope dy/dx is a function of y only; this can be completely symbolically solved using separation of variables to get the [http://en.wikipedia.org/wiki/Logistic_function#In_ecology:_modeling_population_growth logistic function]. > this particular case is quite justified and it is surprising we didn't get any before. I am not sure what the correct formula should be. If this is the division by zero stuff, it's the whole business with the arrowheads having zero length. See #11208. I think Jason has a fix for this, actually, but I don't recall the whole story of what happened once it was reported to mpl.
comment:25 Changed 10 years ago by
Sorry for that noise. Jason is right.
sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2))
That function is independent of x, the next example after this depend on both x and y. I am suspecting the messages we get in
This should be ok. The point is that the slope dy/dx is a function of y only; this can be completely symbolically solved using separation of variables to get the logistic function.
this particular case is quite justified and it is surprising we didn't get any before. I am not sure what the correct formula should be.
If this is the division by zero stuff, it's the whole business with the arrowheads having zero length. See #11208. I think Jason has a fix for this, actually, but I don't recall the whole story of what happened once it was reported to mpl.
comment:26 Changed 10 years ago by
OK I was interpreting the formula another way because I wasn't sure what it is actually computing. Seems like #11208 numpy, mpl and scipy all go hands in hands so if something start happening with one the others may get it too.
Overall the change of behavior of numpy around type conversion is quite annoying.
comment:27 Changed 10 years ago by
It strikes me that the problem may be more an issue of the coercion to numpy types not working properly with numpy-1.6. I will look into that.
comment:28 Changed 10 years ago by
Any ideas why stuff like this is happening with numpy-1.6.x
sage: numpy.arange(10.0)
array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
sage: numpy.linspace(0.0,9.0,10)
array([0.000000000000000, 1.00000000000000, 2.00000000000000,
3.00000000000000, 4.00000000000000, 5.00000000000000,
6.00000000000000, 7.00000000000000, 8.00000000000000,
9.00000000000000], dtype=object)
comment:29 follow-up: ↓ 33 Changed 10 years ago by
It looks like it didn't automatically convert from RR to float, like it should. That's what these two lines should do in sage/rings/real_mpfr.pyx
cdef object numpy_double_interface = {'typestr': '=f8'}
cdef object numpy_object_interface = {'typestr': '|O'}
and this property in that same file:
property __array_interface__:
def __get__(self):
"""
Used for NumPy conversion.
EXAMPLES::
sage: import numpy
sage: numpy.arange(10.0)
array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
sage: numpy.array([1.0, 1.1, 1.2]).dtype
dtype('float64')
sage: numpy.array([1.000000000000000000000000000000000000]).dtype
dtype('object')
"""
if (<RealField_class>self._parent).__prec <= 57: # max size of repr(float)
return numpy_double_interface
else:
return numpy_object_interface
So maybe numpy doesn't do something with __array_interface__ anymore in linspace?
Can you try numpy.linspace(0.0,9.0,float(10)), numpy.linspace(0.0,float(9),10), numpy.linspace(float(0),9.0,10) and various other combinations of float vs. Sage types in the arguments to narrow down what is triggering it?
comment:30 Changed 10 years ago by
I was writing something else:
And more subtly:
sage: a=numpy.linspace(0,9,10) sage: a array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9], dtype=object) sage: type(a[0]) <type 'float'> sage: type(a[9]) <type 'sage.rings.integer.Integer'>
So linspace and arange deal with sage input very differently. There should be some coercion but it is not working properly.
now for what you are asking:
sage: numpy.linspace(0.0,9.0,float(10))
array([0.000000000000000, 1.00000000000000, 2.00000000000000,
3.00000000000000, 4.00000000000000, 5.00000000000000,
6.00000000000000, 7.00000000000000, 8.00000000000000,
9.00000000000000], dtype=object)
sage: numpy.linspace(0.0,float(9),10)
array([0.000000000000000, 1.00000000000000, 2.00000000000000,
3.00000000000000, 4.00000000000000, 5.00000000000000,
6.00000000000000, 7.00000000000000, 8.00000000000000, 9.0], dtype=object)
sage: numpy.linspace(float(0),9.0,10)
array([0.000000000000000, 1.00000000000000, 2.00000000000000,
3.00000000000000, 4.00000000000000, 5.00000000000000,
6.00000000000000, 7.00000000000000, 8.00000000000000,
9.00000000000000], dtype=object)
I know about sage/rings/real_mpfr.pyx and the others that's how I got on thinking of coercion not working.
comment:31 Changed 10 years ago by
Weirder! The code for linspace uses arange inside. No change in the code between the 1.5.1 and 1.6.1 but the juicy important bit is as follow
step = (stop-start)/float(num)
y = _nx.arange(0, num) * step + start
step will become float (unless start or stop is complex or something). But the type of step and start will influence the final array.
comment:32 Changed 10 years ago by
Anyone wants to ask on a numpy mailing list if it is a sign of bug in numpy or something that is clearly under-documented?
comment:33 in reply to: ↑ 29 Changed 10 years ago by
Replying to jason:
So maybe numpy doesn't do something with
__array_interface__anymore in linspace?
I've been tinkering here and this is exactly what is happening. I deleted all the
property __array_interface__:
blocks that were added by 5081-numpy-types.patch and 6506-numpy-types.patch with no apparent effect on what numpy.linspace() returns with numpy-1.6.1 installed. Of course, the deleted blocks do affect what numpy.arange() and numpy.array() return:
sage: numpy.arange(10.0)
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=object)
sage: numpy.array([1.0, 1.1, 1.2]).dtype
dtype('object')
Also the example failure
from scipy import stats stats.uniform(0,15).ppf([0.5,0.7])
from ticket #5081 now again fails with numpy-1.6.x (s-o-g result):
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/storage/strogdon/gentoo/usr/local/portage/sage-on-gentoo/<ipython console> in <module>()
/storage/strogdon/gentoo/usr/lib/python2.7/site-packages/scipy/stats/distributions.pyc in ppf(self, q)
449
450 def ppf(self, q):
--> 451 return self.dist.ppf(q, *self.args, **self.kwds)
452
453 def isf(self, q):
/storage/strogdon/gentoo/usr/lib/python2.7/site-packages/scipy/stats/distributions.pyc in ppf(self, q, *args, **kwds)
1514 goodargs = argsreduce(cond, *((q,)+args+(scale,loc)))
1515 scale, loc, goodargs = goodargs[-2], goodargs[-1], goodargs[:-2]
-> 1516 place(output,cond,self._ppf(*goodargs)*scale + loc)
1517 if output.ndim == 0:
1518 return output[()]
/storage/strogdon/gentoo/usr/lib/python2.7/site-packages/numpy/lib/function_base.pyc in place(arr, mask, vals)
1333
1334 """
-> 1335 return _insert(arr, mask, vals)
1336
1337 def _nanop(op, fill, a, axis=None):
TypeError: array cannot be safely cast to required type
Starting with numpy-1.6.0 scalar upcasting rules were changed http://docs.scipy.org/doc/numpy/reference/ufuncs.html and I'm wondering if this change is interfering with how sage/cython is handling the python __array_interface__ extension, at least relative to numpy.linspace(), and perhaps other numpy functions that are pure python.
comment:34 Changed 10 years ago by
That's the best clue we had in ages Steve. I am not sure when I will be able to dig it further as we have an extra long week end here (Tuesday is a university holyday).
comment:35 Changed 10 years ago by
Here is a minimal example of the problem:
sage: import numpy sage: numpy.array(0.5) array(0.500000000000000, dtype=object) sage: numpy.array(0.5r) array(0.5)
where as before it was
sage: import numpy sage: numpy.array(0.5) array(0.5) sage: numpy.array(0.5r) array(0.5)
comment:36 Changed 10 years ago by
Also, we should incorporate #7831
comment:37 Changed 10 years ago by
Sure Mike do you have any idea on how to fix the coercion problem? The way it's going we'll update directly to 1.7.
comment:38 Changed 10 years ago by
Looking into this it seems that the issue is that in numpy/core/src/multiarray/ctors.c, PyArray_FromAny no longer calls _array_find_type which I believe was responsible for accessing the __array_interface__ attribute on our objects. I'm sending a message to numpy-discussion now.
comment:39 follow-up: ↓ 48 Changed 10 years ago by
comment:40 Changed 10 years ago by
- Keywords sd40.5 added
comment:41 Changed 10 years ago by
Thanks Mike, hopefully we'll get a meaningful answer.
comment:42 Changed 10 years ago by
Looks like 1.6.2 was released last week. I guess I'll update my spkg.
comment:43 Changed 10 years ago by
Looks like I found the commit where the change has been made https://github.com/numpy/numpy/commit/2635398db3f26529ce2aaea4028a8118844f3c48
If you look carefully the call to _array_find_type is still there but it has changed location. In 1.5.1 there was a sequence if(xx==NULL) then call _array_find_type else do some other stuff. In 1.6.1 the structure if(xx!=NULL) then do the other stuff else call _array_find_type.
I will try to write a patch to see if this the cause of our observed behaviour.
comment:44 Changed 10 years ago by
I wrote a small patch to just ignore any code other than _array_find_type in ctors.c no luck so I don't think that's the real source of the problem at all. I thought the presence of a requested_dtype passed to this function could have been the source of the problem but my patch excludes that possibility entirely. So we have to go back to looking.
comment:45 Changed 10 years ago by
Problem still present with numpy-1.6.2
comment:46 Changed 10 years ago by
- Description modified (diff)
New spkg for 1.6.2 rebased on 1.5.1.p1 as well.
comment:47 Changed 10 years ago by
- Summary changed from Update numpy to 1.6.1 to Update numpy to 1.6.2
Mike, I think if you want to ask a question that may have a chance to attract attention on numpy-discussion you should ask: Why do arange and linspace return different dtype object on similar inputs of the same type?
You can then expand on the fact that arange which is completely coded in C respect our request from array interface but linspace which call arange and then do something in python with the result doesn't respect it.
comment:48 in reply to: ↑ 39 Changed 10 years ago by
Replying to mhansen:
See http://mail.scipy.org/pipermail/numpy-discussion/2012-May/062560.html
I believe Mike is correct in that the commit referenced at the above numpy-discussion is the source of the "regression" in being unable to access the Sage-assigned __array_interface__ attribute. I cloned the numpy.git repository, set the master to the commit mentioned my Mike, and from that commit generated a numpy spkg which I installed. Sage manifested the undesirable "dtype=object" when tested. Now if I set the master to the commit just prior to the above mentioned commit and generated a new numpy spkg, then when Sage was tested I got the numpy-1.5.1 behavior. I'm just not sure what's going inside numpy?
comment:49 Changed 10 years ago by
OK so this is indeed the guilty commit. I tried something very simple to see if it was the small bit of code around _array_find_dtype and it was obviously insufficient to uncover the problem. It is quite a big commit,. I am thinking that considering we see an inconsistent behavior that we should just open a bug with numpy.
comment:50 Changed 10 years ago by
- Report Upstream changed from N/A to Reported upstream. No feedback yet.
Thanks Mike for the post on numpy's github tracker https://github.com/numpy/numpy/issues/291 I am not sure how they distribute the load between the github tracker and their trac server http://projects.scipy.org/numpy/report. I just hope they don't ignore it.
comment:51 Changed 10 years ago by
- Report Upstream changed from Reported upstream. No feedback yet. to Reported upstream. Developers acknowledge bug.
It looks like numpy may revert to the old behavior in 1.7.0. There is some traction upstream for this to happen.
comment:52 Changed 9 years ago by
- Report Upstream changed from Reported upstream. Developers acknowledge bug. to Fixed upstream, in a later stable release.
Looks like 1.7.0 will have a fix: https://github.com/numpy/numpy/issues/291. 1.7.0 should be out very soon now.
comment:53 Changed 9 years ago by
I know and I am waiting for that.
comment:54 Changed 9 years ago by
I have been toying with numpy 1.7.0b2 in sage-on-gentoo. I have added a small patch I discussed with Steve Trogdon that I will attach. In the meantime I have gobbles of the following:
sage -t -long -force_lib "devel/sage-main/sage/functions/log.py"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/functions/log.py", line 630:
sage: lambert_w(RDF(1))
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_12[4]>", line 1, in <module>
lambert_w(RDF(Integer(1)))###line 630:
sage: lambert_w(RDF(1))
File "/usr/lib64/python2.7/site-packages/sage/functions/log.py", line 560, in __call__
return BuiltinFunction.__call__(self, 0, args[0], **kwds)
File "function.pyx", line 437, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:5006)
File "/usr/lib64/python2.7/site-packages/sage/functions/log.py", line 607, in _eval_
return self._evalf_(n, z, parent=sage_structure_coerce_parent(z))
File "/usr/lib64/python2.7/site-packages/sage/functions/log.py", line 636, in _evalf_
return scipy.special.lambertw(z, n)
File "lambertw.pyx", line 342, in scipy.special.lambertw.lambertw (scipy/special/lambertw.c:1438)
TypeError: expected a readable buffer object
**********************************************************************
1 items had failures:
1 of 6 in __main__.example_12
***Test Failed*** 1 failures.
For whitespace errors, see the file /home/fbissey/.sage/tmp/log_1583.py
[2.6 s]
Anything using matplotlib spams you to death with these. Without the patch we get the following instead:
sage -t -long -force_lib "devel/sage-main/sage/functions/log.py"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/functions/log.py", line 630:
sage: lambert_w(RDF(1))
Exception raised:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_12[4]>", line 1, in <module>
lambert_w(RDF(Integer(1)))###line 630:
sage: lambert_w(RDF(1))
File "/usr/lib64/python2.7/site-packages/sage/functions/log.py", line 560, in __call__
return BuiltinFunction.__call__(self, 0, args[0], **kwds)
File "function.pyx", line 437, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:5006)
File "/usr/lib64/python2.7/site-packages/sage/functions/log.py", line 607, in _eval_
return self._evalf_(n, z, parent=sage_structure_coerce_parent(z))
File "/usr/lib64/python2.7/site-packages/sage/functions/log.py", line 636, in _evalf_
return scipy.special.lambertw(z, n)
File "lambertw.pyx", line 342, in scipy.special.lambertw.lambertw (scipy/special/lambertw.c:1438)
ValueError: Missing __array_interface__ shape
**********************************************************************
1 items had failures:
1 of 6 in __main__.example_12
***Test Failed*** 1 failures.
For whitespace errors, see the file /home/fbissey/.sage/tmp/log_15682.py
[2.6 s]
It seems that the patch introduce some expectation on the type that are not identified at the present time.
comment:55 Changed 9 years ago by
- Description modified (diff)
Here is a spkg so other people can enjoy. numpy 1.7 also has a new API. By default you can use the old one but we may want to move on to the new one eventually.
comment:56 Changed 9 years ago by
Steve pointed out to me that 1.7.0b2 didn't contain 2 little patches that we need and that will hopefully be in 1.7.0 final. The spkg is updated and there are way fewer doctest failures (matplotlib 1.2.0 will be a major source of doctest noise when we upgrade). So the current failures: 1) cosmetics
sage -t -long -force_lib "devel/sage-main/sage/matrix/matrix1.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/matrix/matrix1.pyx", line 494:
sage: sorted(numpy.typecodes.items())
Expected:
[('All', '?bhilqpBHILQPfdgFDGSUVO'), ('AllFloat', 'fdgFDG'), ('AllInteger', 'bBhHiIlLqQpP'), ('Character', 'c'), ('Complex', 'FDG'), ('Float', 'fdg'), ('Integer', 'bhilqp'), ('UnsignedInteger', 'BHILQP')]
Got:
[('All', '?bhilqpBHILQPefdgFDGSUVOMm'), ('AllFloat', 'efdgFDG'), ('AllInteger', 'bBhHiIlLqQpP'), ('Character', 'c'), ('Complex', 'FDG'), ('Datetime', 'Mm'), ('Float', 'efdg'), ('Integer', 'bhilqp'), ('UnsignedInteger', 'BHILQP')]
**********************************************************************
1 items had failures:
2) lots of divide by 0 warnings - including our old friend riemann.pyx encountered in a previous upgrade.
sage -t -long -force_lib "devel/sage-main/sage/calculus/riemann.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 111:
sage: m = Riemann_Map([f], [fprime], 0) # long time (4 sec)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 595:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 647:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 712:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 801:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 867:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 163:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 214:
sage: isinstance(Riemann_Map([f], [fprime], 0)._repr_(), str) # long time
Expected:
True
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
True
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 228:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 334:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 404:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 447:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 515:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/riemann.pyx", line 543:
sage: m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
sage -t -long -force_lib "devel/sage-main/sage/calculus/interpolators.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/interpolators.pyx", line 52:
sage: m = Riemann_Map([lambda x: ps.value(real(x))], [lambda x: ps.derivative(real(x))],0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/calculus/interpolators.pyx", line 183:
sage: m = Riemann_Map([lambda x: cs.value(real(x))], [lambda x: cs.derivative(real(x))], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
sage -t -long -force_lib "devel/sage-main/sage/symbolic/function.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/symbolic/function.pyx", line 627:
sage: csc(a)
Expected:
array([ inf, 1.18839511, 1.09975017, 7.0861674 , -1.32134871])
Got:
doctest:270: RuntimeWarning: divide by zero encountered in divide
array([ inf, 1.18839511, 1.09975017, 7.0861674 , -1.32134871])
**********************************************************************
sage -t -long -force_lib "devel/sage-main/sage/functions/hyperbolic.py"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/functions/hyperbolic.py", line 610:
sage: arcsech(a)
Expected:
array([ inf, 1.3169579, 0. ])
Got:
doctest:613: RuntimeWarning: divide by zero encountered in divide
array([ inf, 1.3169579, 0. ])
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/functions/hyperbolic.py", line 659:
sage: arccsch(a)
Expected:
array([ inf, 1.44363548, 0.88137359])
Got:
doctest:662: RuntimeWarning: divide by zero encountered in divide
array([ inf, 1.44363548, 0.88137359])
**********************************************************************
sage -t -long -force_lib "devel/sage-main/sage/modules/vector_double_dense.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/modules/vector_double_dense.pyx", line 712:
sage: w.norm(p=-1.6)
Expected:
0.0
Got:
doctest:1992: RuntimeWarning: divide by zero encountered in power
0.0
**********************************************************************
3) odd bits that needs looking at:
sage -t -long -force_lib "devel/sage-main/sage/rings/real_mpfr.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/rings/real_mpfr.pyx", line 1270:
sage: numpy.array([1.000000000000000000000000000000000000]).dtype
Expected:
dtype('object')
Got:
dtype('O')
**********************************************************************
and similarly in rings/integer.pyx, rings/complex_number.pyx and rings/rational.pyx.
4) ????
sage -t -long -force_lib "devel/sage-main/sage/plot/matrix_plot.py"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/plot/matrix_plot.py", line 438:
sage: matrix_plot([[sin(x), cos(x)], [1, 0]])
Expected:
Traceback (most recent call last):
...
ValueError: can not convert entries to floating point numbers
Got:
Traceback (most recent call last):
File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/usr/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_8[40]>", line 1, in <module>
matrix_plot([[sin(x), cos(x)], [Integer(1), Integer(0)]])###line 438:
sage: matrix_plot([[sin(x), cos(x)], [1, 0]])
File "/usr/lib64/python2.7/site-packages/sage/misc/decorators.py", line 456, in wrapper
return func(*args, **kwds)
File "/usr/lib64/python2.7/site-packages/sage/misc/decorators.py", line 456, in wrapper
return func(*args, **kwds)
File "/usr/lib64/python2.7/site-packages/sage/misc/decorators.py", line 534, in wrapper
return func(*args, **options)
File "/usr/lib64/python2.7/site-packages/sage/plot/matrix_plot.py", line 478, in matrix_plot
raise TypeError, "mat must be a Matrix or a two dimensional array"
TypeError: mat must be a Matrix or a two dimensional array
**********************************************************************
and a last one which I am not sure is numpy or sage-on-gentoo or a combination:
sage -t -long -force_lib "devel/sage-main/sage/matrix/matrix_double_dense.pyx"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/matrix/matrix_double_dense.pyx", line 4033:
sage: A.exp(order=2)
Expected:
[51.8888631634 74.6198348038]
[111.929752206 163.818615369]
Got:
[51.9689561987 74.736564567]
[112.104846851 164.073803049]
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.4.rc0/devel/sage-main/sage/matrix/matrix_double_dense.pyx", line 4052:
sage: A.exp(order=2)
Expected:
[-19.6130852955 + 12.5327938535*I 3.81156364812 + 28.891438232*I]
[-32.3827876895 + 21.9087393169*I 2.29565402142 + 44.915581543*I]
Got:
[-19.6146029538 + 12.5177438468*I 3.79496364496 + 28.8837993066*I]
[-32.3835809809 + 21.8842359579*I 2.26963300409 + 44.9013248277*I]
**********************************************************************
comment:57 Changed 9 years ago by
I forgot that one which is another that puzzle me
sage -t -long -force_lib "devel/sage-main/sage/plot/plot3d/implicit_surface.pyx"
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/plot/plot3d/implicit_surface.pyx", line 563:
sage: cube_marcher.y_vertices.any() or cube_marcher.z_vertices.any() # This shouldn't affect the Y or Z vertices.
Expected:
False
Got nothing
**********************************************************************
File "/usr/share/sage/devel/sage-main/sage/plot/plot3d/implicit_surface.pyx", line 458:
sage: cube_marcher.x_vertices.any() # This shouldn't affect the X vertices.
Expected:
False
Got nothing
**********************************************************************
And the matrix_double_dense.pyx filures seem to have disappeared after my last rebuild.
comment:58 Changed 9 years ago by
- Description modified (diff)
- Summary changed from Update numpy to 1.6.2 to Update numpy to 1.7.0
Steve pointed out that the array_interface-shape.patch patch is not necessary as the old numpy 1.5.1 behavior as been restored in full. It doesn't hurt either way but we don't need to add something we don't need.
comment:59 Changed 9 years ago by
After pulling the latest code from 1.7.x maintenance branch I can now see that the patch is still necessary. The pull that Steve had mentioned to me happened on the master not on the 1.7.x maintenance branch so it is not in the current 1.7 branch on github.
comment:60 Changed 9 years ago by
Sorry about the last message after more tests my conclusions are:
- the current tip of the 1.7.x maintenance branch as well as 1.7.0b2 need two extra patches (included in the current spkg)
- the array_interface-shape.patch patch is not necessary after all.
During my early tests I was expecting the two numpy patch to have been merged in the 1.7.x maintenance branch and that lead to confusion.
comment:61 Changed 9 years ago by
Does the rst patch above need to be updated to not talk about the 1.6 bug of converting Sage types?
comment:62 Changed 9 years ago by
At this stage there are no patch to add to sage the rst patch was something I started before I realized that we should do something else. I will put any patch I create for sage in the summary. There are a at least a couple of things we can fix now. The big question in my mind is: what do we do with all those divide by 0 warnings? Last time we had the author of riemann.pyx fix things.
comment:63 Changed 9 years ago by
I'll try taking a look at it. I don't have the most recent Sage yet, but I guess it's not a huge problem right now since numpy 1.7 hasn't been released yet either.
Changed 9 years ago by
tentative fix for matrix_plot.py - not sure it is the right thing to do as we have a change of behavior rather than output.
comment:64 Changed 9 years ago by
- Description modified (diff)
OK for you to have a look in the meantime here are the easy patches. Not completely sure it is the right thing to do in matrix_plot.py. While we are expecting an error we were expecting a different kind of error before.
comment:65 Changed 9 years ago by
I'm unable to get the extra patches included in numpy-1.7.0b2.spkg to apply. The file to be patched in the header of each patch is missing a leading "/" for
patch -p1 <"$patch"
to work. One possibility is to change "-p1" to "-p0" in spkg-install.
comment:66 Changed 9 years ago by
Thanks for testing I forgot to do that and I tested my first version of the spkg but not the second. I will fix this shortly.
comment:67 Changed 9 years ago by
spkg updated in the same location. I just edited the patch as we want the patching command to be uniform with the already existing cygwin patch which is still needed in my opinion.
comment:68 Changed 9 years ago by
Here is some background on the failing test
sage -t -long -force_lib "devel/sage-main/sage/plot/matrix_plot.py"
that's mentioned above above. The failure in Sage of
sage: matrix_plot([[sin(x), cos(x)], [1, 0]])
seems to be associated with the following sequence
sage: import numpy as np sage: np.asarray([[sin(x), cos(x)], [1, 0]], dtype = float)
With numpy-1.5.1 I get
sage: np.asarray([[sin(x), cos(x)], [1, 0]], dtype = float)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/storage/sage/sage-5.5.rc0/<ipython console> in <module>()
/storage/sage/sage-5.5.rc0/local/lib/python2.7/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
282
283 """
--> 284 return array(a, dtype, copy=False, order=order)
285
286 def asanyarray(a, dtype=None, order=None):
ValueError: setting an array element with a sequence.
and with numpy-1.7.0b2
sage: np.asarray([[sin(x), cos(x)], [1, 0]], dtype = float)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/storage/sage/sage-5.5.rc0/<ipython console> in <module>()
/storage/sage/sage-5.5.rc0/local/lib/python2.7/site-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
318
319 """
--> 320 return array(a, dtype, copy=False, order=order)
321
322 def asanyarray(a, dtype=None, order=None):
/storage/sage/sage-5.5.rc0/local/lib/python2.7/site-packages/sage/symbolic/expression.so in sage.symbolic.expression.Expression.__float__ (sage/symbolic/expression.cpp:7023)()
TypeError: unable to simplify to float approximation
So it would appear that the ValueError? when using numpy-1.5.1 is thrown my numpy but the TypeError? when using numpy-1.7.0b2 is thrown by Sage.
comment:69 Changed 9 years ago by
Ondřej just pulled a request with all we need in the 1.7 maintainance branch. Expect new spkg/ebuild early next week based on a git pull.
comment:70 Changed 9 years ago by
There is a now a rc1. I will update when I can.
comment:71 Changed 9 years ago by
- Description modified (diff)
spkg updated. I will soon run test against a vanilla 5.6.rc0 (as opposed to sage-on-gentoo).
comment:72 Changed 9 years ago by
Tests run here is the list of failing doctest
sage -t -long devel/sage-main/sage/calculus/interpolators.pyx # 2 doctests failed
sage -t -long devel/sage-main/sage/misc/inline_fortran.py # 4 doctests failed
sage -t -long devel/sage-main/sage/calculus/riemann.pyx # 14 doctests failed
sage -t -long devel/sage-main/sage/symbolic/function.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/plot/plot_field.py # 1 doctests failed
sage -t -long devel/sage-main/sage/plot/plot3d/implicit_surface.pyx # 2 doctests failed
sage -t -long devel/sage-main/sage/plot/matrix_plot.py # 1 doctests failed
sage -t -long devel/sage-main/sage/matrix/matrix1.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/modules/vector_double_dense.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/rings/complex_number.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/rings/real_mpfr.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/rings/rational.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/rings/integer.pyx # 1 doctests failed
sage -t -long devel/sage-main/sage/functions/hyperbolic.py # 2 doctests failed
There are a couple of new things I believe.
sage -t -long "devel/sage-main/sage/misc/inline_fortran.py"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.6.rc0/devel/sage-main/sage/misc/inline_fortran.py", line 29:
sage: fortran(_example)
Exception raised:
Traceback (most recent call last):
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_1[4]>", line 1, in <module>
fortran(_example)###line 29:
sage: fortran(_example)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/sage/misc/inline_fortran.py", line 21, in __call__
return self.eval(*args, **kwds)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/sage/misc/inline_fortran.py", line 84, in eval
f2py.compile(x, name, extra_args = extra_args, source_fn=fortran_file)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/numpy/f2py/__init__.py", line 40, in compile
s,o = exec_command(c)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/numpy/distutils/exec_command.py", line 197, in exec_command
if _with_python and (0 or sys.stdout.fileno()==-1):
AttributeError: _SpoofOut instance has no attribute 'fileno'
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.6.rc0/devel/sage-main/sage/misc/inline_fortran.py", line 32:
sage: fib(n,int(10))
Exception raised:
Traceback (most recent call last):
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_1[7]>", line 1, in <module>
fib(n,int(Integer(10)))###line 32:
sage: fib(n,int(10))
NameError: name 'fib' is not defined
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.6.rc0/devel/sage-main/sage/misc/inline_fortran.py", line 33:
sage: n
Expected:
array([ 0., 1., 1., 2., 3., 5., 8., 13., 21., 34.])
Got:
array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.6.rc0/devel/sage-main/sage/misc/inline_fortran.py", line 39:
sage: fortran.eval("SYNTAX ERROR !@#$")
Expected:
Traceback (most recent call last):
...
RuntimeError: failed to compile Fortran code:...
Got:
Traceback (most recent call last):
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_1[10]>", line 1, in <module>
fortran.eval("SYNTAX ERROR !@#$")###line 39:
sage: fortran.eval("SYNTAX ERROR !@#$")
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/sage/misc/inline_fortran.py", line 84, in eval
f2py.compile(x, name, extra_args = extra_args, source_fn=fortran_file)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/numpy/f2py/__init__.py", line 40, in compile
s,o = exec_command(c)
File "/home/work/fbissey/sandbox/sage-5.6.rc0/local/lib/python/site-packages/numpy/distutils/exec_command.py", line 197, in exec_command
if _with_python and (0 or sys.stdout.fileno()==-1):
AttributeError: _SpoofOut instance has no attribute 'fileno'
which also happens in 5.5 but I overlooked previously (because in my sage-on-gentoo this test always failed in parallel). This is therefore not caused by #13887. This is new I believe
sage -t -long "devel/sage-main/sage/plot/plot_field.py"
**********************************************************************
File "/home/work/fbissey/sandbox/sage-5.6.rc0/devel/sage-main/sage/plot/plot_field.py", line 49:
sage: r.yvec_array
Expected:
masked_array(data = [0.0 0.707106781187 0.707106781187 0.894427191],
mask = [False False False False],
fill_value = 1e+20)
Got:
masked_array(data = [0.0 0.7071067811865475 0.7071067811865475 0.8944271909999159],
mask = [False False False False],
fill_value = 1e+20)
<BLANKLINE>
Numerical noise and a blank line..
comment:73 Changed 9 years ago by
I just edited my last comment to repair some mismatching between sage version (I ran the tests on files from sage 5.6.rc0 with sage 5.5...)
comment:74 Changed 9 years ago by
Breakage of inline fortran reported upstream at https://github.com/numpy/numpy/issues/2915 as a recent commit look suspicious.
comment:75 Changed 9 years ago by
It's more than suspicious. Reverting one line of this patch in the installed file enables the doctest to complete without errors.
comment:76 Changed 9 years ago by
- Dependencies set to #12415
Numpy upstream pointed me to #12415 which does solve the problem. So depending on it.
comment:77 Changed 9 years ago by
- Dependencies changed from #12415 to #12415, #13992
comment:78 Changed 9 years ago by
Will do. It's not like we are any closer to final on that one.
comment:79 Changed 9 years ago by
Just a word of encouragement to those on this ticket - we've had a number of requests for pandas in Sage lately, but apparently this would be a prereq for easy_install-ing that, so we'd need it.
comment:80 Changed 9 years ago by
Thanks Karl-dieter. Actually if you could provide help with the implicit_surface.py test (cube marcher) or get someone who know the code on that case we possibly could wrap it. We would may be wait for an official 1.7.0 but we'd be ready.
comment:81 follow-up: ↓ 83 Changed 9 years ago by
I reviewed the marching cubes code when it went in. I think here is the problem. With the previous version of numpy:
import numpy numpy.array([[None,None],[None,None]]).any()
returns False, but now returns None.
comment:82 Changed 9 years ago by
I posted to numpy-discussion about it: http://mail.scipy.org/pipermail/numpy-discussion/2013-February/065377.html
comment:83 in reply to: ↑ 81 Changed 9 years ago by
Replying to jason:
I reviewed the marching cubes code when it went in. I think here is the problem. With the previous version of numpy:
import numpy numpy.array([[None,None],[None,None]]).any()returns False, but now returns None.
Yes that correlate with some experiments I have with the marching cube code but I hadn't isolated the problem that deeply. I'll look at the discussion but I am not sure that is a wrong behavior.
comment:84 Changed 9 years ago by
I can see the behavior choice going either way. The question for the numpy folks was if the change was intentional. If not, the follow-up question is whether it should be changed to be backwards compatible.
At any rate, I think fixing the doctests to return nothing is fine.
comment:85 Changed 9 years ago by
Before I do that can I ask about the purpose of test? The comment suggest that the calls have no adverse effects but this is not tested afterwards. Because I do not see the point of the test I would just remove it. Any comment on that?
Once this is sorted, we'll have to see how things combine with #12415. From what I understand from it and some other threads, some "divide by 0 warnings" will be with us forever. We probably should check when they appear but they cannot be hidden anymore (I think evanandel was playing hide and seek with them in riemann.pyx).
comment:86 Changed 9 years ago by
This references the failure in the test
sage -t -long -force_lib "devel/sage-main/sage/plot/matrix_plot.py"
I have some concerns with the following block of code in matrix_plot.py
try:
if sparse:
xy_data_array = mat
else:
xy_data_array = np.asarray(mat, dtype = float)
except TypeError:
raise TypeError, "mat must be a Matrix or a two dimensional array"
except ValueError:
raise ValueError, "can not convert entries to floating point numbers"
if len(xy_data_array.shape) < 2:
raise TypeError, "mat must be a Matrix or a two dimensional array"
The subject failure occurs when the object "mat" is not sparse. In which case, the errors are thrown by
np.asarray(mat, dtype = float)
I have not been able to pass any object "mat" to np.asarray() that returns a TypeError or ValueError
that remotely hints of the cause as being due to
"mat must be a Matrix or a two dimensional array"
I question whether this information can be returned by np.asarray() when used from Sage. I could be wrong though. The referenced failure does return a TypeError
See above. But this error reflects the inability to coerce the symbolic [sin(x), cos(x)] expressions to floats. It is possible to generate a ValueError when the object "mat" contains complex entries:
sage: import numpy
sage: numpy.asarray([[1+1j,2+3j],[1,1]], dtype=float)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-5-2bd7229efabd> in <module>()
----> 1 numpy.asarray([[Integer(1)+ComplexNumber(0, '1'),Integer(2)+ComplexNumber(0, '3')],[Integer(1),Integer(1)]], dtype=float)
/storage/sage/sage-5.7.beta3/local/lib/python2.7/site-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
318
319 """
--> 320 return array(a, dtype, copy=False, order=order)
321
322 def asanyarray(a, dtype=None, order=None):
ValueError: setting an array element with a sequence.
This is they type of error that's returned for the referenced test when using numpy-1.5.1. Sage interprets this error as
"can not convert entries to floating point numbers"
This seems to be a correct interpretation, at least from the python side with the numpy of this ticket:
>>> import numpy
>>> numpy.asarray([[1+1j,2+3j],[1,1]], dtype=float)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/storage/sage/sage-5.7.beta3/local/lib/python2.7/site-packages/numpy/core/numeric.py", line 320, in asarray
return array(a, dtype, copy=False, order=order)
TypeError: can't convert complex to float
although here there is a TypeError?
So it seems to me that regardless of whether there is a TypeError or ValueError the root cause is inability to coerce to a float. The question of whether "mat" is a Matrix or appropriate array is determined by the lines in the code that parse the shape attribute of an object that has been successfully converted to a ndarray. But this is probably minor compared to #12415. Perhaps Mike (mhansen) has some input?
comment:87 Changed 9 years ago by
That would be good to have a little bit more on this although it is indeed not as big as #12415.
In other news 1.7rc2 has been released less than an hour ago. I am trying to get it up as quickly as possible with the changes Jeroen wants.
comment:89 follow-up: ↓ 90 Changed 9 years ago by
When removing the Fortran stuff, you might need to add -fPIC as in #14022.
comment:90 in reply to: ↑ 89 Changed 9 years ago by
comment:91 Changed 9 years ago by
- Cc kini added
comment:92 Changed 9 years ago by
ok build and tested on top of 5.7.beta3 (but without #12415) and there are no new doctests failures. sage-on-gentoo and an older vanilla build got me a couple of latex output slightly different latex output. Very strange but probably steeming from something else.
comment:93 Changed 9 years ago by
OK I completely missed the release which happened 4 days after rc2. New spkg will be available soon.
comment:94 Changed 9 years ago by
- Cc Snark added
comment:95 Changed 9 years ago by
- Description modified (diff)
spkg updated to numpy 1.7.0 final which is identical to 1.7.0.rc2. We need patches for the divide by zero warnings and the marching cubes test.
comment:96 Changed 9 years ago by
rebased one patch against 5.9.beta0, more patches to follow.
comment:97 Changed 9 years ago by
- Description modified (diff)
- Status changed from new to needs_review
More patches. Up to review.
comment:98 Changed 9 years ago by
- Description modified (diff)
comment:99 Changed 9 years ago by
- Description modified (diff)
comment:100 follow-up: ↓ 105 Changed 9 years ago by
- Status changed from needs_review to needs_work
I assume this is caused by Numpy:
sage -t devel/sage/sage/calculus/riemann.pyx
**********************************************************************
File "devel/sage/sage/calculus/riemann.pyx", line 120, in sage.calculus.riemann.Riemann_Map
Failed example:
m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
comment:101 Changed 9 years ago by
It smells like one. There were 14 warnings and I patched 14 warnings in that file and that wasn't one of them. Where the heck does it come from?
comment:102 Changed 9 years ago by
I checked I don't have this one against 5.9.beta2 so it must be triggered by something else. I am OK to add a small patch for it but would very much like to know what ticket triggered it.
comment:103 Changed 9 years ago by
This was with a testing version of sage-5.9.beta4, so there were a bunch of tickets added.
comment:104 Changed 9 years ago by
Not really sure what would trigger something like that from that lot of tickets, my suspicions go with #13031, a change in the way the extension is built. I am assuming you are releasing 5.9.beta3 fairly soon so at least I'll be able to see if it appears on top of it.
comment:105 in reply to: ↑ 100 Changed 9 years ago by
Replying to jdemeyer:
I assume this is caused by Numpy:
sage -t devel/sage/sage/calculus/riemann.pyx ********************************************************************** File "devel/sage/sage/calculus/riemann.pyx", line 120, in sage.calculus.riemann.Riemann_Map Failed example: m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I) Expected nothing Got: doctest:1: RuntimeWarning: divide by zero encountered in divide **********************************************************************
I get this failure here too and I'm testing sage-5.9.beta1 and I don't get the failure associated with the patch for the test starting at line 109
sage: f(t) = e^(I*t) sage: fprime(t) = I*e^(I*t) sage: m = Riemann_Map([f], [fprime], 0) # long time (4 sec)
comment:106 follow-up: ↓ 107 Changed 9 years ago by
I find the whole thing very curious. If you look at the divide patch you'll notice that all the warning generated by Riemann_Map are of the kind Riemann_Map([f], [fprime], 0) - it may be possible for other calls to generate a divide by zero warning but the ending "0" argument in Riemann_Map will automatically give you one. We could check with Eviatar but I am fairly sure that it is a case were you will have one, no matter what. So you both see one that doesn't have a "0", that's possible but that you, Steve, don't see one at line 109 rings alarm bells. Would it be the machine on which you experience https://github.com/cschwan/sage-on-gentoo/issues/208 by any chance?
comment:107 in reply to: ↑ 106 Changed 9 years ago by
Replying to fbissey:
Steve, don't see one at line 109 rings alarm bells. Would it be the machine on which you experience https://github.com/cschwan/sage-on-gentoo/issues/208 by any chance?
No, this is another machine. Vanilla Sage was built in prefix on this machine. And even on the machine associate with https://github.com/cschwan/sage-on-gentoo/issues/208, vanilla Sage does not exhibit the odd behavior of that issue. I get with the prefix-built Sage
$ ./sage -t devel/sage/sage/calculus/riemann.pyx
Running doctests with ID 2013-04-03-16-48-12-0f9b93ae.
Doctesting 1 file.
sage -t devel/sage/sage/calculus/riemann.pyx
**********************************************************************
File "devel/sage/sage/calculus/riemann.pyx", line 119, in sage.calculus.riemann.Riemann_Map
Failed example:
m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "devel/sage/sage/calculus/riemann.pyx", line 163, in sage.calculus.riemann.Riemann_Map.__init__
Failed example:
m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
File "devel/sage/sage/calculus/riemann.pyx", line 228, in sage.calculus.riemann.Riemann_Map._generate_theta_array
Failed example:
m = Riemann_Map([f], [fprime], 0)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
There is no failure associated with the test starting at line 109. However on a fresh start of Sage I get for the test starting at line 109:
sage: f(t) = e^(I*t) sage: fprime(t) = I*e^(I*t) sage: m = Riemann_Map([f], [fprime], 0) # long time (4 sec) /storage/sage/sage-5.9.beta1/local/bin/sage-ipython:1: RuntimeWarning: divide by zero encountered in divide #!/usr/bin/env python
but if I run the commands again without restarting Sage I get no RuntimeWarning. Is this to be expected? Is something being cached somewhere? And oddly with your patch
trac_11334-divide.patch I only get
$ ./sage -t devel/sage/sage/calculus/riemann.pyx Running doctests with ID 2013-04-03-17-14-01-9d1e8739.
Doctesting 1 file.
sage -t devel/sage/sage/calculus/riemann.pyx
**********************************************************************
File "devel/sage/sage/calculus/riemann.pyx", line 120, in sage.calculus.riemann.Riemann_Map
Failed example:
m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I)
Expected nothing
Got:
doctest:1: RuntimeWarning: divide by zero encountered in divide
**********************************************************************
comment:108 follow-up: ↓ 109 Changed 9 years ago by
OK, I think not getting the warning again is expected but I am not sure about the details of why. After my patch the test from line 119 becomes line 120 so that bit is consistent.
What happens if you run the example of line 119/120 in sage? With or without the patch?
comment:109 in reply to: ↑ 108 Changed 9 years ago by
Replying to fbissey:
OK, I think not getting the warning again is expected but I am not sure about the details of why. After my patch the test from line 119 becomes line 120 so that bit is consistent.
What happens if you run the example of line 119/120 in sage? With or without the patch?
It fails, at least on the first try it does
sage: f(t) = e^(I*t) sage: fprime(t) = I*e^(I*t) sage: hf(t) = 0.5*e^(-I*t) sage: hfprime(t) = 0.5*-I*e^(-I*t) sage: m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I) /storage/sage/sage-5.9.beta1/local/bin/sage-ipython:1: RuntimeWarning: divide by zero encountered in divide #!/usr/bin/env python
comment:110 Changed 9 years ago by
OK I am building 5.9.beta2 from scratch on power7 and will try that here as well to see for myself. But 5.9.beta2 s-o-g doesn't suffer from this here.
comment:111 Changed 9 years ago by
Oh wait in sage-on-gentoo I get it when I run the example manually but not when I run the doctest, this is getting weirder.
comment:112 Changed 9 years ago by
OK,
---------------------------------------------------------------------- | Sage Version 5.9.beta2, Release Date: 2013-03-28 | | Type "notebook()" for the browser-based notebook interface. | | Type "help()" for help. | ---------------------------------------------------------------------- ********************************************************************** * * * Warning: this is a prerelease version, and it may be unstable. * * * ********************************************************************** sage: f(t) = e^(I*t) sage: fprime(t) = I*e^(I*t) sage: hf(t) = 0.5*e^(-I*t) sage: hfprime(t) = 0.5*-I*e^(-I*t) sage: m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I) /usr/bin/sage-ipython:1: RuntimeWarning: divide by zero encountered in divide #!/usr/bin/env python
but
---------------------------------------------------------------------- | Sage Version 5.9.beta2, Release Date: 2013-03-28 | | Type "notebook()" for the browser-based notebook interface. | | Type "help()" for help. | ---------------------------------------------------------------------- ********************************************************************** * * * Warning: this is a prerelease version, and it may be unstable. * * * ********************************************************************** sage: f(t) = e^(I*t) sage: fprime(t) = I*e^(I*t) sage: m = Riemann_Map([f], [fprime], 0) /usr/bin/sage-ipython:1: RuntimeWarning: divide by zero encountered in divide #!/usr/bin/env python sage: hf(t) = 0.5*e^(-I*t) sage: hfprime(t) = 0.5*-I*e^(-I*t) sage: m = Riemann_Map([f, hf], [fprime, hfprime], 0.5 + 0.5*I)
so in some conditions numpy warnings are not reset between these two adjacent tests and some other it is. I was going to ask if they could have been run out of order but I think the ticket for that is not happening anytime soon. But that would have explained why you see a failure in 119 but not 109.
comment:113 Changed 9 years ago by
I get it in my build of 5.9.beta2 on power7, I cannot understand how the thing happens.
comment:114 Changed 9 years ago by
I've now tested 5.9.beta3 on two different machines and I get the following numpy-related warnings which I also saw with 5.9.beta2.
./sage -t --long devel/sage/sage/matrix/matrix2.pyx devel/sage/sage/symbolic/expression.pyx
Running doctests with ID 2013-04-04-12-42-24-9b2fa346.
Doctesting 2 files.
sage -t --long devel/sage/sage/matrix/matrix2.pyx
**********************************************************************
File "devel/sage/sage/matrix/matrix2.pyx", line 11476, in sage.matrix.matrix2.Matrix.diagonal.plot
Failed example:
A.plot()
Expected nothing
Got:
doctest:146: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:146: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
doctest:3: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:3: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
doctest:148: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:148: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
doctest:150: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:150: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
<BLANKLINE>
**********************************************************************
1 item had failures:
1 of 7 in sage.matrix.matrix2.Matrix.diagonal.plot
[2000 tests, 1 failure, 38.68 s]
sage -t --long devel/sage/sage/symbolic/expression.pyx
**********************************************************************
File "devel/sage/sage/symbolic/expression.pyx", line 8747, in sage.symbolic.expression.Expression.find_root
Failed example:
f.find_root(-2, 3)
Expected:
1.0
Got:
doctest:146: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:146: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
doctest:3: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:3: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
doctest:148: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:148: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
doctest:150: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
doctest:150: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
1.0
**********************************************************************
1 item had failures:
1 of 29 in sage.symbolic.expression.Expression.find_root
[2041 tests, 1 failure, 36.88 s]
----------------------------------------------------------------------
sage -t --long devel/sage/sage/matrix/matrix2.pyx # 1 doctest failed
sage -t --long devel/sage/sage/symbolic/expression.pyx # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 77.1 seconds
cpu time: 72.3 seconds
cumulative wall time: 75.6 seconds
What is really odd is that the seemingly offending lines when run from the Sage prompt give _no_ warnings. And I don't see where in the .pyx files RuntimeWarnings are explicitly turned on or off. It is as if warnings are only being turned on during doctesting or perhaps something else is up.
comment:115 Changed 9 years ago by
Did install sage and then upgrade numpy? I think it means that some components depending on numpy haven't been rebuilt. I have done my builds by replacing the numpy spkg in spkg/standard and then building from scratch to avoid that kind of problems.
comment:116 Changed 9 years ago by
Yes. I see. That would explain it. Normally I do an install from scratch but didn't here. I'll try again.
comment:117 Changed 9 years ago by
OK, so the above RuntimeWarnings are bogus and don't appear when things are built correctly (from scratch). Curiously,
sage -t --long devel/sage/sage/calculus/riemann.pyx
does not fail when testing using 'make ptestlong', but does yield the 'divide by zero' warning when run individually.
comment:118 Changed 9 years ago by
Yes very curious things like that happens. I am preparing a new set of patch with a different strategy to deal with riemann.pyx. I already uploaded one but I need to make a new patch for other divide warnings not coming from riemann.pyx - people willingly wanting to see 1/0 results.
comment:119 Changed 9 years ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
OK new patch set. If that doesn't work for you we have a serious problem. Back to need review.
comment:120 Changed 9 years ago by
- Reviewers set to Steven Trogdon
- Status changed from needs_review to positive_review
It's odd that one has to resort to such "trickery" to ignore warnings. However, it is not without warrant since the ignore mechanism was already operative at a certain level in riemann.pyx. This works for me here - all "divide by zero" warnings have been removed from output and everything survives make ptestlong. Positive review.
comment:121 Changed 9 years ago by
I suspect the strange behavior we have seen may be a symptom of something deeper on the way the doctesting framework deals with warnings. Anyway it should be sorted by that last couple of patch for this case.
comment:122 Changed 9 years ago by
- Milestone changed from sage-5.9 to sage-5.10
comment:123 Changed 9 years ago by
Changed 9 years ago by
comment:124 Changed 9 years ago by
- Status changed from positive_review to needs_work
11334_noise2.patch (numerical noise on OS X 10.8) needs review.
comment:125 Changed 9 years ago by
- Status changed from needs_work to needs_review
comment:126 Changed 9 years ago by
- Description modified (diff)
- Reviewers changed from Steven Trogdon to Steven Trogdon, Jeroen Demeyer
- Status changed from needs_review to positive_review
When I first saw that I was wondering if it was a combination with python 2.7.4 but I haven't conducted any test on OS X 10.8. The patch looks fine to me and is compatible with the rest. I am taking it as a reviewer patch and putting this back to positive review.
comment:127 Changed 9 years ago by
- Merged in set to sage-5.10.beta0
- Resolution set to fixed
- Status changed from positive_review to closed
CCing fbissey since he's been extremely helpful in previous numpy upgrades.