Opened 10 years ago
Last modified 8 years ago
#11334 closed task
Update numpy to 1.6.1 — at Version 46
Reported by: | jason | Owned by: | tbd |
---|---|---|---|
Priority: | major | Milestone: | sage-5.10 |
Component: | packages: standard | Keywords: | sd40.5 |
Cc: | fbissey, kini, Snark | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | Fixed upstream, in a later stable release. | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
Numpy-1.6.2 is now available.
New spkg: http://spkg-upload.googlecode.com/files/numpy-1.6.2.spkg
Change History (47)
comment:1 Changed 10 years ago by
- Cc fbissey added
comment:2 Changed 10 years ago by
Thx. it looks like numpy has been bumped to 1.6 in Gentoo so I can do some quick compatibility tests. It looks like a flag for gfortran has been added in the ebuild to help with the compiler autodetection madness it may be worth having it sage side too.
Do you know if a scipy release is to follow, if not it would be best to revbump scipy to make sure it is rebuilt after numpy.
Other matter there has been noise about lapack in numpy do we want to disable it, make it optional or keep the status quo?
comment:3 Changed 10 years ago by
I don't remember seeing anything on the scipy lists about an imminent scipy release.
comment:4 Changed 10 years ago by
just in:
sage -t -long -force_lib "devel/sage/doc/en/numerical_sage/numpy.rst" ********************************************************************** File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 97: sage: 2.5*l Expected: array([ 0. , 2.5, 5. , 7.5, 10. , 12.5, 15. , 17.5, 20. , 22.5]) Got: array([0.000000000000000, 2.50000000000000, 5.00000000000000, 7.50000000000000, 10.0000000000000, 12.5000000000000, 15.0000000000000, 17.5000000000000, 20.0000000000000, 22.5000000000000], dtype=object) ********************************************************************** File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 289: sage: xx Expected: array([[ 0. , 0.25, 0.5 , 0.75, 1. ], [ 0. , 0.25, 0.5 , 0.75, 1. ], [ 0. , 0.25, 0.5 , 0.75, 1. ], [ 0. , 0.25, 0.5 , 0.75, 1. ], [ 0. , 0.25, 0.5 , 0.75, 1. ]]) Got: array([[0.000000000000000, 0.250000000000000, 0.500000000000000, 0.750000000000000, 1.00000000000000], [0.000000000000000, 0.250000000000000, 0.500000000000000, 0.750000000000000, 1.00000000000000], [0.000000000000000, 0.250000000000000, 0.500000000000000, 0.750000000000000, 1.00000000000000], [0.000000000000000, 0.250000000000000, 0.500000000000000, 0.750000000000000, 1.00000000000000], [0.000000000000000, 0.250000000000000, 0.500000000000000, 0.750000000000000, 1.00000000000000]], dtype=object) ********************************************************************** File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 295: sage: yy Expected: array([[ 0. , 0. , 0. , 0. , 0. ], [ 0.25, 0.25, 0.25, 0.25, 0.25], [ 0.5 , 0.5 , 0.5 , 0.5 , 0.5 ], [ 0.75, 0.75, 0.75, 0.75, 0.75], [ 1. , 1. , 1. , 1. , 1. ]]) Got: array([[0.000000000000000, 0.000000000000000, 0.000000000000000, 0.000000000000000, 0.000000000000000], [0.250000000000000, 0.250000000000000, 0.250000000000000, 0.250000000000000, 0.250000000000000], [0.500000000000000, 0.500000000000000, 0.500000000000000, 0.500000000000000, 0.500000000000000], [0.750000000000000, 0.750000000000000, 0.750000000000000, 0.750000000000000, 0.750000000000000], [1.00000000000000, 1.00000000000000, 1.00000000000000, 1.00000000000000, 1.00000000000000]], dtype=object) ********************************************************************** File "/usr/share/sage/devel/sage/doc/en/numerical_sage/numpy.rst", line 301: sage: f(xx,yy) Expected: array([[ 0. , 0.0625, 0.25 , 0.5625, 1. ], [ 0.0625, 0.125 , 0.3125, 0.625 , 1.0625], [ 0.25 , 0.3125, 0.5 , 0.8125, 1.25 ], [ 0.5625, 0.625 , 0.8125, 1.125 , 1.5625], [ 1. , 1.0625, 1.25 , 1.5625, 2. ]]) Got: array([[0.000000000000000, 0.0625000000000000, 0.250000000000000, 0.562500000000000, 1.00000000000000], [0.0625000000000000, 0.125000000000000, 0.312500000000000, 0.625000000000000, 1.06250000000000], [0.250000000000000, 0.312500000000000, 0.500000000000000, 0.812500000000000, 1.25000000000000], [0.562500000000000, 0.625000000000000, 0.812500000000000, 1.12500000000000, 1.56250000000000], [1.00000000000000, 1.06250000000000, 1.25000000000000, 1.56250000000000, 2.00000000000000]], dtype=object) ********************************************************************** 2 items had failures: 1 of 55 in __main__.example_0 3 of 13 in __main__.example_1 ***Test Failed*** 4 failures.
there may be more
comment:5 Changed 10 years ago by
- Description modified (diff)
I have cut a new spkg for 1.6.0, I decided to go for plain update for now.
comment:6 Changed 10 years ago by
seems my home install has problems. Probably stuff messed around between python 2.6 and 2.7 because I am working on #9958. Another install didn't report any problems. So I will check on a vanilla sage but I think we won't need to do any patching.
comment:7 Changed 10 years ago by
For some reason the other install didn't upgrade that's why I didn't see anything on the second machine. So right now it looks like we have the following to correct:
sage -t -long -force_lib devel/sage/doc/en/numerical_sage/numpy.rst # 4 doctests failed sage -t -long -force_lib devel/sage/sage/symbolic/function.pyx # 1 doctests failed sage -t -long -force_lib devel/sage/sage/plot/plot_field.py # 1 doctests failed sage -t -long -force_lib devel/sage/sage/plot/matrix_plot.py # 1 doctests failed sage -t -long -force_lib devel/sage/sage/matrix/matrix1.pyx # 1 doctests failed sage -t -long -force_lib devel/sage/sage/numerical/optimize.py # 1 doctests failed sage -t -long -force_lib devel/sage/sage/rings/integer.pyx # 1 doctests failed sage -t -long -force_lib devel/sage/sage/functions/hyperbolic.py # 2 doctests failed sage -t -long -force_lib devel/sage/sage/functions/trig.py # 3 doctests failed sage -t -long -force_lib devel/sage/sage/functions/other.py # 2 doctests failed
The first one is already in the ticket, then we have
sage -t -long -force_lib "devel/sage/sage/symbolic/function.pyx" ********************************************************************** File "/usr/share/sage/devel/sage/sage/symbolic/function.pyx", line 596: sage: csc(a) Expected: array([ inf, 1.18839511, 1.09975017, 7.0861674 , -1.32134871]) Got: doctest:270: RuntimeWarning: divide by zero encountered in divide array([ inf, 1.18839511, 1.09975017, 7.0861674 , -1.32134871]) **********************************************************************
sage -t -long -force_lib "devel/sage/sage/plot/plot_field.py" ********************************************************************** File "/usr/share/sage/devel/sage/sage/plot/plot_field.py", line 216: sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2)).show(aspect_ratio=1) Expected nothing Got: doctest:622: RuntimeWarning: divide by zero encountered in divide doctest:623: RuntimeWarning: invalid value encountered in multiply doctest:624: RuntimeWarning: invalid value encountered in multiply **********************************************************************
sage -t -long -force_lib "devel/sage/sage/plot/matrix_plot.py" ********************************************************************** File "/usr/share/sage/devel/sage/sage/plot/matrix_plot.py", line 383: sage: matrix_plot([[sin(x), cos(x)], [1, 0]]) Expected: Traceback (most recent call last): ... ValueError: can not convert entries to floating point numbers Got: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_8[36]>", line 1, in <module> matrix_plot([[sin(x), cos(x)], [Integer(1), Integer(0)]])###line 383: sage: matrix_plot([[sin(x), cos(x)], [1, 0]]) File "/usr/lib/python2.7/site-packages/sage/misc/decorators.py", line 381, in wrapper return func(*args, **kwds) File "/usr/lib/python2.7/site-packages/sage/misc/decorators.py", line 432, in wrapper return func(*args, **options) File "/usr/lib/python2.7/site-packages/sage/plot/matrix_plot.py", line 423, in matrix_plot raise TypeError, "mat must be a Matrix or a two dimensional array" TypeError: mat must be a Matrix or a two dimensional array **********************************************************************
sage -t -long -force_lib "devel/sage/sage/matrix/matrix1.pyx" ********************************************************************** File "/usr/share/sage/devel/sage/sage/matrix/matrix1.pyx", line 468: sage: sorted(numpy.typecodes.items()) Expected: [('All', '?bhilqpBHILQPfdgFDGSUVO'), ('AllFloat', 'fdgFDG'), ('AllInteger', 'bBhHiIlLqQpP'), ('Character', 'c'), ('Complex', 'FDG'), ('Float', 'fdg'), ('Integer', 'bhilqp'), ('UnsignedInteger', 'BHILQP')] Got: [('All', '?bhilqpBHILQPefdgFDGSUVOMm'), ('AllFloat', 'efdgFDG'), ('AllInteger', 'bBhHiIlLqQpP'), ('Character', 'c'), ('Complex', 'FDG'), ('Datetime', 'Mm'), ('Float', 'efdg'), ('Integer', 'bhilqp'), ('UnsignedInteger', 'BHILQP')] **********************************************************************
sage -t -long -force_lib "devel/sage/sage/numerical/optimize.py" ********************************************************************** File "/usr/share/sage/devel/sage/sage/numerical/optimize.py", line 162: sage: find_minimum_on_interval(f, 1, 5, tol=1e-3) Expected: (-3.28837136189098..., 3.42575079030572...) Got: (-3.28837136189, 3.4257507903057229)
sage -t -long -force_lib "devel/sage/sage/rings/integer.pyx" ********************************************************************** File "/usr/share/sage/devel/sage/sage/rings/integer.pyx", line 4738: sage: numpy.array(2**40).dtype Expected: dtype('int64') Got: dtype('object') **********************************************************************
sage -t -long -force_lib "devel/sage/sage/functions/hyperbolic.py" ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/hyperbolic.py", line 609: sage: arcsech(a) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_23[4]>", line 1, in <module> arcsech(a)###line 609: sage: arcsech(a) File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794) File "/usr/lib/python2.7/site-packages/sage/functions/hyperbolic.py", line 612, in _eval_numpy_ return arccosh(1.0 / x) ZeroDivisionError: float division by zero ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/hyperbolic.py", line 658: sage: arccsch(a) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_26[4]>", line 1, in <module> arccsch(a)###line 658: sage: arccsch(a) File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794) File "/usr/lib/python2.7/site-packages/sage/functions/hyperbolic.py", line 661, in _eval_numpy_ return arcsinh(1.0 / x) ZeroDivisionError: float division by zero **********************************************************************
sage -t -long -force_lib "devel/sage/sage/functions/trig.py" ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/trig.py", line 827: sage: atan2(a, b) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_34[16]>", line 1, in <module> atan2(a, b)###line 827: sage: atan2(a, b) File "function.pyx", line 718, in sage.symbolic.function.GinacFunction.__call__ (sage/symbolic/function.cpp:6475) File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794) File "function.pyx", line 604, in sage.symbolic.function.Function._eval_numpy_ (sage/symbolic/function.cpp:5568) NotImplementedError: The Function arctan2 does not support numpy arrays as arguments ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/trig.py", line 830: sage: atan2(1,a) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_34[17]>", line 1, in <module> atan2(Integer(1),a)###line 830: sage: atan2(1,a) File "function.pyx", line 718, in sage.symbolic.function.GinacFunction.__call__ (sage/symbolic/function.cpp:6475) File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794) File "function.pyx", line 604, in sage.symbolic.function.Function._eval_numpy_ (sage/symbolic/function.cpp:5568) NotImplementedError: The Function arctan2 does not support numpy arrays as arguments ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/trig.py", line 833: sage: atan2(a, 1) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_34[18]>", line 1, in <module> atan2(a, Integer(1))###line 833: sage: atan2(a, 1) File "function.pyx", line 718, in sage.symbolic.function.GinacFunction.__call__ (sage/symbolic/function.cpp:6475) File "function.pyx", line 357, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:3794) File "function.pyx", line 604, in sage.symbolic.function.Function._eval_numpy_ (sage/symbolic/function.cpp:5568) NotImplementedError: The Function arctan2 does not support numpy arrays as arguments **********************************************************************
sage -t -long -force_lib "devel/sage/sage/functions/other.py" ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/other.py", line 211: sage: ceil(a) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_5[20]>", line 1, in <module> ceil(a)###line 211: sage: ceil(a) File "/usr/lib/python2.7/site-packages/sage/functions/other.py", line 251, in __call__ return numpy.ceil(x) AttributeError: ceil ********************************************************************** File "/usr/share/sage/devel/sage/sage/functions/other.py", line 366: sage: floor(a) Exception raised: Traceback (most recent call last): File "/usr/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/usr/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/usr/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_9[15]>", line 1, in <module> floor(a)###line 366: sage: floor(a) File "/usr/lib/python2.7/site-packages/sage/functions/other.py", line 405, in __call__ return numpy.floor(x) AttributeError: floor **********************************************************************
The last three are particularly concerning, could you check what's up with these in numpy?
comment:8 Changed 10 years ago by
This seems to have installed fine on OS X 10.4 PPC. I'm running doctests now (which will take a LONG time) but I did already see the matrix/matrix1.pyx one while testing something else.
comment:9 Changed 10 years ago by
That's good that it at least install. On gentoo there are interactions problem between numpy and scipy to get cblas/blas/lapack right in scipy. I should check if the resulting scipy in vanilla sage suffers from the same problem. http://bugs.gentoo.org/show_bug.cgi?id=371099
The last few failing tests are show stoppers as far as I am concerned.
comment:10 Changed 10 years ago by
Okay, I got pretty much the same failures on a long run. One weird thing... did we change find_minimum_on_interval
away from SciPy recently? This is the full output I get on the (otherwise same as your failure) test for that.
sage: find_minimum_on_interval(f, 1, 5, tol=1e-3) Expected: (-3.28837136189098..., 3.42575079030572...) Got: (-3.28837136189, 3.4257507903057229) GLPK Simplex Optimizer, v4.44 6 rows, 3 columns, 8 non-zeros Preprocessing... 2 rows, 2 columns, 4 non-zeros Scaling... A: min|aij| = 2.400e+01 max|aij| = 5.000e+01 ratio = 2.083e+00 GM: min|aij| = 8.128e-01 max|aij| = 1.230e+00 ratio = 1.514e+00 EQ: min|aij| = 6.606e-01 max|aij| = 1.000e+00 ratio = 1.514e+00 Constructing initial basis... Size of triangular part = 2 * 0: obj = -5.100000000e+01 infeas = 0.000e+00 (0) * 1: obj = -5.225000000e+01 infeas = 0.000e+00 (0) OPTIMAL SOLUTION FOUND
comment:11 Changed 10 years ago by
1.6.1 is now available. I'll have a look sometimes this week.
comment:12 Changed 10 years ago by
Looks like more of the same but maybe a little bit more. There are a few other things that may interfere on my setup and I had to solve that bit: http://bugs.gentoo.org/show_bug.cgi?id=371099 before doing the trial.
comment:13 Changed 10 years ago by
Once I had cleared a few blas problem that I had while solving the gentoo bug. I get exactly the same results. I also get the output from Karl in optimize. This is a verbose output of glpk which get called at some point. You only get to see the such verbose output if there was a failure in the test but not necessarily the test for which glpk is called funnily enough.
comment:14 Changed 10 years ago by
Should we try to move forward on this again? I will have a go at trying it with 4.8.alpha2. Any ideas about the current failures?
comment:15 Changed 10 years ago by
Ok this is still broken in 4.8.alpha2. Some food for thought
sage ---------------------------------------------------------------------- | Sage Version 4.8.alpha2, Release Date: 2011-11-19 | | Type notebook() for the GUI, and license() for information. | ---------------------------------------------------------------------- ********************************************************************** * * * Warning: this is a prerelease version, and it may be unstable. * * * ********************************************************************** sage: import numpy sage: a = numpy.linspace(0,2,6) sage: a array([0.0, 0.4, 0.8, 1.2, 1.6, 2], dtype=object) sage: ceil(a) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /home/fbissey/<ipython console> in <module>() /usr/lib64/python2.7/site-packages/sage/functions/other.py in __call__(self, x, maximum_bits) 250 elif type(x).__module__ == 'numpy': 251 import numpy --> 252 return numpy.ceil(x) 253 254 x_original = x AttributeError: ceil
with numpy-1.5.1 we just have
sage: a array([0.0, 0.4, 0.8, 1.2, 1.6, 2])
I suspect that for this one (functions/other.py) and functions/trig.py the pynac interface may need to be touched. All stuff that takes an numpy array and return a numpy array may have trouble.
comment:16 Changed 10 years ago by
Just to add a bit of info. Same python and ipython
ipython Python 2.7.2 (default, Oct 28 2011, 11:36:16) Type "copyright", "credits" or "license" for more information. IPython 0.10.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]: import numpy In [2]: a = numpy.linspace(0,2,6) In [3]: a Out[3]: array([ 0. , 0.4, 0.8, 1.2, 1.6, 2. ]) In [4]: ceil(a) --------------------------------------------------------------------------- NameError Traceback (most recent call last) /home/fbissey/<ipython console> in <module>() NameError: name 'ceil' is not defined In [5]: numpy.ceil(a) Out[5]: array([ 0., 1., 1., 2., 2., 2.])
So it works in ipython but we don't have the dtype=object attached to the array. Anyone knows what that means actually?
comment:17 Changed 10 years ago by
numpy-1.5.1
sage: numpy.dtype(a[1]) dtype('float64')
numpy-1.6.1
sage: numpy.dtype(a[1]) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/fbissey/<ipython console> in <module>() TypeError: data type not understood
comment:18 Changed 10 years ago by
sage: type(a[0]) <type 'float'> sage: type(a[0:5]) <type 'numpy.ndarray'> sage: type(a[2]) <type 'float'> sage: type(a[3]) <type 'float'> sage: type(a[4]) <type 'float'> sage: type(a[5]) <type 'sage.rings.integer.Integer'>
For some reason numpy.linspace return an array with sage objects rather than just floats. I am guessing it may be deeper than that. Numpy probably uses sage types rather than converting to numpy types. The types of a[0] to a[4] is strange, is there a sage type for floats? I mean I tried numpy.linspace(0,2.0,6) and I got <type 'sage.rings.real_mpfr.RealNumber?'> for a[0] to a[4] and <type 'sage.rings.real_mpfr.RealLiteral?'> for a[5].
Doing a cast numpy.array(a,dtype=float) returns something that will work.
comment:19 Changed 10 years ago by
- Description modified (diff)
- Summary changed from Update numpy to 1.6 to Update numpy to 1.6.1
comment:20 Changed 10 years ago by
I'll be starting to make patches to correct the doctests and document the new behavior in numpy 1.6.x. Some documentation may need updating.
comment:21 Changed 10 years ago by
Actually I am a bit confused by one of the tests:
``plot_slope_field`` takes a function of two variables xvar and yvar (for instance, if the variables are `x` and `y`, take `f(x,y)`), and at representative points `(x_i,y_i)` between xmin, xmax, and ymin, ymax respectively, plots a line with slope `f(x_i,y_i)` (see below). ``plot_slope_field(f, (xvar, xmin, xmax), (yvar, ymin, ymax))`` EXAMPLES: A logistic function modeling population growth:: sage: x,y = var('x y') sage: capacity = 3 # thousand sage: growth_rate = 0.7 # population increases by 70% per unit of time sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2))
That function is independent of x, the next example after this depend on both x and y. I am suspecting the messages we get in this particular case is quite justified and it is surprising we didn't get any before. I am not sure what the correct formula should be.
comment:22 Changed 10 years ago by
I am guessing it should have been
growth_rate*(1-x/capacity)*y
y being the time and x the population in thousands. That satisfy a simple dimensional analysis.
comment:23 Changed 10 years ago by
No, wasn't it right before? That's the standard logistic population model, where y is the population. See http://en.wikipedia.org/wiki/Logistic_function#In_ecology:_modeling_population_growth, for example.
comment:24 Changed 10 years ago by
sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2)) }}}} > That function is independent of x, the next example after this depend on both x and y. I am suspecting the messages we get in This should be ok. The point is that the slope dy/dx is a function of y only; this can be completely symbolically solved using separation of variables to get the [http://en.wikipedia.org/wiki/Logistic_function#In_ecology:_modeling_population_growth logistic function]. > this particular case is quite justified and it is surprising we didn't get any before. I am not sure what the correct formula should be. If this is the division by zero stuff, it's the whole business with the arrowheads having zero length. See #11208. I think Jason has a fix for this, actually, but I don't recall the whole story of what happened once it was reported to mpl.
comment:25 Changed 10 years ago by
Sorry for that noise. Jason is right.
sage: plot_slope_field(growth_rate*(1-y/capacity)*y, (x,0,5), (y,0,capacity*2))
That function is independent of x, the next example after this depend on both x and y. I am suspecting the messages we get in
This should be ok. The point is that the slope dy/dx is a function of y only; this can be completely symbolically solved using separation of variables to get the logistic function.
this particular case is quite justified and it is surprising we didn't get any before. I am not sure what the correct formula should be.
If this is the division by zero stuff, it's the whole business with the arrowheads having zero length. See #11208. I think Jason has a fix for this, actually, but I don't recall the whole story of what happened once it was reported to mpl.
comment:26 Changed 10 years ago by
OK I was interpreting the formula another way because I wasn't sure what it is actually computing. Seems like #11208 numpy, mpl and scipy all go hands in hands so if something start happening with one the others may get it too.
Overall the change of behavior of numpy around type conversion is quite annoying.
comment:27 Changed 10 years ago by
It strikes me that the problem may be more an issue of the coercion to numpy types not working properly with numpy-1.6. I will look into that.
comment:28 Changed 10 years ago by
Any ideas why stuff like this is happening with numpy-1.6.x
sage: numpy.arange(10.0) array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.]) sage: numpy.linspace(0.0,9.0,10) array([0.000000000000000, 1.00000000000000, 2.00000000000000, 3.00000000000000, 4.00000000000000, 5.00000000000000, 6.00000000000000, 7.00000000000000, 8.00000000000000, 9.00000000000000], dtype=object)
comment:29 follow-up: ↓ 33 Changed 10 years ago by
It looks like it didn't automatically convert from RR to float, like it should. That's what these two lines should do in sage/rings/real_mpfr.pyx
cdef object numpy_double_interface = {'typestr': '=f8'} cdef object numpy_object_interface = {'typestr': '|O'}
and this property in that same file:
property __array_interface__: def __get__(self): """ Used for NumPy conversion. EXAMPLES:: sage: import numpy sage: numpy.arange(10.0) array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.]) sage: numpy.array([1.0, 1.1, 1.2]).dtype dtype('float64') sage: numpy.array([1.000000000000000000000000000000000000]).dtype dtype('object') """ if (<RealField_class>self._parent).__prec <= 57: # max size of repr(float) return numpy_double_interface else: return numpy_object_interface
So maybe numpy doesn't do something with __array_interface__
anymore in linspace?
Can you try numpy.linspace(0.0,9.0,float(10))
, numpy.linspace(0.0,float(9),10)
, numpy.linspace(float(0),9.0,10)
and various other combinations of float vs. Sage types in the arguments to narrow down what is triggering it?
comment:30 Changed 10 years ago by
I was writing something else:
And more subtly:
sage: a=numpy.linspace(0,9,10) sage: a array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9], dtype=object) sage: type(a[0]) <type 'float'> sage: type(a[9]) <type 'sage.rings.integer.Integer'>
So linspace and arange deal with sage input very differently. There should be some coercion but it is not working properly.
now for what you are asking:
sage: numpy.linspace(0.0,9.0,float(10)) array([0.000000000000000, 1.00000000000000, 2.00000000000000, 3.00000000000000, 4.00000000000000, 5.00000000000000, 6.00000000000000, 7.00000000000000, 8.00000000000000, 9.00000000000000], dtype=object) sage: numpy.linspace(0.0,float(9),10) array([0.000000000000000, 1.00000000000000, 2.00000000000000, 3.00000000000000, 4.00000000000000, 5.00000000000000, 6.00000000000000, 7.00000000000000, 8.00000000000000, 9.0], dtype=object) sage: numpy.linspace(float(0),9.0,10) array([0.000000000000000, 1.00000000000000, 2.00000000000000, 3.00000000000000, 4.00000000000000, 5.00000000000000, 6.00000000000000, 7.00000000000000, 8.00000000000000, 9.00000000000000], dtype=object)
I know about sage/rings/real_mpfr.pyx and the others that's how I got on thinking of coercion not working.
comment:31 Changed 10 years ago by
Weirder! The code for linspace uses arange inside. No change in the code between the 1.5.1 and 1.6.1 but the juicy important bit is as follow
step = (stop-start)/float(num) y = _nx.arange(0, num) * step + start
step will become float (unless start or stop is complex or something). But the type of step and start will influence the final array.
comment:32 Changed 9 years ago by
Anyone wants to ask on a numpy mailing list if it is a sign of bug in numpy or something that is clearly under-documented?
comment:33 in reply to: ↑ 29 Changed 9 years ago by
Replying to jason:
So maybe numpy doesn't do something with
__array_interface__
anymore in linspace?
I've been tinkering here and this is exactly what is happening. I deleted all the
property __array_interface__:
blocks that were added by 5081-numpy-types.patch and 6506-numpy-types.patch with no apparent effect on what numpy.linspace() returns with numpy-1.6.1 installed. Of course, the deleted blocks do affect what numpy.arange() and numpy.array() return:
sage: numpy.arange(10.0) array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=object) sage: numpy.array([1.0, 1.1, 1.2]).dtype dtype('object')
Also the example failure
from scipy import stats stats.uniform(0,15).ppf([0.5,0.7])
from ticket #5081 now again fails with numpy-1.6.x (s-o-g result):
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) /storage/strogdon/gentoo/usr/local/portage/sage-on-gentoo/<ipython console> in <module>() /storage/strogdon/gentoo/usr/lib/python2.7/site-packages/scipy/stats/distributions.pyc in ppf(self, q) 449 450 def ppf(self, q): --> 451 return self.dist.ppf(q, *self.args, **self.kwds) 452 453 def isf(self, q): /storage/strogdon/gentoo/usr/lib/python2.7/site-packages/scipy/stats/distributions.pyc in ppf(self, q, *args, **kwds) 1514 goodargs = argsreduce(cond, *((q,)+args+(scale,loc))) 1515 scale, loc, goodargs = goodargs[-2], goodargs[-1], goodargs[:-2] -> 1516 place(output,cond,self._ppf(*goodargs)*scale + loc) 1517 if output.ndim == 0: 1518 return output[()] /storage/strogdon/gentoo/usr/lib/python2.7/site-packages/numpy/lib/function_base.pyc in place(arr, mask, vals) 1333 1334 """ -> 1335 return _insert(arr, mask, vals) 1336 1337 def _nanop(op, fill, a, axis=None): TypeError: array cannot be safely cast to required type
Starting with numpy-1.6.0 scalar upcasting rules were changed http://docs.scipy.org/doc/numpy/reference/ufuncs.html and I'm wondering if this change is interfering with how sage/cython is handling the python __array_interface__ extension, at least relative to numpy.linspace(), and perhaps other numpy functions that are pure python.
comment:34 Changed 9 years ago by
That's the best clue we had in ages Steve. I am not sure when I will be able to dig it further as we have an extra long week end here (Tuesday is a university holyday).
comment:35 Changed 9 years ago by
Here is a minimal example of the problem:
sage: import numpy sage: numpy.array(0.5) array(0.500000000000000, dtype=object) sage: numpy.array(0.5r) array(0.5)
where as before it was
sage: import numpy sage: numpy.array(0.5) array(0.5) sage: numpy.array(0.5r) array(0.5)
comment:36 Changed 9 years ago by
Also, we should incorporate #7831
comment:37 Changed 9 years ago by
Sure Mike do you have any idea on how to fix the coercion problem? The way it's going we'll update directly to 1.7.
comment:38 Changed 9 years ago by
Looking into this it seems that the issue is that in numpy/core/src/multiarray/ctors.c
, PyArray_FromAny
no longer calls _array_find_type
which I believe was responsible for accessing the __array_interface__
attribute on our objects. I'm sending a message to numpy-discussion now.
comment:39 Changed 9 years ago by
comment:40 Changed 9 years ago by
- Keywords sd40.5 added
comment:41 Changed 9 years ago by
Thanks Mike, hopefully we'll get a meaningful answer.
comment:42 Changed 9 years ago by
Looks like 1.6.2 was released last week. I guess I'll update my spkg.
comment:43 Changed 9 years ago by
Looks like I found the commit where the change has been made https://github.com/numpy/numpy/commit/2635398db3f26529ce2aaea4028a8118844f3c48
If you look carefully the call to _array_find_type is still there but it has changed location. In 1.5.1 there was a sequence if(xx==NULL) then call _array_find_type else do some other stuff. In 1.6.1 the structure if(xx!=NULL) then do the other stuff else call _array_find_type.
I will try to write a patch to see if this the cause of our observed behaviour.
comment:44 Changed 9 years ago by
I wrote a small patch to just ignore any code other than _array_find_type in ctors.c no luck so I don't think that's the real source of the problem at all. I thought the presence of a requested_dtype passed to this function could have been the source of the problem but my patch excludes that possibility entirely. So we have to go back to looking.
comment:45 Changed 9 years ago by
Problem still present with numpy-1.6.2
comment:46 Changed 9 years ago by
- Description modified (diff)
New spkg for 1.6.2 rebased on 1.5.1.p1 as well.
CCing fbissey since he's been extremely helpful in previous numpy upgrades.