Opened 11 years ago
Closed 11 years ago
#5081 closed enhancement (fixed)
[with patch, positive review] Make numpy play nice with Sage types
Reported by: | jason | Owned by: | was |
---|---|---|---|
Priority: | major | Milestone: | sage-4.1.1 |
Component: | linear algebra | Keywords: | |
Cc: | Merged in: | Sage 4.1.1.rc0 | |
Authors: | Robert Bradshaw | Reviewers: | Jason Grout |
Report Upstream: | Work issues: | ||
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description
We should make numpy understand Sage float and complex types, at least the RDF and CDF types. See the following thread on the numpy list.
http://thread.gmane.org/gmane.comp.python.numeric.general/25251/focus=25273
Attachments (1)
Change History (18)
comment:1 Changed 11 years ago by
comment:2 Changed 11 years ago by
an example of failure:
from scipy import stats stats.uniform(0,15).ppf([0.5,0.7])
Traceback (most recent call last):
File "<stdin>", line 1, in <module> File "/home/Users/fcoelho/.sage/sage_notebook/worksheets/admin/0/code/19.py", line 7, in <module>
stats.uniform(_sage_const_0 ,_sage_const_15 ).ppf([_sage_const_0p5 ,_sage_const_0p7 ])
File "/usr/local/sage-3.4/local/lib/python2.5/site-packages/SQLAlchemy-0.4.6-py2.5.egg/", line 1, in <module>
File "/usr/local/sage-3.4/local/lib/python2.5/site-packages/scipy/stats/distributions.py", line 112, in ppf
return self.dist.ppf(q,*self.args,self.kwds)
File "/usr/local/sage-3.4/local/lib/python2.5/site-packages/scipy/stats/distributions.py", line 563, in ppf
place(output,cond,self._ppf(*goodargs)*scale + loc)
File "/usr/local/sage-3.4/local/lib/python2.5/site-packages/numpy/lib/function_base.py", line 1357, in place
return _insert(arr, mask, vals)
TypeError?: array cannot be safely cast to required type
comment:3 Changed 11 years ago by
- Summary changed from Make numpy play nice with Sage types to [with patch, needs review] Make numpy play nice with Sage types
Solves the integer and real types. Complex types are still an issue which will need to be resolved by fixing NumPy?.
Changed 11 years ago by
comment:4 Changed 11 years ago by
Brilliant! I'm upgrading now so I can test this!
comment:5 Changed 11 years ago by
Is there a reason that you didn't use the same trick on ZZ that you used on RealField? (with regards to not needing to specify whether dtype=object or not)?
comment:6 Changed 11 years ago by
What does the '=' in the type descriptor mean? I couldn't find that documented here: http://docs.scipy.org/doc/numpy/reference/arrays.interface.html
comment:7 Changed 11 years ago by
Ah, I found the answer to my last question here: http://docs.scipy.org/doc/numpy/reference/arrays.dtypes.html#specifying-and-constructing-data-types
'=' means hardware-default endian behavior.
comment:8 Changed 11 years ago by
Things seem to work fine if I put the logic that the patch has in real_mpfr.pyx also in integer.pyx. Is there a reason the logic wasn't included dealing with switching from hardware types to object types?
comment:9 Changed 11 years ago by
http://bugs.python.org/issue1675423 may resolve the complex case.
As for not using the precision trick for ZZ, I was thinking that a single ring should map to a single dtype. A sticky issue is that once you switch from ints to objects, you have different overflow semantics.
I guess numpy folks are already used to this
>>> import numpy >>> numpy.array(2**20) array(1048576) >>> numpy.array(2**20)**2 0 >>> numpy.array(2**40) array(1099511627776L, dtype=int64) >>> numpy.array(2**40)**2 0 >>> numpy.array(2**80) array(1208925819614629174706176L, dtype=object) >>> numpy.array(2**80)**2 1461501637330902918203684832716283019655932542976L
We should maybe have three cases--long (sys.maxint), int64, and object. Of course, the former would only be for 32-bit machines.
- Robert
comment:10 Changed 11 years ago by
- Summary changed from [with patch, needs review] Make numpy play nice with Sage types to [with patch, positive review] Make numpy play nice with Sage types
This looks good, is well documented, and is a long time coming. I'm giving it a positive review.
Regarding:
We should maybe have three cases--long (sys.maxint), int64, and object. Of course, the former would only be for 32-bit machines.
Yes, that would be good. But I don't consider it a show stopper for this patch. It could easily be added on another ticket later.
comment:11 Changed 11 years ago by
- Summary changed from [with patch, positive review] Make numpy play nice with Sage types to [with patch, needs work] Make numpy play nice with Sage types
I'm getting the following test failures:
sage -t -long devel/sage-exp/sage/matrix/matrix1.pyx ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/matrix/matrix1.pyx", line 428: sage: a.numpy() Expected: array([[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]], dtype=object) Got: array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) ********************************************************************** 1 items had failures: 1 of 9 in __main__.example_13 ***Test Failed*** 1 failures. For whitespace errors, see the file /scratch/mvngu/release/sage-4.1.1/tmp/.doctest_matrix1.py [3.5 s] <SNIP> sage -t -long devel/sage-exp/sage/rings/number_field/totallyreal_rel.py ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/rings/number_field/totallyreal_rel.py", line 156: sage: T = sage.rings.number_field.totallyreal_rel.tr_data_rel(F, 2, 2000) Exception raised: Traceback (most recent call last): File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_3[3]>", line 1, in <module> T = sage.rings.number_field.totallyreal_rel.tr_data_rel(F, Integer(2), Integer(2000))###line 156: sage: T = sage.rings.number_field.totallyreal_rel.tr_data_rel(F, 2, 2000) File "/scratch/mvngu/release/sage-4.1.1/local/lib/python/site-packages/sage/rings/number_field/totallyreal_rel.py", line 200, in __init__ adj = pari(Q).qflll()[self.d] File "gen.pyx", line 9174, in sage.libs.pari.gen._pari_trap (sage/libs/pari/gen.c:44241) PariError: unexpected character (2) ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/rings/number_field/totallyreal_rel.py", line 569: sage: enumerate_totallyreal_fields_rel(F, 2, 2000) Exception raised: Traceback (most recent call last): File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_5[4]>", line 1, in <module> enumerate_totallyreal_fields_rel(F, Integer(2), Integer(2000))###line 569: sage: enumerate_totallyreal_fields_rel(F, 2, 2000) File "/scratch/mvngu/release/sage-4.1.1/local/lib/python/site-packages/sage/rings/number_field/totallyreal_rel.py", line 647, in enumerate_totallyreal_fields_rel T = tr_data_rel(F,m,B,a) File "/scratch/mvngu/release/sage-4.1.1/local/lib/python/site-packages/sage/rings/number_field/totallyreal_rel.py", line 200, in __init__ adj = pari(Q).qflll()[self.d] File "gen.pyx", line 9174, in sage.libs.pari.gen._pari_trap (sage/libs/pari/gen.c:44241) PariError: unexpected character (2) ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/rings/number_field/totallyreal_rel.py", line 578: sage: ls = enumerate_totallyreal_fields_rel(F, 2, 10^4) Exception raised: Traceback (most recent call last): File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_5[6]>", line 1, in <module> ls = enumerate_totallyreal_fields_rel(F, Integer(2), Integer(10)**Integer(4))###line 578: sage: ls = enumerate_totallyreal_fields_rel(F, 2, 10^4) File "/scratch/mvngu/release/sage-4.1.1/local/lib/python/site-packages/sage/rings/number_field/totallyreal_rel.py", line 647, in enumerate_totallyreal_fields_rel T = tr_data_rel(F,m,B,a) File "/scratch/mvngu/release/sage-4.1.1/local/lib/python/site-packages/sage/rings/number_field/totallyreal_rel.py", line 200, in __init__ adj = pari(Q).qflll()[self.d] File "gen.pyx", line 9174, in sage.libs.pari.gen._pari_trap (sage/libs/pari/gen.c:44241) PariError: unexpected character (2) ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/rings/number_field/totallyreal_rel.py", line 579: sage: print "ignore this"; ls # random (the second factor is platform-dependent) Exception raised: Traceback (most recent call last): File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_5[7]>", line 1, in <module> print "ignore this"; ls # random (the second factor is platform-dependent)###line 579: sage: print "ignore this"; ls # random (the second factor is platform-dependent) NameError: name 'ls' is not defined ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/rings/number_field/totallyreal_rel.py", line 601: sage: [ f[0] for f in ls ] Exception raised: Traceback (most recent call last): File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_5[8]>", line 1, in <module> [ f[Integer(0)] for f in ls ]###line 601: sage: [ f[0] for f in ls ] NameError: name 'ls' is not defined ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/sage/rings/number_field/totallyreal_rel.py", line 604: sage: [NumberField(ZZx(x[1]), 't').is_galois() for x in ls] Exception raised: Traceback (most recent call last): File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mvngu/release/sage-4.1.1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_5[9]>", line 1, in <module> [NumberField(ZZx(x[Integer(1)]), 't').is_galois() for x in ls]###line 604: sage: [NumberField(ZZx(x[1]), 't').is_galois() for x in ls] NameError: name 'ls' is not defined ********************************************************************** 2 items had failures: 1 of 4 in __main__.example_3 5 of 12 in __main__.example_5 ***Test Failed*** 6 failures. For whitespace errors, see the file /scratch/mvngu/release/sage-4.1.1/tmp/.doctest_totallyreal_rel.py [2.2 s] <SNIP> sage -t -long devel/sage-exp/doc/en/numerical_sage/numpy.rst ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/doc/en/numerical_sage/numpy.rst", line 19: sage: l Expected: array([1, 2, 3], dtype=object) Got: array([1, 2, 3]) ********************************************************************** File "/scratch/mvngu/release/sage-4.1.1/devel/sage-exp/doc/en/numerical_sage/numpy.rst", line 54: sage: l Expected: array([1.00000000000000, 2.00000000000000, 3.00000000000000], dtype=object) Got: array([ 1., 2., 3.]) ********************************************************************** 1 items had failures: 2 of 52 in __main__.example_0 ***Test Failed*** 2 failures. For whitespace errors, see the file /scratch/mvngu/release/sage-4.1.1/tmp/.doctest_numpy.py [1.8 s] <SNIP> ---------------------------------------------------------------------- The following tests failed: sage -t -long devel/sage-exp/sage/matrix/matrix1.pyx # 1 doctests failed sage -t -long devel/sage-exp/sage/rings/number_field/totallyreal_rel.py # 6 doctests failed sage -t -long devel/sage-exp/doc/en/numerical_sage/numpy.rst # 2 doctests failed ---------------------------------------------------------------------- Total time for all tests: 525.6 seconds
comment:12 Changed 11 years ago by
This ticket is related to #6497.
comment:13 Changed 11 years ago by
See #6506 for a patch which fixes most of these issues and discusses another.
comment:14 Changed 11 years ago by
- Summary changed from [with patch, needs work] Make numpy play nice with Sage types to [with patch, needs review] Make numpy play nice with Sage types
All issues fixed in #6506, review of this patch should happen with review of the other.
comment:15 follow-up: ↓ 16 Changed 11 years ago by
- Reviewers set to Jason Grout
- Summary changed from [with patch, needs review] Make numpy play nice with Sage types to [with patch, positive review] Make numpy play nice with Sage types
Positive review, since #6506 is positive review and fixes all issues listed here.
mvngu: are you a reviewer as well?
comment:16 in reply to: ↑ 15 Changed 11 years ago by
Replying to jason:
mvngu: are you a reviewer as well?
No, not really. I often like to apply patches and doctest them to see them break. Seeing doctests break is more fun than seeing all tests passed :-)
comment:17 Changed 11 years ago by
- Merged in set to Sage 4.1.1.rc0
- Resolution set to fixed
- Status changed from new to closed