Opened 11 years ago
Closed 11 years ago
#3324 closed enhancement (fixed)
[with patch, positive review] Matrix_mod2_dense to/from PNG routines
Reported by: | malb | Owned by: | was |
---|---|---|---|
Priority: | major | Milestone: | sage-3.1.2 |
Component: | linear algebra | Keywords: | gd, linear algebra, gf(2), editor_malb |
Cc: | Merged in: | ||
Authors: | Reviewers: | ||
Report Upstream: | Work issues: | ||
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description
1-bit PNGs are a pretty sweet storage format for dense GF(2) matrices
- they are intuitive (you can even look at them)
- they are small (since the data is compressed for you)
- other people wrote fast C libraries to deal with them.
So this patch adds to_png
and from_png
functions to sage.matrix.matrix_mod2_dense
and uses those to implement pickling/unpickling.
Attachments (1)
Change History (22)
comment:1 Changed 11 years ago by
comment:2 Changed 11 years ago by
- Summary changed from [with patch, needs review] Matrix_mod2_dense to/from PNG routines to [with patch, needs work] Matrix_mod2_dense to/from PNG routines
REFEREE REPORT:
I read the code and it looks fine. However it doesn't build/work on any of my test platforms, and that's not so good.
comment:3 Changed 11 years ago by
The code depends on: #3204 (which is in 3.0.3.alpha0) so it won't work against 3.0.2. Sorry for not pointing that out earlier. I don't know about the OSX failure though.
comment:4 Changed 11 years ago by
I've updated the patch to avoid temporary files.
comment:5 Changed 11 years ago by
- Summary changed from [with patch, needs work] Matrix_mod2_dense to/from PNG routines to [with patch, needs review] Matrix_mod2_dense to/from PNG routines
comment:6 Changed 11 years ago by
- Keywords editor_malb added
comment:7 Changed 11 years ago by
wstein, can you review my update until 6/18?
comment:8 Changed 11 years ago by
- Summary changed from [with patch, needs review] Matrix_mod2_dense to/from PNG routines to [with patch, needs work] Matrix_mod2_dense to/from PNG routines
Doesn't work on OS X:
ImportError: dlopen(/Users/was/s/local/lib/python2.5/site-packages/sage/matrix/matrix_mod2_dense.so, 2): Symbol not found: _png_check_sig Referenced from: /Users/was/s/local/lib//libgd.2.dylib Expected in: flat namespace
comment:9 Changed 11 years ago by
The attached patch addresses this issue.
comment:10 Changed 11 years ago by
- Summary changed from [with patch, needs work] Matrix_mod2_dense to/from PNG routines to [with patch, needs review] Matrix_mod2_dense to/from PNG routines
comment:11 Changed 11 years ago by
This does not fix the problem for me:
/Users/was/s/local/lib/python2.5/site-packages/sage/matrix/matrix_space.py in <module>() 36 import matrix_modn_sparse 37 ---> 38 import matrix_mod2_dense 39 #import matrix_mod2_sparse 40 ImportError: dlopen(/Users/was/s/local/lib/python2.5/site-packages/sage/matrix/matrix_mod2_dense.so, 2): Symbol not found: _png_check_sig Referenced from: /Users/was/s/local/lib//libgd.2.dylib Expected in: flat namespace sage:
comment:12 Changed 11 years ago by
- Summary changed from [with patch, needs review] Matrix_mod2_dense to/from PNG routines to [with patch, needs more work] Matrix_mod2_dense to/from PNG routines
comment:13 Changed 11 years ago by
Are you 100% sure that you rebuilt with the patch applied? Note that I just replaced the patch and thus the download manager might have called in somewhat else when downloading. I'm just asking because it works for me on OSX. I'll try on a different machine though.
comment:14 Changed 11 years ago by
I cleanly applied the patch on bsd.math.washington.edu to a 100% fresh clean build of sage-3.0.3. It does not work. You should build sage on bsd, and get the patch to work there:
Loading SAGE library. Current Mercurial branch is: review Exception exceptions.ImportError: 'dlopen(/Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/matrix/matrix_mod2_dense.so, 2): Symbol not found: _png_check_sig\n Referenced from: /Users/was/build/sage-3.0.3/local/lib//libgd.2.dylib\n Expected in: flat namespace\n' in 'sage.rings.polynomial.polynomial_element.Polynomial_generic_dense.__normalize' ignored Exception exceptions.ImportError: 'dlopen(/Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/matrix/matrix_mod2_dense.so, 2): Symbol not found: _png_check_sig\n Referenced from: /Users/was/build/sage-3.0.3/local/lib//libgd.2.dylib\n Expected in: flat namespace\n' in 'sage.rings.polynomial.polynomial_element.Polynomial_generic_dense.__normalize' ignored --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /Users/was/build/sage-3.0.3/local/bin/<ipython console> in <module>() /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/all_cmdline.py in <module>() 12 try: 13 ---> 14 from sage.all import * 15 from sage.calculus.predefined import x 16 preparser(on=True) /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/all.py in <module>() 64 get_sigs() 65 ---> 66 from sage.rings.all import * 67 from sage.matrix.all import * 68 /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/rings/all.py in <module>() 92 93 # Algebraic numbers ---> 94 from qqbar import (AlgebraicRealField, is_AlgebraicRealField, AA, 95 AlgebraicReal, is_AlgebraicReal, 96 AlgebraicField, is_AlgebraicField, QQbar, /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/rings/qqbar.py in <module>() 4734 4735 -> 4736 RR_1_10 = RR(ZZ(1)/10) 4737 QQ_0 = QQ(0) 4738 QQ_1 = QQ(1) /Users/was/build/sage-3.0.3/local/bin/element.pyx in sage.structure.element.RingElement.__div__ (sage/structure/element.c:9074)() /Users/was/build/sage-3.0.3/local/bin/coerce.pyx in sage.structure.coerce.CoercionModel_cache_maps.bin_op_c (sage/structure/coerce.c:5088)() /Users/was/build/sage-3.0.3/local/bin/coerce.pyx in sage.structure.coerce.CoercionModel_cache_maps.canonical_coercion_c (sage/structure/coerce.c:5462)() /Users/was/build/sage-3.0.3/local/bin/coerce.pyx in sage.structure.coerce.CoercionModel_cache_maps.coercion_maps_c (sage/structure/coerce.c:6459)() /Users/was/build/sage-3.0.3/local/bin/coerce.pyx in sage.structure.coerce.CoercionModel_cache_maps.discover_coercion_c (sage/structure/coerce.c:7479)() /Users/was/build/sage-3.0.3/local/bin/parent.pyx in sage.structure.parent.Parent.coerce_map_from_c (sage/structure/parent.c:1061)() /Users/was/build/sage-3.0.3/local/bin/integer_ring.pyx in sage.rings.integer_ring.IntegerRing_class.coerce_map_from_c_impl (sage/rings/integer_ring.c:5224)() /Users/was/build/sage-3.0.3/local/bin/integer.pyx in sage.rings.integer.int_to_Z.__init__ (sage/rings/integer.c:20488)() /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/categories/homset.py in Hom(X, Y, cat) 57 if hasattr(X, '_Hom_'): 58 return X._Hom_(Y, cat) ---> 59 import category_types 60 global _cache 61 key = (X,Y,cat) /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/categories/category_types.py in <module>() 24 from category import * 25 import sage.rings.all ---> 26 import sage.algebras.all 27 28 /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/algebras/all.py in <module>() 26 27 from free_algebra import FreeAlgebra, is_FreeAlgebra ---> 28 from free_algebra_quotient import FreeAlgebraQuotient 29 from quaternion_algebra import (QuaternionAlgebra, QuaternionAlgebraWithInnerProduct, 30 QuaternionAlgebraWithGramMatrix, QuaternionAlgebraWithDiscriminants, /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/algebras/free_algebra_quotient.py in <module>() 34 35 from sage.rings.integer import Integer ---> 36 from sage.modules.free_module import FreeModule 37 from sage.monoids.free_monoid import FreeMonoid 38 from sage.monoids.free_monoid_element import FreeMonoidElement /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/modules/free_module.py in <module>() 133 import module 134 --> 135 import sage.matrix.matrix_space 136 137 import sage.misc.latex as latex /Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/matrix/matrix_space.py in <module>() 36 import matrix_modn_sparse 37 ---> 38 import matrix_mod2_dense 39 #import matrix_mod2_sparse 40 ImportError: dlopen(/Users/was/build/sage-3.0.3/local/lib/python2.5/site-packages/sage/matrix/matrix_mod2_dense.so, 2): Symbol not found: _png_check_sig Referenced from: /Users/was/build/sage-3.0.3/local/lib//libgd.2.dylib Expected in: flat namespace
comment:15 Changed 11 years ago by
State of affairs for editorial meeting 26/06/08
I do have a working copy on bsd and I'm trying to find out what exactly made it work for me.
comment:16 Changed 11 years ago by
Okay, I found it:
if [ `uname` = "Darwin" -a "$SAGE64" = "yes" ]; then echo "Keeping 64 bit OSX libpng.dylib around" else # There is a weird problem on Darwin where most programs # will crash if they see the libpng built as part of this # package instead of the system-wide apple libpng. So # we delete it. # NOTE -- we *only* delete the dylib's. Apps that need libpng can still link it in statically. rm -f "$SAGE_LOCAL"/lib/libpng*dylib rm -f "$SAGE_LOCAL"/lib/libpng*.la fi
from spkg-install}} in libpng. Why do we even install it, if we remove it afterwards? Maybe the headers don't match the system wide libpng and thus linking for gd fails. This problem is also bigger than just my {{{Matrix_mod2_dense
patch. Mabshoff, any idea?
comment:17 Changed 11 years ago by
The problem with libpng.dylib is that certain spkgs like Python and R do not have LDFLAGS and CPPFLAGS set correctly so that on rebuilding them (on upgrade for example) we pick up the systems libpng which is broken. The issue has been corrected for Python, but some more work is needed for R and likely some other spkgs.
Cheers,
Michael
comment:18 Changed 11 years ago by
Guessing from William's failure above gd is also affected here. I believe #3792 will fix that issue.
Cheers,
Michael
comment:19 Changed 11 years ago by
- Summary changed from [with patch, needs more work] Matrix_mod2_dense to/from PNG routines to [with patch, positive review] Matrix_mod2_dense to/from PNG routines
I fixed all the problems on OSX in libpng.spkg, python.spkg and r.spkg and with those fixes the patch applies, builds correctly and all doctests pass on OSX. Positive review pending three other tickets I am about to open.
Cheers,
Michael
comment:20 Changed 11 years ago by
comment:21 Changed 11 years ago by
- Resolution set to fixed
- Status changed from new to closed
Merged in Sage 3.1.2.alpha3
REFEREE CHAT:
This fails to work on OS X PPC:
This fails to work on 32-bit Linux against vanilla 3.0.2 and exactly the same problem on minimal 64-bit debian: