#9568 closed enhancement (duplicate)
Update IML to the newest upstream release, and improve spkg-install
Reported by: | drkirkby | Owned by: | tbd |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | packages: standard | Keywords: | |
Cc: | leif, malb | Merged in: | |
Authors: | Reviewers: | Keshav Kini | |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
Attachments (1)
Change History (18)
comment:1 follow-up: ↓ 2 Changed 9 years ago by
- Type changed from defect to enhancement
comment:2 in reply to: ↑ 1 Changed 9 years ago by
Replying to leif:
This is going to be a fun one, since the upstream source tree is not vanilla. Some files have been patched "in-place", others have been added, and (/but?) there's even a Mercurial repository in it (added by Sage)... :)
I'll take a closer look within the next days.
(I can't change the milestone to 4.6!)
We should also consider whether this package can be built in parallel, so replacing make
with $MAKE
. I just built the current version 1.0.1) ten times in parallel, using $MAKE
rather than make
. It reduced the build time from 13 s to 8 s on my Sun Ultra 27, so hardly a huge benefit.
comment:3 Changed 9 years ago by
Files iml-1.0.1-vanilla/Makefile.in and iml-1.0.1.p13/src/Makefile.in differ Only in iml-1.0.1.p13/src/: cblas.h Files iml-1.0.1-vanilla/configure and iml-1.0.1.p13/src/configure differ Only in iml-1.0.1.p13/src/examples: build.linux Only in iml-1.0.1.p13/src/examples: build.osx Files iml-1.0.1-vanilla/examples/exam-nullspace.c and iml-1.0.1.p13/src/examples/exam-nullspace.c differ Only in iml-1.0.1.p13/src/examples: exam-nullspace2.c Only in iml-1.0.1.p13/src/: gsl_cblas.h Only in iml-1.0.1.p13/src/repl: .libs Only in iml-1.0.1.p13/src/repl: Makefile Only in iml-1.0.1.p13/src/repl: librepl.la Only in iml-1.0.1.p13/src/src: .hg Files iml-1.0.1-vanilla/src/Makefile.am and iml-1.0.1.p13/src/src/Makefile.am differ Files iml-1.0.1-vanilla/src/Makefile.in and iml-1.0.1.p13/src/src/Makefile.in differ Files iml-1.0.1-vanilla/src/RNSop.c and iml-1.0.1.p13/src/src/RNSop.c differ Files iml-1.0.1-vanilla/src/memalloc.c and iml-1.0.1.p13/src/src/memalloc.c differ Files iml-1.0.1-vanilla/src/nonsysolve.c and iml-1.0.1.p13/src/src/nonsysolve.c differ Files iml-1.0.1-vanilla/src/nullspace.c and iml-1.0.1.p13/src/src/nullspace.c differ Files iml-1.0.1-vanilla/src/padiclift.c and iml-1.0.1.p13/src/src/padiclift.c differ Files iml-1.0.1-vanilla/src/padiclift.h and iml-1.0.1.p13/src/src/padiclift.h differ Only in iml-1.0.1.p13/src/src: tinyatlas.h
Perhaps not that hard, but do we want to keep the repository?
Also, the "new" 1.0.3 version (from 2008) brings new functionality, so additions/changes to the Sage library are likely.
comment:4 Changed 9 years ago by
sage-4.5$ grep -i -B5 -A1 iml devel/sage/module_list.py # TODO -- change to use BLAS at some point. Extension('sage.matrix.matrix_integer_dense', sources = ['sage/matrix/matrix_integer_dense.pyx'], # order matters for cygwin!! libraries = ['iml', 'gmp', 'm', 'pari', BLAS, BLAS2]),
From spkg/standard/deps
:
all: $(BASE) \ ... $(INST)/$(IML) \ ... $(INST)/$(IML): $(BASE) $(INST)/$(MPIR) $(INST)/$(GSL) $(INSTALL) "$(SAGE_SPKG) $(IML) 2>&1" "tee -a $(SAGE_LOGS)/$(IML).log" ... # All components that are linked against extensions need to be listed here $(INST)/$(SAGE): $(BASE) \ ... $(INST)/$(IML) \ ...
(These are all occurrences of IML, i.e. only the Sage library depends on it.)
comment:5 follow-up: ↓ 8 Changed 9 years ago by
FYI/status:
... Successfully installed iml-1.0.3.p0 Running the test suite. Running the IML test suite... ... PASS: test-smallentry PASS: test-largeentry ================== All 2 tests passed ================== ... The IML test suite passed without errors. Now cleaning up tmp files. Making Sage/Python scripts relocatable... Making script relocatable Finished installing iml-1.0.3.p0.spkg real 4m30.801s user 0m8.209s sys 0m3.896s leif@portland:~/Sage/sage-4.5$ ./sage -t devel/sage/sage/matrix/matrix_integer_dense.pyx sage -t "devel/sage/sage/matrix/matrix_integer_dense.pyx" [5.5 s] ---------------------------------------------------------------------- All tests passed! Total time for all tests: 5.5 seconds leif@portland:~/Sage/sage-4.5$ ./sage -t -long devel/sage/sage/matrix/matrix_integer_dense.pyx sage -t -long "devel/sage/sage/matrix/matrix_integer_dense.pyx" [18.6 s] ---------------------------------------------------------------------- All tests passed! Total time for all tests: 18.6 seconds
I started with a vanilla 1.0.3, "cleaned up" spkg-install
(dropping all patches and the Debian stuff) and just looked what happens...
(I configured with --prefix=$SAGE_LOCAL --enable-shared
and added the appropriate (Sage) dirs to the preprocessor and linker search paths.)
For a Sage build from scratch, we need to change deps
s.t. IML again(?) depends on ATLAS rather than GSL. Or is ATLAS still(?) considered optional on some systems?
Then I only added a patch for src/src/nullspace.c
, which uses mpz_init_ui()
instead of mpz_init_set_ui()
.
:-)
(Still alpha, though. Looking for what else to carry over.)
-Leif
comment:6 Changed 9 years ago by
I'm attaching a log of a failed build of 1.0.1, which may be due to parallel build issues. I think it would be wise to unset MAKE in this case - at least on the 1.0.1 code. Hard to know with the 1.0.3.
Build environment
- Sun Ultra 27
- Intel W3580 Xeon 3.33 GHz. Quad core. 8 threads.
- 12 GB RAM
- 128-bit ZFS file systems
- 2 x 500 MB mirrored disks for the root file system
- 2 x Hitachi HUA72202-A20N 2 TB disks mirrored disks for the user file system.
- OpenSolaris 2009.06 snv_134 X86
- Sage 4.5.1
- gcc 4.4.4 configured to use the Sun linker and GNU assembler
- A parallel build with
SAGE_PARALLEL_SPKG_BUILD=yes
andMAKE=make -j1000
- Nothing was set in ulimit to limit processes, memory or similar.
Dave
Changed 9 years ago by
Build log from a Sun Ultra 27 with an Intel Xeon processor - showing a strange failure.
comment:7 Changed 9 years ago by
I just rebuilt version 1.0.1 in a loop 100 times. It failed to build once in that 100 times! (That's two failures in total I have observed of this).
Dave
comment:8 in reply to: ↑ 5 Changed 9 years ago by
Replying to leif:
FYI/status:
... Successfully installed iml-1.0.3.p0 Running the test suite. Running the IML test suite... ... PASS: test-smallentry PASS: test-largeentry ================== All 2 tests passed ================== ... The IML test suite passed without errors. Now cleaning up tmp files. Making Sage/Python scripts relocatable... Making script relocatable Finished installing iml-1.0.3.p0.spkg real 4m30.801s user 0m8.209s sys 0m3.896s leif@portland:~/Sage/sage-4.5$ ./sage -t devel/sage/sage/matrix/matrix_integer_dense.pyx sage -t "devel/sage/sage/matrix/matrix_integer_dense.pyx" [5.5 s] ---------------------------------------------------------------------- All tests passed! Total time for all tests: 5.5 seconds leif@portland:~/Sage/sage-4.5$ ./sage -t -long devel/sage/sage/matrix/matrix_integer_dense.pyx sage -t -long "devel/sage/sage/matrix/matrix_integer_dense.pyx" [18.6 s] ---------------------------------------------------------------------- All tests passed! Total time for all tests: 18.6 secondsI started with a vanilla 1.0.3, "cleaned up"
spkg-install
(dropping all patches and the Debian stuff) and just looked what happens... (I configured with--prefix=$SAGE_LOCAL --enable-shared
and added the appropriate (Sage) dirs to the preprocessor and linker search paths.)For a Sage build from scratch, we need to change
deps
s.t. IML again(?) depends on ATLAS rather than GSL. Or is ATLAS still(?) considered optional on some systems?Then I only added a patch for
src/src/nullspace.c
, which usesmpz_init_ui()
instead ofmpz_init_set_ui()
.:-)
(Still alpha, though. Looking for what else to carry over.)
Actually on sage-on-gentoo we have been using iml-1.0.3 for a while. But on some setups we have been hit by a doc test failure which seem to point to iml:
sage -t -force_lib "devel/sage/sage/modular/modsym/space.py" ********************************************************************** File "/opt/sage/devel/sage/sage/modular/modsym/space.py", line 1573: sage: af.intersection_number(ell) Exception raised: Traceback (most recent call last): File "/opt/sage/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/opt/sage/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/opt/sage/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_45[8]>", line 1, in <module> af.intersection_number(ell)###line 1573: sage: af.intersection_number(ell) File "/usr/lib/python2.6/site-packages/sage/modular/modsym/space.py", line 1581, in intersection_number B = M.integral_structure() File "/usr/lib/python2.6/site-packages/sage/modular/modsym/space.py", line 1549, in integral_structure J = self.free_module().intersection(I) File "/usr/lib/python2.6/site-packages/sage/modules/free_module.py", line 3025, in intersection W = B.kernel() File "matrix2.pyx", line 2288, in sage.matrix.matrix2.Matrix.kernel (sage/matrix/matrix2.c:12775) File "matrix2.pyx", line 2586, in sage.matrix.matrix2.Matrix.left_kernel (sage/matrix/matrix2.c:14030) File "matrix_rational_dense.pyx", line 1255, in sage.matrix.matrix_rational_dense.Matrix_rational_dense.right_kernel (sage/matrix/matrix_rational_dense.c:12832) RuntimeError
And the line called in matrix_rational_dense.pyx is:
K = A._rational_kernel_iml().change_ring(QQ)
The tests for matrix_rational_dense.pyx pass. There is a backtrace as well here: http://github.com/cschwan/sage-on-gentoo/issues#issue/3
Francois
comment:9 Changed 9 years ago by
Found out what the issue was in my previous comment. It is not related to iml per see. There is a bug somewhere in ATLAS-3.9.23 which is the version we currently default on in Gentoo. Using another cblas with iml make the issue disappear completely. So as far as we know iml-1.0.3 works fine with sage.
comment:11 Changed 7 years ago by
Oops, looks like I duplicated this at #13027... Leif, do you have an SPKG I can look at?
comment:12 Changed 7 years ago by
comment:14 Changed 7 years ago by
- Reviewers set to Keshav Kini
- Status changed from needs_review to positive_review
comment:15 Changed 7 years ago by
- Milestone changed from sage-5.1 to sage-duplicate/invalid/wontfix
comment:16 Changed 7 years ago by
- Description modified (diff)
- Resolution set to duplicate
- Status changed from positive_review to closed
comment:17 Changed 5 years ago by
Sign of life of IML upstream on #14648 !!1!111!
This is going to be a fun one, since the upstream source tree is not vanilla. Some files have been patched "in-place", others have been added, and (/but?) there's even a Mercurial repository in it (added by Sage)... :)
I'll take a closer look within the next days.
(I can't change the milestone to 4.6!)