Opened 10 years ago
Closed 10 years ago
#13160 closed enhancement (fixed)
upgrade cvxopt to 1.1.5
Reported by: | fbissey | Owned by: | tbd |
---|---|---|---|
Priority: | major | Milestone: | sage-5.3 |
Component: | packages: standard | Keywords: | |
Cc: | Merged in: | sage-5.3.beta1 | |
Authors: | François Bissey | Reviewers: | Volker Braun |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
Upgrading cvxopt to 1.1.5 we can get rid of a patch to correct a wrong behavior that is obvious on OS X. Also I want to simplify the setup.py patch which give rise to overlinking (and in fact potential linking of multiple cblas libraries - yuk!). I am also attempting to install the documentation.
Updated spkg at:
Change History (34)
comment:1 Changed 10 years ago by
comment:2 Changed 10 years ago by
- Description modified (diff)
Experimental spkg now posted and linked in the description. The changes in the spkg have not been committed yet so I can still fiddle with it as necessary.
comment:3 Changed 10 years ago by
- Dependencies set to #10508
- Status changed from new to needs_info
Looks good on OSX 10.6.8 and Sage 5.2.beta1 with #10508 applied.
I tried both lapack/blas, the ones from Apple, and the ones from Atlas 3.10 from #10508, with SAGE_CHECK
on.
By the way, shouldn't #10508 be a dependence of this ticket? (I added it here, tentatively.)
Also, I wonder if there is anything to report upstream here.
comment:4 follow-up: ↓ 5 Changed 10 years ago by
- Dependencies #10508 deleted
#10508 doesn't change the layout of the ATLAS libraries, the current atlas-3.8.4 already provides the cblas library. If the cvxopt spkg were to link to the correct libraries only, it would work with both atlas-3.8.4 and atlas-3.10.0 spkgs.
comment:5 in reply to: ↑ 4 Changed 10 years ago by
Replying to vbraun:
#10508 doesn't change the layout of the ATLAS libraries, the current atlas-3.8.4 already provides the cblas library. If the cvxopt spkg were to link to the correct libraries only, it would work with both atlas-3.8.4 and atlas-3.10.0 spkgs.
When you closed #10509 you said it is fixed by atlas-3.10.0 spkg. And #10508 is the one the provides atlas-3.10.0 spkg.
comment:6 Changed 10 years ago by
The only thing really to check in this ticket is the installation of the documentation. Technically it needs sphinx to build but because up to date build documentation is shipped nothing is done. So the question is whether to require sphinx or just copy the already built documentation. Once I make an informed decision on that I'll finalize the spkg.
comment:7 follow-up: ↓ 9 Changed 10 years ago by
OK I pushed a new spkg on google code it install the already built documentation and has a section commented out on how to build it in spkg-install.
comment:8 Changed 10 years ago by
- Status changed from needs_info to needs_review
comment:9 in reply to: ↑ 7 ; follow-ups: ↓ 10 ↓ 12 Changed 10 years ago by
Replying to fbissey:
OK I pushed a new spkg on google code it install the already built documentation and has a section commented out on how to build it in spkg-install.
Shouldn't it install docs by default? (Currently, you need to set SAGE_SPKG_INSTALL_DOCS
)
Also, I don't understand what you mean by "dependence" on sphinx. Isn't sphinx included in Sage?
comment:10 in reply to: ↑ 9 ; follow-up: ↓ 11 Changed 10 years ago by
Replying to dimpase:
Replying to fbissey:
OK I pushed a new spkg on google code it install the already built documentation and has a section commented out on how to build it in spkg-install.
Shouldn't it install docs by default? (Currently, you need to set
SAGE_SPKG_INSTALL_DOCS
)Also, I don't understand what you mean by "dependence" on sphinx. Isn't sphinx included in Sage?
As far as I know, you only install documentation for spkg if that variable is set. I may be wrong.
Yes sphinx is shipped with sage but if this spkg uses sphinx it has to be listed as one of its dependencies in spkg/standard/deps. In this particular case it is probably ok because cvxopt can be build after sage I think. But in the alternative there are problems as far as I remember from a previous upgrade of numpy way back.
comment:11 in reply to: ↑ 10 Changed 10 years ago by
Replying to fbissey:
OK, I asked on sage-devel
comment:12 in reply to: ↑ 9 Changed 10 years ago by
Replying to dimpase:
Replying to fbissey:
OK I pushed a new spkg on google code it install the already built documentation and has a section commented out on how to build it in spkg-install.
Shouldn't it install docs by default? (Currently, you need to set
SAGE_SPKG_INSTALL_DOCS
)
?? #11197 has sort of delayed most packages actually implementing #10823. I believe that we currently don't want docs installed by default outside of wherever the spkg lives. Or maybe I'm misunderstanding the point of this discussion, my apologies if that's so.
comment:13 follow-up: ↓ 14 Changed 10 years ago by
I didn't remember how the discussion had ended. Probably because I wasn't on the follow up ticket where things were discussed. Should I remove the section about documentation until one day we sort this out?
We should note that some packages already put their doc in $SAGE_LOCAL/share/doc:
frb15@p2n14-c /hpc/scratch/frb15/sandbox/sage-5.1.beta5/local/share/doc :ll total 1792 drwxr-xr-x 2 frb15 z001 131072 Jun 29 13:08 NTL drwxr-xr-x 5 frb15 z001 131072 Jun 29 15:05 ipython drwxr-xr-x 3 frb15 z001 131072 Jun 29 13:13 mpfr drwxr-xr-x 3 frb15 z001 131072 Jun 29 15:12 networkx-1.2 drwxr-xr-x 3 frb15 z001 131072 Jun 29 13:53 ppl drwxr-xr-x 3 frb15 z001 131072 Jun 29 13:53 pwl drwxr-xr-x 2 frb15 z001 131072 Jun 29 15:49 sagetex
I am not even sure those obey SAGE_SPKG_INSTALL_DOCS and some are html. The cvxopt spkg linked in this ticket doesn't need sphinx because the doc is prebuilt in the tarball.
comment:14 in reply to: ↑ 13 Changed 10 years ago by
Replying to fbissey:
I didn't remember how the discussion had ended. Probably because I wasn't on the follow up ticket where things were discussed. Should I remove the section about documentation until one day we sort this out?
no, why, documentation is great to have. But why not remove the pre-built one and make it using sphynx?
(the latter will need a trivial change in SAGE_ROOT/spkg/standard/deps
).
comment:15 Changed 10 years ago by
Can do that but I don't think it will be much different from the one shipped. The first thing that sphinx does when it is run is check whether what is already built is up to date. Rebuilding it again is just a waste of cpu cycles.
I don't really care about it that much myself. So if people are quite happy about rebuilding the doc I'll just remove the pre build from the spkg to save space and produce the other needed patch.
comment:16 follow-up: ↓ 17 Changed 10 years ago by
- Status changed from needs_review to needs_work
There are leftovers of debugging in src/src/C/blas.c, starting from line 295, which reads
printf("scal\n");
This is very strange.
EDIT: I checked the upstream, and they don't have these printfs...
comment:17 in reply to: ↑ 16 Changed 10 years ago by
Replying to dimpase:
There are leftovers of debugging in src/src/C/blas.c, starting from line 295, which reads
printf("scal\n");This is very strange.
EDIT: I checked the upstream, and they don't have these printfs...
Moreover, the upstream files base.c, blas.c, and sparse.c in src/src/C have earlier dates (and different sizes) than the ones in spkg.
And replacing the spkg files by upstream files makes the package seemingly work on OSX10.6.8. I didn't test much, but the file src/examples/doc/chap10/l1svc.py
which gives me crashes with the spkg, works OK with the upstream replacement.
comment:18 Changed 10 years ago by
Sorry wrong spkg I was tracking something on power7 I will correct the source today - hopefully.
comment:19 Changed 10 years ago by
- Status changed from needs_work to needs_review
Corrected the source. Sorry for the inconvenience.
comment:20 Changed 10 years ago by
- Description modified (diff)
On Fedora 17 x86_64:
building 'gsl' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/C gcc -fno-strict-aliasing -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/vbraun/opt/sage-atlas/sage-5.2/local/include/gsl -I/home/vbraun/opt/sage-atlas/sage-5.2/local/include/python2.7 -c C/gsl.c -o build/temp.linux-x86_64-2.7/C/gsl.o C/gsl.c:28:25: fatal error: gsl/gsl_rng.h: No such file or directory compilation terminated.
The header is in Sage but the include path is wrong:
[vbraun@laptop sage-5.2]$ find | grep gsl_rng ./local/include/gsl/gsl_rng.h
The fix is to set If I edit GSL_INC_DIR = SAGE_INCLUDE
in setup.py
, and not SAGE_INCLUDE/gsl
.
I've updated the spkg (see description, only change is the GSL_INC_DIR
) and it builds for me now.
comment:21 Changed 10 years ago by
OK you beat me to it. I was going to post about it. I am guessing the problem is not visible if gsl is installed system wide.
comment:22 Changed 10 years ago by
Just saw that we discovered the same bug within ~5mins. Coincidence? ;-)
comment:23 Changed 10 years ago by
John was the pointer for me really. Didn't notice it before - all the system I tested it with had gsl as standard. I need a box with nothing interesting on it really.
comment:24 Changed 10 years ago by
On hawk, this builds if SAGE_ATLAS_LIB points to the system ATLAS, but not if I use the ATLAS spkg from #10508. Log here.
comment:25 Changed 10 years ago by
shouldn't be -lblas but -lf77blas I guess.
comment:26 Changed 10 years ago by
+if os.environ['UNAME'] == 'CYGWIN': + BLAS_LIB =['blas', 'gfortran'] +elif os.environ['UNAME'] == 'Darwin': + BLAS_LIB =['blas','gfortran','m'] +else: + BLAS_LIB =['blas','cblas','gfortran','atlas','m'] +
should really be
+if os.environ['UNAME'] == 'CYGWIN': + BLAS_LIB =['blas', 'gfortran'] +elif os.environ['UNAME'] == 'Darwin': + BLAS_LIB =['blas','gfortran','m'] +else: + BLAS_LIB =['f77blas','cblas','gfortran','atlas','m'] +
I think it may have escaped because we had libblas before #10508
comment:27 Changed 10 years ago by
I agree and just replaced the spkg (new md5sum = b155f7e4624c2869b9a0e5aa21803435)
Builds on Fedora 17
comment:28 Changed 10 years ago by
Builds and passes self-tests on hawk, both with the system ATLAS and the ATLAS spkg from #10508.
comment:29 Changed 10 years ago by
- Reviewers set to Volker Braun
- Status changed from needs_review to positive_review
Looks good to me!
comment:30 Changed 10 years ago by
Thanks Volker for the last mile. I also just thought that we will need a follow up ticket if we allow people to use ATLAS on OS X.
comment:31 follow-up: ↓ 32 Changed 10 years ago by
So on OS X, if people use the ATLAS spkg, then is this using a mix of Sage's ATLAS and the system's blas? (I'm in the process of building on OS X, including ATLAS from #10508, and cvxopt has built and passes its self-tests. The build hasn't completed, so Sage's test suite hasn't run yet.)
comment:32 in reply to: ↑ 31 Changed 10 years ago by
Replying to jhpalmieri:
So on OS X, if people use the ATLAS spkg, then is this using a mix of Sage's ATLAS and the system's blas? (I'm in the process of building on OS X, including ATLAS from #10508, and cvxopt has built and passes its self-tests. The build hasn't completed, so Sage's test suite hasn't run yet.)
I hope so. It would be a big speed digression to use GSL.
comment:33 Changed 10 years ago by
From what I put in the spkg cvxopt will use the OS X accelerate framework instead of ATLAS. You should note that it is not the only spkg for which it will be the case. linbox will do that as well for example. iml also comes to mind as well and numpy and scipy should be checked. From the sage shell can you check the output of
otool -L local/lib/python/site-packages/cvxopt/blas.so
It should tell you against which library it was linked in some form.
comment:34 Changed 10 years ago by
- Merged in set to sage-5.3.beta1
- Resolution set to fixed
- Status changed from positive_review to closed
I have a spkg almost ready to upload. Changes are uncommitted yet. There are some kinks to check in the installation of the documentation.