Opened 8 years ago
Closed 7 years ago
#18078 closed enhancement (fixed)
CMake experimental package
Reported by: | isuruf | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-6.9 |
Component: | packages: experimental | Keywords: | |
Cc: | Merged in: | ||
Authors: | Isuru Fernando | Reviewers: | François Bissey |
Report Upstream: | N/A | Work issues: | |
Branch: | cb96c05 (Commits, GitHub, GitLab) | Commit: | cb96c05fd4703db60f0d6762127245f981639b44 |
Dependencies: | Stopgaps: |
Description (last modified by )
Sage experimental package for CMake
CMake version 3.2.3
Source can be found here, http://www.cmake.org/files/v3.2/cmake-3.2.3.tar.gz
Change History (50)
comment:1 Changed 8 years ago by
Branch: | → u/isuruf/cmake_optional_package |
---|
comment:2 Changed 8 years ago by
Commit: | → 2f172e0c13681493aaa7c8f4ab334dac55450571 |
---|---|
Component: | PLEASE CHANGE → packages: optional |
Description: | modified (diff) |
Type: | PLEASE CHANGE → enhancement |
comment:3 Changed 8 years ago by
Please word-wrap SPKG.txt
to 72 characters per line.
If it's ready for review, don't forget to set this ticket to needs_review
.
comment:5 Changed 8 years ago by
Commit: | 2f172e0c13681493aaa7c8f4ab334dac55450571 → 2b9f83e5439bca4355cfce2e5e04cd8cfd13913c |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
2b9f83e | Word-wrap cmake/SPKG.txt
|
comment:6 Changed 8 years ago by
Authors: | → Isuru Fernando |
---|---|
Status: | new → needs_review |
comment:7 follow-up: 9 Changed 8 years ago by
The installation works on Linux, at least. How does one check it is not broken?
comment:8 Changed 8 years ago by
Status: | needs_review → needs_work |
---|
WARNING: spkg-install is not executable, making it executable
comment:9 follow-up: 15 Changed 8 years ago by
Replying to dimpase:
The installation works on Linux, at least. How does one check it is not broken?
cmake
has a test-suite run with make test
. So a spkg-check
file should be added to run this.
One more remark: you don't really need to split up $MAKE
and $MAKE install
in two steps, just $MAKE install
is sufficient.
comment:10 Changed 8 years ago by
Doesn't build on OSX 10.10.3 with the latest XCode:
27%] Building CXX object Source/CMakeFiles/OSXScriptLauncher.dir/CPack/OSXScriptLauncher.cxx.o [ 27%] Building CXX object Utilities/cmjsoncpp/CMakeFiles/cmjsoncpp.dir/src/lib_json/json_writer.cpp.o In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/dispatch/dispatch.h:50:0, from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFStream.h:15, from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFPropertyList.h:13, from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:55, from /usr/local/src/sage/sage/local/var/tmp/sage/build/cmake-3.2.1/src/Source/CPack/OSXScriptLauncher.cxx:17: /usr/local/src/sage/sage/local/include/dispatch/object.h:321:1: error: 'DISPATCH_UNAVAILABLE' does not name a type DISPATCH_UNAVAILABLE ^ /usr/local/src/sage/sage/local/include/dispatch/object.h:358:1: error: 'DISPATCH_UNAVAILABLE' does not name a type DISPATCH_UNAVAILABLE ^ /usr/local/src/sage/sage/local/include/dispatch/object.h:387:1: error: 'DISPATCH_UNAVAILABLE' does not name a type DISPATCH_UNAVAILABLE ^ /usr/local/src/sage/sage/local/include/dispatch/object.h:418:1: error: 'DISPATCH_UNAVAILABLE' does not name a type DISPATCH_UNAVAILABLE ^ make[2]: *** [Source/CMakeFiles/OSXScriptLauncher.dir/CPack/OSXScriptLauncher.cxx.o] Error 1 make[1]: *** [Source/CMakeFiles/OSXScriptLauncher.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... Linking CXX static library libcmjsoncpp.a [ 27%] Built target cmjsoncpp make: *** [all] Error 2 Error building CMake. real 7m15.276s user 5m26.048s sys 1m33.521s ************************************************************************ Error installing package cmake-3.2.1 ************************************************************************ ...
I suspect that we don't need the fully blown OSX cmake (we don't need OSXScriptLauncher probably), perhaps some options need to be used while building on OSX...
comment:11 Changed 8 years ago by
My experience with kitware stuff on OS X is they assume clang
and most other stuff is in the too hard basket. I was helping someone get paraview on their OS X box and we had to give up using gcc
starting 10.10.2. I will have a look if we can do something at booststrap time but I am not especially hopeful.
comment:12 Changed 8 years ago by
Current error is while building internal version of curl
which is of course available by default on OS X. But using the system curl just push the problem further away. Attentive people will notice that the problematic header is the one we copy and modify for gcc install on 10.10.
I hate to do that, but my suggestion is for this to set CC=clang CXX=clang++
on OS X.
comment:13 Changed 8 years ago by
OK, it is worse than I thought. Not only do we get a gcc
that will sometimes break, when gcc
breaks so does clang
because of that little header. I am wondering if we should remove that header once gcc
is built. It looks like a lot of stuff that will want that header will break regardless. Thoughts?
comment:14 Changed 8 years ago by
Commit: | 2b9f83e5439bca4355cfce2e5e04cd8cfd13913c → 72a020d1f2628dd7fc7595b9cf0485a14d6fb426 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
72a020d | Make cmake/spkg-install executable
|
comment:15 Changed 8 years ago by
Replying to jdemeyer:
Replying to dimpase:
The installation works on Linux, at least. How does one check it is not broken?
cmake
has a test-suite run withmake test
. So aspkg-check
file should be added to run this.
cmake
has a test suite that can be run using ctest
. I wrote a spkg-check
and ran the tests on Linux. 3/382 tests fail, two of which is related to curl, and the other about a sleep command. I'm not sure whether these options will be required by packages in the future.
23 - curl (Failed)
178 - CTestTestUpload (Failed)
218 - CTestTestStopTime (Failed)
One more remark: you don't really need to split up
$MAKE
and$MAKE install
in two steps, just$MAKE install
is sufficient.
Thanks. I will make this change.
comment:16 Changed 8 years ago by
Commit: | 72a020d1f2628dd7fc7595b9cf0485a14d6fb426 → 3aae9bf7905d0cb5ec63eaceef8ee9f590c87463 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
3aae9bf | CMake : Merge make and make install steps into one
|
comment:17 follow-up: 18 Changed 8 years ago by
At the moment I would recommend people who want cmake on OS X to download it from kitware and make sure that the cmake
command is in their PATH
. We could put some words of that effect in spkg-install
. I don't see this working on OS X anytime soon. We would need a breakthrough with gcc
or clang
.
comment:18 follow-up: 19 Changed 8 years ago by
Replying to fbissey:
At the moment I would recommend people who want cmake on OS X to download it from kitware and make sure that the
cmake
command is in theirPATH
. We could put some words of that effect inspkg-install
. I don't see this working on OS X anytime soon. We would need a breakthrough withgcc
orclang
.
I won't be surprised if this cmake won't be useful within Sage, as it might assume too much of the native OSX toolchain.
cmake was mentioned by me in a GSoC 2015 discussion regarding getting CSymPy into Sage; I expressed my doubts that cmake, which is used to configure CSymPy, is easy to deal with in Sage, based on some vague memories about cmake/Sage difficulties I saw. Now I recall that that was about building cmake in Sage on OSX.
comment:19 Changed 8 years ago by
Replying to dimpase:
Replying to fbissey:
At the moment I would recommend people who want cmake on OS X to download it from kitware and make sure that the
cmake
command is in theirPATH
. We could put some words of that effect inspkg-install
. I don't see this working on OS X anytime soon. We would need a breakthrough withgcc
orclang
.I won't be surprised if this cmake won't be useful within Sage, as it might assume too much of the native OSX toolchain.
cmake was mentioned by me in a GSoC 2015 discussion regarding getting CSymPy into Sage; I expressed my doubts that cmake, which is used to configure CSymPy, is easy to deal with in Sage, based on some vague memories about cmake/Sage difficulties I saw. Now I recall that that was about building cmake in Sage on OSX.
I don't think there is a difficulty per see in using cmake
on OS X. The trouble building cmake
and in fact gcc
and in turn a bunch of other stuff is solely the fault of apple pushing its own stuff and not caring. gcc
is currently slightly broken on OS X and unless the gcc
team gets something together it will just get worse. Not being able to build cmake
is just one of the many symptoms of gcc
cracking while apple is pushing hard.
comment:20 Changed 8 years ago by
cmake 3.2.2 has been released, so this ticket ought to be adjusted to this version, I suppose.
comment:21 follow-up: 22 Changed 8 years ago by
@fbissey: and the problem is that we really need GCC for Sage, because we need a Fortran compiler and clang
doesn't (yet) support Fortran.
comment:22 follow-up: 24 Changed 8 years ago by
Replying to jdemeyer:
@fbissey: and the problem is that we really need GCC for Sage, because we need a Fortran compiler and
clang
doesn't (yet) support Fortran.
I know. Although I have been trying very hard to think about fortran code in sage apart from lapack. I think the only fortran bits where you need a fortran compiler are in atlas, scipy and unfortunately for OS X: R.
We use the accelerate framework instead of Atlas on OS X, one less fortran thing.
A fortran compiler is needed while compiling scipy :( lots of fortran code in there.
R is more pernicious, you need a fortran compiler on OS X because there is a bug preventing us to use accelerate.
flang is just a dot on the horizon so far. At least they say they started work on it.
comment:23 Changed 8 years ago by
Milestone: | sage-6.6 → sage-6.7 |
---|
comment:24 follow-up: 25 Changed 8 years ago by
Replying to fbissey:
Replying to jdemeyer:
@fbissey: and the problem is that we really need GCC for Sage, because we need a Fortran compiler and
clang
doesn't (yet) support Fortran.I know. Although I have been trying very hard to think about fortran code in sage apart from lapack. I think the only fortran bits where you need a fortran compiler are in atlas, scipy and unfortunately for OS X: R.
We use the accelerate framework instead of Atlas on OS X, one less fortran thing.
A fortran compiler is needed while compiling scipy :( lots of fortran code in there.
R is more pernicious, you need a fortran compiler on OS X because there is a bug preventing us to use accelerate.
flang is just a dot on the horizon so far. At least they say they started work on it.
I don't even see dots, apart from some github repos untouched since 2013 or so.
We can consider going back to having a dedicated gfortran package, as it used to be the case before OSX 10.7 (IIRC) broken xcode trouble started, but otherwise use xcode's compilers.
comment:25 follow-up: 28 Changed 8 years ago by
Replying to dimpase:
We can consider going back to having a dedicated gfortran package, as it used to be the case before OSX 10.7 (IIRC) broken xcode trouble started, but otherwise use xcode's compilers.
It's not just a matter of using gfortran
. It's also a matter of linking code together (good luck linking a library containing pieces compiled by clang
and other pieces by gfortran
).
comment:26 follow-up: 27 Changed 8 years ago by
About CMake: it's very ironic that the cmake
authors wrote a build system which is supposed to make it easy to build cross-platform software, but they cannot make cmake
itself build properly! That's really not good advertising for cmake
.
comment:27 Changed 8 years ago by
Replying to jdemeyer:
About CMake: it's very ironic that the
cmake
authors wrote a build system which is supposed to make it easy to build cross-platform software, but they cannot makecmake
itself build properly! That's really not good advertising forcmake
.
they didn't use autotools, I suppose, that's why ;-)
comment:28 Changed 8 years ago by
Replying to jdemeyer:
Replying to dimpase:
We can consider going back to having a dedicated gfortran package, as it used to be the case before OSX 10.7 (IIRC) broken xcode trouble started, but otherwise use xcode's compilers.
It's not just a matter of using
gfortran
. It's also a matter of linking code together (good luck linking a library containing pieces compiled byclang
and other pieces bygfortran
).
And there are still parts of Sage which currently don't build with clang
...
Otherwise, w.r.t. gfortran
on MacOS X: Revive f2c
! ;-)
comment:29 Changed 8 years ago by
Thanks for trying to compile cmake.
A few comments:
- if csympy is the only package that requires cmake and cmake is impossible to build, we can also support an alternative build system in csympy
- I really like cmake, so let me try to fix these issues first, as I would prefer to keep cmake as the build system (but it needs to reliably build on OS X, both with clang and with gcc)
- I independently discovered lots of similar issues with building cmake with gcc on OS X 10.10 (https://github.com/hashdist/hashstack/issues/746), and just like you noticed, it builds fine with the native clang. As you can see, you can easily tell cmake not to compile in various things and rather reuse them from the system or Sage. I really believe all these issues can be fixed and I am also sure Kitware is itself interested in cmake being able to build, e.g. Robert Maynard from Kitware already commented on my issue above with some tips.
So I will have a look in the coming days and I will update this issue.
comment:30 Changed 8 years ago by
I guess this is still pretty much work in progress, but we certainly should use/tweak a couple of configure options for bootstrapping within Sage, such as for example
--parallel=n bootstrap cmake in parallel, where n is number of nodes [1] --no-system-libs use all cmake-provided third-party libraries (default) --system-zlib use system-installed zlib library --no-system-zlib use cmake-provided zlib library (default) --system-bzip2 use system-installed bzip2 library --no-system-bzip2 use cmake-provided bzip2 library (default) ...
comment:31 Changed 8 years ago by
Commit: | 3aae9bf7905d0cb5ec63eaceef8ee9f590c87463 → c8778a93f2076dd8343e533511a7e6758b1803ab |
---|
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
e2c3532 | Added CMake v3.2.1 optional package
|
1cbb417 | Word-wrap cmake/SPKG.txt
|
79d84cc | Make cmake/spkg-install executable
|
4a219ec | CMake : Merge make and make install steps into one
|
2870d55 | CMake: use sage's bzip2 and zlib
|
fe8eb4a | Test CMake installation
|
c8778a9 | Patch for building CMake on OSX 10.10
|
comment:32 Changed 8 years ago by
A lot has changed in the handling of optional packages recently, this should be rebased on top of Sage 6.8.beta5.
comment:33 Changed 8 years ago by
Commit: | c8778a93f2076dd8343e533511a7e6758b1803ab → 6fea35f5c78f637a50b1cf35d7fa2fb36a58af77 |
---|
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
eadda32 | Added CMake v3.2.1 optional package
|
a84859e | Word-wrap cmake/SPKG.txt
|
feadaf3 | Make cmake/spkg-install executable
|
919f12e | CMake : Merge make and make install steps into one
|
268dee9 | CMake: use sage's bzip2 and zlib
|
def4c8a | Test CMake installation
|
c73f9e2 | Patch for building CMake on OSX 10.10
|
6fea35f | CMake : Add type as experimental
|
comment:34 Changed 8 years ago by
Is the OS X patch still needed with cmake 3.2.3
? If not I suggest bumping to that and I will test it on my OS X box. If it builds I think we can review it positively as an experimental package. While reviewing we have pushed at the optional level of quality in review rather than experimental in my opinion.
comment:35 Changed 8 years ago by
I realise that I thought it was experimental
because that's what the type says in the git commit. But this ticket says "optional" and I think we are getting there so the type will have to be changed too.
comment:36 Changed 8 years ago by
Commit: | 6fea35f5c78f637a50b1cf35d7fa2fb36a58af77 → 0096eed76cf7cc2fc99953192dc78807501e8f3a |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
0096eed | CMake : updated version to 3.2.3
|
comment:37 Changed 8 years ago by
Component: | packages: optional → packages: experimental |
---|---|
Description: | modified (diff) |
Status: | needs_work → needs_review |
Summary: | CMake optional package → CMake experimental package |
Yes, the patch is still needed for 3.2.3
I added experimental because even with the patch 6 tests out of 387 tests in OSX 10.10 fail. Once it is ready to become optional, I'll update the type to optional.
comment:38 Changed 8 years ago by
Status: | needs_review → needs_work |
---|
Invalid checksum for cached file /usr/local/src/sage-config/upstream/cmake-3.2.3.tar.gz, deleting
comment:39 Changed 8 years ago by
Commit: | 0096eed76cf7cc2fc99953192dc78807501e8f3a → b65bb43a99a3290e10ccf6f752eb160aafb95e89 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
b65bb43 | CMake : update checksums
|
comment:40 Changed 8 years ago by
Sorry about that, chrome seems to be playing tricks on me. It's updated now.
comment:41 Changed 8 years ago by
It seems that the curl library included with CMake is outdated and needs to be built separately for CMake to work in OSX.
comment:42 Changed 8 years ago by
Milestone: | sage-6.7 → sage-6.8 |
---|
Does it really need curl
? OSX does ship curl
just fine...
comment:43 Changed 8 years ago by
There are some features that involve downloading. That's why curl is used.
We can use system installed curl
for OSX and build curl
for other systems.
comment:44 Changed 8 years ago by
curl
is really very standard thing on Linux, installed by default. I don't know why cmake
needs it for anything but Windows...
comment:45 Changed 8 years ago by
Commit: | b65bb43a99a3290e10ccf6f752eb160aafb95e89 → d71e284bcb8127523ffff4a0229fa152c0064976 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
d71e284 | CMake : Use system curl
|
comment:46 Changed 8 years ago by
I pushed in a commit to use system curl when building cmake.
@dimpase, yes, there's really no need to build curl as it is installed in OSX and Linux. Building cmake in windows is not supported, so no use there either.
comment:47 Changed 8 years ago by
Commit: | d71e284bcb8127523ffff4a0229fa152c0064976 → cb96c05fd4703db60f0d6762127245f981639b44 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
cb96c05 | Use system-curl only in OSX and build curl in Linux
|
comment:48 Changed 8 years ago by
Status: | needs_work → needs_review |
---|
comment:49 Changed 7 years ago by
Milestone: | sage-6.8 → sage-6.9 |
---|---|
Reviewers: | → François Bissey |
Status: | needs_review → positive_review |
Let's get this in, this is experimental after all.
comment:50 Changed 7 years ago by
Branch: | u/isuruf/cmake_optional_package → cb96c05fd4703db60f0d6762127245f981639b44 |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
New commits:
Added CMake v3.2.1 optional package