CMake experimental package

[isuruf]

Sage experimental package for CMake

CMake version 3.2.3

Source can be found here, ​http://www.cmake.org/files/v3.2/cmake-3.2.3.tar.gz

[isuruf]

New commits:

​2f172e0

Added CMake v3.2.1 optional package

[jdemeyer]

Please word-wrap SPKG.txt to 72 characters per line.

If it's ready for review, don't forget to set this ticket to needs_review.

[jdemeyer]

And don't forget to fill in your real name in the "Author" field.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2b9f83e

Word-wrap cmake/SPKG.txt

[dimpase]

The installation works on Linux, at least. How does one check it is not broken?

[jdemeyer]

WARNING: spkg-install is not executable, making it executable

[jdemeyer]

Replying to dimpase:

The installation works on Linux, at least. How does one check it is not broken?

cmake has a test-suite run with make test. So a spkg-check file should be added to run this.

One more remark: you don't really need to split up $MAKE and $MAKE install in two steps, just $MAKE install is sufficient.

[dimpase]

Doesn't build on OSX 10.10.3 with the latest XCode:

 27%] Building CXX object Source/CMakeFiles/OSXScriptLauncher.dir/CPack/OSXScriptLauncher.cxx.o
[ 27%] Building CXX object Utilities/cmjsoncpp/CMakeFiles/cmjsoncpp.dir/src/lib_json/json_writer.cpp.o
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/dispatch/dispatch.h:50:0,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFStream.h:15,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFPropertyList.h:13,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:55,
                 from /usr/local/src/sage/sage/local/var/tmp/sage/build/cmake-3.2.1/src/Source/CPack/OSXScriptLauncher.cxx:17:
/usr/local/src/sage/sage/local/include/dispatch/object.h:321:1: error: 'DISPATCH_UNAVAILABLE' does not name a type
 DISPATCH_UNAVAILABLE
 ^
/usr/local/src/sage/sage/local/include/dispatch/object.h:358:1: error: 'DISPATCH_UNAVAILABLE' does not name a type
 DISPATCH_UNAVAILABLE
 ^
/usr/local/src/sage/sage/local/include/dispatch/object.h:387:1: error: 'DISPATCH_UNAVAILABLE' does not name a type
 DISPATCH_UNAVAILABLE
 ^
/usr/local/src/sage/sage/local/include/dispatch/object.h:418:1: error: 'DISPATCH_UNAVAILABLE' does not name a type
 DISPATCH_UNAVAILABLE
 ^
make[2]: *** [Source/CMakeFiles/OSXScriptLauncher.dir/CPack/OSXScriptLauncher.cxx.o] Error 1
make[1]: *** [Source/CMakeFiles/OSXScriptLauncher.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Linking CXX static library libcmjsoncpp.a
[ 27%] Built target cmjsoncpp
make: *** [all] Error 2
Error building CMake.

real	7m15.276s
user	5m26.048s
sys	1m33.521s
************************************************************************
Error installing package cmake-3.2.1
************************************************************************
...

I suspect that we don't need the fully blown OSX cmake (we don't need OSXScriptLauncher probably), perhaps some options need to be used while building on OSX...

[fbissey]

My experience with kitware stuff on OS X is they assume clang and most other stuff is in the too hard basket. I was helping someone get paraview on their OS X box and we had to give up using gcc starting 10.10.2. I will have a look if we can do something at booststrap time but I am not especially hopeful.

[fbissey]

Current error is while building internal version of curl which is of course available by default on OS X. But using the system curl just push the problem further away. Attentive people will notice that the problematic header is the one we copy and modify for gcc install on 10.10.

I hate to do that, but my suggestion is for this to set CC=clang CXX=clang++ on OS X.

[fbissey]

OK, it is worse than I thought. Not only do we get a gcc that will sometimes break, when gcc breaks so does clang because of that little header. I am wondering if we should remove that header once gcc is built. It looks like a lot of stuff that will want that header will break regardless. Thoughts?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​72a020d

Make cmake/spkg-install executable

[isuruf]

Replying to jdemeyer:

Replying to dimpase:

The installation works on Linux, at least. How does one check it is not broken?

cmake has a test-suite run with make test. So a spkg-check file should be added to run this.

cmake has a test suite that can be run using ctest. I wrote a spkg-check and ran the tests on Linux. 3/382 tests fail, two of which is related to curl, and the other about a sleep command. I'm not sure whether these options will be required by packages in the future. 23 - curl (Failed) 178 - CTestTestUpload (Failed) 218 - CTestTestStopTime (Failed)

One more remark: you don't really need to split up $MAKE and $MAKE install in two steps, just $MAKE install is sufficient.

Thanks. I will make this change.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3aae9bf

CMake : Merge make and make install steps into one

[fbissey]

At the moment I would recommend people who want cmake on OS X to download it from kitware and make sure that the cmake command is in their PATH. We could put some words of that effect in spkg-install. I don't see this working on OS X anytime soon. We would need a breakthrough with gcc or clang.

[dimpase]

Replying to fbissey:

At the moment I would recommend people who want cmake on OS X to download it from kitware and make sure that the cmake command is in their PATH. We could put some words of that effect in spkg-install. I don't see this working on OS X anytime soon. We would need a breakthrough with gcc or clang.

I won't be surprised if this cmake won't be useful within Sage, as it might assume too much of the native OSX toolchain.

cmake was mentioned by me in a GSoC 2015 discussion regarding getting CSymPy into Sage; I expressed my doubts that cmake, which is used to configure CSymPy, is easy to deal with in Sage, based on some vague memories about cmake/Sage difficulties I saw. Now I recall that that was about building cmake in Sage on OSX.

[fbissey]

Replying to dimpase:

Replying to fbissey:

At the moment I would recommend people who want cmake on OS X to download it from kitware and make sure that the cmake command is in their PATH. We could put some words of that effect in spkg-install. I don't see this working on OS X anytime soon. We would need a breakthrough with gcc or clang.

I won't be surprised if this cmake won't be useful within Sage, as it might assume too much of the native OSX toolchain.

cmake was mentioned by me in a GSoC 2015 discussion regarding getting CSymPy into Sage; I expressed my doubts that cmake, which is used to configure CSymPy, is easy to deal with in Sage, based on some vague memories about cmake/Sage difficulties I saw. Now I recall that that was about building cmake in Sage on OSX.

I don't think there is a difficulty per see in using cmake on OS X. The trouble building cmake and in fact gcc and in turn a bunch of other stuff is solely the fault of apple pushing its own stuff and not caring. gcc is currently slightly broken on OS X and unless the gcc team gets something together it will just get worse. Not being able to build cmake is just one of the many symptoms of gcc cracking while apple is pushing hard.

[dimpase]

cmake 3.2.2 has been released, so this ticket ought to be adjusted to this version, I suppose.

[jdemeyer]

@fbissey: and the problem is that we really need GCC for Sage, because we need a Fortran compiler and clang doesn't (yet) support Fortran.

[fbissey]

Replying to jdemeyer:

@fbissey: and the problem is that we really need GCC for Sage, because we need a Fortran compiler and clang doesn't (yet) support Fortran.

I know. Although I have been trying very hard to think about fortran code in sage apart from lapack. I think the only fortran bits where you need a fortran compiler are in atlas, scipy and unfortunately for OS X: R.

We use the accelerate framework instead of Atlas on OS X, one less fortran thing.

A fortran compiler is needed while compiling scipy :( lots of fortran code in there.

R is more pernicious, you need a fortran compiler on OS X because there is a bug preventing us to use accelerate.

flang is just a dot on the horizon so far. At least they say they started work on it.

[dimpase]

Replying to fbissey:

Replying to jdemeyer:

@fbissey: and the problem is that we really need GCC for Sage, because we need a Fortran compiler and clang doesn't (yet) support Fortran.

I know. Although I have been trying very hard to think about fortran code in sage apart from lapack. I think the only fortran bits where you need a fortran compiler are in atlas, scipy and unfortunately for OS X: R.

We use the accelerate framework instead of Atlas on OS X, one less fortran thing.

A fortran compiler is needed while compiling scipy :( lots of fortran code in there.

R is more pernicious, you need a fortran compiler on OS X because there is a bug preventing us to use accelerate.

flang is just a dot on the horizon so far. At least they say they started work on it.

I don't even see dots, apart from some github repos untouched since 2013 or so.

We can consider going back to having a dedicated gfortran package, as it used to be the case before OSX 10.7 (IIRC) broken xcode trouble started, but otherwise use xcode's compilers.

[jdemeyer]

Replying to dimpase:

We can consider going back to having a dedicated gfortran package, as it used to be the case before OSX 10.7 (IIRC) broken xcode trouble started, but otherwise use xcode's compilers.

It's not just a matter of using gfortran. It's also a matter of linking code together (good luck linking a library containing pieces compiled by clang and other pieces by gfortran).

[jdemeyer]

About CMake: it's very ironic that the cmake authors wrote a build system which is supposed to make it easy to build cross-platform software, but they cannot make cmake itself build properly! That's really not good advertising for cmake.

[dimpase]

Replying to jdemeyer:

About CMake: it's very ironic that the cmake authors wrote a build system which is supposed to make it easy to build cross-platform software, but they cannot make cmake itself build properly! That's really not good advertising for cmake.

they didn't use autotools, I suppose, that's why ;-)

[leif]

Replying to jdemeyer:

Replying to dimpase:

We can consider going back to having a dedicated gfortran package, as it used to be the case before OSX 10.7 (IIRC) broken xcode trouble started, but otherwise use xcode's compilers.

It's not just a matter of using gfortran. It's also a matter of linking code together (good luck linking a library containing pieces compiled by clang and other pieces by gfortran).

And there are still parts of Sage which currently don't build with clang...

Otherwise, w.r.t. gfortran on MacOS X: Revive f2c! ;-)

[certik]

Thanks for trying to compile cmake.

A few comments:

• if csympy is the only package that requires cmake and cmake is impossible to build, we can also support an alternative build system in csympy

• I really like cmake, so let me try to fix these issues first, as I would prefer to keep cmake as the build system (but it needs to reliably build on OS X, both with clang and with gcc)

• I independently discovered lots of similar issues with building cmake with gcc on OS X 10.10 (​https://github.com/hashdist/hashstack/issues/746), and just like you noticed, it builds fine with the native clang. As you can see, you can easily tell cmake not to compile in various things and rather reuse them from the system or Sage. I really believe all these issues can be fixed and I am also sure Kitware is itself interested in cmake being able to build, e.g. Robert Maynard from Kitware already commented on my issue above with some tips.

So I will have a look in the coming days and I will update this issue.

[leif]

I guess this is still pretty much work in progress, but we certainly should use/tweak a couple of configure options for bootstrapping within Sage, such as for example

  --parallel=n            bootstrap cmake in parallel, where n is
                          number of nodes [1]

  --no-system-libs        use all cmake-provided third-party libraries
                          (default)

  --system-zlib           use system-installed zlib library
  --no-system-zlib        use cmake-provided zlib library (default)
  --system-bzip2          use system-installed bzip2 library
  --no-system-bzip2       use cmake-provided bzip2 library (default)

  ...

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​e2c3532	Added CMake v3.2.1 optional package
​1cbb417	Word-wrap cmake/SPKG.txt
​79d84cc	Make cmake/spkg-install executable
​4a219ec	CMake : Merge make and make install steps into one
​2870d55	CMake: use sage's bzip2 and zlib
​fe8eb4a	Test CMake installation
​c8778a9	Patch for building CMake on OSX 10.10

[jdemeyer]

A lot has changed in the handling of optional packages recently, this should be rebased on top of Sage 6.8.beta5.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​eadda32	Added CMake v3.2.1 optional package
​a84859e	Word-wrap cmake/SPKG.txt
​feadaf3	Make cmake/spkg-install executable
​919f12e	CMake : Merge make and make install steps into one
​268dee9	CMake: use sage's bzip2 and zlib
​def4c8a	Test CMake installation
​c73f9e2	Patch for building CMake on OSX 10.10
​6fea35f	CMake : Add type as experimental

[fbissey]

Is the OS X patch still needed with cmake 3.2.3? If not I suggest bumping to that and I will test it on my OS X box. If it builds I think we can review it positively as an experimental package. While reviewing we have pushed at the optional level of quality in review rather than experimental in my opinion.

[fbissey]

I realise that I thought it was experimental because that's what the type says in the git commit. But this ticket says "optional" and I think we are getting there so the type will have to be changed too.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0096eed

CMake : updated version to 3.2.3

[isuruf]

Yes, the patch is still needed for 3.2.3

I added experimental because even with the patch 6 tests out of 387 tests in OSX 10.10 fail. Once it is ready to become optional, I'll update the type to optional.

[jdemeyer]

Invalid checksum for cached file /usr/local/src/sage-config/upstream/cmake-3.2.3.tar.gz, deleting

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b65bb43

CMake : update checksums

[isuruf]

Sorry about that, chrome seems to be playing tricks on me. It's updated now.

[isuruf]

It seems that the curl library included with CMake is outdated and needs to be built separately for CMake to work in OSX.

[dimpase]

Does it really need curl? OSX does ship curl just fine...

[isuruf]

There are some features that involve downloading. That's why curl is used.

We can use system installed curl for OSX and build curl for other systems.

[dimpase]

curl is really very standard thing on Linux, installed by default. I don't know why cmake needs it for anything but Windows...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d71e284

CMake : Use system curl

[isuruf]

I pushed in a commit to use system curl when building cmake.

@dimpase, yes, there's really no need to build curl as it is installed in OSX and Linux. Building cmake in windows is not supported, so no use there either.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cb96c05

Use system-curl only in OSX and build curl in Linux

[fbissey]

Let's get this in, this is experimental after all.