Opened 3 years ago

Closed 3 years ago

#21567 closed defect (fixed)

Allow Sage to build on OS X 10.12 (Sierra)

Reported by: jhpalmieri Owned by:
Priority: blocker Milestone: sage-7.4
Component: build Keywords:
Cc: mstreng Merged in:
Authors: John Palmieri Reviewers: François Bissey
Report Upstream: N/A Work issues:
Branch: 2d85bc9 (Commits) Commit: 2d85bc965d3f751ff0175a2d2fd09a737250de74
Dependencies: Stopgaps:

Description

Trying to build Sage from scratch on OS X 10.12 results in a broken Python:

    Failed to build these modules:
    _scproxy    

We've seen this before -- see #17174 and #17169.

Change History (59)

comment:1 Changed 3 years ago by mstreng

  • Cc mstreng added

comment:2 Changed 3 years ago by klee

On Macbook Pro with macOS Sierra, making the latest Sage beta I got this:

[gcc-4.9.3.p1] The directory that should contain system headers does not exist:
[gcc-4.9.3.p1]   /usr/include
[gcc-4.9.3.p1] make[6]: *** [stmp-fixinc] Error 1
[gcc-4.9.3.p1] make[5]: *** [all-stage1-gcc] Error 2
[gcc-4.9.3.p1] make[4]: *** [stage1-bubble] Error 2
[gcc-4.9.3.p1] make[3]: *** [all] Error 2
[gcc-4.9.3.p1] 
[gcc-4.9.3.p1] real	5m10.515s
[gcc-4.9.3.p1] user	4m4.134s
[gcc-4.9.3.p1] sys	0m59.514s
[gcc-4.9.3.p1] ************************************************************************
[gcc-4.9.3.p1] Error installing package gcc-4.9.3.p1
[gcc-4.9.3.p1] ************************************************************************

I have no idea how to fix this.

comment:3 Changed 3 years ago by klee

I am retrying after installing Xcode commandline tools.

comment:4 follow-up: Changed 3 years ago by klee

It does not finish building gcc-4.9.3.p1. It seems to have fallen in an infinite loop.

comment:5 in reply to: ↑ 4 ; follow-up: Changed 3 years ago by jdemeyer

Replying to klee:

It seems to have fallen in an infinite loop.

Are you sure? Maybe it just needs more time.

comment:6 Changed 3 years ago by jhpalmieri

It built on my machine, for what that's worth. I ran into problems later, as noted above. You could also try reinstalling Xcode in addition to the command line tools.

comment:7 in reply to: ↑ 5 Changed 3 years ago by klee

Replying to jdemeyer:

Replying to klee:

It seems to have fallen in an infinite loop.

Are you sure? Maybe it just needs more time.

You are right. It is not compiling the same file again and again.

comment:8 follow-up: Changed 3 years ago by klee

After long compilation, make finally failed with

[setuptools-24.0.2] ImportError: No module named _scproxy

I could not find a message in the log about the failure of building _scproxy itself.

comment:9 in reply to: ↑ 8 Changed 3 years ago by jhpalmieri

Replying to klee:

I could not find a message in the log about the failure of building _scproxy itself.

Did you look in the python build log?

comment:10 Changed 3 years ago by klee

Yes, it is there

Failed to build these modules:
_scproxy 

Sorry for the noise. I didn't help you :-) I have never inspected deep into the build process before.

comment:11 Changed 3 years ago by klee

I hope that this issue was solved in the latest Python 2.7.12. The relevant ticket is #19735.

comment:12 Changed 3 years ago by fbissey

I must say I haven't tried yet a straight build on Sierra. I'll have to look in my logs for python, because I did a clang build (failing building sagelib but completely different problems).

comment:13 follow-up: Changed 3 years ago by jhpalmieri

Is there any reason to believe that the new Python will help to build Sage on OS X 10.12? I can build Python 2.7.10 on OS X 10.12 outside of Sage's environment and _scproxy builds fine; it's only with Sage's gcc (etc.) that Python has problems.

comment:14 in reply to: ↑ 13 Changed 3 years ago by klee

Replying to jhpalmieri:

Is there any reason to believe that the new Python will help to build Sage on OS X 10.12?

A reason not so well grounded... By unguided internet search, I learned that _scproxy is Mac-specific and related with other problems. So I expected somehow new Python did something about that.

comment:15 Changed 3 years ago by jhpalmieri

To be clear: I have no idea how to fix this. I don't understand why the fixes worked when this sort of problem occurred before (as mentioned in the summary). But if people make suggestions, I am happy to try them out. Or if you want to see some log files or other information, I can post it. It's important that we fix this fast, since Mac OS X is one of our supported platforms and Sage doesn't build on the newest OS X release.

comment:16 Changed 3 years ago by fbissey

The last time it was fixed by a gcc upgrade. Why not try #21603?

comment:17 Changed 3 years ago by jhpalmieri

It was fixed by a gcc upgrade plus what looks like a hack -- from the gcc spkg-install script:

# On OS X 10.10 there is random ObjC stuff in a C header
if { uname -sr | grep 'Darwin 14\.' ;} &>/dev/null; then
    if [ -f /usr/include/dispatch/object.h ]; then
        mkdir -p "$SAGE_LOCAL/include/dispatch"
        sed 's+typedef void (\^dispatch_block_t)(void)+typedef void* dispatch_block_t+' \
            /usr/include/dispatch/object.h > "$SAGE_LOCAL/include/dispatch/object.h"
    else
        echo "Warning: header /usr/include/dispatch/object.h not found, not applying fix."
    fi
fi

I tried changing this so it would also apply if it matched Darwin 16., but it didn't help. I'm a little wary of #21603 because of #20350, but I'll give it a try later today.

comment:18 Changed 3 years ago by fbissey

Yes that hack is partly because the headers in OS X have objective C bits mixed in (and not properly screened by define) and apple doesn't see it as a problem. I do remember the stuff we found when doing that upgrade.

The ABI problem is, let's say inconvenient. However compiling everything with the same c++ compiler should work, so long as we don't involve c++ libraries from the host system.

One thing I found in my first attempt at building sage with clang on OS X (10.11 at the time) was that clang++ and g++ (for 4.9.3) do not use the same ABI. So I couldn't compile linbox with g++ while givaro had been compiled with clang++. g++ offered me an option to change ABI but that lead to another issue where object code was not recognized as x86_64 arch. #21603 may improve on that (cross fingers).

comment:19 Changed 3 years ago by fbissey

More cruft I am guessing

[python2-2.7.10.p2] gcc -fno-strict-aliasing -I/Users/fbissey/build/sage/local/var/tmp/sage/build/python2-2.7.10.p2/include -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -I/Users/fbissey/build/sage/local/include -I. -IInclude -I./Include -I/usr/local/include -I/Users/fbissey/build/sage/local/var/tmp/sage/build/python2-2.7.10.p2/src/Include -I/Users/fbissey/build/sage/local/var/tmp/sage/build/python2-2.7.10.p2/src -c ./Mac/Modules/_scproxy.c -o build/temp.macosx-10.9-x86_64-2.7/./Mac/Modules/_scproxy.o
[python2-2.7.10.p2] In file included from /usr/include/Availability.h:176:0,
[python2-2.7.10.p2]                  from /usr/include/stdio.h:65,
[python2-2.7.10.p2]                  from Include/Python.h:33,
[python2-2.7.10.p2]                  from ./Mac/Modules/_scproxy.c:5:
[python2-2.7.10.p2] /System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:53:34: error: expected ',' or '}' before '__attribute__'
[python2-2.7.10.p2]      kCFISO8601DateFormatWithYear API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0)) = (1UL << 0),
[python2-2.7.10.p2]                                   ^
[python2-2.7.10.p2] /System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:80:126: error: 'introduced' undeclared here (not in a function)
[python2-2.7.10.p2]  CFDateFormatterRef CFDateFormatterCreateISO8601Formatter(CFAllocatorRef allocator, CFISO8601DateFormatOptions formatOptions) API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0));
[python2-2.7.10.p2]                                                                                                                               ^
[python2-2.7.10.p2] /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'deprecated' undeclared here (not in a function)
[python2-2.7.10.p2]  const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
[python2-2.7.10.p2]                                        ^
[python2-2.7.10.p2] /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'message' undeclared here (not in a function)
[python2-2.7.10.p2]  const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
[python2-2.7.10.p2]                                        ^
[python2-2.7.10.p2] In file included from /System/Library/Frameworks/Security.framework/Headers/Security.h:81:0,
[python2-2.7.10.p2]                  from /System/Library/Frameworks/SystemConfiguration.framework/Headers/SCPreferences.h:35,
[python2-2.7.10.p2]                  from /System/Library/Frameworks/SystemConfiguration.framework/Headers/SystemConfiguration.h:124,
[python2-2.7.10.p2]                  from ./Mac/Modules/_scproxy.c:6:
[python2-2.7.10.p2] /System/Library/Frameworks/Security.framework/Headers/Authorization.h:194:7: error: variably modified 'bytes' at file scope
[python2-2.7.10.p2]   char bytes[kAuthorizationExternalFormLength];
[python2-2.7.10.p2]        ^
[python2-2.7.10.p2] ./Mac/Modules/_scproxy.c: In function 'get_proxy_settings':
[python2-2.7.10.p2] ./Mac/Modules/_scproxy.c:67:50: warning: the comparison will always evaluate as 'true' for the address of 'kSCPropNetProxiesExcludeSimpleHostnames' will never be NULL [-Waddress]
[python2-2.7.10.p2]      if (&kSCPropNetProxiesExcludeSimpleHostnames != NULL) {
[python2-2.7.10.p2]                                                   ^
[python2-2.7.10.p2] building '_tkinter' extension

Not sure #21603 will help in the end. Diving in that header file.

comment:20 follow-up: Changed 3 years ago by fbissey

The mixing of objective C with C in system headers is deep with that one. I am almost ready to suggest we just compile python with clang on OS X. Should work.

comment:21 Changed 3 years ago by jhpalmieri

I tried #21603 and got the same issue with Python: Failed to build these modules: _scproxy.

comment:22 in reply to: ↑ 20 Changed 3 years ago by jhpalmieri

Replying to fbissey:

The mixing of objective C with C in system headers is deep with that one. I am almost ready to suggest we just compile python with clang on OS X. Should work.

Interesting idea. I built python with clang and things mostly worked. r and psutil failed to build. I got around that by building r with clang and basically ignoring psutil. I two some doctest failures, but they were not repeatable. I'm going to run doctests again to see what happens.

The r failure:

gcc -std=gnu99 -I. -I../../../src/include -I../../../src/include -I.  -DLOCALEDIR=\"\" -DLOCALEALIAS_PATH=\"\" -DIN_LIBINTL -DHAVE_CONFIG_H -I/System/Library/Frameworks/CoreFoundation.framework/Headers  -fPIC  -g -O2    -c langprefs.c -o langprefs.o
In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CFStream.h:11:0,
                 from /System/Library/Frameworks/CoreFoundation.framework/Headers/CFPropertyList.h:13,
                 from langprefs.c:30:
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:74: error: 'introduced' undeclared here (not in a function)
 const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                                                          ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:90: error: 'deprecated' undeclared here (not in a function)
 const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                                                                          ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:107: error: 'message' undeclared here (not in a function)
 const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                                                                                           ^
make[7]: *** [langprefs.o] Error 1
make[6]: *** [all-yes] Error 2
make[5]: *** [make.intl] Error 2
make[5]: *** Waiting for unfinished jobs....

The psutil failure:

    gcc -fno-strict-aliasing -I/Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/python2-2.7.10.p2/include -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -DPSUTIL_VERSION=431 -I/Users/palmieri/Desktop/Sage_stuff/git/sage/local/include/python2.7 -c psutil/_psutil_osx.c -o build/temp.macosx-10.9-x86_64-2.7/psutil/_psutil_osx.o
    In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:53:0,
                     from psutil/_psutil_osx.c:36:
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:53:34: error: expected ',' or '}' before '__attribute__'
         kCFISO8601DateFormatWithYear API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0)) = (1UL << 0),
                                      ^
    In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:53:0,
                     from psutil/_psutil_osx.c:36:
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:80:161: error: 'introduced' undeclared here (not in a function)
     CFDateFormatterRef CFDateFormatterCreateISO8601Formatter(CFAllocatorRef allocator, CFISO8601DateFormatOptions formatOptions) API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0));
                                                                                                                                                                     ^
    In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CFStream.h:11:0,
                     from /System/Library/Frameworks/CoreFoundation.framework/Headers/CFPropertyList.h:13,
                     from /System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:60,
                     from psutil/_psutil_osx.c:36:
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:90: error: 'deprecated' undeclared here (not in a function)
     const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                                                                              ^
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:107: error: 'message' undeclared here (not in a function)
     const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                                                                                               ^
    error: command 'gcc' failed with exit status 1
    Running setup.py install for psutil: finished with status 'error'
Cleaning up...
  Removing source in /var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-d3w_bP-build
Command "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/bin/python -u -c "import setuptools, tokenize;__file__='/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-d3w_bP-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-zZTY0f-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-d3w_bP-build/
Exception information:
Traceback (most recent call last):
  File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/pip/commands/install.py", line 317, in run
    prefix=options.prefix_path,
  File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/pip/req/req_set.py", line 742, in install
    **kwargs
  File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/pip/req/req_install.py", line 880, in install
    spinner=spinner,
  File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 718, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/bin/python -u -c "import setuptools, tokenize;__file__='/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-d3w_bP-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-zZTY0f-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-d3w_bP-build/

Now I see that if I build that with clang, it works too. (In all of these cases, I just put CC=clang some appropriate place in spkg-install.)

comment:23 Changed 3 years ago by jhpalmieri

This basically works for me:

  • build/pkgs/psutil/spkg-install

    diff --git a/build/pkgs/psutil/spkg-install b/build/pkgs/psutil/spkg-install
    index c1a2289..6134562 100755
    a b  
    11#!/usr/bin/env bash
    22
     3if [ "$UNAME" = "Darwin" ] && [ $MACOSX_VERSION -ge 16 ]; then
     4    echo "OS X 10.$[$MACOSX_VERSION-4] Building with clang."
     5    CC=clang
     6fi
     7
    38cd src && $PIP_INSTALL .
  • build/pkgs/python2/spkg-install

    diff --git a/build/pkgs/python2/spkg-install b/build/pkgs/python2/spkg-install
    index e679291..683d5b8 100755
    a b if [ "$UNAME" = Darwin ]; then 
    5656    mkdir "$CUR"/include
    5757    cp -rp "${xcode}"/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift-migrator/sdk/MacOSX.sdk/usr/include/openssl "$CUR"/include
    5858    export CFLAGS="-I$CUR/include"
     59    if [ $MACOSX_VERSION -ge 16 ]; then
     60        echo "OS X 10.$[$MACOSX_VERSION-4] Building with clang."
     61        CC=clang
     62    fi
    5963elif [ "$UNAME" = SunOS ]; then
    6064    # Enable some C99 features on Solaris. This in particular enables
    6165    # the isinf() and isfinite() functions. It works both for C and
  • build/pkgs/r/spkg-install

    diff --git a/build/pkgs/r/spkg-install b/build/pkgs/r/spkg-install
    index 8601108..5b984fb 100755
    a b if [ "$UNAME" = "Darwin" ]; then 
    8080    if [ $MACOSX_VERSION -ge 14 ]; then
    8181        echo "OS X 10.$[$MACOSX_VERSION-4] Configuring R without aqua support."
    8282        R_CONFIGURE="--with-aqua=no $R_CONFIGURE"
    83     fi
     83    fi
     84    if [ $MACOSX_VERSION -ge 16 ]; then
     85        echo "OS X 10.$[$MACOSX_VERSION-4] Building with clang."
     86        CC=clang
     87    fi
    8488fi
    8589
    8690if [ "$UNAME" = "CYGWIN" ]; then

With this change, Sage builds and doctests almost pass. The one exception is that I occasionally get a "bad exit" for a seemingly random file. On two different attempts at make ptestlong I've seen

sage -t --long --warn-long 61.3 src/doc/ru/tutorial/tour_groups.rst  # Bad exit: 2
sage -t --long --warn-long 61.3 src/sage/matrix/matrix_integer_dense.pyx  # Timed out (and interrupt failed)

and

sage -t --long --warn-long 67.6 src/sage/rings/infinity.py  # Bad exit: 2

A third time I didn't see any of these problems.

comment:24 Changed 3 years ago by fbissey

Actually something I haven't look into. Since the last beta we build openblas on OS X but I haven't checked if we are using it or still use Accelerate. Could you check? I'd rather stick with Accelerate personally.

comment:25 Changed 3 years ago by jhpalmieri

openblas is being built on OS X. What should I check to figure out if it's actually being used?

comment:26 Changed 3 years ago by fbissey

Try

otool -L local/lib/libiml.dylib

comment:27 Changed 3 years ago by jhpalmieri

$ otool -L local/lib/libiml.dylib
local/lib/libiml.dylib:
	/Users/palmieri/Desktop/TESTING/sage/local/lib/libiml.0.dylib (compatibility version 2.0.0, current version 2.1.0)
	/Users/palmieri/Desktop/TESTING/sage/local/lib/libgmp.16.dylib (compatibility version 23.0.0, current version 23.2.0)
	/Users/palmieri/Desktop/TESTING/sage/local/lib/libopenblas_haswellp-r0.2.15.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
	/Users/palmieri/Desktop/TESTING/sage/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

comment:28 Changed 3 years ago by fbissey

Openblas in use then - minor detail, we should get a switch to use Accelerate but that's low priority.

Sierra makes Yosemite look like a picnic. Now the ants have arrived. Unless someone bundles some nice libc headers for OS X (I think intel may be doing this, I think they have their own libc) we are being pushed to clang.

sage is in no condition to be compiled with clang/clang++/gfortran right now, I don't think we can rely clang/g++/gfortran either at the moment. Using clang (not clang++) in a few key package is acceptable but in the long term, unless something happen in the GNU camp, we should prepare the move to clang/clang++/gfortran (or available fortran compiler coming along).

comment:29 follow-up: Changed 3 years ago by jhpalmieri

I can push my current branch with the changes in comment:23 if you think that's a good idea.

comment:30 in reply to: ↑ 29 Changed 3 years ago by fbissey

Replying to jhpalmieri:

I can push my current branch with the changes in comment:23 if you think that's a good idea.

Yes, that would allow people on Sierra to build sage even if we don't push it in this form to release.

I think we should aim for something like clang/g++/gfortran for packages and gcc/g++/gfortran for sagelib. clang/clang++/gfortran for package and gcc/clang++/gfortran for sagelib would be even better but I don't believe we can pull it off. But I should give it a go to be sure since I have already done the package side.

comment:31 Changed 3 years ago by jhpalmieri

  • Branch set to u/jhpalmieri/clang

comment:32 Changed 3 years ago by jhpalmieri

  • Commit set to 2d85bc965d3f751ff0175a2d2fd09a737250de74

Okay, here it is. I won't mark it ready for review in case you want to do something different, but this does work for me.


New commits:

2d85bc9On OS X 10.12, use clang for compiling Python, psutil, R.

comment:33 Changed 3 years ago by fbissey

Gone back to my clang build. After the first error with clang in sage/finance/time_series.pyx I switched to CC=gcc CXX=clang++. I got, first the same error as with the _scproxy compilation, followed by 10 to 20 thousands lines of error messages and warnings (I am serious, I don't think I ever got so many from a single compilation with any compilers on any systems) I am glad that mac terminal is set to "infinite" scrollback.

I think for now, limiting the usage of clang [and strictly no clang++] to a few packages will be ok.

comment:34 follow-up: Changed 3 years ago by jdemeyer

Just mentioning another option (without claiming that it will work): you could build the Objective-C and Objective-C++ compilers of GCC and use those. Those will probably be ABI-compatible with the other GCC compilers.

comment:35 in reply to: ↑ 34 ; follow-up: Changed 3 years ago by fbissey

Replying to jdemeyer:

Just mentioning another option (without claiming that it will work): you could build the Objective-C and Objective-C++ compilers of GCC and use those. Those will probably be ABI-compatible with the other GCC compilers.

That's worth a shot, I guess.

comment:36 in reply to: ↑ 35 Changed 3 years ago by jdemeyer

Replying to fbissey:

Replying to jdemeyer:

Just mentioning another option (without claiming that it will work): you could build the Objective-C and Objective-C++ compilers of GCC and use those. Those will probably be ABI-compatible with the other GCC compilers.

That's worth a shot, I guess.

To do that, you need to change the --enable-languages option in build/pkgs/gcc/spkg-install.

comment:37 Changed 3 years ago by fbissey

Yes. I am familiar with the option, thank you (and the option are objc and obj-c++). My little laptop takes a bit of time to do a full compile of gcc and sage for that matter, so it will take a little bit of time on my side once I am ready.

comment:38 Changed 3 years ago by jhpalmieri

So if I want to try this, should I use --enable-languages=objc,obj-c++,fortran ?

comment:39 Changed 3 years ago by fbissey

I don't believe you can skip c and c++. So it would have to be --enable-languages=c,c++,objc,obj-c++,fortran.

comment:40 Changed 3 years ago by jhpalmieri

Okay, and then do I have to tell certain packages to use objc instead of gcc? Or set CC=objc somewhere, or something like that?

comment:41 Changed 3 years ago by fbissey

I guess. It is exploratory at this stage, I don't know if the objective c compiler will take kindly to c option for example. One thing I tried yesterday was CC=clang++ which errored on -std=c99 - it stopped right there. g++ would probably have told me "not a valid option" and go on anyway, not clang++, it just stopped.

comment:42 Changed 3 years ago by jhpalmieri

I can't even get gcc 4.9.3 to build with --enable-languages=c,c++,objc,obj-c++,fortran

Last edited 3 years ago by jhpalmieri (previous) (diff)

comment:43 Changed 3 years ago by fbissey

What's the problem?

comment:44 Changed 3 years ago by jhpalmieri

Here is the last bit of the log file (in a serial build):

checking if /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/xgcc -B/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/bin/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/lib/ -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/include -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/sys-include    supports -c -o file.o... (cached) yes
checking whether the /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/xgcc -B/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/bin/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/lib/ -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/include -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/sys-include    linker (/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/collect-ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin16.0.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/xgcc -B/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/bin/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/lib/ -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/include -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/sys-include    and cc understand -c and -o together... yes
checking whether make sets $(MAKE)... yes
checking stdio.h usability... yes
checking stdio.h presence... yes
checking for stdio.h... yes
checking for ANSI C header files... (cached) yes
checking sched.h usability... yes
checking sched.h presence... no
configure: WARNING: sched.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: sched.h: proceeding with the compiler's result
checking for sched.h... yes
checking whether the target supports thread-local storage... no
checking for exception model to use... call frame
configure: updating cache ./config.cache
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: executing default-1 commands
config.status: executing libtool commands
/bin/sh ./libtool  --mode=compile /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/xgcc -B/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/bin/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/lib/ -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/include -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/sys-include    /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/NXConstStr.m -c \
	   -I. -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc   -g -O2 -W -Wall -Wwrite-strings -Wstrict-prototypes -DIN_GCC -DIN_TARGET_LIBS -fno-strict-aliasing -fexceptions -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../gcc -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../gcc/config -I../.././gcc -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../libgcc -I../libgcc -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../include  -fgnu-runtime \
	   -o NXConstStr.lo
libtool: compile:  /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/xgcc -B/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/gcc-build/./gcc/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/bin/ -B/Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/lib/ -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/include -isystem /Users/palmieri/Desktop/TESTING/sage/local/x86_64-apple-darwin16.0.0/sys-include /Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/NXConstStr.m -c -I. -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc -g -O2 -W -Wall -Wwrite-strings -Wstrict-prototypes -DIN_GCC -DIN_TARGET_LIBS -fno-strict-aliasing -fexceptions -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../gcc -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../gcc/config -I../.././gcc -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../libgcc -I../libgcc -I/Users/palmieri/Desktop/TESTING/sage/local/var/tmp/sage/build/gcc-4.9.3.p1/src/libobjc/../include -fgnu-runtime  -fno-common -DPIC -o .libs/NXConstStr.o
/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T//ccC1ir79.s:95:17: error: non-local symbol required in directive
        .no_dead_strip L_OBJC_Module
                       ^
make[5]: *** [NXConstStr.lo] Error 1
make[4]: *** [all-target-libobjc] Error 2
make[3]: *** [all] Error 2

comment:45 Changed 3 years ago by fbissey

OK not good. Could be better with 6.2.0 but that's a long shot.

comment:46 Changed 3 years ago by jhpalmieri

It built with 6.2.0, but I don't see a binary for objc or anything similar in local/bin. How do I use objective c?

comment:47 Changed 3 years ago by fbissey

Apparently gcc for objective c :) https://gcc.gnu.org/onlinedocs/gcc-4.5.3/gcc/Objective_002dC-and-Objective_002dC_002b_002b-Dialect-Options.html so that should give parity with clang in terms of language feature. Cross fingers that it work.

comment:48 Changed 3 years ago by jhpalmieri

No luck for me getting Python to build correctly, but I'm probably getting the CFLAGS wrong.

comment:49 Changed 3 years ago by fbissey

Same errors or different ones?

comment:50 Changed 3 years ago by jhpalmieri

If I use (at the end of an if [ "$UNAME" = Darwin ] block)

export CFLAGS="-I$CUR/include -fgnu-runtime "

then Python builds but I get

Failed to build these modules:
_scproxy

Same if I use

export CFLAGS="-I$CUR/include -fnext-runtime "

comment:51 Changed 3 years ago by fbissey

I think we may be on the wrong track, remember #18254? I will play around with MACOSX_DEPLOYMENT_TARGET currently stuck on 10.9 as the maximum. Somehow something gone "boing" in my install, so make distclean before restarting gcc then takes some time...

comment:52 Changed 3 years ago by fbissey

I think this is #18254 all over again except that setting MACOSX_DEPLOYMENT_TARGET to 10.9 (or 10.12) doesn't fix the problem. Unsetting failed me too.

comment:53 Changed 3 years ago by fbissey

And because we are using rpath we need to set it at 10.5 minimum - not unset it. I think we are cornered into going clang. Any of the other people in brew, macports and al using gcc have suggestions? Couldn't find any on google.

comment:54 follow-up: Changed 3 years ago by vbraun

The failure in matrix_integer_dense.pyx is also very common on OSX 10.11 right now (possibly related to openblas, not sure). Why not go with comment:23, its definitely an improvement over not building.

comment:55 in reply to: ↑ 54 Changed 3 years ago by fbissey

Replying to vbraun:

The failure in matrix_integer_dense.pyx is also very common on OSX 10.11 right now (possibly related to openblas, not sure). Why not go with comment:23, its definitely an improvement over not building.

matrix_integer_dense.pyx needs its own ticket. It wants to be compiled as both a c and a c++ file at the same time. module_list.py gives it -std=c99, linbox is pulled in its dependencies from somewhere and bring -std=gnu++11 and the cythonized file ends up in .cpp. Removing the -std=c99 from module_list.py and adding language =c++ doesn't make a difference. Something else puts back the std=c99 (multi_modular.pxd?). This is my current obstacle in porting to clang at #12426.

comment:56 Changed 3 years ago by fbissey

I am otherwise OK to go with the branch rather than not building.

Last edited 3 years ago by fbissey (previous) (diff)

comment:57 Changed 3 years ago by fbissey

  • Authors set to John Palmieri
  • Reviewers set to François Bissey
  • Status changed from new to needs_review

comment:58 Changed 3 years ago by fbissey

  • Status changed from needs_review to positive_review

Making it positive if you want to include it.

comment:59 Changed 3 years ago by vbraun

  • Branch changed from u/jhpalmieri/clang to 2d85bc965d3f751ff0175a2d2fd09a737250de74
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.