#25113 closed defect (fixed)
OSX is f*ed up sometimes
Reported by:  vbraun  Owned by:  

Priority:  blocker  Milestone:  sage8.2 
Component:  build  Keywords:  
Cc:  fbissey  Merged in:  
Authors:  Volker Braun  Reviewers:  François Bissey 
Report Upstream:  N/A  Work issues:  
Branch:  53315d1 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
The latest XCode broke Sage
sage t long src/doc/de/tutorial/tour_advanced.rst # 2 doctests failed sage t long src/doc/en/tutorial/tour_advanced.rst # 2 doctests failed sage t long src/doc/fr/tutorial/tour_advanced.rst # 2 doctests failed sage t long src/doc/ja/tutorial/tour_advanced.rst # 2 doctests failed sage t long src/doc/pt/tutorial/tour_advanced.rst # 2 doctests failed sage t long src/doc/ru/tutorial/tour_advanced.rst # 2 doctests failed sage t long src/sage/rings/polynomial/groebner_fan.py # 70 doctests failed sage t long src/sage/rings/polynomial/multi_polynomial_ideal.py # 1 doctest failed
Specifically, as reported on sagedevel:
sage t long src/sage/rings/polynomial/multi_polynomial_ideal.py ********************************************************************** File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 3304, in sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_fan Failed example: g.reduced_groebner_bases() Exception raised: Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/VANILLA/sage8.2.rc0/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 551, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/VANILLA/sage8.2.rc0/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 961, in compile_and_execute exec(compiled, globs) File "<doctest sage.rings.polynomial.multi_polynomial_ideal.NCPolynomialIdeal.groebner_fan[3]>", line 1, in <module> g.reduced_groebner_bases() File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/VANILLA/sage8.2.rc0/local/lib/python2.7/sitepackages/sage/rings/polynomial/groebner_fan.py", line 1064, in reduced_groebner_bases X = [ReducedGroebnerBasis(self, [S(f) for f in G[i].split(',')], G[i]) for i in range(len(G))] File "sage/structure/parent.pyx", line 920, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9734) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 145, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4555) raise File "sage/structure/coerce_maps.pyx", line 140, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4417) return C._element_constructor(x) File "sage/rings/polynomial/multi_polynomial_libsingular.pyx", line 982, in sage.rings.polynomial.multi_polynomial_libsingular.MPolynomialRing_libsingular._element_constructor_ (build/cythonized/sage/rings/polynomial/multi_polynomial_libsingular.cpp:12001) raise TypeError("Could not find a mapping of the passed element to this ring.") TypeError: Could not find a mapping of the passed element to this ring. **********************************************************************
Change History (44)
comment:1 Changed 3 years ago by
 Cc fbissey added
comment:2 Changed 3 years ago by
 Summary changed from OSX is f*ed up to OSX is f*ed up sometimes
comment:3 Changed 3 years ago by
Which version of xcode?
comment:4 Changed 3 years ago by
The latest, naturally.
$ softwareupdate history Display Name Version Date    iTunes 12.7.4 03/04/2018 at 23:54:12 macOS High Sierra 10.13.4 Update 03/04/2018 at 23:54:12 Command Line Tools (macOS High Sierra version 10.13) for Xcode 9.3 03/04/2018 at 23:54:12 ...
comment:5 Changed 3 years ago by
And yes, I did run make distclean
, and then ./bootstrap
, ./configure CC=cc CXX=c++
And I don't have cchache
installed.
comment:6 Changed 3 years ago by
What else:
$ sysctl n machdep.cpu.brand_string Intel(R) Xeon(R) CPU E52697 v2 @ 2.70GHz
So it could be that some other CPUs are f*cked, but not this one.
comment:7 Changed 3 years ago by
Interesting possibility
$ sysctl n machdep.cpu.brand_string Intel(R) Core(TM) i54258U CPU @ 2.40GHz
comment:8 Changed 3 years ago by
Other possibilities  I have yasm
prebuilt, so I don't build it over and over again; as well, I have prebuilt autotools, and ran ./bootstrap
before building.
And next to no optional packages, it could be there is some rot there.
comment:9 Changed 3 years ago by
I did a clean build no optional packages. yasm
would be intriguing since it is only involved in gmp/mpir.
comment:10 Changed 3 years ago by
Probably we should do everything with SAGE_CHECK=yes
. I imagine this would show any problem with yasm while testing MPIR.
However, with this setting I get an error while testing cython
. Perhaps that's where the real problem is  however I don't know whether this worked before for cython
on OSX, or there were always errors...
comment:11 Changed 3 years ago by
On one machine:
$ softwareupdate history Display Name Version Date    macOS High Sierra 10.13.4 Update 03/30/2018, 08:36:15 Command Line Tools (macOS High Sierra version 10.13) for Xcode 9.3 03/30/2018, 08:36:15 iTunes 12.7.4 03/30/2018, 07:44:17
and
$ sysctl n machdep.cpu.brand_string Intel(R) Core(TM) i77700K CPU @ 4.20GHz
On another machine: similar for softwareupdate history
and
$ sysctl n machdep.cpu.brand_string Intel(R) Core(TM) i54260U CPU @ 1.40GHz
comment:12 Changed 3 years ago by
I saw the failures building from a fresh tarball, so I didn't run ./bootstrap
or anything else, just unpacked it and did make ptestlong
. (I am pretty sure that I also did make distclean
followed by make ptestlong
and got the same errors.)
comment:13 Changed 3 years ago by
Perhaps a way to test whether this is CPUdependent is to build with SAGE_FAT_BINARY on?
comment:14 Changed 3 years ago by
Right now I have one machine with a still functioning build (because I was doing incremental upgrades). I can try to break it, but once it's been broken, I can't repeat the test. Should I try to break it by doing ./sage f singular
? ./sage f some_other_package
? What would help to diagnose this problem?
comment:15 Changed 3 years ago by
I just tried ./sage ba
and doctests still pass. I won't try anything else for now.
comment:16 Changed 3 years ago by
Can you build yasm using gcc and use it for a fresh build (instead of letting Sage build yasm)? (yasm is like curl, configure will check if it is in the path and won't build it if it is there. I presume you can just create a link to such yasm you have built with gcc...)
comment:17 Changed 3 years ago by
By gcc
, you mean an actual gcc
and not /usr/bin/gcc
, which is actually clang
? I have a copy of gcc6.4.0
lying around, and I used that to build yasm
. Now a Sage build (using clang, not gcc) is underway.
comment:18 Changed 3 years ago by
Sure, I meant real gcc, not a fake one.
comment:19 followup: ↓ 21 Changed 3 years ago by
Compiling with SAGE_INSTALL_GCC=yes fixes it for me...
osx:~ vbraun$ sysctl n machdep.cpu.brand_string Intel(R) Core(TM) i73720QM CPU @ 2.60GHz
comment:20 Changed 3 years ago by
Using my own copy of yasm, I didn't run full doctests, but:
$ ./sage tp src/doc/*/tutorial/  sage t src/doc/ru/tutorial/tour_advanced.rst # 2 doctests failed sage t src/doc/ja/tutorial/tour_advanced.rst # 2 doctests failed sage t src/doc/en/tutorial/tour_advanced.rst # 2 doctests failed sage t src/doc/de/tutorial/tour_advanced.rst # 2 doctests failed sage t src/doc/fr/tutorial/tour_advanced.rst # 2 doctests failed sage t src/doc/pt/tutorial/tour_advanced.rst # 2 doctests failed 
comment:21 in reply to: ↑ 19 Changed 3 years ago by
Replying to vbraun:
Compiling with SAGE_INSTALL_GCC=yes fixes it for me...
osx:~ vbraun$ sysctl n machdep.cpu.brand_string Intel(R) Core(TM) i73720QM CPU @ 2.60GHz
I also get no failures with Sage 8.1, which uses gcc instead of clang.
comment:22 Changed 3 years ago by
Are all these errors point at gfan? One cannot tell without seeing errors in nongfan dirs...
Does anyone know how to run gfan's testsuite? It would be a very good idea to do so, also adding spkgcheck for gfan would be great...
There seems to be no make targets for this, is it done by some other tool?
comment:23 Changed 3 years ago by
 Description modified (diff)
comment:24 Changed 3 years ago by
 Branch set to u/vbraun/osx_is_f_ed_up_sometimes
comment:25 Changed 3 years ago by
 Commit set to 53315d1e86fea4c159edf73fb97c6cc1e4949a1a
FWIW gfan has a "make check" target but at least on gfan 0.6.2 it also fails with gcc on OSX.
The fact that this is archdependent suggests that it is a bug in the clang optimizer.
In any case, we should just use gcc until this is fixed.
New commits:
53315d1  Use GCC to build on OSX until XCode works again

comment:26 Changed 3 years ago by
 Status changed from new to needs_review
For testing you need to run autoconf (and not download the packaged confball)
comment:27 Changed 3 years ago by
Should it be inside the [...]
associated with the test for darwin? Rather than just out? It looks to me like it will be applied to all OSes.
comment:28 followup: ↓ 29 Changed 3 years ago by
No, the inside [] is not expanded in the configure.ac > configure step. The end of the block is the fi
comment:29 in reply to: ↑ 28 Changed 3 years ago by
Replying to vbraun:
No, the inside [] is not expanded in the configure.ac > configure step. The end of the block is the
fi
Of course! Silly me, I wasn't thinking properly.
comment:30 Changed 3 years ago by
I also tripped over that one on the first go ;)
comment:31 Changed 3 years ago by
I'd rather this branch be on a stopgap ticket while this one remains open until a proper fix is found.
comment:32 Changed 3 years ago by
How about simply makeing a separate ticket for the proper fix to gfan?
comment:33 Changed 3 years ago by
This is now ￼#25118: gfan fails when compiled with XCode 9.3
comment:34 Changed 3 years ago by
So can somebody set this to positive review?
comment:35 Changed 3 years ago by
 Reviewers set to François Bissey
 Status changed from needs_review to positive_review
LGTM.
comment:36 Changed 3 years ago by
Also on the ancient laptop tests fail like on i5/i7
$ sysctl n machdep.cpu.brand_string Intel(R) Core(TM)2 Duo CPU L9400 @ 1.86GHz
interestingly, clang is newer: Apple LLVM version 9.1.0 (clang902.0.39.1)
comment:37 Changed 3 years ago by
 Branch changed from u/vbraun/osx_is_f_ed_up_sometimes to 53315d1e86fea4c159edf73fb97c6cc1e4949a1a
 Resolution set to fixed
 Status changed from positive_review to closed
comment:38 followup: ↓ 39 Changed 3 years ago by
 Commit 53315d1e86fea4c159edf73fb97c6cc1e4949a1a deleted
I must say I don't understand the rationale for this haste. Previous versions of Xcode are available, just don't use the broken one!
After all, it's what we do with gcc, we are not rushing to use the latest greatest, we are fine with gcc 5.4 still, right?
(One annoying part is that the older versions are harder to deploy, as they need to be downloaded from https://developer.apple.com/download/more/ but other than that...)
comment:39 in reply to: ↑ 38 Changed 3 years ago by
Replying to dimpase:
I must say I don't understand the rationale for this haste. Previous versions of Xcode are available, just don't use the broken one!
After all, it's what we do with gcc, we are not rushing to use the latest greatest, we are fine with gcc 5.4 still, right?
I am assuming most people will be up to date and using the latest version they can. OS X users usually do. I am not sure how easy or hard it is to downgrade and stay there. Generally it is not nice to tell users to downgrade what they have.
You are right we could be pinpointing the version in question but actually maintaining a blacklist is quite costly.
comment:40 Changed 3 years ago by
condaforge stays with Xcode 6, iirc.
No, I think on OSX people are not rushing to get the latest...
comment:41 Changed 3 years ago by
By the way, apparently Apple has been changing contents of their "Command line tools" bundles without bumping up versions, not sure whether it's the 1st accident of this sort, or not.
Namely, on my old laptop I installed Command Line Tools (macOS High Sierra version 10.13) for Xcode 9.3 on the 31st of March, and it had Apple LLVM version 9.1.0 (clang902.0.39.1)
(sic!), giving a broken gfan.
Yesterday I went and installed Command Line Tools (macOS High Sierra version 10.13) for Xcode 9.2 from their website,
and it has Apple LLVM version 9.0.0 (clang900.0.39.2)
, which results in OK gfan (with just one test broken, as on Ubuntu).
Could you people please install Command Line Tools (macOS High Sierra version 10.13) for Xcode 9.3 from https://developer.apple.com/download/more/ and test? (I presume you will get clang902.0.39.1 this way).
comment:42 Changed 3 years ago by
We should arguing here and move that bit to #25118. Still that numbering makes you wonder if someone did build and ship the wrong level of tools or something.
comment:43 Changed 3 years ago by
Apparently the version of LLVM (from the same package!) depends upon the arch. I just tried installing Command Line Tools (macOS High Sierra version 10.13) for Xcode 9.3 on the Xeon box, but it did not result in a newer version.
comment:44 Changed 3 years ago by
Anyhow, I'm very close to have a complete fix for this on #25118.
I cannot reproduce this