Opened 2 years ago
Closed 20 months ago
#30651 closed enhancement (invalid)
Meta-ticket: support macOS 11 Big Sur
Reported by: | slelievre | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | porting | Keywords: | macOS |
Cc: | dimpase, jhpalmieri, mkoeppe, slelievre, vbraun, was | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #30494 | Stopgaps: |
Description (last modified by )
This is to support macOS 11 Big Sur.
- #30752 gmp instead of mpir
- #30929 GH Actions: Update macos test environments
- #30745 homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local
- #31050 upgrade wheel
MACOSX_DEPLOYMENT_TARGET
problems - https://groups.google.com/g/sage-devel/c/Sj3YBVJoHQM/m/Zt-Ih_hJAwAJ- #31128 MR50: Upgrade cffi to 1.14.4
- #31134 upgrade setuptools and setuptools_scm
- #30589 Upgrade Python to 3.9.1, pip to 20.3.3
- #31166 numpy: Fix for macOS 11 (Big Sur)
- #31183 scipy: fix for macOS 11 (Big Sur)
- #31326
homebrew-macos-11.0-xcode-minimal
:scipy
build fails
Related:
References:
Attachments (7)
Change History (77)
comment:1 Changed 2 years ago by
Milestone: | sage-9.2 → sage-9.3 |
---|
comment:2 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:4 follow-ups: 5 14 Changed 2 years ago by
I've actually gotten this to build for me without patching Sage in any way, just building 9.2.rc0. Homebrew setup:
% brew list autoconf gd haskell-stack libpng ncurses r automake gdbm icu4c libtasn1 nettle re2c bdw-gc gettext ilmbase libtiff ninja readline bison ghc intltool libtool openblas sphinx-doc boost ghostscript isl libunistring openexr sqlite bzip2 git jansson libxml2 openssl@1.1 suite-sparse cmake glib jpeg meson p11-kit tbb curl glpk libatomic_ops metis pandoc texinfo doxygen gmp libevent mpc pcre unbound fontconfig gnutls libffi mpfi pcre2 webp freetype gpatch libidn2 mpfr pkg-config xz gcc gsl libmpc mpir ppl yasm gcc@9 guile libmpdclient nauty python@3.8 zeromq
Then I used the command
$ ./configure \ --with-system-readline=no \ --with-system-zlib=no \ --with-system-gfortran=no \ --with-system-openblas=no \ --with-system-bzip2=no
after which make
worked but make ptestlong
showed a bunch of failures, all like this:
Expected nothing Got: ld: warning: dylib (/usr/local/lib/libgmp.dylib) was built for newer macOS version (10.15) than being linked (10.9) ld: warning: dylib (/usr/local/lib/libmpfr.dylib) was built for newer macOS version (10.15) than being linked (10.9) ld: warning: dylib (/usr/local/lib/libgmpxx.dylib) was built for newer macOS version (10.15) than being linked (10.9)
Maybe because MACOSX_DEPLOYMENT_TARGET
is being set? I could try building Sage's mpfr
and gmp
. But in any case, I got Sage to build, unfortunately by not using a bunch of the system versions of things.
comment:5 Changed 2 years ago by
Replying to jhpalmieri:
Maybe because
MACOSX_DEPLOYMENT_TARGET
is being set? I could try building Sage'smpfr
andgmp
.
I could try that, but it won't work because our mpir
is broken when using Xcode 12, with the usual error, and the latest release of mpir
(git clone git://github.com/wbhart/mpir.git mpir
) doesn't help.
comment:6 follow-up: 8 Changed 2 years ago by
mpir is basically not developed any more (for the time being at least), so we should not use it.
What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.
comment:7 Changed 2 years ago by
Replying to gh-wdjoyner:
I'm trying to compile sage-9.2.rc0 on big sur with xcode 12.0.1.
I ran sage -f openblas. Here's the tail of that:
[...]
David, could you please replace this 10-screen comment by an attachement?
comment:8 follow-ups: 9 11 Changed 2 years ago by
Replying to dimpase:
mpir is basically not developed any more (for the time being at least), so we should not use it.
It's a standard package, so I'm confused about what the alternatives are.
Anyway, I hacked the spkg-install.in script for mpir
, as in #30729, and I got it to build. That eliminated those doctest failures. I'm left with two:
sage -t --long --random-seed=0 src/sage/misc/gperftools.py ********************************************************************** File "src/sage/misc/gperftools.py", line 118, in sage.misc.gperftools.Profiler._libc Failed example: Profiler()._libc() Exception raised: Traceback (most recent call last): File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute exec(compiled, globs) File "<doctest sage.misc.gperftools.Profiler._libc[1]>", line 1, in <module> Profiler()._libc() File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/misc/gperftools.py", line 129, in _libc raise ImportError('failed to open libc') ImportError: failed to open libc
and
sage -t --long --random-seed=0 src/sage/tests/parigp.py Timed out
What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.
I haven't changed Sage, so it's whatever gets set in sage-env: 10.9, I think.
comment:9 follow-up: 10 Changed 2 years ago by
Replying to jhpalmieri:
Replying to dimpase:
mpir is basically not developed any more (for the time being at least), so we should not use it.
It's a standard package, so I'm confused about what the alternatives are.
One can choose between gmp and mpir. Both are standard, but only one is needed.
Use the configure flag --with-mp=gmp
or with-mp=mpir
to choose.
Likewise, one can choose between ATLAS and OpenBLAS, and both are standard.
Use --with-blas=openblas
or --with-blas=atlas
for that.
In that case #30350 proposes to remove ATLAS.
comment:10 follow-up: 17 Changed 2 years ago by
Sorry, I led us on a bit of a wild goose chase for the past few comments. Back in comment:4, I said the right thing:
I could try building Sage's mpfr and gmp.
But then (maybe I was tired or distracted), I started building Sage's mpir
(and also Sage's gmp
, which was appropriate). There is no reason to care about mpir
: just use --with-system-gmp=no
and Sage will build its own gmp
and mpfr
.
comment:11 Changed 2 years ago by
Replying to jhpalmieri:
Replying to dimpase:
What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.
I haven't changed Sage, so it's whatever gets set in sage-env: 10.9, I think.
Just a quick remark - MACOSX_DEPLOYMENT_TARGET
only gets set in builds configured --without-system-python
. (See #30724, which makes this code in sage-env
cleaner.)
comment:12 follow-up: 13 Changed 2 years ago by
Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own, so MACOSX_DEPLOYMENT_TARGET
is relevant. (Also, Sage's Python can't find various system libraries to build the corresponding modules, which is why I'm passing the flags --with-system-zlib=no --with-system-bzip2=no
to configure
.)
comment:13 follow-up: 15 Changed 2 years ago by
Replying to jhpalmieri:
Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own
Is this because of dependencies of the python3
package? In this case, #30559 may be relevant
comment:14 Changed 2 years ago by
Replying to jhpalmieri:
I've actually gotten this to build for me without patching Sage in any way, just building 9.2.rc0.
...
Then I used the command
./configure --with-system-readline=no --with-system-zlib=no --with-system-gfortran=no --with-system-openblas=no --with-system-bzip2=no
make
worked.
This configure command followed by make gave me a functioning sage, so thank you!
R didn't compile but I don't use it.
comment:15 Changed 2 years ago by
Replying to mkoeppe:
Replying to jhpalmieri:
Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own
Is this because of dependencies of the
python3
package?
Yes, sqlite is not successfully detected. I'll try turning that dependency off.
comment:16 Changed 2 years ago by
Sage built using the system Python after modifying the dependencies to avoid sqlite. I used
./configure --with-system-gfortran=no --with-system-openblas=no --with-system-gmp=no
Unfortunately the documentation won't build and many doctests fail , in both cases with a bus error. For example:
File "src/sage/algebras/quatalg/quaternion_algebra.py", line 107, in sage.algebras.quatalg.quaternion_algebra.QuaternionAlgebraFactory Failed example: QuaternionAlgebra(sqrt(-1), sqrt(-3)) Exception raised: Traceback (most recent call last): File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute exec(compiled, globs) File "<doctest sage.algebras.quatalg.quaternion_algebra.QuaternionAlgebraFactory[4]>", line 1, in <module> QuaternionAlgebra(sqrt(-Integer(1)), sqrt(-Integer(3))) File "sage/structure/factory.pyx", line 367, in sage.structure.factory.UniqueFactory.__call__ (build/cythonized/sage/structure/factory.c:2181) key, kwds = self.create_key_and_extra_args(*args, **kwds) File "sage/structure/factory.pyx", line 471, in sage.structure.factory.UniqueFactory.create_key_and_extra_args (build/cythonized/sage/structure/factory.c:3402) return self.create_key(*args, **kwds), {} File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/algebras/quatalg/quaternion_algebra.py", line 246, in create_key if a == 0 or b == 0: File "sage/symbolic/expression.pyx", line 3182, in sage.symbolic.expression.Expression.__nonzero__ (build/cythonized/sage/symbolic/expression.cpp:20543) res = self.test_relation() File "sage/symbolic/expression.pyx", line 3296, in sage.symbolic.expression.Expression.test_relation (build/cythonized/sage/symbolic/expression.cpp:21404) if self.lhs().is_algebraic() and self.rhs().is_algebraic(): File "sage/symbolic/expression.pyx", line 2257, in sage.symbolic.expression.Expression.is_algebraic (build/cythonized/sage/symbolic/expression.cpp:15979) ex = sage.rings.all.QQbar(self) File "sage/structure/parent.pyx", line 900, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9336) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4515) return C._element_constructor(x) File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/rings/qqbar.py", line 1496, in _element_constructor_ return x._algebraic_(QQbar) File "sage/symbolic/expression.pyx", line 1754, in sage.symbolic.expression.Expression._algebraic_ (build/cythonized/sage/symbolic/expression.cpp:12658) return algebraic(self, field) File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 1227, in algebraic return AlgebraicConverter(field)(ex) File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 218, in __call__ return self.arithmetic(ex, operator) File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 1061, in arithmetic return self.field(base**expt) File "sage/structure/element.pyx", line 2057, in sage.structure.element.Element.__pow__ (build/cythonized/sage/structure/element.c:14456) return coercion_model.bin_op(left, right, pow) File "sage/structure/coerce.pyx", line 1196, in sage.structure.coerce.CoercionModel.bin_op (build/cythonized/sage/structure/coerce.c:10205) return (<Action>action)._act_(y, x) File "sage/categories/action.pyx", line 173, in sage.categories.action.Action._act_ (build/cythonized/sage/categories/action.c:3900) cpdef _act_(self, g, x): File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/rings/qqbar.py", line 6054, in _act_ target_arg.sin() * target_abs) File "sage/rings/real_mpfi.pyx", line 4557, in sage.rings.real_mpfi.RealIntervalFieldElement.sin (build/cythonized/sage/rings/real_mpfi.c:26988) sig_on() cysignals.signals.SignalError: Bus error
comment:17 Changed 2 years ago by
Replying to jhpalmieri:
Sorry, I led us on a bit of a wild goose chase for the past few comments. Back in comment:4, I said the right thing:
I could try building Sage's mpfr and gmp.But then (maybe I was tired or distracted), I started building Sage's
mpir
(and also Sage'sgmp
, which was appropriate). There is no reason to care aboutmpir
: just use--with-system-gmp=no
and Sage will build its owngmp
andmpfr
.
They cannot be, and are not, "installed together" - mpir
uses the "gmp compatibility" mode where it installs gmp-compatible libs etc. You need to run ./configure with
--with-mp=gmp to select gmp as the package to use for
mp, (= multiple precision), as the default for
mp is
mpir`.
Default mp
should be changed to gmp
, I think, as mpir
is getting bitrotten more and more.
See #30752 (where a lie about mpir/gmp told by ./configure -h
is uncovered, as well)
comment:18 Changed 2 years ago by
Priority: | major → critical |
---|
comment:19 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:20 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:21 Changed 2 years ago by
Cc: | was added |
---|
comment:22 Changed 2 years ago by
I am having problems building scipy
. This is after installing #31050, #30589, upgrades to setuptools
and setuptools_scm
, plus patches (related to parsing the OS X version number) to cffi
and numpy
. Then scipy
fails with
/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so gfortran: warning: couldn't understand version 11 ld: library not found for -lgcc_s.10.4 collect2: error: ld returned 1 exit status error: Command "/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so" failed with exit status 1 Building wheel for scipy (PEP 517): finished with status 'error' ERROR: Failed building wheel for scipy
comment:23 Changed 2 years ago by
Try doing export MACOSX_DEPLOYMENT_TARGET=10.16
to fix the issue until gfortran understands 11
comment:24 Changed 2 years ago by
Or you can set SYSTEM_VERSION_COMPAT=1
which would make the system return 10.16
automatically
comment:25 Changed 2 years ago by
I don't know how to set these variables in such a way to get Sage to build. (This is with the changes listed in comment:22.)
- If I set them globally, then
pillow
fails: it can't findzlib
. - If I set them just in the
spkg-install.in
script forscipy
, then it fails with the messageERROR: scipy-1.5.4-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
comment:26 Changed 2 years ago by
If I set them globally, then pillow fails: it can't find zlib.
What's the error?
comment:27 Changed 2 years ago by
Now I'm seeing a different error when building pillow, not about zlib:
clang: error: the clang compiler does not support '-march=native'
Details: I did
% export MACOSX_DEPLOYMENT_TARGET=10.16 % make distclean; ./configure; make pillow
Same error whether I use vanilla Sage 9.3.beta5, or that + #30589, or that + #30589 + new versions of setuptools
and setuptools_scm
. Pillow builds successfully if I then do
% unset MACOSX_DEPLOYMENT_TARGET % make distclean; ./configure; make pillow
comment:29 Changed 2 years ago by
If I instead do export SYSTEM_VERSION_COMPAT=1
, the Pillow build gets farther but ends with
ERROR: Pillow-8.0.1-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
Changed 2 years ago by
Attachment: | pillow-8.0.1-SYSTEM_VERSION_COMPAT=1.log added |
---|
built with SYSTEM_VERSION_COMPAT=1
comment:31 Changed 2 years ago by
Try patching pip at https://github.com/pypa/pip/blob/master/src/pip/_vendor/packaging/tags.py#L438 by doing,
if version == (10, 16): version = (11, 0)
comment:32 Changed 2 years ago by
That patch has no obvious effect when setting MACOSX_DEPLOYMENT_TARGET or SYSTEM_VERSION_COMPAT or both: I see the same errors.
comment:34 Changed 2 years ago by
I did the same edit independently. With SYSTEM_VERSION_COMPAT=1
, Pillow now builds! With MACOSX_DEPLOYMENT_TARGET=10.16
, it does not: same error as before.
comment:35 Changed 2 years ago by
And in case it matters, this is all with 9.3.beta5 + #30589 + updated setuptools and setuptools_scm.
comment:36 follow-up: 48 Changed 2 years ago by
After running make pillow
, I tried make numpy
and got an unexpected error:
Package 'numpy' is currently not installed Uninstalling 'numpy' with legacy uninstaller Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/var/tmp/sage/build/numpy-1.19.4/src/../lapack_conf.py", line 7, in <module> pc_blas = pkgconfig.parse('cblas blas') File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 248, in parse _raise_if_not_exists(package) File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 103, in _raise_if_not_exists raise PackageNotFoundError(package) pkgconfig.pkgconfig.PackageNotFoundError: cblas not found
Missing dependency? After running just make
, it has now reached numpy
and gotten past this problem.
comment:37 Changed 2 years ago by
scipy
failed, unfortunately:
/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so gfortran: warning: couldn't understand version 11 ld: library not found for -lgcc_s.10.4 collect2: error: ld returned 1 exit status error: Command "/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so" failed with exit status 1 Building wheel for scipy (PEP 517): finished with status 'error'
numpy
had a warning in its log, but it built successfully:
building library "npymath" sources gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11 gfortran: warning: couldn't understand version 11
Changed 2 years ago by
Attachment: | scipy-1.5.4.log added |
---|
comment:38 Changed 2 years ago by
Is this with SYSTEM_COMPAT_VERSION=1
or MACOSX_DEPLOYMENT_TARGET=10.16
. Are these set globally or in the spkg-install.in of scipy?
comment:39 Changed 2 years ago by
I know that Homebrew fixes scipy by setting
MACOSX_DEPLOYMENT_TARGET=11.0
If the OS is big sur.
comment:41 Changed 2 years ago by
to elaborate on my previous comment, if I run
make scipy
then it fails. However, if I run
MACOSX_DEPLOYMENT_TARGET=11.0 make scipy
then it succeeds with no issues. I don't know too much about makefiles but a simple fix could be to set MACOSX_DEPLOYMENT_TARGET if the operating system is Big Sur and then unset it after scipy installs.
comment:42 follow-up: 44 Changed 2 years ago by
I can get Sage to build on OS X 11.1 with the following changes:
- #30589
- upgrade setuptools and setuptools_scm (not sure if this is necessary)
- patch cffi and numpy: without patching, I get an error during cffi's
setup.py
when evaluating this:return tuple(map(int, get_config_var("MACOSX_DEPLOYMENT_TARGET").split('.')))
The problem is thatget_config_var(...)
returns the int 11 rather than the string "11", so I patched it to dostr(get_config_var(...))
. Similar with numpy. - add
export MACOSX_DEPLOYMENT_TARGET=11.0
in thespkg-install.in
script forscipy
.
I did not set any global environment variables.
There were some doctest failures, I think all due to gfortran. For example:
File "src/sage/misc/inline_fortran.py", line 114, in sage.misc.inline_fortran._import_module_from_path_impl Failed example: fortran(code, globals()) Exception raised: Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/misc/inline_fortran.py", line 170, in eval out = subprocess.check_output(cmd, stderr=subprocess.STDOUT) File "/usr/local/Cellar/python@3.9/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/local/Cellar/python@3.9/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/bin/python3', '-c', 'import numpy.f2py; numpy.f2py.main()', '-c', '-m', 'fortran_module', 'fortran_module.f', '--quiet', '--f77exec=sage-inline-fortran', '--f90exec=sage-inline-fortran']' returned non-zero exit status 1. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute exec(compiled, globs) File "<doctest sage.misc.inline_fortran._import_module_from_path_impl[1]>", line 1, in <module> fortran(code, globals()) File "sage/misc/lazy_import.pyx", line 360, in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:4032) return self.get_object()(*args, **kwds) File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/misc/inline_fortran.py", line 76, in __call__ return self.eval(*args, **kwds) File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/misc/inline_fortran.py", line 172, in eval raise RuntimeError( RuntimeError: failed to compile Fortran code: b'gfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\nerror: Command "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/bin/sage-inline-fortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib /var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/src.macosx-11-x86_64-3.9/fortran_modulemodule.o /var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/src.macosx-11-x86_64-3.9/fortranobject.o /var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/fortran_module.o -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -lgfortran -o ./fortran_module.cpython-39-darwin.so" failed with exit status 1\n' ********************************************************************** File "src/sage/misc/inline_fortran.py", line 117, in sage.misc.inline_fortran._import_module_from_path_impl Failed example: fib(a, 10) Exception raised: Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute exec(compiled, globs) File "<doctest sage.misc.inline_fortran._import_module_from_path_impl[4]>", line 1, in <module> fib(a, Integer(10)) NameError: name 'fib' is not defined ********************************************************************** File "src/sage/misc/inline_fortran.py", line 118, in sage.misc.inline_fortran._import_module_from_path_impl Failed example: a Expected: array([ 0., 1., 1., 2., 3., 5., 8., 13., 21., 34.]) Got: array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.]) ********************************************************************** 1 item had failures: 3 of 10 in sage.misc.inline_fortran._import_module_from_path_impl
comment:43 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:44 follow-up: 45 Changed 2 years ago by
Replying to jhpalmieri:
- upgrade setuptools and setuptools_scm (not sure if this is necessary)
Could you open a ticket for this?
comment:45 Changed 2 years ago by
Replying to mkoeppe:
Replying to jhpalmieri:
- upgrade setuptools and setuptools_scm (not sure if this is necessary)
Could you open a ticket for this?
Done at #31134.
comment:46 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:47 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:48 Changed 2 years ago by
Replying to jhpalmieri:
After running
make pillow
, I triedmake numpy
and got an unexpected error:Package 'numpy' is currently not installed Uninstalling 'numpy' with legacy uninstaller Traceback (most recent call last): File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/var/tmp/sage/build/numpy-1.19.4/src/../lapack_conf.py", line 7, in <module> pc_blas = pkgconfig.parse('cblas blas') File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 248, in parse _raise_if_not_exists(package) File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 103, in _raise_if_not_exists raise PackageNotFoundError(package) pkgconfig.pkgconfig.PackageNotFoundError: cblas not foundMissing dependency? After running just
make
, it has now reachednumpy
and gotten past this problem.
I opened #31135 for this.
comment:49 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:50 follow-up: 51 Changed 2 years ago by
I opened an issue with numpy
for a fix as was done with cffi
.
comment:51 follow-up: 54 Changed 2 years ago by
Replying to jhpalmieri:
I opened an issue with
numpy
for a fix as was done withcffi
.
Should be fixed in numpy
1.20, which is in release-candidate stage.
comment:52 Changed 2 years ago by
I found a patch for gfortran on Intel Macs at
https://github.com/gcc-mirror/gcc/commit/96de87b99bf8fd1c46df373bbcc2f7d76db716ad
I'll try to open a pull request tomorrow with Homebrew to see if they can patch GCC. That should hopefully fix all Fortran issues until GCC 11 comes out.
comment:53 Changed 2 years ago by
I noticed this morning that homebrew had updated their gcc. With the newest gcc formula you still need
MACOSX_DEPLOYMENT_TARGET=11.0 make scipy
in order for scipy to build, but now if you do
MACOSX_DEPLOYMENT_TARGET=11.0 ./sage -t src/sage/misc/inline_fortran.py
then the tests all pass.
comment:54 Changed 2 years ago by
Replying to jhpalmieri:
Replying to jhpalmieri:
I opened an issue with
numpy
for a fix as was done withcffi
.Should be fixed in
numpy
1.20, which is in release-candidate stage.
We will have to backport the fix because numpy 1.20 has dropped support for Python 3.6, which I think we want to continue supporting in Sage 9.3
comment:56 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:57 Changed 2 years ago by
I was just curious if anyone was planning to open a ticket for scipy. I don't know enough about the sage build system, but it's possible to check if the user is on Big Sur by doing
sw_vers | grep 11
and then based off of the result you could set MACOSX_DEPLOYMENT_TARGET=11.0
comment:58 follow-up: 60 Changed 2 years ago by
During the Sage build process, Sage already defines an environment variable MACOSX_VERSION
by
MACOSX_VERSION=`uname -r | awk -F. '{print $1}'`
This should be '20' for Big Sur.
I'll open a scipy
ticket.
comment:59 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:60 Changed 2 years ago by
Replying to jhpalmieri:
During the Sage build process, Sage already defines an environment variable
MACOSX_VERSION
byMACOSX_VERSION=`uname -r | awk -F. '{print $1}'`This should be '20' for Big Sur.
Actually, this is not correct: this variable is not universally defined anymore, so Sage packages that try to use it end up printing warnings. For example, r
:
./spkg-install: line 90: [: -ge: unary operator expected ./spkg-install: line 94: [: -ge: unary operator expected
See #31186.
comment:61 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:62 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:63 Changed 2 years ago by
Fails on a the as-vanilla-as-possible buildbot machine while building scipy:
Building modules... Building module "mvn"... Constructing wrapper function "mvnun"... value,inform = mvnun(lower,upper,means,covar,[maxpts,abseps,releps]) Constructing wrapper function "mvnun_weighted"... value,inform = mvnun_weighted(lower,upper,means,weights,covar,[maxpts,abseps,releps]) Constructing wrapper function "mvndst"... error,value,inform = mvndst(lower,upper,infin,correl,[maxpts,abseps,releps]) Constructing COMMON block support for "dkblck"... ivls Wrote C/API module "mvn" to file "build/src.macosx-10.9-x86_64-3.9/scipy/stats/mvnmodule.c" Fortran 77 wrappers are saved to "build/src.macosx-10.9-x86_64-3.9/scipy/stats/mvn-f2pywrappers.f" adding 'build/src.macosx-10.9-x86_64-3.9/build/src.macosx-10.9-x86_64-3.9/scipy/stats/fortranobject.c' to sources. adding 'build/src.macosx-10.9-x86_64-3.9/build/src.macosx-10.9-x86_64-3.9/scipy/stats' to include_dirs. adding 'build/src.macosx-10.9-x86_64-3.9/scipy/stats/mvn-f2pywrappers.f' to sources. building extension "scipy.ndimage._nd_image" sources building extension "scipy.ndimage._ni_label" sources building extension "scipy.ndimage._ctest" sources building extension "scipy.ndimage._ctest_oldapi" sources building extension "scipy.ndimage._cytest" sources building extension "scipy._lib._ccallback_c" sources building extension "scipy._lib._test_ccallback" sources building extension "scipy._lib._fpumode" sources building extension "scipy._lib.messagestream" sources creating build/src.macosx-10.9-x86_64-3.9/scipy/_lib Traceback (most recent call last): File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module> main() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel self.run_setup() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 253, in run_setup super(_BuildMetaLegacyBackend, File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 145, in run_setup exec(compile(code, __file__, 'exec'), locals()) File "setup.py", line 583, in <module> setup_package() File "setup.py", line 579, in setup_package setup(**metadata) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/core.py", line 169, in setup return old_setup(**new_attr) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup return distutils.core.setup(**attrs) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/core.py", line 148, in setup dist.run_commands() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/command/dist_info.py", line 31, in run egg_info.run() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/egg_info.py", line 24, in run self.run_command("build_src") File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 144, in run self.build_sources() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 161, in build_sources self.build_extension_sources(ext) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 318, in build_extension_sources sources = self.generate_sources(sources, ext) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 378, in generate_sources source = func(extension, build_dir) File "scipy/_lib/setup.py", line 30, in get_messagestream_config if config_cmd.check_func('open_memstream', decl=True, call=True): File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/config.py", line 321, in check_func self._check_compiler() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/config.py", line 80, in _check_compiler self.fcompiler = new_fcompiler(compiler=self.fcompiler, File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 880, in new_fcompiler compiler = get_default_fcompiler(plat, requiref90=requiref90, File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 851, in get_default_fcompiler compiler_type = _find_existing_fcompiler(matching_compiler_types, File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 802, in _find_existing_fcompiler c.customize(dist) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 507, in customize get_flags('opt', oflags) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 498, in get_flags flags.extend(getattr(self.flag_vars, tag)) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/environment.py", line 37, in __getattr__ return self._get_var(name, conf_desc) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/environment.py", line 53, in _get_var var = self._hook_handler(name, hook) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 705, in _environment_hook return hook() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/gnu.py", line 229, in get_flags_opt v = self.get_version() File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 426, in get_version version = CCompiler.get_version(self, force=force, ok_status=ok_status) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 90, in <lambda> m = lambda self, *args, **kw: func(self, *args, **kw) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 657, in CCompiler_get_version version = matcher(output) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/gnu.py", line 278, in version_match v = self.gnu_version_match(version_string) File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/gnu.py", line 80, in gnu_version_match raise ValueError(err + version_string) ValueError: A valid Fortran version was not found in this string: 9.2.0 Preparing wheel metadata: finished with status 'error'
comment:66 Changed 2 years ago by
For what it's worth, I am due for an upgrade and a test run of a Time Machine copy of my current (old) Mac onto a new Big Sur machine on an M1 chip apparently works - I'm not sure how, presumably using Rosetta somehow. Do we have any other data points about upgrading the Mac but leaving a working Sage install in place? (I guess upgrading that install would fall squarely under this ticket, though.)
comment:67 Changed 23 months ago by
Priority: | critical → major |
---|
comment:68 Changed 23 months ago by
Milestone: | sage-9.3 → sage-duplicate/invalid/wontfix |
---|---|
Status: | new → needs_review |
I think we can close the ticket now - this task is done
comment:70 Changed 20 months ago by
Resolution: | → invalid |
---|---|
Status: | positive_review → closed |
I'm trying to compile sage-9.2.rc0 on big sur with xcode 12.0.1.
I ran
sage -f openblas
. Here's the tail of that:(see the openblas attachment).
After trying lots of things such as
the compilation of
sage -f gfortran
stopped with an error compiling R.However, Sage did start and crash with this error:
(for the rest, see the error-in-start attachment).
It looks like Sage's Python doesn't work, right? Wrong:
In case it's any use, here is what homebrew claims it has installed: