Opened 2 years ago

Last modified 10 days ago

#30592 positive_review enhancement

Meta-ticket: Port to Apple silicon

Reported by: Matthias Köppe Owned by:
Priority: critical Milestone: sage-duplicate/invalid/wontfix
Component: porting Keywords:
Cc: Dima Pasechnik, Isuru Fernando, Karl-Dieter Crisman, François Bissey, William Stein Merged in:
Authors: Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

... on top of homebrew, which provides gfortran 11 and many usable packages.

Tickets:

  • #31258 Upgrade cmake to 3.19.3
  • #31838 gfortran: Accept system gfortran 11
  • #31905 cvxopt: Correct link directory
  • #32796 gmp: update to 6.2.1
  • #32815 Update numpy to 1.21.3, networkx to 2.6.3
  • #33191 Fix build errors on Apple Silicon with homebrew
  • #33195 iml: Refresh build scripts with current autotools
  • #33194 openblas: Update to 0.3.19
  • #33816 Update gcc/gfortran to 12.x + Apple Silicon support, update openblas to 0.3.21, gsl to 2.7.1

Discussions:

See also:

  • #30494: Meta-ticket: Support Xcode 12
  • #30651: Meta-ticket: support macOS 11 Big Sur

Attachments (9)

config.log (317.0 KB) - added by GMS 10 months ago.
install.log (235.4 KB) - added by GMS 10 months ago.
sagelib-9.5.rc1.log (9.4 MB) - added by GMS 10 months ago.
config.2.log (284.1 KB) - added by GMS 10 months ago.
install.2.log (79.5 KB) - added by GMS 10 months ago.
openblas-0.3.19.log (12.5 MB) - added by GMS 10 months ago.
config.3.log (358.7 KB) - added by GMS 10 months ago.
install.3.log (151.7 KB) - added by GMS 10 months ago.
ptestlong.log (562.6 KB) - added by GMS 10 months ago.

Change History (89)

comment:1 Changed 2 years ago by Samuel Lelièvre

Does GitHub Actions provide that kind of machines for CI?

Somewhat related:

comment:2 Changed 2 years ago by Matthias Köppe

Milestone: sage-9.2sage-9.3

comment:3 Changed 2 years ago by Matthias Köppe

Cc: Isuru Fernando Karl-Dieter Crisman François Bissey added
Description: modified (diff)

comment:4 Changed 2 years ago by Matthias Köppe

Cc: William Stein added
Description: modified (diff)

comment:5 Changed 23 months ago by Woodstock

gFortran is now working natively on Apple Silicon (ARM64) GNU Fortran (Homebrew GCC 10.2.0_2) 10.2.1 20201220

as part of GCC.

gcc: stable 10.2.0 (bottled), HEAD GNU compiler collection https://gcc.gnu.org/

Last edited 23 months ago by Woodstock (previous) (diff)

comment:6 Changed 23 months ago by Matthias Köppe

Description: modified (diff)

comment:7 Changed 23 months ago by Matthias Köppe

Description: modified (diff)

comment:8 Changed 20 months ago by Matthias Köppe

Milestone: sage-9.3sage-9.4

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

comment:9 Changed 18 months ago by Karl-Dieter Crisman

If I wanted to help test this, what could I do? I tried upgrading my Sage yesterday (finally!) but ran into problems with pari and some other packages; then I did make distclean and now it won't even install gmp, wonderful lines

[gmp-6.2.0] configure: error: Oops, mp_limb_t is 64 bits, but the assembler code
[gmp-6.2.0] in this configuration expects 32 bits.

that look scary. Anyway, I'd be quite interested in testing anything directly from source that does not use Homebrew, which after all the warnings many years ago about Macports and Fink conflicts I'm skeptical of.

comment:10 Changed 18 months ago by Dima Pasechnik

Homebrew is the most popular "missing package manager" for macOS, and I really do not understand why you are sceptical about. Sage on x86_64 Homebrew mostly works, with many, many packages used rather than built from source. (Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

Macports doesn't work with Sage, and Fink is mostly dead, IMHO

Homebrew surely does have ready Apple M1 GMP, cf. https://formulae.brew.sh/formula/gmp#default

Maintaining the Sage status quo - porting patches from Hombrew back to Sage, etc, it's such a waste of time if you ask me.

comment:11 in reply to:  10 ; Changed 18 months ago by Karl-Dieter Crisman

Homebrew is the most popular "missing package manager" for macOS, and I really do not understand why you are sceptical about. Sage on x86_64 Homebrew mostly works, with many, many packages used rather than built from source. (Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

I guess bad experiences in the past with the other "managers" we will not name again - and, separately, liking to maintain ability for Mac users to download source and just type make. But I will give it a try.


However, as to the main topic of this ticket, is this simply "use Homebrew/Rosetta?" or is there presumably some other testing I can do to help?

comment:12 in reply to:  10 ; Changed 18 months ago by William Stein

Replying to dimpase:

(Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

Just curious -- is there work underway to make sagemath itself a homebrew package, so one could do "brew install sagemath"? That would be pretty awesome.

comment:13 in reply to:  12 Changed 18 months ago by Dave Morris

Making sage into a homebrew package is on the wishlist. (I don't think anyone is doing anything about it.) I think further discussion of this should go to #29395.

comment:14 in reply to:  12 Changed 18 months ago by Dima Pasechnik

Replying to was:

Replying to dimpase:

(Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

Just curious -- is there work underway to make sagemath itself a homebrew package, so one could do "brew install sagemath"? That would be pretty awesome.

I played with creating Homebrew formulae a while ago, see https://github.com/sagemath/homebrew-science/commits/master

It's not that hard - and in fact e.g. arb and flint made it into mainline Homebrew since I touched this last time. It's more annoying that we sometimes need a variation of what's in Homebrew, e.g. it ships single-threaded Pari, but we need multi-threaded (as that's what giac requires).

comment:15 in reply to:  11 ; Changed 18 months ago by Dima Pasechnik

Replying to kcrisman:

However, as to the main topic of this ticket, is this simply "use Homebrew/Rosetta?" or is there presumably some other testing I can do to help?

no, no Rosetta, just pure, unfiltered, Homebrew. (Rosetta is an x86_64 emulator, as you might recall)

Give it a try please, I'd like to know how far we are there.

comment:16 Changed 18 months ago by Dima Pasechnik

by the way, this ticket says "need gfortran first" (for M1), but it's already there. https://formulae.brew.sh/formula/gcc (it's gcc/gfortran 11)

comment:17 Changed 18 months ago by Matthias Köppe

Description: modified (diff)

comment:18 in reply to:  15 ; Changed 18 months ago by Karl-Dieter Crisman

(Rosetta is an x86_64 emulator, as you might recall)

Exactly; so homebrew will be using ARM stuff natively in this case?

Give it a try please, I'd like to know how far we are there.

Wish I hadn't destroyed my working install :( but I think I have a few old binaries I can use if under duress. That's why I waited for the semester to finish!


I'll post on devel if I achieve anything interesting. One note in prereqs: With respect to BLAS (mentioned in one of the threads) I get

==> Pouring openblas--0.3.15_1.big_sur.bottle.tar.gz
==> Caveats
openblas is keg-only, which means it was not symlinked into /usr/local,
because macOS provides BLAS in Accelerate.framework.

For compilers to find openblas you may need to set:
  export LDFLAGS="-L/usr/local/opt/openblas/lib"
  export CPPFLAGS="-I/usr/local/opt/openblas/include"

For pkg-config to find openblas you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

==> Summary
🍺  /usr/local/Cellar/openblas/0.3.15_1: 23 files, 120MB

so hopefully accelerate will continue to work here for Sage as an option.

comment:19 in reply to:  18 Changed 18 months ago by Dima Pasechnik

Replying to kcrisman:

(Rosetta is an x86_64 emulator, as you might recall)

Exactly; so homebrew will be using ARM stuff natively in this case?

yes. They have a setup for this now (IIRC they use /opt/local for native M1, and /usr/local for Rosetta)

Give it a try please, I'd like to know how far we are there.

Wish I hadn't destroyed my working install :( but I think I have a few old binaries I can use if under duress. That's why I waited for the semester to finish!


I'll post on devel if I achieve anything interesting. One note in prereqs: With respect to BLAS (mentioned in one of the threads) I get

==> Pouring openblas--0.3.15_1.big_sur.bottle.tar.gz
==> Caveats
openblas is keg-only, which means it was not symlinked into /usr/local,
because macOS provides BLAS in Accelerate.framework.

For compilers to find openblas you may need to set:
  export LDFLAGS="-L/usr/local/opt/openblas/lib"
  export CPPFLAGS="-I/usr/local/opt/openblas/include"

For pkg-config to find openblas you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

==> Summary
🍺  /usr/local/Cellar/openblas/0.3.15_1: 23 files, 120MB

so hopefully accelerate will continue to work here for Sage as an option.

it's not an option if you want to use more external Homebrew packages, which depend on (open)blas. Anyhow, sourcing .homebrew-build-env should take care of setting everything, including Homebrew's OpenBLAS, to be used by Sage.

comment:20 Changed 18 months ago by Karl-Dieter Crisman

Description: modified (diff)

comment:21 Changed 18 months ago by Karl-Dieter Crisman

We may also need to add openjpeg to the list here.

comment:22 in reply to:  21 Changed 18 months ago by Dima Pasechnik

Replying to kcrisman:

We may also need to add openjpeg to the list here.

I see no need for this. openjpeg is an optional dep of Pillow, and thus a problem may arise from a broken install of openjpeg - or no problem arises if openjpeg is not installed. But it's not Sage's job to do quality control of every package on a user box...

comment:23 Changed 16 months ago by Marc Culler

While we have had a bit of trouble getting complete information, it appears that the current version of the SageMath app (based on rc0) for macOS works fine in the Intel emulation mode on M1 macs. This was not always the case, but we are now taking care to avoid AVX, AVX2 and BMI instructions unless they are enabled at runtime by the "fat" version of gmp. This appears to eliminate the crashes that were seen before in emulation mode.

comment:24 Changed 15 months ago by Matthias Köppe

Milestone: sage-9.4sage-9.5

comment:25 Changed 13 months ago by Dima Pasechnik

Description: modified (diff)

comment:26 Changed 13 months ago by Matthias Köppe

Description: modified (diff)

comment:27 Changed 12 months ago by Dima Pasechnik

Apparently our Openblas build fails on M1, see https://groups.google.com/g/sage-support/c/XwGF584P7EY/m/MsN6mfWkAQAJ

Last edited 12 months ago by Samuel Lelièvre (previous) (diff)

comment:28 Changed 11 months ago by Matthias Köppe

It would probably be a good idea if people who have access to M1 hardware test the current beta, as we are getting closer to the 9.5 release.

comment:29 Changed 11 months ago by Woodstock

Keen to test. Any build instructions for M1/Pro/Max?

Or is the a binary?

comment:30 Changed 11 months ago by Matthias Köppe

Follow the instructions for build from source in README.md, using homebrew; just mildly updated in #29784

comment:31 Changed 11 months ago by Woodstock

@mkoeppe - apologies can’t seem to find the brew instructions. Do I just build from src with —HEAD?

comment:32 Changed 11 months ago by Dima Pasechnik

have homebrew installed, then

source .homebrew-build-env
./bootstrap
./configure

pay attention to last pages of output of two latter commands, install packages they recommend, in the latter case, rerun

./bootstrap && ./configure

then

make build

comment:33 Changed 10 months ago by Matthias Köppe

comment:34 Changed 10 months ago by Matthias Köppe

Priority: majorcritical

comment:35 Changed 10 months ago by Matthias Köppe

From the logs posted there: Build failure primecountpy

  creating build/temp.macosx-11.6-arm64-3.9/primecountpy
  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -g -O2 -g -O2 -I/opt/homebrew/opt/openssl@3/include -I/Users/gms/sage/sage-9.5.rc1/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/cysignals -I/Users/gms/sage/sage-9.5.rc1/local/var/lib/sage/venv-python3.9.9/include/python3.9 -c primecountpy/primecount.cpp -o build/temp.macosx-11.6-arm64-3.9/primecountpy/primecount.o
  g++ -bundle -undefined dynamic_lookup -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/opt/homebrew/opt/openssl@3/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/var/lib/sage/venv-python3.9.9/lib -L. -L/Users/gms/sage/sage-9.5.rc1/local/lib -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/opt/homebrew/opt/openssl@3/lib -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/opt/homebrew/opt/openssl@3/lib -g -O2 -I/opt/homebrew/opt/openssl@3/include build/temp.macosx-11.6-arm64-3.9/primecountpy/primecount.o -L/Users/gms/sage/sage-9.5.rc1/local/var/lib/sage/venv-python3.9.9/lib -lprimesieve -lprimecount -o build/lib.macosx-11.6-arm64-3.9/primecountpy/primecount.cpython-39-darwin.so -lpari
  ld: library not found for -lprimesieve
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/usr/bin/g++' failed with exit code 1
  Building wheel for primecountpy (pyproject.toml): finished with status 'error'
  ERROR: Failed building wheel for primecountpy
Failed to build primecountpy
ERROR: Failed to build one or more wheels

According to config.log posted there, that's with system cmake and system primesieve, and primecount installed from SPKG.

Last edited 10 months ago by Matthias Köppe (previous) (diff)

comment:36 Changed 10 months ago by Matthias Köppe

Build failure m4ri

checking whether to build with PNG support... yes
checking for pkg-config... /opt/homebrew/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for PNG... yes
...
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I./m4ri   -I/opt/homebrew/opt/openssl@3/include    -g -O2 -I/Users/gms/sage/sage-9.5.rc1/local/include -g -fPIC -Wall -pedantic -c -o m4ri/io.lo m4ri/io.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I./m4ri -I/opt/homebrew/opt/openssl@3/include -g -O2 -I/Users/gms/sage/sage-9.5.rc1/local/include -g -fPIC -Wall -pedantic -c m4ri/io.c  -fno-common -DPIC -o m4ri/.libs/io.o
m4ri/io.c:28:10: fatal error: 'png.h' file not found
#include <png.h>
         ^~~~~~~
1 error generated.

Same error for tachyon.

Last edited 10 months ago by Matthias Köppe (previous) (diff)

comment:37 in reply to:  35 ; Changed 10 months ago by François Bissey

Replying to mkoeppe:

According to config.log posted there, that's with system cmake and system primesieve, and primecount installed from SPKG.

Do you mean sage primesieve and primecount rather than "system".

comment:38 Changed 10 months ago by Matthias Köppe

Build failure libhomfly

/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"libhomfly\" -DPACKAGE_TARNAME=\"libhomfly\" -DPACKAGE_VERSION=\"1.02r6\" -DPACKAGE_STRING=\"libhomfly\ 1.02r6\" -DPACKAGE_BUGREPORT=\"mmarco@unizar.es\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libhomfly\" -DVERSION=\"1.02r6\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I.   -I/opt/homebrew/opt/openssl@3/include  -g -O2 -c -o control.lo control.c
control.c:8:10: fatal error: 'gc.h' file not found
#include <gc.h>
         ^~~~~~

From top-level config.log:

configure:19776: $PKG_CONFIG --exists --print-errors "bdw-gc >= 7.6.4"
configure:19779: $? = 0
configure:19819: result: yes
configure:19917: will use system package and not install SPKG gc

This appears to be another case of our configure forgetting to tell packages where the stuff is that it found.

comment:39 in reply to:  37 Changed 10 months ago by Matthias Köppe

Replying to fbissey:

Replying to mkoeppe:

According to config.log posted there, that's with system cmake and system primesieve, and primecount installed from SPKG.

Do you mean sage primesieve and primecount rather than "system".

No, primesieve is from homebrew

comment:40 Changed 10 months ago by François Bissey

OK, I am now parsing that sentence correctly :)

comment:41 Changed 10 months ago by Matthias Köppe

Proposed fix in #33191

comment:42 Changed 10 months ago by Matthias Köppe

Description: modified (diff)

comment:43 Changed 10 months ago by Matthias Köppe

Next failure, reported in https://groups.google.com/g/sage-release/c/8ctwsUGl6LQ/m/j660ZbCmAAAJ: iml

Default checking path = /Users/gms/sage/sage-9.5.rc1/local /usr /usr/local
checking for GMP >= 3.1.1... found
checking whether GMP is 4.0 or greater... yes
[...]
/bin/bash ../libtool --tag=CC   --mode=link gcc -I///opt/homebrew/include -I.  -I. -g -O2 -version-info 1:1:1  -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib -L/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath,/Users/gms/sage/sage-9.5.rc1/local/lib  -lm -o libiml.la -rpath /Users/gms/sage/sage-9.5.rc1/local/lib libiml_la-basisop.lo libiml_la-certsolve.lo libiml_la-error.lo libiml_la-latreduce.lo libiml_la-memalloc.lo libiml_la-mtrans.lo libiml_la-nonsysolve.lo libiml_la-padiclift.lo libiml_la-reconsolu.lo libiml_la-RNSop.lo libiml_la-nullspace.lo -L///opt/homebrew/lib -lgmp -L/Users/gms/sage/sage-9.5.rc1/local/lib -lgmp -L/Users/gms/sage/sage-9.5.rc1/local/lib -L/opt/homebrew/Cellar/openblas/0.3.19/lib -lopenblas   
libtool: link: gcc -dynamiclib -Wl,-flat_namespace -Wl,-undefined -Wl,suppress -o .libs/libiml.0.dylib  .libs/libiml_la-basisop.o .libs/libiml_la-certsolve.o .libs/libiml_la-error.o .libs/libiml_la-latreduce.o .libs/libiml_la-memalloc.o .libs/libiml_la-mtrans.o .libs/libiml_la-nonsysolve.o .libs/libiml_la-padiclift.o .libs/libiml_la-reconsolu.o .libs/libiml_la-RNSop.o .libs/libiml_la-nullspace.o   -L/Users/gms/sage/sage-9.5.rc1/local/lib -lm -L///opt/homebrew/lib -lgmp -L/opt/homebrew/Cellar/openblas/0.3.19/lib -lopenblas  -O2 -Wl,-rpath -Wl,/Users/gms/sage/sage-9.5.rc1/local/lib -Wl,-rpath -Wl,/Users/gms/sage/sage-9.5.rc1/local/lib   -install_name  /Users/gms/sage/sage-9.5.rc1/local/lib/libiml.0.dylib -compatibility_version 2 -current_version 2.1 -Wl,-single_module
ld: warning: dylib (/opt/homebrew/opt/gcc/lib/gcc/11/libgfortran.5.dylib) was built for newer macOS version (11.5) than being linked (11.0)
ld: warning: dylib (/opt/homebrew/opt/gcc/lib/gcc/11/libgomp.1.dylib) was built for newer macOS version (11.5) than being linked (11.0)
ld: file not found: @rpath/libgcc_s.1.1.dylib for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[5]: *** [libiml.la] Error 1

comment:44 Changed 10 months ago by François Bissey

I have seen that one posted before. I am not sure how to fix it without talking to homebrew people about how they build their gcc/gfortran.

comment:45 Changed 10 months ago by Matthias Köppe

comment:46 Changed 10 months ago by GMS

I do not know if this can be related:

After sudo ln -s /opt/homebrew/lib lib in /usr/local (see https://github.com/apaffenholz/homebrew-polymake/issues/3) this works

brew install apaffenholz/polymake/polymake

(suggested by ./configure).

comment:47 Changed 10 months ago by Matthias Köppe

Description: modified (diff)

comment:48 Changed 10 months ago by GMS

Description: modified (diff)

I have restarted the whole process, with the fix #33191.

iml was still failing, so I have done

MACOSX_DEPLOYMENT_TARGET=11.2 make

as indicated in https://groups.google.com/g/sage-devel/c/6LldEeJu3ZA/m/BpNMpD-yAAAJ and there is no error about installing packages, but the building fails. Logs attached as soon as I find how to do it.

comment:49 Changed 10 months ago by Matthias Köppe

Update of iml (#33195) might be worth trying

Changed 10 months ago by GMS

Attachment: config.log added

Changed 10 months ago by GMS

Attachment: install.log added

Changed 10 months ago by GMS

Attachment: sagelib-9.5.rc1.log added

comment:50 Changed 10 months ago by Matthias Köppe

So iml went through without errors?

comment:51 Changed 10 months ago by Matthias Köppe

The errors in sagelib are:

ld: illegal thread local variable reference to regular symbol __ZN3NTL8ZZ_pInfoE for architecture arm64

comment:52 in reply to:  50 Changed 10 months ago by GMS

Replying to mkoeppe:

So iml went through without errors?

Yes!

comment:53 Changed 10 months ago by Matthias Köppe

Description: modified (diff)

comment:54 Changed 10 months ago by Matthias Köppe

If you have a chance, please test if #33195 fixes iml (without setting MACOSX_DEPLOYMENT_TARGET)

comment:55 in reply to:  54 Changed 10 months ago by GMS

Replying to mkoeppe:

If you have a chance, please test if #33195 fixes iml (without setting MACOSX_DEPLOYMENT_TARGET)

I have cloned the git repository.

Could you please tell me how to apply #33195?

comment:56 Changed 10 months ago by Matthias Köppe

git remote add trac git://trac.sagemath.org/sage.git
git fetch trac u/mkoeppe/iml__update_to_1_0_5
git merge -m "Merge #33195" FETCH_HEAD
./bootstrap
./configure --enable-download-from-upstream-url

comment:57 in reply to:  56 Changed 10 months ago by GMS

Replying to mkoeppe:

git remote add trac git://trac.sagemath.org/sage.git
git fetch trac u/mkoeppe/iml__update_to_1_0_5
git merge -m "Merge #33195" FETCH_HEAD
./bootstrap
./configure --enable-download-from-upstream-url

Thanks, make running.

comment:58 Changed 10 months ago by GMS

[iml-1.0.4p2.p2] successfully installed. So your patch works.

comment:59 Changed 10 months ago by Matthias Köppe

Thanks for testing!

comment:60 Changed 10 months ago by Matthias Köppe

Next it would be worth checking if #33194 (branch u/mkoeppe/openblas__update_to_0_3_19) fixes the build of openblas when ./configure --enable-download-from-upstream-url --without-system-openblas is in use.

comment:61 Changed 10 months ago by GMS

real	26m34.164s
user	113m56.740s
sys	13m19.283s
Sage build/upgrade complete!

And thanks to you (all of you).

comment:62 in reply to:  61 ; Changed 10 months ago by Matthias Köppe

Replying to gh-GMS103:

real	26m34.164s
user	113m56.740s
sys	13m19.283s
Sage build/upgrade complete!

And thanks to you (all of you).

Great! So the issue with ntl from comment:51 didn't show up in this build.

comment:63 in reply to:  60 Changed 10 months ago by GMS

Replying to mkoeppe:

Next it would be worth checking if #33194 (branch u/mkoeppe/openblas__update_to_0_3_19) fixes the build of openblas when ./configure --enable-download-from-upstream-url --without-system-openblas is in use.

make running.

comment:64 in reply to:  62 Changed 10 months ago by GMS

Replying to mkoeppe:

Replying to gh-GMS103:

real	26m34.164s
user	113m56.740s
sys	13m19.283s
Sage build/upgrade complete!

And thanks to you (all of you).

Great! So the issue with ntl from comment:51 didn't show up in this build.

No, everything went smoothly.

comment:65 Changed 10 months ago by GMS

After

git fetch trac u/mkoeppe/openblas__update_to_0_3_19
git merge -m "Merge #33194" FETCH_HEAD
./bootstrap 
./configure --enable-download-from-upstream-url --without-system-openblas

make gives an error for openblas:

[openblas-0.3.19]   make[6]: *** No rule to make target `../kernel/arm64/amin.S', needed by `samin_k.o'.  Stop.
Last edited 10 months ago by GMS (previous) (diff)

Changed 10 months ago by GMS

Attachment: config.2.log added

Changed 10 months ago by GMS

Attachment: install.2.log added

Changed 10 months ago by GMS

Attachment: openblas-0.3.19.log added

comment:66 Changed 10 months ago by Matthias Köppe

This looks similar to https://trac.macports.org/ticket/61700; it sounds like we would have to configure openblas differently. We can probably get this from homebrew's build script. This won't make it into Sage 9.5.

Thanks again for testing!

comment:67 Changed 10 months ago by GMS

And make ptestlong (after make with #33191 and #33195 but wihtout #33194) gives

----------------------------------------------------------------------
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/functions/error.py  # 1 doctest failed
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/matrix/matrix_gf2e_dense.pyx  # 1 doctest failed
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/rings/complex_arb.pyx  # 2 doctests failed
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/rings/real_arb.pyx  # 5 doctests failed
----------------------------------------------------------------------
Total time for all tests: 1294.6 seconds
    cpu time: 7872.1 seconds
    cumulative wall time: 9610.5 seconds
Features detected for doctesting: dvipng,ffmpeg,graphviz,imagemagick,pandoc,pdf2svg,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.plot,sage.rings.number_field,sage.rings.real_double,sage.symbolic,sagemath_doc_html,sphinx
Pytest is not installed, skip checking tests that rely on it.
make: *** [ptestlong] Error 1

Changed 10 months ago by GMS

Attachment: config.3.log added

Changed 10 months ago by GMS

Attachment: install.3.log added

Changed 10 months ago by GMS

Attachment: ptestlong.log added

comment:68 Changed 10 months ago by GMS

Just in case, things I think can be taken into account (besides #33191 and #33195):

  1. Install everything ./configure (or ./configure --enable-download-from-upstream-url) recommends with brew install [...].
    • Be aware of the 2 versions of homebrew: the new arm64 should be used.
    • In particular, to install polymake do
      cd /usr/local
      sudo ln -s /opt/homebrew/lib lib
      brew install apaffenholz/polymake/polymake
      
      (see https://github.com/apaffenholz/homebrew-polymake/issues/3)
    • Do not forget source .homebrew-build-env afterwards.
  2. Before make, do:
    eval "$(/opt/homebrew/bin/brew shellenv)"
    export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"
    export LDFLAGS="-L/opt/homebrew/opt/openssl@3/lib -L/opt/homebrew/opt/openblas/lib"
    export CPPFLAGS="-I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/openblas/include"
    export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig:/opt/homebrew/opt/openblas/lib/pkgconfig"
    
    (recommended by brew, but can be overlooked; perhaps to be put in .homebrew-build-env?)
  3. export MAKE="make -j16" works fine on Apple M1 (all 8 cores at 100% most of the time).

comment:69 Changed 10 months ago by Matthias Köppe

The PKG_CONFIG_PATH addition for openssl is already taken care of by .homebrew-build-env. I don't think we need the changes to the other variables

comment:70 Changed 10 months ago by Matthias Köppe

Thanks for writing this summary - it will be helpful for those who find their way here from https://wiki.sagemath.org/ReleaseTours/sage-9.5#Sources

comment:71 Changed 10 months ago by Matthias Köppe

New report of build failures with 9.5.rc3 on M1 with macOS 12.1: https://groups.google.com/g/sage-release/c/1YV0dNtEZz4/m/2kQ-fKfaAQAJ

From the singular log:

libtool: compile:  g++ -std=gnu++11 -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../.. -I../../libpolys -I../../libpolys -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src/factory -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src/factory/include -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src -I/Users/admin/SageMath/Git/sage/local/include -I/Users/admin/SageMath/Git/sage/local/include/ -I/Users/admin/SageMath/Git/sage/local/include -g -O2 -pipe -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fno-threadsafe-statics -funroll-loops -fno-rtti -Qunused-arguments -c mpr_inout.cc  -fno-common -DPIC -o .libs/mpr_inout.o
In file included from mpr_inout.cc:11:
In file included from ../../kernel/mod2.h:13:
../../libpolys/misc/auxiliary.h:70:2: error: "Unexpected SIZEOF_LONG"
#error "Unexpected SIZEOF_LONG"

(resolved by removing stuff in /usr/local)

Last edited 10 months ago by Matthias Köppe (previous) (diff)

comment:72 Changed 10 months ago by Frédéric Chapoton

Milestone: sage-9.5sage-9.6

bump to 9.6

comment:73 Changed 7 months ago by Matthias Köppe

Milestone: sage-9.6sage-9.7

comment:74 Changed 2 months ago by Matthias Köppe

Milestone: sage-9.7sage-9.8

comment:75 Changed 2 months ago by Matthias Köppe

Description: modified (diff)

comment:76 Changed 12 days ago by John Palmieri

Maybe we can close this.

comment:77 Changed 11 days ago by Dima Pasechnik

how about #31905 ?

comment:78 in reply to:  77 Changed 11 days ago by John Palmieri

Replying to Dima Pasechnik:

how about #31905 ?

I think we now agree that's not an issue. I don't know what to do with #30494, but maybe we don't need both this ticket and that one. So I propose closing this one and thinking about what to do with #30494 separately.

comment:79 Changed 10 days ago by Dima Pasechnik

Milestone: sage-9.8sage-duplicate/invalid/wontfix
Status: newneeds_review

comment:80 Changed 10 days ago by Dima Pasechnik

Reviewers: Dima Pasechnik
Status: needs_reviewpositive_review

close as done

Note: See TracTickets for help on using tickets.