Opened 21 months ago

Last modified 2 months ago

#31505 needs_review enhancement

Extend system package scripts to handle macports; tox.ini: Add macports environment

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-9.8
Component: porting Keywords:
Cc: Samuel Lelièvre, gh-kliem, John Palmieri, Dima Pasechnik, Sébastien Labbé, Volker Braun, Zachary L Scherr Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: Not yet reported upstream; Will do shortly. Work issues:
Branch: u/mkoeppe/tox_ini__add_macports_environment (Commits, GitHub, GitLab) Commit: 476fa33e16780466a04556f1c87532465dbde898
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

Based on #30504, which added MacPorts package information, we update the system package scripts to include support for MacPorts.

Current Sage builds just fine on a system with MacPorts. To test this, we add local-macports environments to tox.ini:

tox -e local-macports-minimal

or

tox -e local-macports-standard             # uses /usr/bin/python3

or

tox -e local-macports-standard-python3.10  # uses python3.10 from macports

Each of these commands creates an isolated installation of MacPorts in a subdirectory of .tox (using https://guide.macports.org/#installing.macports.source.multiple ); it will not use or modify /opt/local.

(There is also tox -e local-macports-optlocal-standard, which uses sudo to install things into the default location -- use at your own risk.)

Requested at

Attachments (8)

sagelib-9.3.beta9.log (10.4 MB) - added by John Palmieri 21 months ago.
config.log (303.0 KB) - added by John Palmieri 21 months ago.
openblas-0.3.13.log (8.9 MB) - added by John Palmieri 20 months ago.
config.2.log (263.7 KB) - added by John Palmieri 20 months ago.
for nohomebrew macports-minimal
sagelib-9.3.rc2.log (17.2 MB) - added by John Palmieri 20 months ago.
sagelib-9.3.rc2-delta.log (32.0 KB) - added by John Palmieri 20 months ago.
p1 (30.8 KB) - added by Dima Pasechnik 3 months ago.
sysconfig1
p2 (26.3 KB) - added by Dima Pasechnik 3 months ago.
sysconf2

Change History (180)

comment:1 Changed 21 months ago by Matthias Köppe

Branch: u/mkoeppe/tox_ini__add_macports_environment

comment:2 Changed 21 months ago by Matthias Köppe

Authors: Matthias Koeppe
Commit: 72c92412c8bd9fa903c24084ef2a42590eb30ae7
Description: modified (diff)
Status: newneeds_review

New commits:

ab5819btox.ini: Add local-macports
8131271Add macports.txt
45dec6abuild/pkgs/_prereq/distros: Update file comments
8e1393cbuild/bin/sage-print-system-package-command: Handle macports install
72c9241m4/sage_check_osx_supported.m4: Do not exit with error if fink or macports is found

comment:3 Changed 21 months ago by Matthias Köppe

Cc: gh-kliem added

comment:4 Changed 21 months ago by git

Commit: 72c92412c8bd9fa903c24084ef2a42590eb30ae7b6f1596e6e250f2f4a3bf537051154f4fc1933f2

Branch pushed to git repo; I updated commit sha1. New commits:

b6f1596tox.ini (local-macports-optlocal): Use sudo

comment:5 Changed 21 months ago by git

Commit: b6f1596e6e250f2f4a3bf537051154f4fc1933f2218989037968aa4d94c2b151cb3183bd5840b184

Branch pushed to git repo; I updated commit sha1. New commits:

e9eeb0bbuild/bin/sage-print-system-package-command (macports): Handle setup-build-env
2189890build/bin/sage-guess-package-system: Detect macports

comment:6 Changed 21 months ago by Matthias Köppe

Dependencies: #30504

comment:7 Changed 21 months ago by git

Commit: 218989037968aa4d94c2b151cb3183bd5840b1844acfc5c8bed0227762a73ff23204bdce2e88e5cf

Branch pushed to git repo; I updated commit sha1. New commits:

4f7398eUpdate repology info
338674930504: Rename and remove distros files from #29129 and #30619
7c272ecAdd MacPorts package info
8c9f53c30504: Add _recommended package info for MacPorts
bf0a4d230504: Use zmq-devel in MacPorts for zeromq
e42448930504: Fix gcc gfortran openblas info for MacPorts
b8b7092build/pkgs/{gcc,gfortran}/distros/macports.txt: Fixup
4acfc5cMerge #30504

comment:8 Changed 21 months ago by Matthias Köppe

Cc: John Palmieri Dima Pasechnik added
Description: modified (diff)

comment:9 Changed 21 months ago by Matthias Köppe

Summary: tox.ini: Add macports environmentExtend system package scripts to handle macports; tox.ini: Add macports environment

comment:10 Changed 21 months ago by Matthias Köppe

Description: modified (diff)
Summary: Extend system package scripts to handle macports; tox.ini: Add macports environmentRemove outdated macports ban; Extend system package scripts to handle macports; tox.ini: Add macports environment

comment:11 Changed 21 months ago by Matthias Köppe

Priority: majorcritical

comment:12 Changed 21 months ago by Matthias Köppe

Cc: Sébastien Labbé added

comment:13 Changed 21 months ago by Samuel Lelièvre

No need for a .macports-build-env file similar to .homebrew-build-env?

comment:14 Changed 21 months ago by Samuel Lelièvre

Description: modified (diff)

comment:15 Changed 21 months ago by Matthias Köppe

I don't think we need this - only 1 command is needed to prepare the build

comment:16 Changed 21 months ago by Samuel Lelièvre

The tox -e ... commands run ./bootstrap && ./configure && make or similar.

Could they run ./bootstrap -q && ./configure -q && make -s V=0 instead?

comment:17 Changed 21 months ago by Samuel Lelièvre

Oh in fact they already do. Sorry for the noise. I'm trying this out and will report.

comment:18 Changed 21 months ago by Matthias Köppe

Yes, make runs with V=0, but for the others I really want to see the output.

comment:19 Changed 21 months ago by Matthias Köppe

For more quiet, you can use tox -p auto -e local-macports-standard -- config.status (logs will appear in .tox/local-macports-standard/log

comment:20 Changed 21 months ago by Samuel Lelièvre

In a fresh clone, I ran:

$ git trac checkout 31505
$ git trac fetch 31496 && git merge FETCH_HEAD -m "Merge 31496"
$ git trac fetch 31335 && git merge FETCH_HEAD -m "Merge 31335"
$ git trac fetch 31474 && git merge FETCH_HEAD -m "Merge 31474"

and then got this for local-macports-minimal (I'm a bit surprised it succeeds so fast):

$ tox -p auto -e local-macports-minimal -- config.status
✔ OK local-macports-minimal in 32 minutes, 29.252 seconds
_________________________ summary _________________________
  local-macports-minimal: commands succeeded
  congratulations :)

As for local-macports-standard, it is taking a lot longer. I'll report when it's done.

comment:21 Changed 21 months ago by Matthias Köppe

Cc: Volker Braun added

comment:22 Changed 21 months ago by Matthias Köppe

Let's get this into 9.3 please

comment:23 Changed 21 months ago by Matthias Köppe

Cc: Zachary L Scherr added

comment:24 Changed 21 months ago by John Palmieri

Cc: Zachary L Scherr removed

I tried tox -e local-macports-standard on OS X Big Sur, and it seems to have gotten stuck installing sagelib. I can't find the logs/pkgs directory, but sagelib is not listed in .tox/local-macports-standard/local/var/lib/sage/installed: the most recently modified file there is for networkx. There is also no appropriate directory in local/lib/python3.8/site-packages/. It looks like it's been stuck for hours now. (I'm not sure why it decided to use the Xcode Python 3.8 vs. the MacPorts? Python 3.9, by the way.)

comment:25 Changed 21 months ago by John Palmieri

Cc: Zachary L Scherr added

comment:26 Changed 21 months ago by John Palmieri

Oh, the package log files are in their usual place. Here is the sagelib log file (after I hit ctrl-C to end the process).

Changed 21 months ago by John Palmieri

Attachment: sagelib-9.3.beta9.log added

comment:27 in reply to:  24 Changed 21 months ago by Matthias Köppe

Replying to jhpalmieri:

(I'm not sure why it decided to use the Xcode Python 3.8 vs. the MacPorts? Python 3.9, by the way.)

MacPorts does not install a python3 executable, only versioned executables python3.8, python3.9.

Unclear why it gets stuck in the build -- look like yet another library threads vs. python multiprocessing problem, like the ones that we chased in the parallel docbuild ticket not so long ago. Does it go away if you repeat the build (it will not start from scratch)

comment:28 Changed 21 months ago by git

Commit: 4acfc5c8bed0227762a73ff23204bdce2e88e5cff68ed3c5bb9ce276af78de075bab7c15abe4899f

Branch pushed to git repo; I updated commit sha1. New commits:

f68ed3ctox.ini: Add configuration factors macports-python{3.8,3.9}

comment:29 Changed 21 months ago by Matthias Köppe

Now you can do tox -e local-macports-standard-python3.9

comment:30 Changed 21 months ago by Matthias Köppe

Description: modified (diff)

comment:31 Changed 21 months ago by John Palmieri

tox -e local-macports-standard-python3.9 doesn't work for me: it ends with

Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of 'make all-build'):

* package:         gmpy2-2.1.0b5
  last build time: Mar 26 18:32
  log file:        /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/logs/pkgs/gmpy2-2.1.0b5.log
  build directory: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/.tox/local-macports-standard-python3.9/local/var/tmp/sage/build/gmpy2-2.1.0b5

* package:         rpy2-3.3.6
  last build time: Mar 26 18:40
  log file:        /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/logs/pkgs/rpy2-3.3.6.log
  build directory: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/.tox/local-macports-standard-python3.9/local/var/tmp/sage/build/rpy2-3.3.6

* package:         cysignals-1.10.2
  last build time: Mar 26 18:42
  log file:        /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/logs/pkgs/cysignals-1.10.2.log
  build directory: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/.tox/local-macports-standard-python3.9/local/var/tmp/sage/build/cysignals-1.10.2

* package:         pynac-0.7.27.p1
  last build time: Mar 26 18:43
  log file:        /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/logs/pkgs/pynac-0.7.27.p1.log
  build directory: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/.tox/local-macports-standard-python3.9/local/var/tmp/sage/build/pynac-0.7.27.p1

* package:         cvxopt-1.2.5
  last build time: Mar 26 18:44
  log file:        /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/logs/pkgs/cvxopt-1.2.5.log
  build directory: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/.tox/local-macports-standard-python3.9/local/var/tmp/sage/build/cvxopt-1.2.5

It is safe to delete any log files and build directories, but they
contain information that is helpful for debugging build problems.
WARNING: If you now run 'make' again, the build directory of the
same version of the package will, by default, be deleted. Set the
environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this.

make[1]: *** [all-build] Error 1
make: *** [build] Error 2

rpy2:

clang: error: unsupported option '-fopenmp'

gmpy2:

In file included from src/gmpy2.c:461:
./src/gmpy2.h:81:12: fatal error: 'gmp.h' file not found

cysignals:

ld: illegal thread local variable reference to regular symbol _PARI_SIGINT_block for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/bin/gcc' failed with exit code 1
Building wheel for cysignals (setup.py): finished with status 'error'

etc.

comment:32 Changed 21 months ago by Matthias Köppe

Thanks for testing! Is this on a system that also has homebrew in /usr/local?

comment:33 Changed 21 months ago by git

Commit: f68ed3c5bb9ce276af78de075bab7c15abe4899fc6534e9fd80565e35398df4f84ce6005e8897f9b

Branch pushed to git repo; I updated commit sha1. New commits:

c6534e9Merge tag '9.3.rc0' into t/31505/tox_ini__add_macports_environment

comment:34 Changed 21 months ago by Matthias Köppe

For making sure that no homebrew stuff from /usr/local leaks in, I have some work in #31552, #31562, #31567

comment:35 Changed 21 months ago by John Palmieri

Yes, it has homebrew in /usr/local.

comment:36 Changed 21 months ago by Matthias Köppe

Could you post config.log please?

comment:37 Changed 21 months ago by John Palmieri

Here's config.log from .tox/local-macports-standard/log.

Changed 21 months ago by John Palmieri

Attachment: config.log added

comment:38 Changed 20 months ago by Matthias Köppe

config.log has a lot of /usr/local - for gmp, ntl, boost.

comment:39 Changed 20 months ago by Matthias Köppe

Dependencies: #30504#31567

comment:40 Changed 20 months ago by git

Commit: c6534e9fd80565e35398df4f84ce6005e8897f9b336a49381df30edc01e32516e2f62e9e88c689ee

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

4403924src/sage/interfaces/r.py: Mark all tests # optional - rpy2
405ebb9More # optional - rpy2
b7c1576Merge tag '9.3.beta8' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional
ce3b35fMore # optional - r
49c10c1Revert "build/pkgs/{r,rpy2}: Downgrade to optional"
a67300bMerge #30383
f0a5fb5configure.ac: Add option --disable-r
f04c134Merge branch 't/30383/new_package_type__optional_enabled_by_default' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional
7966b66Merge #31409
336a493Merge branch 't/31567/tox_ini__add_local_macos_nohomebrew_environments' into t/31505/tox_ini__add_macports_environment

comment:41 Changed 20 months ago by John Palmieri

I tried with the new branch and I'm still not succeeding. More problematically, and this has been true throughout, the process never ends — when it hits a problem, it just hangs and I have to hit ctrl-C to stop it.

comment:42 in reply to:  41 ; Changed 20 months ago by Matthias Köppe

Replying to jhpalmieri:

More problematically, and this has been true throughout, the process never ends — when it hits a problem, it just hangs and I have to hit ctrl-C to stop it.

Could this just be the behavior of make -k? I use this throughout in runs via tox.

comment:43 in reply to:  42 Changed 20 months ago by Matthias Köppe

Replying to mkoeppe:

Could this just be the behavior of make -k? I use this throughout in runs via tox.

I've opened #31574 to stop imposing this as a default

comment:44 Changed 20 months ago by John Palmieri

I don't know. For example, when there is an error building sagelib, that particular process never ends: there is no conclusion in the sagelib log file until I halt it, and then it ends with KeyboardInterrupt. Is that a symptom of using make -k? I've used make -k before when building Sage and it has always stopped when it was supposed to.

comment:45 Changed 20 months ago by Matthias Köppe

That sounds more like another instance of the multiprocessing bugs that we encountered recently in the docbuild, just this time affecting the parallel build of extension modules...

comment:46 Changed 20 months ago by Matthias Köppe

Probably better to first try out #31567

comment:47 Changed 20 months ago by git

Commit: 336a49381df30edc01e32516e2f62e9e88c689eecf91f972e1d34ea30d725b217fe71adae324fe8d

Branch pushed to git repo; I updated commit sha1. New commits:

b5edde2tox.ini (macos-nohomebrew): Fix typo in PILLOW_...
dffcdbctox.ini (macos-nohomebrew): Fix up pillow build by setting ZLIB_ROOT
3779a25tox.ini (macos-nohomebrew): Disable more libraries to remove dependencies on homebrew
7ff26dfgit grep -l SAGE_SUITESPARSE | xargs sed -i .bak s/SAGE_SUITESPARSE_LOCALINSTALL/SAGE_SUITESPARSE_PREFIX/g
52537d1build/pkgs/cvxopt/spkg-install.in: Actually use the value of SAGE_SUITESPARSE_PREFIX
ed1a089Merge #31584
cf91f97Merge #31567

comment:48 Changed 20 months ago by John Palmieri

It seems that the macports builds don't exclude homebrew by default, so I had to use tox -e macos-nohomebrew-local-macports-minimal — I hope I got that right. I don't see many traces of /usr/local in the log files, but openblas failed to build, ending with

perl ./gensymbol osx x86_64 _ 1 0  0 0 0 0 "" "" 1 0 1 1 1 1 > osx.def
gfortran-mp-10 -O2 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64 -fPIC -msse3 -mssse3 -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc1/.tox/macos-nohomebrew-local-macports-minimal/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc1/.tox/macos-nohomebrew-local-macports-minimal/local/lib  -all_load -headerpad_max_install_names -install_name "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc1/.tox/macos-nohomebrew-local-macports-minimal/local/var/tmp/sage/build/openblas-0.3.13/src/exports/../libopenblas.0.dylib" -dynamiclib -o ../libopenblas_atomp-r0.3.13.dylib ../libopenblas_atomp-r0.3.13.a -Wl,-exported_symbols_list,osx.def  
/bin/sh: gfortran-mp-10: command not found

I'll attach the log file.

Last edited 20 months ago by John Palmieri (previous) (diff)

Changed 20 months ago by John Palmieri

Attachment: openblas-0.3.13.log added

comment:49 Changed 20 months ago by Matthias Köppe

config.log too please

Changed 20 months ago by John Palmieri

Attachment: config.2.log added

for nohomebrew macports-minimal

comment:50 Changed 20 months ago by John Palmieri

I don't know if there are any helpful files in .../local/var/tmp/build/sage/openblas/src, but I can provide those, too.

comment:51 Changed 20 months ago by Matthias Köppe

No, I won't need those. I found the problem already

comment:52 Changed 20 months ago by git

Commit: cf91f972e1d34ea30d725b217fe71adae324fe8dda6f752e0e38fd1a72617f728193a628a8d4ffdd

Branch pushed to git repo; I updated commit sha1. New commits:

3287a01build/bin/sage-print-system-package-command (macports): Recommend FC as configure arg, not environment variable
da6f752tox.ini (macports): Pass FC as configure argument; add variants macports-gcc_{spkg,9,10,11}

comment:53 in reply to:  48 Changed 20 months ago by Matthias Köppe

Replying to jhpalmieri:

It seems that the macports builds don't exclude homebrew by default,

Yes, the additional isolation work to ignore homebrew in /usr/local is only done if you use the factors macos-nohomebrew.

so I had to use tox -e macos-nohomebrew-local-macports-minimal — I hope I got that right

This variant should work better now. What went wrong in the previous version is that it prescribed using the MacPorts?-provided gfortran compiler, but in the configuration macports-minimal that one is not actually installed.

comment:54 Changed 20 months ago by John Palmieri

Great, macos-nohomebrew-local-macports-minimal now works. The standard version did not, although I'm trying again with the latest branch. (I've now lost the old log files, but a few packages complained about missing png.h.)

Should the macports recipes automatically include nohomebrew? Conversely, and I haven't tried seriously installing macports on a system, so I don't know if this is an issue, but should homebrew builds also try to exclude macports?

comment:55 in reply to:  54 Changed 20 months ago by Matthias Köppe

Replying to jhpalmieri:

Great, macos-nohomebrew-local-macports-minimal now works.

Thanks for testing!

Should the macports recipes automatically include nohomebrew?

I thought about this yesterday but decided against it. I think it's useful for developers to be able to reproduce the build behavior on user machines that have both homebrew and macports installed.

Conversely, and I haven't tried seriously installing macports on a system, so I don't know if this is an issue, but should homebrew builds also try to exclude macports?

I haven't tried yet with a global installation of macports. Given that macports by default does not install into /usr/local, there is much less potential for macports leaking into the build, although there could be packages that check the standard macports install location explicitly.

comment:56 Changed 20 months ago by John Palmieri

Probably for another ticket: Sage's build system doesn't find the macports gpatch program. The file build/pkgs/patch/distros/macports.txt says the right thing, but it looks like spkg-configure.m4 doesn't look for a gpatch executable, only patch.

comment:57 Changed 20 months ago by John Palmieri

Would this fix it?

  • build/pkgs/patch/spkg-configure.m4

    diff --git a/build/pkgs/patch/spkg-configure.m4 b/build/pkgs/patch/spkg-configure.m4
    index 02adb910d9..31bb84dc9d 100644
    a b  
    11SAGE_SPKG_CONFIGURE(
    22    [patch], [
    33       AC_CACHE_CHECK([for GNU patch >= 2.7.0], [ac_cv_path_PATCH], [
    4         AC_PATH_PROGS_FEATURE_CHECK([PATCH], [patch], [
     4        AC_PATH_PROGS_FEATURE_CHECK([PATCH], [patch gpatch], [
    55            patch_version=`$ac_path_PATCH --version 2>&1 \
    66                | $SED -n -e 's/GNU patch *\([[0-9]]*\.[[0-9]]*\.[[0-9]]*\)/\1/p'`
    77            AS_IF([test -n "$patch_version"], [

comment:58 Changed 20 months ago by Matthias Köppe

This alone would not be a good fix. configure would certainly find gpatch after this change -- but then we would have to make sure that all of our code that uses patch instead uses the environment variable PATCH -- and also audit all SPKG whether they do the same. (In any case, it would be a different ticket.)

comment:59 Changed 20 months ago by John Palmieri

I opened #31595 for the gpatch issue.

comment:60 Changed 20 months ago by Matthias Köppe

I have encountered a problem with local-macports-standard-python3.9: rpy2 build fails with clang: error: unsupported option '-fopenmp' - this is now #31596

comment:61 Changed 20 months ago by John Palmieri

I see that too, with macos-nohomebrew-local-macports-standard.

comment:62 Changed 20 months ago by Matthias Köppe

I am testing with EXTRA_CONFIGURE_ARGS="--disable-r" now

comment:63 Changed 20 months ago by John Palmieri

I'm running into mild problems with macos-nohomebrew-local-macports-standard, in particular after I run it once, hit some problems, and then run tox -r -e ...:

Error: Failed to activate icu: Image error: /Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc0/.tox/macos-nohomebrew-local-macports-stan already exists and does not belong to a registered port.  Unable to activate port icu. Use 'port -f activate icu' to force the activation.

Is there really supposed to be a file .tox/macos-nohomebrew-local-macports-stan (which there is), or was the correct name truncated and then not handled properly by tox -r ...?

comment:64 Changed 20 months ago by Matthias Köppe

No, this looks like a bug in the MacPorts? scripts, truncating some filenames.

comment:65 in reply to:  44 Changed 20 months ago by John Palmieri

Replying to jhpalmieri:

I don't know. For example, when there is an error building sagelib, that particular process never ends: there is no conclusion in the sagelib log file until I halt it, and then it ends with KeyboardInterrupt. Is that a symptom of using make -k? I've used make -k before when building Sage and it has always stopped when it was supposed to.

I'm hitting this problem on another computer, this time running OS X Catalina. I don't know what to do except interrupt the process.

comment:66 Changed 20 months ago by Dima Pasechnik

please rebase

comment:67 Changed 20 months ago by git

Commit: da6f752e0e38fd1a72617f728193a628a8d4ffdd439d7464e10e1115c25a5f7cd499b8fa68e07a9b

Branch pushed to git repo; I updated commit sha1. New commits:

439d746Merge tag '9.3.rc2' into t/31505/tox_ini__add_macports_environment

comment:68 Changed 20 months ago by Dima Pasechnik

I see a bug of sorts - /usr/local leaks into macports build, via GMP:

#define ABSOLUTE_GMP_H "///usr/local/include/gmp.h"

and this results in fpylll not finding gmp.h, for some reason

comment:69 Changed 20 months ago by Dima Pasechnik

and gmp in /opt/local is not recognised by ./configure

Last edited 20 months ago by Dima Pasechnik (previous) (diff)

comment:70 Changed 20 months ago by Matthias Köppe

Yes, this should be fixed via #31348 (as in its ticket description, not the branch just added there)

comment:71 Changed 20 months ago by Matthias Köppe

Given that the /usr/local leakage only affects users who have both MacPorts? and /usr/local installed, perhaps we can get this ticket in as is?

comment:72 Changed 20 months ago by John Palmieri

I wish that macos-nohomebrew-local-macports-standard would succeed on at least one machine for me, but I keep hitting problems with sagelib. I'll try one more time and then upload the sagelib log file, but it will probably look like the one I uploaded before.

comment:73 Changed 20 months ago by Matthias Köppe

Do you set MAKE to make -j... or SAGE_NUM_THREADS by any chance? If so, it would be worth testing if the problem goes away if you unset these variables.

Another thing to try could be export EXTRA_CONFIGURE_ARGS="--enable-editable", which uses a simpler build system

comment:74 Changed 20 months ago by John Palmieri

I unset MAKE and MAKEFLAGS, and other than taking longer, I see the same thing. I'm attaching the sagelib log file (before hitting ctrl-c). I'll try with --enable-editable.

Changed 20 months ago by John Palmieri

Attachment: sagelib-9.3.rc2.log added

comment:75 Changed 20 months ago by John Palmieri

With --enable-editable, it did at least end when building the Sage library failed. The end of the log file looks like this:

    g++ -bundle -undefined dynamic_lookup -Wl,-headerpad,0x1000 -L/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib -Wl,-rpath,/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib -O2 -g -march=native build/temp.macosx-10.14.6-x86_64-3.8/sage/symbolic/expression.o -lpynac -lgmp -o build/lib.macosx-10.14.6-x86_64-3.8/sage/symbolic/expression.cpython-38-darwin.so -lpari
    ld: warning: dylib (/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/libpynac.dylib) was built for newer macOS version (10.15) than being linked (10.14.6)
    ld: warning: dylib (/usr/local/lib/libgmp.dylib) was built for newer macOS version (10.15) than being linked (10.14.6)
    error: command '/usr/bin/gcc' failed with exit code 1
ERROR: Command errored out with exit status 1: /Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/src/setup.py'"'"'; __file__='"'"'/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --no-user-cfg develop --no-deps Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 189, in _main
    status = self.run(options, args)
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 178, in wrapper
    return func(self, options, args)
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 391, in run
    installed = install_given_reqs(
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 80, in install_given_reqs
    requirement.install(
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 764, in install
    install_editable_legacy(
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/operations/install/editable_legacy.py", line 49, in install_editable
    call_subprocess(
  File "/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 258, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: /Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/src/setup.py'"'"'; __file__='"'"'/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --no-user-cfg develop --no-deps Check the logs for full command output.
Removed build tracker: '/private/var/folders/00/zrss7sdj3bzd23f2_gnspy2h0000gp/T/pip-req-tracker-jrbc7dbe'

real	5m28.311s
user	49m7.815s
sys	1m20.961s
Version 0, edited 20 months ago by John Palmieri (next)

comment:76 Changed 20 months ago by Matthias Köppe

Thanks for testing!

This error: command '/usr/bin/gcc' failed with exit code 1 without an actual gcc error message is really puzzling.

Also in the previous logs that you sent, there was such a message at the end but I didn't see an actual error either.

Could you try what happens now if you do SKIP_SYSTEM_PKG_INSTALL=1 SKIP_BOOTSTRAP=1 SKIP_CONFIGURE=1 EXTRA_CONFIGURE_ARGS="--enable-editable --disable-r" tox -e macos-nohomebrew-local-macports-standard (note, no -r for tox, so this should be fast)

Changed 20 months ago by John Palmieri

Attachment: sagelib-9.3.rc2-delta.log added

comment:77 Changed 20 months ago by John Palmieri

It fails again. I've attached the new part of the sagelib log.

comment:78 Changed 20 months ago by Matthias Köppe

Now error messages are there!

    building 'sage.libs.readline' extension
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -O2 -g -march=native -Isage/cpython -I/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/src -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -I/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/python3.8/site-packages/numpy/core/include -I/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c sage/libs/readline.c -o build/temp.macosx-10.14.6-x86_64-3.8/sage/libs/readline.o
    sage/libs/readline.c:1451:36: error: use of undeclared identifier 'rl_catch_signals'
      __pyx_t_1 = __Pyx_PyInt_From_int(rl_catch_signals); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 88, __pyx_L1_error)
    gcc -bundle -undefined dynamic_lookup -Wl,-headerpad,0x1000 -L/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib -Wl,-rpath,/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib -O2 -g -march=native build/temp.macosx-10.14.6-x86_64-3.8/sage/rings/complex_double.o -L/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/macports/lib -lm -lgmp -lpari -lgsl -lm -lopenblas -o build/lib.macosx-10.14.6-x86_64-3.8/sage/rings/complex_double.cpython-38-darwin.so -lpari
    ld: warning: dylib (/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/macports/lib/libgmp.dylib) was built for newer macOS version (10.15) than being linked (10.14.6)
    ld: warning: dylib (/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/macports/lib/libgsl.dylib) was built for newer macOS version (10.15) than being linked (10.14.6)
    ld: warning: dylib (/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/macports/lib/libopenblas.dylib) was built for newer macOS version (10.15) than being linked (10.14.6)
    ld: illegal data reference in _avma to thread local variable _avma in dylib /Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/.tox/macos-nohomebrew-local-macports-standard/local/lib/libpari.dylib for architecture x86_64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command '/usr/bin/gcc' failed with exit code 1
Last edited 20 months ago by Matthias Köppe (previous) (diff)

comment:79 Changed 20 months ago by Matthias Köppe

The readline issue is probably just #29000 again.

(Or we may want to create a .macports-build-env script, after all. https://guide.macports.org/#installing.shell only mentions setting PATH - but we may need to also set CPATH and LIBRARY_PATH if we want libraries to be found that do not come with .pc files.)

comment:80 Changed 20 months ago by John Palmieri

At this point I'm certainly happy removing the ban, but since I can't get the macports-standard tox build to work, I can't positively review the rest.

comment:81 Changed 20 months ago by Matthias Köppe

Dependencies: #31567#31567, #31641
Summary: Remove outdated macports ban; Extend system package scripts to handle macports; tox.ini: Add macports environmentExtend system package scripts to handle macports; tox.ini: Add macports environment

OK, I have split this part out to #31641

comment:82 Changed 20 months ago by Matthias Köppe

Milestone: sage-9.3sage-9.4
Priority: criticalmajor

comment:83 Changed 20 months ago by Matthias Köppe

Status: needs_reviewneeds_work
Work issues: rebase on #31534, #31641

comment:84 Changed 18 months ago by git

Commit: 439d7464e10e1115c25a5f7cd499b8fa68e07a9bc787c3cb69c0644f119facea4084459976b25db6

Branch pushed to git repo; I updated commit sha1. New commits:

617e8ebtox.ini: In -maximal environments, use IGNORE_MISSING_SYSTEM_PACKAGES=yes for all non-current distributions
ab785aftox.ini: Use IGNORE_MISSING_SYSTEM_PACKAGES=no for rolling distributions, but IGNORE_MISSING_SYSTEM_PACKAGES=yes for unstable distributions
afc7266Merge tag '9.4.beta0' into t/31534/tox_ini__in__maximal_environments__use_ignore_missing_system_packages_yes_for_all_non_current_distributions
c787c3cMerge branch 't/31534/tox_ini__in__maximal_environments__use_ignore_missing_system_packages_yes_for_all_non_current_distributions' into t/31505/tox_ini__add_macports_environment

comment:85 Changed 18 months ago by Matthias Köppe

Dependencies: #31567, #31641#31534
Work issues: rebase on #31534, #31641

comment:86 Changed 17 months ago by git

Commit: c787c3cb69c0644f119facea4084459976b25db627b0064337dc42705da89a67518d17d67fa82dfa

Branch pushed to git repo; I updated commit sha1. New commits:

27b0064Merge tag '9.4.beta4' into t/31505/tox_ini__add_macports_environment

comment:87 Changed 17 months ago by Matthias Köppe

Dependencies: #31534

comment:88 Changed 17 months ago by Samuel Lelièvre

How can we move forward here? Does this still need work? Or is it ready for review?

comment:89 Changed 17 months ago by Matthias Köppe

Dependencies: #31786

I'd think after #31786 is merged, we can update build/pkgs/gfortran/distros/macports.txt and perhaps create a .macports-build-env as suggested in comment:13, comment:79.

comment:90 Changed 16 months ago by Matthias Köppe

Milestone: sage-9.4sage-9.5

comment:91 Changed 13 months ago by Samuel Lelièvre

Ticket #31786 was merged in Sage 9.4.beta6.

comment:92 Changed 11 months ago by Matthias Köppe

Milestone: sage-9.5sage-9.6

comment:93 Changed 8 months ago by Matthias Köppe

Milestone: sage-9.6sage-9.7

comment:94 Changed 8 months ago by git

Commit: 27b0064337dc42705da89a67518d17d67fa82dfae7e72fa9d4a1ad49ec6a5f8f6459156bba92812c

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

e21e053tox.ini: Add local-macports
2de11f7Add macports.txt
68dc2a2build/pkgs/_prereq/distros: Update file comments
a299cedbuild/bin/sage-print-system-package-command: Handle macports install
73cf4e2tox.ini (local-macports-optlocal): Use sudo
73a339dbuild/bin/sage-print-system-package-command (macports): Handle setup-build-env
a193556build/bin/sage-guess-package-system: Detect macports
f9ea55etox.ini: Add configuration factors macports-python{3.8,3.9}
55910b9build/bin/sage-print-system-package-command (macports): Recommend FC as configure arg, not environment variable
e7e72fatox.ini (macports): Pass FC as configure argument; add variants macports-gcc_{spkg,9,10,11}

comment:95 Changed 8 months ago by Matthias Köppe

Dependencies: #31786
Description: modified (diff)

comment:96 Changed 8 months ago by John Palmieri

I must be doing something wrong:

% tox -e local-macports-standard
_____________________________________________________ summary _____________________________________________________
ERROR:   local-macports-standard: unresolvable substitution(s):
    commands: 'EXTRA_SAGE_PACKAGES'
Environment variables are missing or defined recursively.

comment:97 Changed 8 months ago by git

Commit: e7e72fa9d4a1ad49ec6a5f8f6459156bba92812c4bf4e4577aa070b0f3fb581e1cfad545225edb9f

Branch pushed to git repo; I updated commit sha1. New commits:

4bf4e45tox.ini (macports): Fix up use of ALL_EXTRA_SAGE_PACKAGES

comment:98 Changed 8 months ago by Matthias Köppe

Thanks for testing! I hadn't tested it yet after rebasing

comment:99 Changed 8 months ago by John Palmieri

Thanks, now it's building. I don't know why it (Macports) is building Python 2.7, but I happened to look just now, and that's what it's doing.

Last edited 8 months ago by John Palmieri (previous) (diff)

comment:100 Changed 8 months ago by John Palmieri

I don't know if this part of the description is still accurate:

tox -e local-macports-standard             # uses /usr/bin/python unless configured otherwise

There is no /usr/bin/python on my OS X machine, just /usr/bin/python3.

comment:101 Changed 8 months ago by Matthias Köppe

tox -e local-macports-standard just failed for me (on macOS Big Sur 11.6) with

--->  Applying patches to gcc10
--->  Configuring gcc10
Warning: Applying '--without-build-config' workaround to Xcode none / CLT 13.2.0.0.1.1638488800
Warning: If versions > 12.5 please check if it is still required
--->  Building gcc10                                     
Error: Failed to build gcc10: command execution failed   
Error: See /Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-standard/macports/var/macports/logs/_Users_mkoeppe_s_sage_sage-rebasing_worktree-rebase_.tox_local-macports-standard_macports_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc10/gcc10/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port gcc10 failed

comment:102 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:103 in reply to:  101 Changed 8 months ago by John Palmieri

Replying to mkoeppe:

tox -e local-macports-standard just failed for me (on macOS Big Sur 11.6) with

--->  Applying patches to gcc10
--->  Configuring gcc10
Warning: Applying '--without-build-config' workaround to Xcode none / CLT 13.2.0.0.1.1638488800
Warning: If versions > 12.5 please check if it is still required
--->  Building gcc10                                     
Error: Failed to build gcc10: command execution failed   
Error: See /Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-standard/macports/var/macports/logs/_Users_mkoeppe_s_sage_sage-rebasing_worktree-rebase_.tox_local-macports-standard_macports_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc10/gcc10/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port gcc10 failed

I see the same.

comment:104 Changed 8 months ago by git

Commit: 4bf4e4577aa070b0f3fb581e1cfad545225edb9f18a394628b793358016cad8327fa31473c9a6d72

Branch pushed to git repo; I updated commit sha1. New commits:

8c62dedtox.ini (local-macports): Update macports base version to 2.7.2
18a3946build/pkgs/gfortran/distros/macports.txt: Switch to gcc11

comment:105 Changed 8 months ago by Matthias Köppe

With the change to gcc11, I am getting a bit farther.

--->  Building R
Error: Failed to build R: command execution failed       
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_R/R/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port R failed

apparently because of a failure in some pdf docbuild

:info:build Error: compiling TeX file 'reshape.tex' failed with message:
:info:build Running 'texi2dvi' on 'reshape.tex' failed.
:info:build LaTeX errors:
:info:build ! LaTeX Error: File `fancyvrb.sty' not found.
:info:build Type X to quit or <RETURN> to proceed,
:info:build or enter new name. (Default extension: sty)
:info:build ! Emergency stop.
:info:build <read *> 
:info:build          
:info:build l.19 \begingroup
:info:build                 ^^M
:info:build !  ==> Fatal error occurred, no output PDF file produced!
:info:build Execution halted
:info:build make[1]: *** [vignettes] Error 1
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_R/R/work/R-4.1.3/src/library'
:info:build make: *** [vignettes] Error 2

comment:106 Changed 8 months ago by John Palmieri

I see

Warning: texlive-bin installs files outside the common directory structure.
--->  Installing texlive-bin @2022.62882_0+x11
--->  Activating texlive-bin @2022.62882_0+x11
Error: Failed to activate texlive-bin: can't create directory "/Library/TeX/Distributions/.FactoryDefaults/MacPorts-Users_palmieri_Desktop_Sage_sage_builds_TESTING_sage-9.6.rc0_.tox_local-macports-standard_macports-TeXLive": permission denied
Error: See /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.6.rc0/.tox/local-macports-standard/macports/var/macports/logs/_Users_palmieri_Desktop_Sage_sage_builds_TESTING_sage-9.6.rc0_.tox_local-macports-standard_macports_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_tex_texlive-bin/texlive-bin/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port R failed

I don't like the idea of this trying to install anything in /Library, and that looks like a problem.

comment:107 Changed 8 months ago by Matthias Köppe

Yes, it looks like not all packages are prepared for installation in a custom prefix.

I've tried installing in the default prefix (/opt/local) with sudo (that's tox -e local-macports-optlocal-standard), and installation of texlive went through without error. Just did sudo /opt/local/bin/port install texlive-latex-extra to get some of the missing files.

comment:108 Changed 8 months ago by Matthias Köppe

Report Upstream: N/ANot yet reported upstream; Will do shortly.

comment:109 Changed 8 months ago by git

Commit: 18a394628b793358016cad8327fa31473c9a6d727ab30b4dad469b6f15438148f689252c640ac6d9

Branch pushed to git repo; I updated commit sha1. New commits:

7ab30b4tox.ini (macports): By default use FC=gfortran-mp-11

comment:110 Changed 8 months ago by git

Commit: 7ab30b4dad469b6f15438148f689252c640ac6d9b91993318a58e73ace783a838b171b1f8208c9c0

Branch pushed to git repo; I updated commit sha1. New commits:

b919933build/bin/sage-print-system-package-command (macports): Also update to gcc11 here

comment:111 Changed 8 months ago by Matthias Köppe

This gets me to the Sage distribution build. Next error: rpy2 fails

  gcc -bundle -undefined dynamic_lookup -Wl,-headerpad,0x1000 -L/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -Wl,-rpath,/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -L/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -Wl,-rpath,/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -g -O2 build/temp.macosx-10.14-x86_64-3.8/build/temp.macosx-10.14-x86_64-3.8/_rinterface_cffi_api.o -L/opt/local/lib -L/opt/local/lib -lpcre2-8 -llzma -lbz2 -lz -ldl -lm -liconv -licuuc -licui18n -o build/lib.macosx-10.14-x86_64-3.8/_rinterface_cffi_api.abi3.so -fopenmp -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -F/opt/local/Library/Frameworks/R.framework/.. -framework R
  clang: error: unsupported option '-fopenmp'

comment:112 Changed 8 months ago by Matthias Köppe

Also sagelib fails

4 warnings generated.
ld: illegal data reference in _avma to thread local variable _avma in dylib /Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib/libpari.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: Command "gcc -bundle -undefined dynamic_lookup -Wl,-headerpad,0x1000 -L/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -Wl,-rpath,/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -g -O2 build/temp.macosx-10.14-x86_64-3.8/build/cythonized/sage/libs/pari/convert_sage_complex_double.o -L/opt/local/lib -lgsl -lpari -lgmp -lm -lopenblas -o build/lib.macosx-10.14-x86_64-3.8/sage/libs/pari/convert_sage_complex_double.cpython-38-darwin.so -lpari" failed with exit status 1

comment:113 Changed 8 months ago by John Palmieri

With tox -e local-macports-minimal, gfortran fails. This unfortunately happens on this machine with ordinary Sage if I use ./configure --with-system-gfortran=no with the error message

Error: configuring for an unreleased macOS version x86_64-apple-darwin21.4.0

I don't know why this is called "unreleased".

comment:114 Changed 8 months ago by Matthias Köppe

I guess our gcc/gfortran package is just too old.

Unfortunately we don't see this on our GH Actions because there are no macOS 12 runners yet. Apparently one can request access - https://github.community/t/macos-12-runner-not-available/242185/9

comment:115 Changed 8 months ago by Matthias Köppe

I've requested access to the macOS 12 runners for the sagemath GH organization.

comment:116 in reply to:  111 Changed 8 months ago by Matthias Köppe

Replying to mkoeppe:

This gets me to the Sage distribution build. Next error: rpy2 fails

  gcc -bundle -undefined dynamic_lookup -Wl,-headerpad,0x1000 -L/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -Wl,-rpath,/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -L/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -Wl,-rpath,/Users/mkoeppe/s/sage/sage-rebasing/worktree-rebase/.tox/local-macports-optlocal-standard/local/lib -g -O2 build/temp.macosx-10.14-x86_64-3.8/build/temp.macosx-10.14-x86_64-3.8/_rinterface_cffi_api.o -L/opt/local/lib -L/opt/local/lib -lpcre2-8 -llzma -lbz2 -lz -ldl -lm -liconv -licuuc -licui18n -o build/lib.macosx-10.14-x86_64-3.8/_rinterface_cffi_api.abi3.so -fopenmp -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -F/opt/local/Library/Frameworks/R.framework/.. -framework R
  clang: error: unsupported option '-fopenmp'

This compiler flag comes from the configuration of the macports-provided R.

(sage-buildsh) mkoeppe@egret:src$ /opt/local/bin/R CMD config --cppflags
-I/opt/local/Library/Frameworks/R.framework/Resources/include
(sage-buildsh) mkoeppe@egret:src$ /opt/local/bin/R CMD config --ldflags
-fopenmp -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -F/opt/local/Library/Frameworks/R.framework/.. -framework R -L/opt/local/lib -lpcre2-8 -llzma -lbz2 -lz -ldl -lm -liconv -licuuc -licui18n
(sage-buildsh) mkoeppe@egret:src$ /opt/local/bin/R CMD config CC
/opt/local/bin/clang-mp-13

comment:117 in reply to:  113 Changed 7 months ago by Matthias Köppe

Replying to jhpalmieri:

gfortran fails. This unfortunately happens on this machine with ordinary Sage if I use ./configure --with-system-gfortran=no with the error message

Error: configuring for an unreleased macOS version x86_64-apple-darwin21.4.0

I've opened #33750 for the upgrade of gcc/gfortran

comment:118 Changed 7 months ago by John Palmieri

With 9.6.rc3 + this ticket, tox -e local-macports-minimal more or less worked for me: the first time through it hung, but then I hit ctrl-c and ran it again, and it finished. tox -e local-macports-standard did not work.

comment:119 Changed 7 months ago by John Palmieri

Running tox -e local-macports-standard ends with:

--->  Computing dependencies for gd2
--->  Dependencies to be installed: fontconfig freetype brotli cmake curl libheif aom git gdk-pixbuf2 gobject-introspection cairo libjpeg-turbo tiff lerc rav1e cargo cargo-bootstrap rust libgit2 cargo-c x265 webp
Error: The following dependencies were not installed because all of them have unmet dependencies (likely due to a dependency cycle): fontconfig freetype brotli cmake curl libheif aom git gdk-pixbuf2 gobject-introspection cairo libjpeg-turbo tiff lerc rav1e cargo cargo-bootstrap rust libgit2 cargo-c x265 webp
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port gd2 failed
ERROR: InvocationError for command /bin/bash -c 'case "" in 1|y*|Y*);; *)  port selfupdate &&  port upgrade outdated && PACKAGES=$(build/bin/sage-get-system-packages macports $(PATH=build/bin:$PATH build/bin/sage-package list --has-file=spkg-configure.m4 :standard:) _bootstrap    ); eval $(build/bin/sage-print-system-package-command macports  --yes --no-install-recommends install $PACKAGES) || [ "$IGNORE_MISSING_SYSTEM_PACKAGES" = yes ] && echo "(ignoring errors)" ;; esac' (exited with code 1)
________________________________________ summary _________________________________________
ERROR:   local-macports-standard: commands failed

comment:120 Changed 7 months ago by Matthias Köppe

I removed /opt/local after the upgrade to macOS Monterey, and tox -e local-macports-optlocal-minimal did a fresh installation of macports. The build succeeded.

comment:121 Changed 7 months ago by git

Commit: b91993318a58e73ace783a838b171b1f8208c9c06544ae26b0acdf0a24f411c80dc75a25a2e3b760

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

9b63cdbtox.ini: Add configuration factors macports-python{3.8,3.9}
c0cfbbfbuild/bin/sage-print-system-package-command (macports): Recommend FC as configure arg, not environment variable
1c99ba2tox.ini (macports): Pass FC as configure argument; add variants macports-gcc_{spkg,9,10,11}
8b36b30tox.ini (macports): Fix up use of ALL_EXTRA_SAGE_PACKAGES
5942923tox.ini (local-macports): Update macports base version to 2.7.2
5e97276build/pkgs/gfortran/distros/macports.txt: Switch to gcc11
9fedf39tox.ini (macports): By default use FC=gfortran-mp-11
ead3515build/bin/sage-print-system-package-command (macports): Also update to gcc11 here
336cd17WIP
6544ae2tox.ini (macports): Remove variants that tried to use real gcc

comment:122 Changed 7 months ago by Matthias Köppe

(That's after rebasing on 9.6rc3)

comment:123 in reply to:  119 Changed 7 months ago by Matthias Köppe

Replying to jhpalmieri:

Running tox -e local-macports-standard ends with:

--->  Computing dependencies for gd2
--->  Dependencies to be installed: fontconfig freetype brotli cmake curl libheif aom git gdk-pixbuf2 gobject-introspection cairo libjpeg-turbo tiff lerc rav1e cargo cargo-bootstrap rust libgit2 cargo-c x265 webp

I'm getting the same error with tox -e local-macports-optlocal-standard.

comment:124 Changed 7 months ago by git

Commit: 6544ae26b0acdf0a24f411c80dc75a25a2e3b76027eff2ba095a385db8ab70434e1a4f8661bc6295

Branch pushed to git repo; I updated commit sha1. New commits:

27eff2bbuild/pkgs/libgd/distros/macports.txt: Disable

comment:125 Changed 7 months ago by git

Commit: 27eff2ba095a385db8ab70434e1a4f8661bc6295c6cd79f10c19a26175cdb8b16fe30e52b9e827d0

Branch pushed to git repo; I updated commit sha1. New commits:

c6cd79fbuild/pkgs/python3/distros/macports.txt: Use python310

comment:126 Changed 7 months ago by Matthias Köppe

Now it ends with Error: The following dependencies were not installed because all of them have unmet dependencies (likely due to a dependency cycle): curl brotli cmake

comment:127 Changed 7 months ago by git

Commit: c6cd79f10c19a26175cdb8b16fe30e52b9e827d0a9094850ec0c9b206a224871401a5d6eb33a6962

Branch pushed to git repo; I updated commit sha1. New commits:

a909485Disable more broken macports packages

comment:128 Changed 7 months ago by git

Commit: a9094850ec0c9b206a224871401a5d6eb33a696299feb0db45fac76f048c56b8cb083442b99cb1b6

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

99feb0dDisable more broken macports packages

comment:129 Changed 7 months ago by git

Commit: 99feb0db45fac76f048c56b8cb083442b99cb1b67a2d87519020f7ce44dc74ab74784b12246d73fb

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7a2d875Disable more broken macports packages

comment:130 Changed 7 months ago by git

Commit: 7a2d87519020f7ce44dc74ab74784b12246d73fb1fcbecf1893fb400a73eb9c55363a47b787ea4f0

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

1fcbecfDisable more broken macports packages

comment:131 Changed 7 months ago by Matthias Köppe

Now all packages that run into the issue with cmake appear to be disabled. Macports package installation finished without error now.

comment:132 Changed 7 months ago by Matthias Köppe

There are a few installation notes

  gpatch has the following notes:
    The patch tool provided by this port is prefixed with the character 'g' by default making it available as 'gpatch'.
    
    If you want to use 'patch' and other GNU coreutils by default, add this directory to the front of your PATH environment variable:
        /opt/local/libexec/gnubin/

  py310-tox has the following notes:
    To make the Python 3.10 version of tox the one that is run when you execute the commands without a version suffix, e.g. 'tox', run:
    
    port select --set tox tox310
  py310-virtualenv has the following notes:
    The executable is installed as '/opt/local/bin/virtualenv-3.10'. To symlink it to '/opt/local/bin/virtualenv', run:
    
        sudo port select --set virtualenv virtualenv310
  python310 has the following notes:
    To make this the default Python or Python 3 (i.e., the version run by the 'python' or 'python3' commands), run one or both of:
    
        sudo port select --set python python310
        sudo port select --set python3 python310

In particular, tox -e local-macports-optlocal-standard uses /usr/bin/python3 as the python.

comment:133 Changed 7 months ago by Matthias Köppe

tox -e local-macports-optlocal-standard succeeds.

comment:134 Changed 7 months ago by Matthias Köppe

tox -e local-macports-optlocal-python3.10-standard warns:

checking ... whether /opt/local/bin/python3.10 is good... 
configure: WARNING: this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://trac.sagemath.org/ticket/31132

comment:135 Changed 7 months ago by Matthias Köppe

Status: needs_workneeds_review

comment:136 Changed 7 months ago by git

Commit: 1fcbecf1893fb400a73eb9c55363a47b787ea4f01e8e4bce49ff91a359049ee5a8cd3c50196049b9

Branch pushed to git repo; I updated commit sha1. New commits:

1e8e4bcbuild/bin/sage-print-system-package-command [macports]: Do not describe variants that do not work

comment:137 in reply to:  134 ; Changed 7 months ago by Matthias Köppe

Replying to mkoeppe:

tox -e local-macports-optlocal-python3.10-standard warns:

checking ... whether /opt/local/bin/python3.10 is good... 
configure: WARNING: this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://trac.sagemath.org/ticket/31132

... and gmpy2 fails to build

comment:138 in reply to:  137 Changed 7 months ago by Matthias Köppe

Replying to mkoeppe:

Replying to mkoeppe:

tox -e local-macports-optlocal-python3.10-standard warns:

checking ... whether /opt/local/bin/python3.10 is good... 
configure: WARNING: this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://trac.sagemath.org/ticket/31132

... and gmpy2, cysignals, cvxopt fail to build - similar to comment:31

comment:139 Changed 7 months ago by git

Commit: 1e8e4bce49ff91a359049ee5a8cd3c50196049b95bfc840d5de9bed57f2b22c8eb5c8f53a90a27d4

Branch pushed to git repo; I updated commit sha1. New commits:

5bfc840tox.ini (macports): Set CPATH, LIBRARY_PATH

comment:140 Changed 7 months ago by Matthias Köppe

This fixes gmpy2, cvxopt

comment:141 Changed 7 months ago by git

Commit: 5bfc840d5de9bed57f2b22c8eb5c8f53a90a27d430c326c981897a58dcf1b12f1395ae1065be57d8

Branch pushed to git repo; I updated commit sha1. New commits:

30c326cbuild/pkgs/pari/distros/macports.txt: Disable

comment:142 Changed 7 months ago by git

Commit: 30c326c981897a58dcf1b12f1395ae1065be57d8c69872294366bb80b1560505e2ca45cce170db4d

Branch pushed to git repo; I updated commit sha1. New commits:

c698722tox.ini (macports): Use isysroot

comment:143 Changed 7 months ago by Matthias Köppe

Now tox -e local-macports-optlocal-python3.10-standard succeeds. The trick was to block out the homebrew installation in /usr/local.

comment:144 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:145 Changed 4 months ago by git

Commit: c69872294366bb80b1560505e2ca45cce170db4d476fa33e16780466a04556f1c87532465dbde898

Branch pushed to git repo; I updated commit sha1. New commits:

476fa33Merge tag '9.7.beta7' into t/31505/tox_ini__add_macports_environment

comment:146 Changed 4 months ago by Matthias Köppe

tox -e local-macports-optlocal-standard-python3.10 -- ptest builds without problems, and there is only a small number of doctest failures.

sage -t --random-seed=37353794756962291469199267391330052516 src/doc/en/constructions/calculus.rst  # Timed out
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/dynamics/arithmetic_dynamics/projective_ds.py  # Timed out
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/interfaces/macaulay2.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/libs/libecm.pyx  # 7 doctests failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/matrix/constructor.pyx  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/misc/sageinspect.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/plot/plot3d/parametric_plot3d.py  # Timed out
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/attach.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/display/formatter.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/interface_magic.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/interpreter.py  # 2 doctests failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/ipython_tests.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/rich_output/backend_ipython.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/repl/ipython_extension.py  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/structure/sage_object.pyx  # 1 doctest failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/tests/cmdline.py  # 4 doctests failed
sage -t --random-seed=37353794756962291469199267391330052516 src/sage/typeset/ascii_art.py  # 1 doctest failed

Most of these failures are from the warning "UserWarning?: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv."

The only real failures are:

sage -t --random-seed=37353794756962291469199267391330052516 src/sage/libs/libecm.pyx
**********************************************************************
File "src/sage/libs/libecm.pyx", line 33, in sage.libs.libecm
Failed example:
    ecmfactor(2^128+1,1000,sigma=227140902)
Expected:
    (True, 5704689200685129054721, 227140902)
Got:
    (False, None)
**********************************************************************
File "src/sage/libs/libecm.pyx", line 91, in sage.libs.libecm.ecmfactor
Failed example:
    ecmfactor(N, 2e5)
Expected:
    (True, 2349023, ...)
Got:
    (True, 187072209578355573530071658587684226515959365500927, 0)
**********************************************************************
File "src/sage/libs/libecm.pyx", line 113, in sage.libs.libecm.ecmfactor
Failed example:
    ecmfactor(N, 1e3)
Expected:
    (False, None)
Got:
    (True, 170141183460469231731687303715884105727, 0)
**********************************************************************
File "src/sage/libs/libecm.pyx", line 130, in sage.libs.libecm.ecmfactor
Failed example:
    ecmfactor(N, 100, verbose=True)
Expected:
    Performing one curve with B1=100
    Found factor in step 1: 11
    (True, 11, ...)
Got:
    Performing one curve with B1=100
    Found factor in step 1: 479270685891287906588011710623477140055626627230262983486227930615309559797523855924635580611607650107391
    (True,
     479270685891287906588011710623477140055626627230262983486227930615309559797523855924635580611607650107391,
     0)
**********************************************************************
File "src/sage/libs/libecm.pyx", line 134, in sage.libs.libecm.ecmfactor
Failed example:
    ecmfactor(N/11, 100, verbose=True)
Expected:
    Performing one curve with B1=100
    Found no factor.
    (False, None)
Got:
    Performing one curve with B1=100
    Found factor in step 1: 43570062353753446053455610056679740005056966111842089407838902783209959981593077811330507328327968191581
    (True,
     43570062353753446053455610056679740005056966111842089407838902783209959981593077811330507328327968191581,
     0)
**********************************************************************
File "src/sage/libs/libecm.pyx", line 144, in sage.libs.libecm.ecmfactor
Failed example:
    alarm(0.5); ecmfactor(2^521-1, 1e7)
Expected:
    Traceback (most recent call last):
    ...
    AlarmInterrupt
Got:
    (True,
     6864797660130609714981900799081393217269435300143305409394463459185543183397656052122559640661454554977296311391480858037121987999716643812574028291115057151,
     0)
**********************************************************************
File "src/sage/libs/libecm.pyx", line 151, in sage.libs.libecm.ecmfactor
Failed example:
    ecmfactor(1, 100)
Expected:
    (True, 1, ...)
Got:
    (False, None)
**********************************************************************
2 items had failures:
   1 of   9 in sage.libs.libecm
   6 of  21 in sage.libs.libecm.ecmfactor
    [28 tests, 7 failures, 0.93 s]

comment:147 Changed 4 months ago by Matthias Köppe

Good enough to get this in?

comment:148 Changed 3 months ago by Dima Pasechnik

does macports have its own ecm here in these errors? If so, we can avoid them by testing ecm more in spkg-configure.

comment:149 Changed 3 months ago by Matthias Köppe

Yes, there is an ecm.

comment:150 Changed 3 months ago by Matthias Köppe

Do you want to work on this? I don't, hence my question in comment:147

comment:151 in reply to:  150 Changed 3 months ago by Dima Pasechnik

Replying to mkoeppe:

Do you want to work on this? I don't, hence my question in comment:147

I'll see if I can fix this quickly. If not, it can be merged as is, I guess.

comment:152 Changed 3 months ago by John Palmieri

I tried tox -e local-macports-standard and got

Error: Failed to install cmake-bootstrap: MacPorts requires root privileges for this action
Error: See /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.7.beta8/.tox/local-macports-standard/macports/var/macports/logs/_Users_palmieri_Desktop_Sage_sage_builds_TESTING_sage-9.7.beta8_.tox_local-macports-standard_macports_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_cmake-bootstrap/cmake-bootstrap/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port gcc11 failed

Since I wasn't using local-macports-optlocal-standard, I wasn't expecting to need root privileges.

comment:153 Changed 3 months ago by Matthias Köppe

I can confirm this. It seems to be a new macports bug.

comment:154 Changed 3 months ago by Dima Pasechnik

regarding ecm errors, are they only for the library interface, whereas sage/interfaces/ecm.py tests OK?

comment:155 Changed 3 months ago by Matthias Köppe

If I was careful in comment:146 where I summarized the failures, then yes

comment:156 in reply to:  153 Changed 3 months ago by Dima Pasechnik

Replying to mkoeppe:

I can confirm this. It seems to be a new macports bug.

https://ports.macports.org/port/cmake-bootstrap/ says to run sudo port install cmake-bootstrap - no idea why.

I'm trying to test with sudo now.

comment:157 Changed 3 months ago by Dima Pasechnik

How does one pass --enable-build-as-root to tox?

comment:158 Changed 3 months ago by Matthias Köppe

You can set the environment variable EXTRA_CONFIGURE_ARGS

comment:159 Changed 3 months ago by Dima Pasechnik

sudo EXTRA_CONFIGURE_ARGS="--enable-build-as-root" tox -e local-macports-standard is very slow (single-thread?), and seems to hang on installing Pari - this is after installing all the pre-reqs - which was also very slow, and included building clang from source (sic!).

Can't it use XCode's clang instead?

comment:160 Changed 3 months ago by Dima Pasechnik

anyway, macports in .tox/ was installed,so I chown'd it to the normal user, and proceeded without sudo. This pushed the build further,but still it is single thread. Waiting for tests to finish.

comment:161 in reply to:  159 Changed 3 months ago by Matthias Köppe

Replying to dimpase:

sudo EXTRA_CONFIGURE_ARGS="--enable-build-as-root" tox -e local-macports-standard is very slow (single-thread?)

You could have set MAKE="make -j16"

Last edited 3 months ago by Matthias Köppe (previous) (diff)

comment:162 Changed 3 months ago by Dima Pasechnik

with the latest beta8 merged, I am getting a sagelib building error

    error: Command "gcc -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -Wno-unused-result 
-Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System
/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Develop
er/Library/Frameworks/Python3.framework/Versions/3.8/Headers -Werror=implicit-function-declaration -g -O2
 -I///Users/dima/software/sage/.tox/local-macports-standard/macports/include -Isage/cpython -Isage/libs/n
tl -Isage/libs/arb -I/Users/dima/software/sage/.tox/local-macports-standard/local/lib/python3.8/site-pack
ages/cypari2 -Isage/libs/flint -I/Users/dima/software/sage/.tox/local-macports-standard/local/lib/python3
.8/site-packages/cysignals -I/Users/dima/software/sage/src -I/Users/dima/software/sage/.tox/local-macport
s-standard/local/lib/python3.8/site-packages/numpy/core/include -I/Applications/Xcode.app/Contents/Develo
per/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -I/Users/dima/software/sage/.tox/
local-macports-standard/local/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Pyt
hon3.framework/Versions/3.8/include/python3.8 -c sage/rings/number_field/number_field_element_quadratic.c
pp -o build/temp.macosx-10.14-x86_64-cpython-38/sage/rings/number_field/number_field_element_quadratic.o 
-std=c++11" failed with exit status 1
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /Users/dima/software/sage/.tox/local-macports-standard/local/bin/python3 -c '
    exec(compile('"'"''"'"''"'"'
    # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
    #
    # - It imports setuptools before invoking setup.py, to enable projects that directly
    #   import from `distutils.core` to work with newer packaging standards.
    # - It provides a clear error message when setuptools is not installed.
    # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
    #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
    #     manifest_maker: standard file '"'"'-c'"'"' not found".
    # - It generates a shim setup.py, for handling setup.cfg-only projects.
    import os, sys, tokenize
    
    try:
        import setuptools
    except ImportError as error:
        print(
            "ERROR: Can not execute `setup.py` since setuptools is not available in "
            "the build environment.",
            file=sys.stderr,
        )
        sys.exit(1)
    
    __file__ = %r
    sys.argv[0] = __file__
    
    if os.path.exists(__file__):
        filename = __file__
        with tokenize.open(__file__) as f:
            setup_py_code = f.read()
    else:
        filename = "<auto-generated setuptools caller>"
        setup_py_code = "from setuptools import setup; setup()"
    
    exec(compile(setup_py_code, filename, "exec"))
    '"'"''"'"''"'"' % ('"'"'/Users/dima/software/sage/src/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' --no-user-cfg develop --no-deps
    cwd: /Users/dima/software/sage/src/
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

comment:163 Changed 3 months ago by Matthias Köppe

The actual g++ error message is missing in what you pasted here

comment:164 Changed 3 months ago by Dima Pasechnik

right - I thought it's some kind of Python packaging error :-)

    INFO: compile options: '-I///Users/dima/software/sage/.tox/local-macports-standard/macports/include -
Isage/cpython -Isage/libs/ntl -Isage/libs/arb -I/Users/dima/software/sage/.tox/local-macports-standard/lo
cal/lib/python3.8/site-packages/cypari2 -Isage/libs/flint -I/Users/dima/software/sage/.tox/local-macports
-standard/local/lib/python3.8/site-packages/cysignals -I/Users/dima/software/sage/src -I/Users/dima/softw
are/sage/.tox/local-macports-standard/local/lib/python3.8/site-packages/numpy/core/include -I/Application
s/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -I/Use
rs/dima/software/sage/.tox/local-macports-standard/local/include -I/Applications/Xcode.app/Contents/Devel
oper/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c'
    extra options: '-std=c++11'
    INFO: gcc: sage/rings/number_field/number_field_element_quadratic.cpp
    In file included from sage/rings/number_field/number_field_element_quadratic.cpp:837:
    In file included from sage/libs/flint/flint_wrap.h:42:
    ///Users/dima/software/sage/.tox/local-macports-standard/macports/include/flint/fmpz_mod_poly.h:306:12: error: use of undeclared identifier '_fmpz_poly_deflation'; did you mean 'fmpz_mod_poly_deflation'?
        return _fmpz_poly_deflation(input->coeffs, input->length);
               ^~~~~~~~~~~~~~~~~~~~
               fmpz_mod_poly_deflation
    ///Users/dima/software/sage/.tox/local-macports-standard/macports/include/flint/fmpz_mod_poly.h:303:7: note: 'fmpz_mod_poly_deflation' declared here
    ulong fmpz_mod_poly_deflation(const fmpz_mod_poly_t input,
          ^
    ///Users/dima/software/sage/.tox/local-macports-standard/macports/include/flint/fmpz_mod_poly.h:306:33: error: cannot initialize a parameter of type 'const fmpz_mod_poly_struct *' with an lvalue of type 'fmpz *const' (aka 'long *const')
        return _fmpz_poly_deflation(input->coeffs, input->length);
...

macports comes with flint 2.9.0, so it seems there is some kind of incompatibility with our 2.8.4.

comment:165 Changed 3 months ago by Matthias Köppe

The problem is that -I///Users/dima/software/sage/.tox/local-macports-standard/macports/include at the front of the command line. It's not supposed to be there

comment:166 in reply to:  165 Changed 3 months ago by Dima Pasechnik

Replying to mkoeppe:

The problem is that -I///Users/dima/software/sage/.tox/local-macports-standard/macports/include at the front of the command line. It's not supposed to be there

that's probably what tox is doing to create an isolated environment, no?

comment:167 Changed 3 months ago by Matthias Köppe

No, this is the install location of macports within the tox environment. I set these variables:

+    local-macports:           MP_PREFIX={envdir}/macports
+    local-macports:   PATH={env:MP_PREFIX}/bin:/usr/bin:/bin:/usr/sbin:/sbin
+    local-macports:   CPATH={env:MP_PREFIX}/include
+    local-macports:   LIBRARY_PATH={env:MP_PREFIX}/lib
+    local-macports:   SETENV=eval $(build/bin/sage-print-system-package-command {env:SYSTEM} setup-build-env)

but not CFLAGS.

comment:168 Changed 3 months ago by Dima Pasechnik

Then it's done by macports, not by tox.

comment:169 Changed 3 months ago by Matthias Köppe

Could you check what the macports-installed python3 gives when you ask it python3 -m sysconfig

comment:170 Changed 3 months ago by Dima Pasechnik

there are 2 installations of python3.10 in .tox/local-macports-standard/:

dima@oucl13243 sage %  .tox/local-macports-standard/macports/bin/python3.10
Python 3.10.6 (main, Aug 27 2022, 17:58:26) [Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 
dima@oucl13243 sage %  .tox/local-macports-standard/bin/python3
Python 3.10.6 (main, Aug 11 2022, 13:49:25) [Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

The latter is the only python3 (without .10), though.

Changed 3 months ago by Dima Pasechnik

Attachment: p1 added

sysconfig1

Changed 3 months ago by Dima Pasechnik

Attachment: p2 added

sysconf2

comment:171 Changed 3 months ago by Dima Pasechnik

attached are their outputs when run with -m sysconfig.

(there is also python2.7.18 - hopefully we have stopped all python2's used in our build by now...)

comment:172 Changed 2 months ago by Matthias Köppe

Milestone: sage-9.7sage-9.8
Note: See TracTickets for help on using tickets.