Opened 13 months ago

Closed 7 months ago

#32113 closed defect (fixed)

GH Actions: conda-forge tests broken

Reported by: mkoeppe Owned by:
Priority: critical Milestone: sage-9.5
Component: porting Keywords:
Cc: isuruf, vdelecroix, dimpase, gh-kliem Merged in:
Authors: Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: f3116a1 (Commits, GitHub, GitLab) Commit: f3116a125a3283b84240c783d3d19900ca740614
Dependencies: #30766, #32894 Stopgaps:

Status badges

Description

docker-conda-forge-minimal, docker-conda-forge-standard (https://github.com/sagemath/sage/runs/2967006198?check_suite_focus=true)

  [python3-3.9.5]   LD_LIBRARY_PATH=/sage/local/var/tmp/sage/build/python3-3.9.5/src CC='/opt/conda/bin/x86_64-conda-linux-gnu-cc -pthread' LDSHARED='/opt/conda/bin/x86_64-conda-linux-gnu-cc -pthread -shared -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -L. -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib  ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall' 	_TCLTK_INCLUDES='' _TCLTK_LIBS='' 	./python -E ./setup.py  build
  [python3-3.9.5]   Traceback (most recent call last):
  [python3-3.9.5]     File "/sage/local/var/tmp/sage/build/python3-3.9.5/src/./setup.py", line 33, in <module>
  [python3-3.9.5]       from distutils import log
  [python3-3.9.5]     File "/opt/conda/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 80, in create_module
  [python3-3.9.5]       return importlib.import_module('._distutils', 'setuptools')
  [python3-3.9.5]     File "/opt/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
  [python3-3.9.5]       return _bootstrap._gcd_import(name[level:], package, level)
  [python3-3.9.5]     File "/opt/conda/lib/python3.9/site-packages/setuptools/__init__.py", line 16, in <module>
  [python3-3.9.5]       from setuptools.extern.six import PY3, string_types
  [python3-3.9.5]     File "/opt/conda/lib/python3.9/site-packages/setuptools/extern/__init__.py", line 50, in load_module
  [python3-3.9.5]       raise ImportError(
  [python3-3.9.5]   ImportError: The 'six' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
  [python3-3.9.5]   make[3]: *** [Makefile:641: sharedmods] Error 1

conda-forge-ubuntu (https://github.com/sagemath/sage/runs/2967003736?check_suite_focus=true):

local-conda-forge-ubuntu-minimal run-test: commands[5] | bash -c 'if [ ! -x /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-minimal/conda/bin/conda ]; then curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh -C - -o /home/runner/work/sage/sage/.tox/Caches/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh && bash /home/runner/work/sage/sage/.tox/Caches/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh -b -f -p /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-minimal/conda; fi'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   160  100   160    0     0    935      0 --:--:-- --:--:-- --:--:--   935

100   634  100   634    0     0   1962      0 --:--:-- --:--:-- --:--:--  1962

 22 69.9M   22 15.6M    0     0  15.6M      0  0:00:04 --:--:--  0:00:04 15.6M
 61 69.9M   61 42.8M    0     0  20.8M      0  0:00:03  0:00:02  0:00:01 25.7M
100 69.9M  100 69.9M    0     0  25.0M      0  0:00:02  0:00:02 --:--:-- 30.2M
PREFIX=/home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-minimal/conda
Unpacking payload ...
ERROR   Expected environment not found at prefix: /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-minimal/conda
ERROR   Aborting.
/home/runner/work/sage/sage/.tox/Caches/Miniforge3-Linux-x86_64.sh: line 365: /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-minimal/conda/preconda.tar.bz2: No such file or directory

Likewise on local-conda-forge-macos, https://github.com/sagemath/sage/runs/2967003146?check_suite_focus=true

Change History (34)

comment:1 Changed 13 months ago by isuruf

Can you try removing SETUPTOOLS_USE_DISTUTILS=local for now?

comment:2 Changed 13 months ago by mkoeppe

Sure, we can conditionalize this on CONDA_PREFIX like in build/pkgs/pillow/spkg-install.in

comment:3 Changed 12 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5

comment:4 Changed 11 months ago by mkoeppe

  • Dependencies set to #32558, #32557

comment:5 Changed 10 months ago by isuruf

  • Authors set to Isuru Fernando
  • Branch set to u/isuruf/conda
  • Commit set to 330db42a75b978c7737b72266345f9fc56c57974
  • Status changed from new to needs_review

Added a branch. Needs to be tested after merging in the dependencies.

comment:6 Changed 10 months ago by git

  • Commit changed from 330db42a75b978c7737b72266345f9fc56c57974 to ccdd330e2646fbb6de5d256f73dd00d303f4e702

Branch pushed to git repo; I updated commit sha1. New commits:

ec983bdfix for conda python
e193a82Need rpath too when building python and override CONFIGURE_LDFLAGS only
ccdd330allow building with different compilers on debian/ubuntu

comment:7 Changed 10 months ago by git

  • Commit changed from ccdd330e2646fbb6de5d256f73dd00d303f4e702 to b85cfba08b0156f670dfc8dffb9e122dee30a6b0

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

a95daa0Need rpath too when building python and override CONFIGURE_LDFLAGS only
b85cfbaallow building with different compilers on debian/ubuntu

comment:8 Changed 10 months ago by git

  • Commit changed from b85cfba08b0156f670dfc8dffb9e122dee30a6b0 to 2d4421c5b4887f6665b6fa225975a1b2c5044bb6

Branch pushed to git repo; I updated commit sha1. New commits:

2d4421cupdate compiling on debian patch

comment:9 Changed 10 months ago by mkoeppe

  • Cc dimpase added

comment:10 Changed 10 months ago by git

  • Commit changed from 2d4421c5b4887f6665b6fa225975a1b2c5044bb6 to 9a38a29c123aa808fdafeb20d14b4fe39f542443

Branch pushed to git repo; I updated commit sha1. New commits:

9a38a29run 2to3 manually for nose

comment:11 Changed 10 months ago by isuruf

  • Dependencies changed from #32558, #32557 to #32558, #32557, #32424

New commits:

9a38a29run 2to3 manually for nose

comment:12 Changed 10 months ago by dimpase

how about dropping nose? All the packages that used it have swithed to pytests.

Last edited 10 months ago by dimpase (previous) (diff)

comment:13 Changed 10 months ago by isuruf

pytest is optional at the moment. Let's start a vote to drop nose and promote pytest to standard

comment:14 Changed 10 months ago by mkoeppe

  • Dependencies changed from #32558, #32557, #32424 to #32558, #32557, #32424, #32595

The latest run, with 9.3.beta5, of local-conda-forge-macos-standard (https://github.com/sagemath/sage/runs/3863933907?check_suite_focus=true) looks better but uses XCode's python3, as #31539 is still unresolved.

comment:15 Changed 10 months ago by mkoeppe

  • Branch changed from u/isuruf/conda to u/mkoeppe/conda

comment:16 Changed 10 months ago by mkoeppe

  • Commit changed from 9a38a29c123aa808fdafeb20d14b4fe39f542443 to ee6256df89af258d039ee9175b1b7116a52c8128

Not sure if my rebase of the rpath business will work as intended


New commits:

a579725Need rpath too when building python and override CONFIGURE_LDFLAGS only
e458a39allow building with different compilers on debian/ubuntu
ee6256dupdate compiling on debian patch

comment:17 Changed 10 months ago by mkoeppe

https://github.com/mkoeppe/sage/runs/3986317613?check_suite_focus=true

local-conda-forge-macos-minimal create: /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal
local-conda-forge-macos-minimal run-test-pre: PYTHONHASHSEED='3748675463'
local-conda-forge-macos-minimal run-test: commands[0] | bash -c 'if [ ! -d /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/Library/Caches ]; then mkdir -p /Users/runner/work/sage/sage/.tox/Caches && mkdir -p /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/Library && ln -sf /Users/runner/work/sage/sage/.tox/Caches /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/Library/; fi'
local-conda-forge-macos-minimal run-test: commands[1] | bash -c 'mkdir -p /Users/runner/work/sage/sage/.tox/Caches/conda_pkgs /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda'
local-conda-forge-macos-minimal run-test: commands[2] | bash -c 'echo >  /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/.condarc "pkgs_dirs:"'
local-conda-forge-macos-minimal run-test: commands[3] | bash -c 'echo >> /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/.condarc "  - /Users/runner/work/sage/sage/.tox/Caches/conda_pkgs"'
local-conda-forge-macos-minimal run-test: commands[4] | bash -c 'cat condarc.yml >> /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/.condarc'
local-conda-forge-macos-minimal run-test: commands[5] | bash -c 'if [ ! -x /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/bin/conda ]; then mkdir /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/conda-meta && curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh -C - -o /Users/runner/work/sage/sage/.tox/Caches/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh && bash /Users/runner/work/sage/sage/.tox/Caches/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh -b -f -p /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda; fi'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   161  100   161    0     0    800      0 --:--:-- --:--:-- --:--:--   805

100   635  100   635    0     0   1819      0 --:--:-- --:--:-- --:--:--  1819

 13 41.5M   13 5778k    0     0  8161k      0  0:00:05 --:--:--  0:00:05 8161k
100 41.5M  100 41.5M    0     0  31.2M      0  0:00:01  0:00:01 --:--:-- 57.9M
PREFIX=/Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda
Unpacking payload ...
ERROR   Configuration file specified but does not exist at 'condarc.yml'
ERROR   Aborting.
/Users/runner/work/sage/sage/.tox/Caches/Miniforge3-MacOSX-x86_64.sh: line 364: /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/preconda.tar.bz2: No such file or directory
ERROR: InvocationError for command /bin/bash -c 'if [ ! -x /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/bin/conda ]; then mkdir /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda/conda-meta && curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh -C - -o /Users/runner/work/sage/sage/.tox/Caches/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh && bash /Users/runner/work/sage/sage/.tox/Caches/Miniforge3-$(uname | sed s/^Darwin/MacOSX/;)-x86_64.sh -b -f -p /Users/runner/work/sage/sage/.tox/local-conda-forge-macos-minimal/conda; fi' (exited with code 1)

comment:18 Changed 8 months ago by git

  • Commit changed from ee6256df89af258d039ee9175b1b7116a52c8128 to e6b912b18d36ed3d3256021543cf2f41af5c4ad2

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7030a0fNeed rpath too when building python and override CONFIGURE_LDFLAGS only
5af7789allow building with different compilers on debian/ubuntu
e5d336fupdate compiling on debian patch
e6b912btox.ini: Do not set environment variable CONDARC

comment:19 Changed 8 months ago by mkoeppe

  • Authors changed from Isuru Fernando to Isuru Fernando, Matthias Koeppe
  • Dependencies #32558, #32557, #32424, #32595 deleted

comment:20 Changed 8 months ago by mkoeppe

  • Priority changed from major to critical

comment:21 Changed 8 months ago by mkoeppe

The last commit fixes ERROR Configuration file specified but does not exist at 'condarc.yml'

comment:22 Changed 8 months ago by mkoeppe

With tox -e docker-conda-forge-minimal I get:

  [openblas-0.3.18]   /opt/conda/bin/x86_64-conda-linux-gnu-cc -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DUSE_TLS -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.18\" -msse3 -mssse3 -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I. -O2 -DSMALL_MATRIX_OPT -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DUSE_TLS -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.18\" -msse3 -mssse3 -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I.. -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -w -o linktest linktest.c ../libopenblas_atomp-r0.3.18.so -L/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0 -L/opt/conda/bin/../lib/gcc -L/sage/local/lib/../lib -L/sage/local/lib/../lib -L/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/lib/../lib -L/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../lib -L/opt/conda/bin/../x86_64-conda-linux-gnu/sysroot/lib/../lib -L/opt/conda/bin/../x86_64-conda-linux-gnu/sysroot/usr/lib/../lib -L/sage/local/lib -L/sage/local/lib -L/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/lib -L/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../.. -L/opt/conda/bin/../x86_64-conda-linux-gnu/sysroot/lib -L/opt/conda/bin/../x86_64-conda-linux-gnu/sysroot/usr/lib -Wl,-rpath,/opt/conda/lib  -lgfortran -lm -lquadmath -lm -lc   && echo OK.
  [openblas-0.3.18]   /opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: ../libopenblas_atomp-r0.3.18.so: undefined reference to `GOMP_parallel'
  [openblas-0.3.18]   /opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: ../libopenblas_atomp-r0.3.18.so: undefined reference to `omp_get_thread_num_'
  [openblas-0.3.18]   /opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: ../libopenblas_atomp-r0.3.18.so: undefined reference to `omp_get_num_threads_'
  [openblas-0.3.18]   /opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: ../libopenblas_atomp-r0.3.18.so: undefined reference to `GOMP_task'
  [openblas-0.3.18]   /opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: ../libopenblas_atomp-r0.3.18.so: undefined reference to `omp_get_thread_num'
  [openblas-0.3.18]   collect2: error: ld returned 1 exit status
  [openblas-0.3.18]   make[4]: *** [Makefile:186: ../libopenblas_atomp-r0.3.18.so] Error 1
  [openblas-0.3.18]   make[4]: Target 'so' not remade because of errors.
  [openblas-0.3.18]   make[3]: *** [Makefile:122: shared] Error 2
  [openblas-0.3.18]   ********************************************************************************
  [openblas-0.3.18]   Error building openblas-0.3.18
  [openblas-0.3.18]   ********************************************************************************

comment:23 Changed 8 months ago by mkoeppe

With tox -e docker-conda-forge-standard I get a build error with primecount. That's now #33054.

comment:24 Changed 8 months ago by mkoeppe

  • Dependencies set to #30766

comment:25 Changed 8 months ago by git

  • Commit changed from e6b912b18d36ed3d3256021543cf2f41af5c4ad2 to c0b5b67966eb27ccd18f76c5bc932c2ce0468d43

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

63078acbuild/pkgs/terminado: Update to 0.12.1
22de5dcbuild/pkgs/notebook: Update to 6.4.6
a0fe18bbuild/pkgs/nbclient: Update to 0.5.4
ef3fc7fbuild/pkgs/jupyter_client: Update to 7.1.0
a593bc9build/pkgs/jupyter_core: Update to 4.9.1
66c16b1build/pkgs/importlib_metadata: Update to 4.8.2
02aa94cMerge #32930
249357dsage.cpython.atexit: fix for python 3.10 (#30766)
df9f1d4Pass float(0.5) to time.sleep
c0b5b67Merge #30766

comment:26 Changed 8 months ago by mkoeppe

  • Dependencies changed from #30766 to #30766, #32894

comment:27 Changed 8 months ago by mkoeppe

I've put the commits for fixing the python3 spkg build on #33055.

Here on this ticket, I'll focus on getting the conda-forge-standard builds to work (using conda's python) - the python3 patches are not needed here.

comment:28 Changed 8 months ago by git

  • Commit changed from c0b5b67966eb27ccd18f76c5bc932c2ce0468d43 to f3116a125a3283b84240c783d3d19900ca740614

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

de7de85cython is not in toolchain!
43d481bbump to 0.1.0
f66821bcysignals are needed
ae37b4edeprecate sage.interfaces.primecount, not just remove
d3ae5f4primecount is on conda, too
3a8c7feprimesieve is on conda, too
74b3845allow float inputs for prime_pi
591be34Merge #32894
049a5f6tox.ini: Do not set environment variable CONDARC
f3116a1tox.ini (conda): Force use of conda's python3

comment:29 Changed 8 months ago by mkoeppe

  • Authors changed from Isuru Fernando, Matthias Koeppe to Matthias Koeppe

comment:30 Changed 8 months ago by mkoeppe

  • Reviewers set to https://github.com/mkoeppe/sage/actions/runs/1605145773

comment:31 Changed 8 months ago by mkoeppe

  • Cc gh-kliem added

comment:32 Changed 8 months ago by dimpase

  • Status changed from needs_review to positive_review

lgtm

comment:33 Changed 8 months ago by mkoeppe

  • Reviewers changed from https://github.com/mkoeppe/sage/actions/runs/1605145773 to Dima Pasechnik

Thanks!

comment:34 Changed 7 months ago by vbraun

  • Branch changed from u/mkoeppe/conda to f3116a125a3283b84240c783d3d19900ca740614
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.