Opened 8 weeks ago

Last modified 9 days ago

#33800 needs_review enhancement

cibuildwheel workflow for sagemath-objects, sagemath-categories

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.7
Component: distribution Keywords:
Cc: dimpase, gh-kliem, gh-tobiasdiez, saraedum Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/cibuildwheel_workflow_for_sagemath_objects__sagemath_categories_etc_ (Commits, GitHub, GitLab) Commit: 926bfcdbdb253d47ffd2e420c64fa08fb302a98b
Dependencies: #29549, #33983, #34017 Stopgaps:

Status badges

Description (last modified by mkoeppe)

We rename the GH Actions workflow sdist.yml to dist.yml and add wheel-building for the distributions sagemath-objects and sagemath-categories for macOS (Intel) and Linux (x86_64, i686) using cibuildwheel.

The necessary libraries are built using the Sage distribution.

Example run: https://github.com/mkoeppe/sage/actions/runs/2522295930

Follow-ups:

  • After #33812 (Refactor distributions sagemath-{objects,categories}), make the built wheel for sagemath-objects available for the wheel building of sagemath-categories
  • Possibly use --prefix=$(pwd)/wheelhouse/local-ARCH --with-sage-venv so that SAGE_LOCAL gets copied in/out of the container

Change History (25)

comment:1 Changed 8 weeks ago by mkoeppe

  • Branch set to u/mkoeppe/cibuildwheel_workflow_for_sagemath_objects__sagemath_categories_etc_

comment:2 Changed 8 weeks ago by git

  • Commit set to 64b46a15032a8b23c7f0dec2bcf6f828f2c84923

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3133c8e.github/workflows/sdist.yml: Only build wheels for sagemath-objects, -categories
3d5850asage.env.cython_aliases: Handle ecl via default_required_modules
e77efb3pkgs/sagemath-objects/MANIFEST.in: Add missing sage.misc.package_dir
434d539pkgs/sagemath-objects/MANIFEST.in: Add sage.misc.sagedoc
64b46a1.github/workflows/dist.yml: Rename from sdist.yml

comment:3 Changed 8 weeks ago by git

  • Commit changed from 64b46a15032a8b23c7f0dec2bcf6f828f2c84923 to 025200be3f7ea29ffa82b7183c5924291fe863b7

Branch pushed to git repo; I updated commit sha1. New commits:

474104b.github/workflows/dist.yml: Install gmp before building the wheel
025200b.github/workflows/dist.yml: Install gmp, mpfr, mpc before building the wheel

comment:4 Changed 8 weeks ago by mkoeppe

The method of installing the required system packages (gmp, mpfr, mpc) with whatever is provided by the target distribution does not work because the manylinux distributions are deliberately ancient. Getting error: #error "GMPY2 requires MPC 1.0.3 or later." (https://github.com/mkoeppe/sage/runs/6300752059).

Also the macOS arm64 cross-build does not work: fatal error: 'gmp.h' file not found (https://github.com/mkoeppe/sage/runs/6300752117)

comment:5 Changed 8 weeks ago by mkoeppe

  • Dependencies set to #33803

comment:6 Changed 8 weeks ago by mkoeppe

  • Work issues set to Rebase on #33803

comment:7 Changed 8 weeks ago by mkoeppe

We could add a section [tool.cibuildwheel] to pkg/*/pyproject.toml.m4 and add conda installation + conda package installation there as before-all.

comment:8 Changed 3 weeks ago by git

  • Commit changed from 025200be3f7ea29ffa82b7183c5924291fe863b7 to 79549d975cb2ad5000a519ad3de0bba0f1c98d0a

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

4d54d21.github/workflows/sdist.yml: Build wheels too
431492d.github/workflows/sdist.yml: Set CIBW_PROJECT_REQUIRES_PYTHON
1e14375.github/workflows/sdist.yml: Only build wheels for sagemath-objects, -categories
152aca2.github/workflows/dist.yml: Rename from sdist.yml
f0bd1cf.github/workflows/dist.yml: Install gmp before building the wheel
6f91ddc.github/workflows/dist.yml: Install gmp, mpfr, mpc before building the wheel
79549d9.github/workflows/dist.yml: Build prereqs of wheels using Sage distro

comment:9 Changed 3 weeks ago by git

  • Commit changed from 79549d975cb2ad5000a519ad3de0bba0f1c98d0a to 42e991af51c8d1617dab27c1522d2c070303d4a9

Branch pushed to git repo; I updated commit sha1. New commits:

08acac4build/pkgs/sage_setup/dependencies: Add jinja2
42e991a.github/workflows/dist.yml: One package at a time, build out of pkgs/

comment:10 Changed 3 weeks ago by mkoeppe

  • Dependencies #33803 deleted
  • Work issues Rebase on #33803 deleted

comment:12 Changed 3 weeks ago by mkoeppe

They are using straightforward cibuildwheel

comment:13 Changed 3 weeks ago by dimpase

but they need to have gmp installed first.

comment:14 Changed 3 weeks ago by mkoeppe

Sure, they install the libraries from source with a custom script. In Sage, we can just use our spkg scripts for the same purpose.

comment:15 Changed 9 days ago by mkoeppe

  • Dependencies set to #29549

comment:16 Changed 9 days ago by mkoeppe

  • Dependencies changed from #29549 to #29549, #33983

comment:17 Changed 9 days ago by git

  • Commit changed from 42e991af51c8d1617dab27c1522d2c070303d4a9 to c7418788db37b2febd6f0b7635a8b1ebed9b544f

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

3fd1cb5.github/workflows/dist.yml: Use pkgconfig, not pkg-config on centos
f9407af.github/workflows/dist.yml: Set ACLOCAL_PATH
fcbf814.github/workflows/dist.yml: Use cibuildwheel==2.7.0
77affa8tox.ini: Add manylinux_2_28
dc352d9.github/workflows/dist.yml: Use --enable-build-as-root
ac0e301do not use prefix
7999353update PATH
07a018bUse CIBW_BUILD_FRONTEND=build
e653ff1Build wheels from sdists
c741878Unpack sdist for cibuildwheel

comment:18 Changed 9 days ago by git

  • Commit changed from c7418788db37b2febd6f0b7635a8b1ebed9b544f to ae12df2b96ac0fd42ef4eb033d7ae7d6d90d1031

Branch pushed to git repo; I updated commit sha1. New commits:

ae12df2.github/workflows/dist.yml: Disable musllinux for now

comment:19 Changed 9 days ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Description modified (diff)
  • Summary changed from cibuildwheel workflow for sagemath-objects, sagemath-categories etc. to cibuildwheel workflow for sagemath-objects, sagemath-categories

comment:20 Changed 9 days ago by git

  • Commit changed from ae12df2b96ac0fd42ef4eb033d7ae7d6d90d1031 to 65bc0f1d07c22716529af0117c3c938e84d22405

Branch pushed to git repo; I updated commit sha1. New commits:

65bc0f1.github/workflows/dist.yml: Add comment

comment:21 Changed 9 days ago by git

  • Commit changed from 65bc0f1d07c22716529af0117c3c938e84d22405 to 036cf704fa4514d6865b7707e7847c343e6963ab

Branch pushed to git repo; I updated commit sha1. New commits:

036cf70.github/workflows/dist.yml: Build Linux x86_64 and i686 in separate jobs

comment:22 Changed 9 days ago by mkoeppe

  • Description modified (diff)

comment:23 Changed 9 days ago by mkoeppe

  • Cc saraedum added
  • Status changed from new to needs_review

comment:24 Changed 9 days ago by mkoeppe

  • Dependencies changed from #29549, #33983 to #29549, #33983, #34017

comment:25 Changed 9 days ago by git

  • Commit changed from 036cf704fa4514d6865b7707e7847c343e6963ab to 926bfcdbdb253d47ffd2e420c64fa08fb302a98b

Branch pushed to git repo; I updated commit sha1. New commits:

58d62f4build/bin/write-dockerfile.sh: ADD src/VERSION.txt
926bfcdMerge #34017
Note: See TracTickets for help on using tickets.