Opened 23 months ago

Last modified 4 months ago

#29060 new task

Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.5
Component: build Keywords: docker, ContinuousIntegration, sd111
Cc: vbraun, fbissey, isuruf, dimpase, embray, saraedum, arojas, slelievre, gh-sheerluck Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

Testing of source and binary distributions relies too much on manual testing by people.

We propose to set up Dockerfiles and CI scripts that document and test the expected capabilities of source and binary Sage distributions; and of downstream distribution packaging.

Testing of source trees and source distributions: (in progress)

With the important changes brought by #27330 (Meta-ticket: spkg-configure: Try to use as many system packages as possible), sage-the-distribution has become less monolithic; it now interacts in a more complex way with distribution packaging. We propose to add infrastructure for testing correct installation of sage source distributions on a variety of platforms.

  1. We collect information about distribution packages systematically and store it in on a per-SPKG basis in build/pkgs/SPKG/distros/. (Right now this information is scattered - in downstream sage distribution packagers' build scripts, trac tickets, replies to bug reports in google groups, the sage installation manual, personal knowledge...) This is part of #29053, for debian/fedora/arch/conda (with follow-up tickets #29105, #29106 for other platforms).
  1. We generate well-defined test environments in the form of Dockerfiles. This makes it possible to test, on one's development computer, the correct installation of sage-the-distribution on a variety of platforms and configurations. #29053 provides build/bin/write-dockerfile.sh that generates the Dockerfile using the information in 1. The minimal configuration has just the packages that are needed for a build to succeed. The standard configuration installs all distribution packages that sage knows how to use.
  1. Running tests on many test environments is automated using tox in #29053. The top-level tox.ini file defines the test environments. Using tox's factor conditions (https://tox.readthedocs.io/en/latest/config.html#complex-factor-conditions), there is a succinct description of many test environments. For example, we can run tox -e docker-debian-stretch-minimal,docker-arch-latest-standard.
  1. #29087 automates running the tox tests for a selection of 34 platforms/configurations using a GitHub Actions workflow on every git push to a GitHub repository. An example run: https://github.com/mkoeppe/sage/actions/runs/32812271

Tickets:

  • #29053: Add debian/fedora/arch/conda package information to build/pkgs, generate Dockerfiles and installation help; add tox.ini --- this helps test that all spkg-configure.m4 scripts work as expected
  • #29066: Add Dockerfile that sets up sage prerequisites on Arch Linux
  • #29104: Add to tox.ini some homebrew toxenvs (with a fresh install not in /usr/local)
  • #29143: Add to tox.ini 32-bit linux environments and other architectures supported by docker
  • #29129: Add more system packages to build/pkgs/SPKG/distros/
  • #29273: add system packages info for R and others
  • #29106: Add cygwin package information
  • #29295: Add GitHub CI workflow for cygwin
  • #29087: Add GitHub Actions workflow for testing spkg-configure / build on various Linux distributions
  • #29367: tox.ini: Add tests for python2
  • #29415: tox.ini: Add environment local-conda
  • #29417: tox.ini, GitHub CI workflow - minor fixes
  • #29401: Add documentation of tox and GitHub actions workflow to developer's manual
  • #29366: archlinux: Add missing system packages (arch.txt), add arch section to installation manual, fix detection of system packages
  • #29341: GitHub CI workflow: Add testing of optional packages
  • #29403: ci-cygwin.yml: Multi-stage build
  • #29530: Make Docker images generated by GitHub CI workflows available; minor CI fixes
  • #29669: For Sage 9.2: Remove Python 2 support from the build system and CI scripts
  • #29757: .github/workflows/tox-gcc_spkg.yml: Remove python2
  • #29901: .github/workflows/: Test all optional packages, add workflow for testing all experimental packages
  • #29815: .github/workflows/tox-gcc_spkg.yml: Disable tests on some platforms that fail --without-system-gcc
  • #29105: Add gentoo package information, extend dockerfile generator to gentoo
  • #29851: tox.ini: debian-bullseye, -sid have Python3.8 instead of 3.7
  • #30064: Fix tox-docker builds broken by #29884
  • #29091: tox.ini: Add raspbian-buster
  • #29929: tox.ini: Various CI fixes
  • #30195: tox.ini: Add centos-6, docker-nobootstrap, manylinux-*
  • #30044: tox.ini: Test on voidlinux
  • #30144: tox.ini / tox.yml: Move code for tagging/pushing docker images into tox.ini
  • #29130: Add nix package information, extend dockerfile generator to nix
  • #30216 GH Actions: Test Windows with WSL
  • #30923: tox.ini: Add environments local-sudo-ubuntu-standard, etc.
  • #30619: update opensuse.txt across build/pkgs/
  • #30154: cygwin: CI fixes
  • #28745: Add SAGE_ROOT/environment.yml for "conda env create -f".
  • #31211: ci-cygwin*.yml: Fix packaging up of SAGE_LOCAL/var/tmp/sage/build/*
  • #30933: GH Actions: Repair upload of docker images
  • #31088: GH Actions: WSL workflow: No space left on device
  • #31413: GH Actions: Add workflow with configure --enable-editable
  • #30845: tox.ini, GH Actions: Add test for conda without SPKG
  • #30726: GH Actions: Test optional/experimental packages with Xcode 12
  • #29527: tox.ini: Add docker-opensuse
  • #30505: tox.ini: Add wsl as a technology, to complement local and docker
  • #30924: tox.ini: Add cygwin as a technology, to complement local and docker
  • #29536: Make docker images from GitHub CI workflow and regular Sage Docker images interoperable
  • #30213: tox.ini: Add scientificlinux
  • #30217: tox.ini: Add tests with Ubuntu + deadsnakes
  • test gentoo "prefix" builds on top of some Linux distribution
  • #29416: tox.ini: Add local-copy
  • #30506: GH Actions: Use cancel-all-previous-workflow-runs
  • #21469: Enable VPATH builds (several independent build trees connected to one source tree)
  • #29283: tox.ini: Add test environments using LXC
  • #29159: Add to tox.ini a toxenv docker-linuxbrew
  • #29535: Customize CI workflows on GitHub Actions using specially structured tag names

Related meta-tickets:

  • #29356: Meta-ticket: more system pkgs info for various linuxes
  • #29146: Meta-ticket: Refactor/improve system package related scripts, tox.ini, build/bin/write_dockerfile.sh

Ideas without tickets so far:

  • Testing that it is possible to build without error when upgrading with git from some list of previous releases.
  • Test that packages can be cleanly uninstalled

Testing of the sage binary distributions: moved to #31133


Testing of downstream packaging of sage: (planning stage)


Symptoms:

  • #29055: Installing any optional package on the binary distribution triggers full recompile of sagelib
  • #29051: Paths configured in installed numpy site.cfg [DEFAULT/ALL] do not affect scipy

Change History (45)

comment:1 follow-up: Changed 23 months ago by saraedum

  • Cc slelievre added
  • Keywords docker ContinuousIntegration added

I like the general idea of doing more CI a lot. Probably we should split this ticket up though or turn it into a meta ticket that tracks similar CI improvements such as #28457, #24854, #25262.

For this to be actually noticed by people we probably would need some integration with Trac. If I understood embray correctly, adding something like that, similar to the patchbot status, would not be difficult. (Since #28457 has been ready for a while, it might be a good first candidate for such an integration.)

I would propose to use GitLab CI for probably all the things you describe. We already have some CI setup there that has been mostly stable recently (though most people don't know about it https://gitlab.com/sagemath/dev/trac/pipelines?page=1&scope=all since it's not visible in trac yet.) I had made some experiments with macOS & Windows there that were not completely disappointing. For macOS you cannot use docker but still GitLab CI works, see e.g. #25980. Windows can be run inside docker (on Windows) but two years ago, the performance was not sufficient, see #25805. Without docker it worked fine though.

For our Linux CI needs, we can mostly use the free GitLab runners but last time we tried we could also easily get plenty of free credits on Google Cloud. For macOS & Windows, I had at some point built a PoC runner that slelievre volunteered to host; unfortunately, I never finished that project.

comment:2 in reply to: ↑ 1 Changed 23 months ago by mkoeppe

  • Summary changed from Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages to Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages

Replying to saraedum:

I like the general idea of doing more CI a lot. Probably we should split this ticket up though or turn it into a meta ticket that tracks similar CI improvements such as #28457, #24854, #25262.

For this to be actually noticed by people we probably would need some integration with Trac.

Yes, that would be nice. But it would already help if this were run at the time that betas and releases are prepared. I don't think it needs to be run on every ticket.

comment:3 Changed 23 months ago by mkoeppe

  • Description modified (diff)

comment:4 Changed 23 months ago by mkoeppe

  • Description modified (diff)

comment:5 Changed 23 months ago by mkoeppe

  • Description modified (diff)

comment:6 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:7 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:8 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:9 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:10 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:11 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:12 follow-up: Changed 22 months ago by dimpase

This needs a writeup to explain how all these cogs: tox, docker, github actions, etc. fit together and move.

comment:13 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:14 in reply to: ↑ 12 Changed 22 months ago by mkoeppe

Replying to dimpase:

This needs a writeup to explain how all these cogs: tox, docker, github actions, etc. fit together and move.

Good idea, please take a look at the revised description and let me know what needs more detail.

comment:15 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:16 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:17 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:18 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:19 Changed 22 months ago by dimpase

On some branches there are also lists of <distro> packages in build/pkgs/<distro>.txt which appear to be used for some kind of bootstrapping. What are these?

comment:20 Changed 22 months ago by mkoeppe

build/pkgs/DISTRO.txt contains the minimal requirements for building sage from a source distribution and build/pkgs/DISTRO-bootstrap.txt are the additional requirements so that ./bootstrap works. In #29124 I propose to create pseudo-spkgs to capture these, instead, to make this a bit more uniform.

comment:21 Changed 22 months ago by mkoeppe

Comments on the top of build/pkgs/debian.txt explain this.

comment:22 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:23 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:24 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:25 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:26 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:27 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:28 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:29 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:30 Changed 19 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

comment:31 Changed 18 months ago by mkoeppe

  • Description modified (diff)

comment:32 Changed 18 months ago by mkoeppe

  • Cc gh-sheerluck added
  • Description modified (diff)

comment:33 Changed 17 months ago by mkoeppe

  • Description modified (diff)

comment:34 Changed 15 months ago by mkoeppe

  • Description modified (diff)

comment:35 Changed 15 months ago by mkoeppe

  • Description modified (diff)

comment:36 Changed 15 months ago by mkoeppe

  • Description modified (diff)

comment:37 Changed 15 months ago by mkoeppe

  • Description modified (diff)

comment:38 Changed 13 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:39 Changed 13 months ago by mkoeppe

  • Description modified (diff)

comment:40 Changed 13 months ago by mkoeppe

  • Description modified (diff)

comment:41 Changed 12 months ago by mkoeppe

  • Keywords sd111 added

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

comment:42 Changed 11 months ago by mkoeppe

  • Description modified (diff)

comment:43 Changed 9 months ago by mkoeppe

  • Description modified (diff)

comment:44 Changed 7 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Moving to 9.4, as 9.3 has been released.

comment:45 Changed 4 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5
Note: See TracTickets for help on using tickets.