Opened 3 years ago

Last modified 4 months ago

#29536 new enhancement

Make docker images from tox/GitHub CI workflow, regular Sage Docker images, gitpod, CoCalc interoperable

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-9.8
Component: docker Keywords: ci, docker, ContinuousIntegration, sd109
Cc: gh-kliem, Dima Pasechnik, Erik Bray, Julian Rüth, Samuel Lelièvre, Frédéric Chapoton, Tobias Diez, William Stein, Marc Culler, Nathan Dunfield Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

  1. Adjust directory names to something that works for both cases ...
    • docker/Dockerfile currently has:
      • SAGE_ROOT=/home/sage/sage
      • SAGE_LOCAL=$SAGE_ROOT/local
    • Docker images built with tox/GH Actions (including Gitpod) and also computop/sage have:
      • SAGE_ROOT=/sage
      • SAGE_LOCAL=/sage/local
    • After this ticket:
      • SAGE_ROOT=/sage
      • SAGE_LOCAL=/opt/sage
    • CoCalc Docker image has:
      • SAGE_ROOT=/usr/local/sage
      • SAGE_LOCAL=/usr/local/sage/local
  1. #33851: Define a script package representing "useful stuff to have on a Docker image"
  1. Replace all apt-get in docker/Dockerfile by using a Docker image built using build/bin/write-dockerfile as the base image.
  1. By default, docker/Dockerfile would use configure --without-system-... a lot so that CI is done against "the reference distribution"
  1. Move build/bin/write-dockerfile.sh into docker/
  1. Add a mechanism (environment variable) to tox -e docker... to copy SAGE_LOCAL from a given Docker image.

Change History (35)

comment:1 Changed 3 years ago by Matthias Köppe

Description: modified (diff)

comment:2 Changed 3 years ago by Dima Pasechnik

by the way, we don't seem to have in docs an explanation regarding file shared between docker images and regular file system of the host system.

comment:3 Changed 3 years ago by Matthias Köppe

Keywords: sd109 added

comment:4 Changed 3 years ago by Matthias Köppe

Description: modified (diff)

comment:5 Changed 3 years ago by Erik Bray

I'm confused--should this say "GitHub" in the title or "GitLab"? I don't know of any GitHub CI docker images.

comment:6 in reply to:  5 Changed 3 years ago by Dima Pasechnik

Replying to embray:

I'm confused--should this say "GitHub" in the title or "GitLab"? I don't know of any GitHub CI docker images.

please see https://doc.sagemath.org/html/en/developer/index.html#testing-on-multiple-platforms

we now have a builder of docker images for multiple platforms, configurable, and by default hosted on github.

comment:7 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:8 Changed 2 years ago by Matthias Köppe

Milestone: sage-wishlistsage-9.2

comment:9 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:10 Changed 2 years ago by Matthias Köppe

Cc: Frédéric Chapoton added

comment:11 Changed 2 years ago by Matthias Köppe

Milestone: sage-9.2sage-9.3

comment:12 Changed 2 years ago by Matthias Köppe

Dependencies: #30960

comment:13 Changed 2 years ago by Matthias Köppe

Cc: Tobias Diez added

comment:14 Changed 20 months ago by Matthias Köppe

Milestone: sage-9.3sage-9.4

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

comment:15 Changed 20 months ago by Matthias Köppe

Description: modified (diff)
Summary: Make docker images from GitHub CI workflow and regular Sage Docker images interoperableMake docker images from tox/GitHub CI workflow and regular Sage Docker images interoperable

comment:16 Changed 17 months ago by Matthias Köppe

Milestone: sage-9.4sage-9.5

comment:17 Changed 14 months ago by Dima Pasechnik

Given that GitLab Docker build is broken for ages, can we arrange for GitHub-built docker images to be pushed to the main Docker repo, or at least create some sort of advertised Docker repo on GitHub?

comment:18 Changed 13 months ago by Matthias Köppe

Cc: William Stein added

comment:19 Changed 11 months ago by Matthias Köppe

Milestone: sage-9.5sage-9.6

comment:20 Changed 11 months ago by Matthias Köppe

Description: modified (diff)
Summary: Make docker images from tox/GitHub CI workflow and regular Sage Docker images interoperableMake docker images from tox/GitHub CI workflow, regular Sage Docker images, and gitpod interoperable

comment:21 Changed 8 months ago by Matthias Köppe

Milestone: sage-9.6sage-9.7

comment:22 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

Bikeshedding time! Let's pick SAGE_ROOT, SAGE_LOCAL. My proposal is in the ticket description.

(The only important constraint for me is: Don't nest SAGE_LOCAL within SAGE_ROOT - for flexibility such as bind-mounting a new version of the source tree over SAGE_ROOT without hiding SAGE_LOCAL.)

comment:23 Changed 7 months ago by Matthias Köppe

Dependencies: #30960#33740
Description: modified (diff)

comment:24 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:25 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:26 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:27 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:28 Changed 7 months ago by Matthias Köppe

Branch: u/mkoeppe/make_docker_images_from_tox_github_ci_workflow__regular_sage_docker_images__and_gitpod_interoperable

comment:29 Changed 7 months ago by git

Commit: 58e112ab36ecec775199528c66e4b6e5535ad953

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

10338a7Change ci as well
0cf7334Add sage-setup as install-requires dep
6d2b149Use relative paths to not install src package
1dba3baMaybe thats the correct shebang?
c1b2fbfRevert install_requires change
ad56ba3Go back to two pip installs for now
b42f5f5Readd how to use specific python version
87f360cReadd name arg
01eb374Merge #33740
58e112abootstrap-conda: Use script package _develop instead of hardcoded list of dev tools; remove unused RECOMMENDED

comment:30 Changed 7 months ago by Matthias Köppe

Branch: u/mkoeppe/make_docker_images_from_tox_github_ci_workflow__regular_sage_docker_images__and_gitpod_interoperable
Commit: 58e112ab36ecec775199528c66e4b6e5535ad953
Dependencies: #33740

comment:31 Changed 6 months ago by Matthias Köppe

Summary: Make docker images from tox/GitHub CI workflow, regular Sage Docker images, and gitpod interoperableMake docker images from tox/GitHub CI workflow, regular Sage Docker images, gitpod, CoCalc interoperable

comment:32 Changed 6 months ago by Matthias Köppe

Description: modified (diff)

comment:33 Changed 6 months ago by Matthias Köppe

Description: modified (diff)

comment:34 Changed 6 months ago by Matthias Köppe

Cc: Marc Culler Nathan Dunfield added

comment:35 Changed 4 months ago by Matthias Köppe

Milestone: sage-9.7sage-9.8
Note: See TracTickets for help on using tickets.