#30044 closed enhancement (fixed)

tox.ini: Test on voidlinux

Reported by: mkoeppe Owned by:
Priority: minor Milestone: sage-9.2
Component: porting Keywords:
Cc: dimpase, mjo, gh-kliem, slelievre Merged in:
Authors: Nicolò Piazzalunga, Matthias Koeppe Reviewers: Jonathan Kliem
Report Upstream: N/A Work issues:
Branch: d43730c (Commits, GitHub, GitLab) Commit: d43730cdbe9bb09b1045ba16daad841f83419642
Dependencies: #30064 Stopgaps:

Status badges

Description (last modified by mkoeppe)

Prompted by installation trouble reported at https://groups.google.com/d/msg/sage-devel/kiEh8JgbWeQ/Oqe77sQTCAAJ

we add infrastructure to test on voidlinux installations via Docker by repeating the steps done for Slackware in #29354

To test:

tox -e docker-voidlinux-standard

References:

Change History (17)

comment:1 Changed 17 months ago by mkoeppe

  • Description modified (diff)

comment:2 Changed 17 months ago by mkoeppe

  • Branch set to u/mkoeppe/tox_ini__test_on_voidlinux

comment:3 Changed 17 months ago by mkoeppe

  • Commit set to 0faf817d3cff061ab08b46916f4ec654ede2ef80
  • Dependencies set to #30064

New commits:

a31b522build/bin/{sage-guess-package-system,sage-print-system-package-command,build/bin/write-dockerfile.sh}: Handle voidlinux (xbps).
49bdc21tox.ini: Add voidlinux
e01c37dbuild/pkgs/void-bootstrap.txt: New
0faf817build/bin/write-dockerfile.sh [voidlinux]: Fix typo

comment:4 Changed 17 months ago by git

  • Commit changed from 0faf817d3cff061ab08b46916f4ec654ede2ef80 to bc5c2dbd8e979b6780e13e1a246aefdc6fed8dfe

Branch pushed to git repo; I updated commit sha1. New commits:

539c182build/make/install: Do not depend on src/bin/sage-version.sh
761092cMerge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884
f2efa6asrc/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist
b7bf43bbuild/bin/write-dockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884
bc5c2dbMerge branch 't/30064/fix_tox_docker_builds_broken_by__29884' into t/30044/tox_ini__test_on_voidlinux

comment:5 Changed 17 months ago by git

  • Commit changed from bc5c2dbd8e979b6780e13e1a246aefdc6fed8dfe to 8043c9c4adc2d2848d0ef585f88a0d537fc13793

Branch pushed to git repo; I updated commit sha1. New commits:

8043c9cbuild/pkgs/void-bootstrap.txt: Add gettext-devel

comment:6 Changed 17 months ago by git

  • Commit changed from 8043c9c4adc2d2848d0ef585f88a0d537fc13793 to 9f4a232aee35913b1134063171a64dc9914c0f95

Branch pushed to git repo; I updated commit sha1. New commits:

f2fe15bMerge tag '9.2.beta5' into t/30044/tox_ini__test_on_voidlinux
83c240eAdd more voidlinux packages
9f4a232Add voidlinux package for giac

comment:7 Changed 17 months ago by git

  • Commit changed from 9f4a232aee35913b1134063171a64dc9914c0f95 to d43730cdbe9bb09b1045ba16daad841f83419642

Branch pushed to git repo; I updated commit sha1. New commits:

d43730cremove glib2 (typo?)

comment:8 Changed 17 months ago by mkoeppe

  • Description modified (diff)

comment:9 Changed 17 months ago by mkoeppe

  • Authors set to Nicolò Piazzalunga, Matthias Koeppe
  • Cc gh-kliem added
  • Status changed from new to needs_review

This works well enough to reproduce at least some of the reported build errors. Needs review

comment:10 Changed 16 months ago by mkoeppe

  • Cc slelievre added

comment:11 Changed 16 months ago by mkoeppe

Let's get this in please

comment:12 Changed 16 months ago by gh-kliem

I'm running some tests to see whether voidlinux somehow works with github actions:

https://github.com/kliem/sage/pull/22/checks

comment:13 follow-up: Changed 16 months ago by gh-kliem

In principle this seems capable of reproducing problems. However, I'm getting errors in log file artifacts. Did I do something wrong? Is it a problem with github? Is it caused by this ticket?

comment:14 Changed 16 months ago by gh-kliem

  • .github/workflows/tox.yml

    diff --git a/.github/workflows/tox.yml b/.github/workflows/tox.yml
    index d6f46bd7b1..27f2f34dd4 100644
    a b jobs: 
    3636      fail-fast: false
    3737      max-parallel: 20
    3838      matrix:
    39         tox_system_factor: [ubuntu-trusty, ubuntu-xenial, ubuntu-bionic, ubuntu-eoan, ubuntu-focal, debian-jessie, debian-stretch, debian-buster, debian-bullseye, debian-sid, linuxmint-17, linuxmint-18, linuxmint-19, linuxmint-19.3, fedora-26, fedora-27, fedora-28, fedora-29, fedora-30, fedora-31, fedora-32, centos-7, centos-8, gentoo, archlinux-latest, slackware-14.2, conda-forge, ubuntu-bionic-i386, ubuntu-eoan-i386, debian-buster-i386, centos-7-i386]
     39        tox_system_factor: [debian-buster, voidlinux]
    4040        tox_packages_factor: [minimal, standard]
    4141    env:
    4242      TOX_ENV: docker-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    jobs: 
    9494            done || echo "(Ignoring errors)"
    9595          fi
    9696        if: always()
    97 
    98   local-macos:
    99 
    100     runs-on: macos-latest
    101     strategy:
    102       fail-fast: false
    103       max-parallel: 4
    104       matrix:
    105         tox_system_factor: [homebrew-macos, homebrew-macos-python3_xcode, homebrew-macos-python3_xcode-nokegonly, homebrew-macos-python3_pythonorg, homebrew-macos-python3_xcode-gcc_spkg, conda-forge-macos]
    106         tox_packages_factor: [minimal, standard]
    107     env:
    108       TOX_ENV: local-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    109       LOGS_ARTIFACT_NAME: logs-commit-${{ github.sha }}-tox-local-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    110     steps:
    111       - uses: actions/checkout@v2
    112       - name: Install test prerequisites
    113         run: |
    114           brew install tox
    115       - name: Install python3 from python.org
    116         # As of 2020-03-30 (https://github.com/actions/virtual-environments/blob/master/images/macos/macos-10.15-Readme.md),
    117         # Python 3.7.7 is installed on GitHub Actions runners.  But we install our own copy from the python.org binary package.
    118         run: |
    119           curl -o python3.pkg https://www.python.org/ftp/python/3.7.7/python-3.7.7-macosx10.9.pkg
    120           sudo installer -verbose -pkg python3.pkg -target /
    121         if: contains(matrix.tox_system_factor, 'python3_pythonorg')
    122       - name: Build and test with tox
    123         # We use a high parallelization on purpose in order to catch possible parallelization bugs in the build scripts.
    124         # For doctesting, we use a lower parallelization to avoid timeouts.
    125         run: |
    126           MAKE="make -j12" tox -e $TOX_ENV -- SAGE_NUM_THREADS=4 $TARGETS
    127       - name: Prepare logs artifact
    128         run: |
    129           mkdir -p "artifacts/$LOGS_ARTIFACT_NAME"; cp -r .tox/*/log "artifacts/$LOGS_ARTIFACT_NAME"
    130         if: always()
    131       - uses: actions/upload-artifact@v1
    132         with:
    133           path: artifacts
    134           name: ${{ env.LOGS_ARTIFACT_NAME }}
    135         if: always()
    136       - name: Print out logs for immediate inspection
    137         # and markup the output with GitHub Actions logging commands
    138         run: |
    139           .github/workflows/scan-logs.sh "artifacts/$LOGS_ARTIFACT_NAME"
    140         if: always()
    141 
    142   local-ubuntu:
    143 
    144     runs-on: ubuntu-latest
    145     strategy:
    146       fail-fast: false
    147       max-parallel: 1
    148       matrix:
    149         tox_system_factor: [conda-forge-ubuntu]
    150         tox_packages_factor: [minimal, standard]
    151     env:
    152       TOX_ENV: local-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    153       LOGS_ARTIFACT_NAME: logs-commit-${{ github.sha }}-tox-local-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    154     steps:
    155       - uses: actions/checkout@v2
    156       - name: Install test prerequisites
    157         run: |
    158           sudo DEBIAN_FRONTEND=noninteractive apt-get update
    159           sudo DEBIAN_FRONTEND=noninteractive apt-get install python-tox
    160       - name: Build and test with tox
    161         # We use a high parallelization on purpose in order to catch possible parallelization bugs in the build scripts.
    162         # For doctesting, we use a lower parallelization to avoid timeouts.
    163         run: |
    164           MAKE="make -j12" tox -e $TOX_ENV -- SAGE_NUM_THREADS=4 $TARGETS
    165       - name: Prepare logs artifact
    166         run: |
    167           mkdir -p "artifacts/$LOGS_ARTIFACT_NAME"; cp -r .tox/*/log "artifacts/$LOGS_ARTIFACT_NAME"
    168         if: always()
    169       - uses: actions/upload-artifact@v1
    170         with:
    171           path: artifacts
    172           name: ${{ env.LOGS_ARTIFACT_NAME }}
    173         if: always()
    174       - name: Print out logs for immediate inspection
    175         # and markup the output with GitHub Actions logging commands
    176         run: |
    177           .github/workflows/scan-logs.sh "artifacts/$LOGS_ARTIFACT_NAME"
    178         if: always()
    179 
    180   dist:
    181 
    182     runs-on: ubuntu-latest
    183     steps:
    184       - uses: actions/checkout@v2
    185         with:
    186           fetch-depth: 500
    187       - name: fetch tags
    188         run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*
    189       - name: Install bootstrap prerequisites
    190         run: |
    191           sudo DEBIAN_FRONTEND=noninteractive apt-get update
    192           sudo DEBIAN_FRONTEND=noninteractive apt-get install $(sed "s/#.*//;" build/pkgs/debian-bootstrap.txt)
    193       - name: Bootstrap with sage-update-version
    194         # We set SAGE_ROOT and SAGE_SRC by hand
    195         # because 'sage -sh' does not work with an unconfigured tree,
    196         # giving: Error: SAGE_SCRIPTS_DIR is set to a bad value
    197         run: |
    198           git config --global user.email "nobody@example.com"
    199           git config --global user.name "Sage GitHub CI"
    200           SAGE_ROOT=. SAGE_SRC=./src src/bin/sage-update-version $(git describe)
    201       - name: make dist
    202         run: |
    203           ./configure && make dist
    204       - uses: actions/upload-artifact@v2
    205         with:
    206           path: "dist/*.tar.gz"
    207           name: dist
    208 
    209   local-macos-nohomebrew:
    210 
    211     needs: [dist]
    212 
    213     runs-on: macos-latest
    214     strategy:
    215       fail-fast: false
    216       max-parallel: 4
    217       matrix:
    218         tox_system_factor: [macos-nobootstrap, macos-nobootstrap-python3_pythonorg]
    219         tox_packages_factor: [minimal]
    220     env:
    221       TOX_ENV: local-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    222       LOGS_ARTIFACT_NAME: logs-commit-${{ github.sha }}-tox-local-${{ matrix.tox_system_factor }}-${{ matrix.tox_packages_factor }}
    223     steps:
    224       - uses: actions/checkout@v2
    225         if: "!contains(matrix.tox_system_factor, 'nobootstrap')"
    226       - uses: actions/download-artifact@v2
    227         with:
    228           path: .
    229           name: dist
    230         if: contains(matrix.tox_system_factor, 'nobootstrap')
    231       - name: Unpack sage dist
    232         run: |
    233           tar xf sage*.tar.gz --strip-components=1
    234         if: contains(matrix.tox_system_factor, 'nobootstrap')
    235       - name: Move homebrew away
    236         run: |
    237           (cd /usr/local && for a in bin etc include lib opt sbin share; do mv $a $a-moved; done)
    238       - name: Install test prerequisites
    239         run: |
    240           /usr/bin/python3 -m pip install --user tox
    241       - name: Install python3 from python.org
    242         # As of 2020-03-30 (https://github.com/actions/virtual-environments/blob/master/images/macos/macos-10.15-Readme.md),
    243         # Python 3.7.7 is installed on GitHub Actions runners.  But we install our own copy from the python.org binary package.
    244         run: |
    245           curl -o python3.pkg https://www.python.org/ftp/python/3.7.7/python-3.7.7-macosx10.9.pkg
    246           sudo installer -verbose -pkg python3.pkg -target /
    247         if: contains(matrix.tox_system_factor, 'python3_pythonorg')
    248       - name: Build and test with tox
    249         # We use a high parallelization on purpose in order to catch possible parallelization bugs in the build scripts.
    250         # For doctesting, we use a lower parallelization to avoid timeouts.
    251         run: |
    252           MAKE="make -j12" tox -e $TOX_ENV -- SAGE_NUM_THREADS=4 $TARGETS
    253       - name: Prepare logs artifact
    254         run: |
    255           mkdir -p "artifacts/$LOGS_ARTIFACT_NAME"; cp -r .tox/*/log "artifacts/$LOGS_ARTIFACT_NAME"
    256         if: always()
    257       - uses: actions/upload-artifact@v1
    258         with:
    259           path: artifacts
    260           name: ${{ env.LOGS_ARTIFACT_NAME }}
    261         if: always()
    262       - name: Print out logs for immediate inspection
    263         # and markup the output with GitHub Actions logging commands
    264         run: |
    265           .github/workflows/scan-logs.sh "artifacts/$LOGS_ARTIFACT_NAME"
    266         if: always()

comment:15 in reply to: ↑ 13 Changed 16 months ago by mkoeppe

Replying to gh-kliem:

In principle this seems capable of reproducing problems. However, I'm getting errors in log file artifacts. Did I do something wrong? Is it a problem with github? Is it caused by this ticket?

It's a problem with github. I have had a lot of trouble with it in the past weeks.

comment:16 Changed 16 months ago by gh-kliem

  • Reviewers set to Jonathan Kliem
  • Status changed from needs_review to positive_review

LGTM.

comment:17 Changed 16 months ago by vbraun

  • Branch changed from u/mkoeppe/tox_ini__test_on_voidlinux to d43730cdbe9bb09b1045ba16daad841f83419642
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.