#31216 closed enhancement (fixed)

tox.ini (local): Add environment variables to skip system package installs and other steps, add mechanism for a local interactive shell

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.3
Component: porting Keywords:
Cc: gh-tobiasdiez, gh-kliem, dimpase Merged in:
Authors: Matthias Koeppe Reviewers: Tobias Diez
Report Upstream: N/A Work issues:
Branch: 2d84b5a (Commits, GitHub, GitLab) Commit: 2d84b5ab5ac0741e872fb5a15539c2bbf935fe86
Dependencies: #30944 Stopgaps:

Status badges

Description (last modified by mkoeppe)

(from #31064)

We add the following to the local-... environments:

  • an environment variable that can be passed to tox to skip system package installs, directly reusing a previously set up system
    SKIP_SYSTEM_PKG_INSTALL=yes tox -e local-homebrew-macos-standard -- config.status
    SKIP_SYSTEM_PKG_INSTALL=yes tox -e local-homebrew-macos-standard -- config.status
    
    This can save time and also give developers more control for experiments with system packages.
  • a target that gives an interactive shell in the tox environment:
    tox -e local-homebrew-macos-standard -- bash
    tox -e local-direct -- bash
    SKIP_SYSTEM_PKG_INSTALL=yes SKIP_BOOTSTRAP=1 SKIP_CONFIGURE=1 tox -e local-homebrew-macos-standard -- bash
    

Change History (25)

comment:1 Changed 20 months ago by mkoeppe

  • Dependencies set to #30944

comment:2 Changed 20 months ago by mkoeppe

  • Branch set to u/mkoeppe/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell

comment:3 Changed 20 months ago by mkoeppe

  • Commit set to 13f58fc7445bca7f54cc80efea51c1082a615542
  • Description modified (diff)

Last 10 new commits:

e5fe752Merge tag '9.3.beta4' into t/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available
78ff9d5src/sage/misc/package.py: Add one more # optional - build
a44042fMerge branch 't/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available' into t/29124/script-packages-prereq-toolchain-bootstrap
64bde5fMerge tag '9.3.beta5' into t/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available
e9a7572src/sage/misc/package.py: Improve source formatting
c7bcda9Merge branch 't/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available' into t/29124/script-packages-prereq-toolchain-bootstrap
9988c5fci-cygwin*.yml: Adjust to new script packages _bootstrap, _prereq
ab19133Merge branch 't/29124/script-packages-prereq-toolchain-bootstrap' into t/30944/tox__improve_local_sudo_ubuntu_standard
c6ad06eMerge branch 't/30944/tox__improve_local_sudo_ubuntu_standard' into t/31216/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell
13f58fctox.ini (local): Run an interactive shell if posargs are bash

comment:4 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:5 Changed 20 months ago by git

  • Commit changed from 13f58fc7445bca7f54cc80efea51c1082a615542 to ace4ef6674eaf29ff6a590ce0bcc5943f36c342c

Branch pushed to git repo; I updated commit sha1. New commits:

ace4ef6tox.ini (local): Add option SKIP_SYSTEM_PKG_INSTALL=1

comment:6 Changed 20 months ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Status changed from new to needs_review

comment:7 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:8 follow-up: Changed 20 months ago by gh-tobiasdiez

On a first glance this looks good. Thanks! Could you please also add documentation for these tox commands / switches. Also I was wondering if one could add a tox -e local-homebrew-macos-standard -- install-system-packages that only installs the system packages?

comment:9 Changed 20 months ago by git

  • Commit changed from ace4ef6674eaf29ff6a590ce0bcc5943f36c342c to ddc4f0b63ae13eee632663d7cb57c2b6d8207eb8

Branch pushed to git repo; I updated commit sha1. New commits:

ddc4f0btox.ini (local): Handle environment variables SKIP_BOOTSTRAP, SKIP_CONFIGURE

comment:10 Changed 20 months ago by mkoeppe

  • Description modified (diff)
  • Summary changed from tox.ini (local): Add environment variables to skip system package installs, mechanism for a local interactive shell to tox.ini (local): Add environment variables to skip system package installs and other steps, add mechanism for a local interactive shell

comment:11 Changed 20 months ago by git

  • Commit changed from ddc4f0b63ae13eee632663d7cb57c2b6d8207eb8 to 6253bd822f2dd515979875d3aea4beaf8de0d0ec

Branch pushed to git repo; I updated commit sha1. New commits:

bb6aec4src/doc/en/developer/portability_testing.rst: Updates
6253bd8src/doc/en/developer/portability_testing.rst: New section on SKIP_... environment variables

comment:12 in reply to: ↑ 8 ; follow-up: Changed 20 months ago by mkoeppe

Replying to gh-tobiasdiez:

Could you please also add documentation for these tox commands / switches.

Done

Also I was wondering if one could add a tox -e local-homebrew-macos-standard -- install-system-packages that only installs the system packages?

I have instead added more SKIP_... options.

comment:13 in reply to: ↑ 12 ; follow-up: Changed 19 months ago by gh-tobiasdiez

Replying to mkoeppe:

I have instead added more SKIP_... options.

Thanks! Would it also make sense to add one to skip make? (I remember you once told me how to do this, I think, it was via config.status or something - but maybe the version using skip is closer in spirit anyway).

I'll review this once #30944 is merged in develop.

comment:14 in reply to: ↑ 13 ; follow-up: Changed 19 months ago by mkoeppe

Replying to gh-tobiasdiez:

Would it also make sense to add one to skip make?

You can just pass a make target that does nothing - such as Makefile

comment:15 Changed 19 months ago by gh-tobiasdiez

  • Status changed from needs_review to needs_work

local-sudo-standard run-test: commands[1] | bash -c 'case "" in 1|y*|Y*);; *) eval $(build/bin/sage-print-system-package-command $(build/bin/sage-guess-package-system) --sudo update) ;;'/usr/bin/bash: -c: line 1: syntax error: unexpected end of file

I think there is a "esac" missing at the end of these two lines:

+    local-{root,sudo}:    bash -c 'case "{env:SKIP_SYSTEM_PKG_INSTALL:}" in 1|y*|Y*);; *) eval $(build/bin/sage-print-system-package-command {env:SYSTEM} {env:__SUDO:} update) ;;'
+    local-{root,sudo}:    bash -c 'case "{env:SKIP_SYSTEM_PKG_INSTALL:}" in 1|y*|Y*);; *) PACKAGES=$(build/bin/sage-get-system-packages {env:SYSTEM} $(PATH=build/bin:$PATH build/bin/sage-package list {env:SAGE_PACKAGE_LIST_ARGS}) _bootstrap); eval $(build/bin/sage-print-system-package-command {env:SYSTEM} {env:__SUDO:} --yes --no-install-recommends install $PACKAGES) || [ "$IGNORE_MISSING_SYSTEM_PACKAGES" = yes ] && echo "(ignoring errors)" ;;'

comment:16 in reply to: ↑ 14 Changed 19 months ago by gh-tobiasdiez

Replying to mkoeppe:

Replying to gh-tobiasdiez:

Would it also make sense to add one to skip make?

You can just pass a make target that does nothing - such as Makefile

tox -e local-sudo-standard Makefile with SKIP_BOOTSTRAP: yes and SKIP_CONFIGURE: yes still runs bootstrap and base-toolchain in the make command, which then fails since configure was not called before.

comment:17 Changed 19 months ago by mkoeppe

Ah, of course. Make that configure.ac instead of Makefile...

comment:18 Changed 19 months ago by gh-tobiasdiez

That worked, thanks! Apart from the missing "esac" mentioned above, this looks good to me.

comment:19 Changed 19 months ago by git

  • Commit changed from 6253bd822f2dd515979875d3aea4beaf8de0d0ec to 2d84b5ab5ac0741e872fb5a15539c2bbf935fe86

Branch pushed to git repo; I updated commit sha1. New commits:

2d84b5atox.ini: Add missing esac

comment:20 Changed 19 months ago by mkoeppe

  • Status changed from needs_work to needs_review

comment:22 in reply to: ↑ 21 Changed 19 months ago by mkoeppe

Replying to gh-tobiasdiez:

Is the prefix [mkoeppe@sage sage]$ really needed / what's its purpose?

It's a typical shell prompt on posix systems. The second sage indicates the current directory. This is consistent with other parts of the Sage developer guide, for example https://doc.sagemath.org/html/en/developer/doctesting.html

comment:23 Changed 19 months ago by gh-tobiasdiez

  • Status changed from needs_review to positive_review

Thanks, then it looks good to me!

comment:24 Changed 19 months ago by mkoeppe

  • Reviewers set to Tobias Diez

Thanks!

comment:25 Changed 19 months ago by vbraun

  • Branch changed from u/mkoeppe/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell to 2d84b5ab5ac0741e872fb5a15539c2bbf935fe86
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.