Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#30622 closed enhancement (fixed)

build/bin/sage-spkg: Handle an optional argument to set the installation tree (default SAGE_LOCAL)

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.3
Component: build Keywords: sd111
Cc: jhpalmieri, mjo, gh-kliem Merged in:
Authors: Matthias Koeppe Reviewers: Jonathan Kliem
Report Upstream: N/A Work issues:
Branch: 7ea44f7 (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

We handle a new optional argument to sage-spkg, which sets the installation tree - the new variable SAGE_INST_LOCAL (default: $SAGE_LOCAL) for the package installation scripts. (The script sage-spkg-uninstall already accepts this optional argument.)

We also move the definition of SAGE_SPKG_INST and SAGE_SPKG_WHEELS out of build/bin/install into sage-spkg (and into Makefile.in for script packages). This is so that they can depend on SAGE_INST_LOCAL.

Thus, the installation location of the installation records now depends on the installation tree - it is $SAGE_INST_LOCAL/var/lib/sage/{installed,scripts,wheels}.

This is preparation for #29013 - which proposes to install Python packages in $SAGE_VENV instead of $SAGE_LOCAL.

Change History (37)

comment:1 Changed 2 years ago by mkoeppe

Description: modified (diff)
Summary: build/bin/sage-spkg: Allow overriding SAGE_LOCAL by SAGE_SPKG_PREFIXbuild/bin/sage-spkg: Allow overriding SAGE_LOCAL by SAGE_INST_LOCAL

comment:2 Changed 2 years ago by mkoeppe

Dependencies: #29500

comment:3 Changed 2 years ago by mkoeppe

Branch: u/mkoeppe/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix

comment:4 Changed 2 years ago by git

Commit: 8fc7f9ca20d6665425d0332070c6e7e3dc3968c9

Branch pushed to git repo; I updated commit sha1. New commits:

f2e7075Merge tag '9.2.beta13' into t/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels
8fc7f9cMerge branch 't/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels' into t/30622/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix

comment:5 Changed 2 years ago by mkoeppe

Dependencies: #29500#29500, #29386

comment:6 Changed 2 years ago by mkoeppe

Dependencies: #29500, #29386#29500, #29386, #30657

comment:7 Changed 2 years ago by git

Commit: 8fc7f9ca20d6665425d0332070c6e7e3dc3968c9a713c390686ce8da37f0c0268059d3b522c0d0d3

Branch pushed to git repo; I updated commit sha1. New commits:

0f1b7aabuild/make/install: Do not set SAGE_SPKG_INST here; set it in build/make/Makefile.in
1baf8eaWIP
bd0fd2fMerge branch 't/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels' into t/30657/fix_up__sage__p_
2e20f23build/bin/sage-spkg: Move SAGE_SPKG_{INST,SCRIPTS,WHEELS} setting here from build/bin/install, make them relative to SAGE_INST_LOCAL
be2417cMerge tag '9.2.beta14' into t/30657/fix_up__sage__p_
8ee7ee2build/make/install: Set SAGE_SPKG_WHEELS also here so that script packages still work
a713c39Merge branch 't/30657/fix_up__sage__p_' into t/30622/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix

comment:8 Changed 2 years ago by mkoeppe

Authors: Matthias Koeppe
Last edited 2 years ago by mkoeppe (previous) (diff)

comment:9 Changed 2 years ago by git

Commit: a713c390686ce8da37f0c0268059d3b522c0d0d3333c5b5668b7290f074bcf65fc421a58a3cd74eb

Branch pushed to git repo; I updated commit sha1. New commits:

8ef2508Merge tag '9.3.beta2' into t/30622/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix
333c5b5build/bin/sage-spkg: Accept installation tree as an optional argument, like sage-spkg-uninstall

comment:10 Changed 2 years ago by mkoeppe

Dependencies: #29500, #29386, #30657
Description: modified (diff)
Summary: build/bin/sage-spkg: Allow overriding SAGE_LOCAL by SAGE_INST_LOCALbuild/bin/sage-spkg: Handle an optional argument to set the installation tree (default SAGE_LOCAL)

comment:11 Changed 2 years ago by git

Commit: 333c5b5668b7290f074bcf65fc421a58a3cd74ebdbbd614955b84bb7379f5222b3717051abc8ab14

Branch pushed to git repo; I updated commit sha1. New commits:

dbbd614build/bin/sage-spkg: From bash -x back to bash

comment:12 Changed 2 years ago by mkoeppe

Cc: jhpalmieri mjo added
Status: newneeds_review

comment:13 Changed 2 years ago by mkoeppe

Status: needs_reviewneeds_work

comment:14 Changed 2 years ago by git

Commit: dbbd614955b84bb7379f5222b3717051abc8ab14cb151691167c0317214ddc4657ea5efc2691776c

Branch pushed to git repo; I updated commit sha1. New commits:

cb15169Makefile, build/make/Makefile.in: Fix 'make list'

comment:15 Changed 2 years ago by mkoeppe

Status: needs_workneeds_review

comment:16 Changed 2 years ago by mkoeppe

Cc: gh-kliem added

comment:17 Changed 2 years ago by gh-kliem

Reviewers: https://github.com/kliem/sage/pull/29/checks

comment:18 Changed 2 years ago by mkoeppe

Keywords: sd111 added

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

comment:19 Changed 2 years ago by gh-kliem

Reviewers: https://github.com/kliem/sage/pull/29/checksJonathan Kliem
Status: needs_reviewpositive_review

LGTM.

comment:20 Changed 2 years ago by mkoeppe

Thank you!

comment:21 Changed 2 years ago by vbraun

Status: positive_reviewneeds_work

Merge conflict

comment:22 Changed 2 years ago by mkoeppe

Work issues: merge #30375

comment:23 Changed 2 years ago by git

Commit: cb151691167c0317214ddc4657ea5efc2691776c9c9b3bf8a52544285935dac66e0589769bac363c

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

9c5e8a2add references to `bin/sage-build-env` and `bin/sage-build-env-config`
4b8933bremove incorrect `$` in `build/bin/sage-build-env`
071b8ccRevert "add references to `bin/sage-build-env` and `bin/sage-build-env-config`"
c51e752correct comments to makefile
aa7ecd6fix that we might have changed $FCFLAGS during configure
94326dbbetter solution for dealing with -ffree-form
87dae61keep FCFLAGS during gfortran configure
7577dd4added quotes
3bb3099prepare preciouos variables
9c9b3bfMerge commit '3bb309944b7e8542b2ac88ed3c9d9a60e68644d7' of git://trac.sagemath.org/sage into t/30622/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix

comment:24 Changed 2 years ago by mkoeppe

Dependencies: #30375
Status: needs_workpositive_review
Work issues: merge #30375

Merged #30375

comment:25 Changed 2 years ago by vbraun

Branch: u/mkoeppe/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix9c9b3bf8a52544285935dac66e0589769bac363c
Resolution: fixed
Status: positive_reviewclosed

comment:26 Changed 2 years ago by vbraun

Branch: 9c9b3bf8a52544285935dac66e0589769bac363cu/mkoeppe/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix
Resolution: fixed
Status: closednew

Breaks making source tarballs:

Sage version 9.3.beta5, release date 2020-12-27
Cloning into '/home/release/Sage/tmp/sage-9.3.beta5'...
done.
Enumerating objects: 737573, done.
Counting objects: 100% (737573/737573), done.
Delta compression using up to 16 threads
Compressing objects: 100% (702757/702757), done.
Writing objects: 100% (737573/737573), done.
Total 737573 (delta 588187), reused 141018 (delta 0), pack-reused 0
Checking connectivity: 737573, done.
Finished cloning Sage sources
Makefile:22: *** This Makefile needs to be invoked by build/make/install.  Stop.
Traceback (most recent call last):
  File "/home/release/opt/bin/git-releasemgr", line 16, in <module>
    cmdline.launch()
  File "/home/release/opt/git-trac-command/git_trac/releasemgr/cmdline.py", line 166, in launch
    app.release(args.version, check=args.check)
  File "/home/release/opt/git-trac-command/git_trac/releasemgr/app.py", line 315, in release
    create_tarball()
  File "/home/release/opt/git-trac-command/git_trac/releasemgr/make_release.py", line 15, in create_tarball
    check_call(['sage', '-sdist'])
  File "/home/release/opt/conda/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sage', '-sdist']' returned non-zero exit status 2

comment:27 Changed 2 years ago by git

Commit: 9c9b3bf8a52544285935dac66e0589769bac363c062ef926118ea779aec0af92530079f140f276bc

Branch pushed to git repo; I updated commit sha1. New commits:

36e8f7dMerge tag '9.3.beta5' into t/30622/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix
062ef92build/make/Makefile.in: Fix up 'make dist'

comment:28 Changed 2 years ago by mkoeppe

Status: newneeds_review

comment:29 Changed 2 years ago by mkoeppe

Dependencies: #30375

comment:30 Changed 2 years ago by gh-kliem

There is two small things that could be fixed along the way:

  • make dist does not appear in make --help.
  • local/bin/sage-sdist claims it should be called by spkg/bin/sage, but it is really run by local/bin/sage.

The error message This Makefile needs to be invoked by build/make/install seems strange, as there is at least one other legal way to call the Makefile, as we discovered here. If it is not included in the message, I think the comment should be more verbose.

comment:31 Changed 2 years ago by git

Commit: 062ef926118ea779aec0af92530079f140f276bc6eb8f26b9edfa806c8b5e2453d6b6f6e0f54b5ab

Branch pushed to git repo; I updated commit sha1. New commits:

6eb8f26build/bin/sage-sdist: Update file comment

comment:32 in reply to:  30 Changed 2 years ago by mkoeppe

Replying to gh-kliem:

There is two small things that could be fixed along the way:

  • make dist does not appear in make --help.

My version of make here on macOS does not list any targets on make --help. In any case, beyond the scope of the ticket.

  • local/bin/sage-sdist claims it should be called by spkg/bin/sage, but it is really run by local/bin/sage.

Updated.

comment:33 Changed 2 years ago by git

Commit: 6eb8f26b9edfa806c8b5e2453d6b6f6e0f54b5ab7ea44f7f8faf65d0061d97b53a53e73151716d94

Branch pushed to git repo; I updated commit sha1. New commits:

7ea44f7build/make/Makefile.in: Expand comment

comment:34 Changed 2 years ago by gh-kliem

Status: needs_reviewpositive_review

Back to positive review.

comment:35 Changed 2 years ago by mkoeppe

Thanks.

comment:36 Changed 2 years ago by vbraun

Branch: u/mkoeppe/build_bin_sage_spkg__allow_overriding_sage_local_by_sage_spkg_prefix7ea44f7f8faf65d0061d97b53a53e73151716d94
Resolution: fixed
Status: positive_reviewclosed

comment:37 Changed 2 years ago by gh-kliem

Commit: 7ea44f7f8faf65d0061d97b53a53e73151716d94

Apparently we broke sage -b with this ticket.

Note: See TracTickets for help on using tickets.