Opened 10 months ago

Closed 4 months ago

#29896 closed defect (duplicate)

fix make download

Reported by: dimpase Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: distribution Keywords: download
Cc: saliola, mkoeppe, mjo, slelievre, vbraun Merged in:
Authors: Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/fix_make_download (Commits, GitHub, GitLab) Commit: 039fed7ec5f15feade1ec8368c106fcbe2c71152
Dependencies: #30865 Stopgaps:

Status badges

Change History (23)

comment:1 Changed 10 months ago by saliola

  • Cc saliola added

comment:2 Changed 10 months ago by jhpalmieri

One option:

  • Makefile

    diff --git a/Makefile b/Makefile
    index 822d1d316f..74863704b1 100644
    a b buildbot-python3: 
    6161       $(MAKE)
    6262
    6363# Preemptively download all standard upstream source tarballs.
    64 download:
     64download: build/make/Makefile
    6565       export SAGE_ROOT=$$(pwd) && \
    66        export PATH=$$SAGE_ROOT/src/bin:$$PATH && \
     66       export PATH=$$SAGE_ROOT/build/bin:$$SAGE_ROOT/src/bin:$$PATH && \
    6767       ./src/bin/sage-download-upstream
    6868
    6969dist: build/make/Makefile

Then (after #29316) users will be warned that they have to run ./configure first. For me, if I've run ./configure, this modified version of make download works.

comment:3 Changed 10 months ago by jhpalmieri

Although there is another issue: make download says that it downloads the standard upstream source tarballs, but it actually downloads tarballs for standard, optional, and experimental (after querying) packages. At least one of the experimental database packages is pretty big, so maybe we don't want to do this. With #20104, we could do this:

  • src/bin/sage-download-upstream

    diff --git a/src/bin/sage-download-upstream b/src/bin/sage-download-upstream
    index c5d9afe31e..226806ab7d 100755
    a b  
    11#!/usr/bin/env bash
    22
    3 for pkg in $SAGE_ROOT/build/pkgs/*
     3for pkg in `sage --package list :standard:`
    44do
    5     if [ -d $pkg ]; then
     5    if [ -d $SAGE_ROOT/build/pkgs/$pkg ]; then
    66        sage-spkg -d `basename $pkg`
    77    fi
    88done

comment:4 Changed 10 months ago by jhpalmieri

  • Cc mkoeppe added

comment:5 Changed 10 months ago by jhpalmieri

I could also imagine trying to use the results from ./configure to decide which packages to download, which to skip because they are already available from the system. But if someone wants to download once and then distribute to colleagues on a thumb drive, it's a good idea to have all of the standard packages.

comment:6 Changed 9 months ago by mkoeppe

  • Cc mjo added

comment:7 follow-up: Changed 9 months ago by jhpalmieri

Should this be using the same logic as make dist to determine what to download?

comment:8 in reply to: ↑ 7 Changed 9 months ago by mjo

Replying to jhpalmieri:

Should this be using the same logic as make dist to determine what to download?

That's my instinct. We probably don't need both make download-for-sdist and make download in that case.

comment:9 Changed 6 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:10 Changed 6 months ago by mkoeppe

  • Dependencies set to #30846

comment:11 Changed 6 months ago by mkoeppe

  • Branch set to u/mkoeppe/fix_make_download

comment:12 Changed 6 months ago by mkoeppe

  • Authors set to John Palmieri, Matthias Koeppe
  • Commit set to 896544fa0d1ccad2526f871afca56945563f9459

The branch "fixes" make download (and also cleans up src/bin/ a bit more, on top of #30846 and similar earlier tickets).

I don't use this myself, so I don't know what is the best way to proceed regarding "standard" vs. "all" vs. "configured" packages. Perhaps they should be 3 different make targets.


New commits:

044fcc0Move sage-list-packages from src/bin (sagelib) to build/bin (sage_bootstrap)
896544fMove sage-download-upstream from src/bin (sagelib) to build/bin (sage_bootstrap)

comment:13 Changed 5 months ago by slelievre

  • Cc slelievre added
  • Keywords download added
  • Priority changed from minor to major

In case that is relevant to the present ticket: our release manager says: "If there were a command-line switch to download all standard + optional + experimental packages then I could test that..." (i.e. test they all download properly, to avoid missing upstream tarballs on our download mirrors after package upgrade tickets get merged).

comment:14 Changed 5 months ago by git

  • Commit changed from 896544fa0d1ccad2526f871afca56945563f9459 to 039fed7ec5f15feade1ec8368c106fcbe2c71152

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

e09d7daMove sage-download-upstream from src/bin (sagelib) to build/bin (sage_bootstrap)
ca87916Makefile: Update documentation of 'make download'
039fed7src/doc/en/installation/source.rst: Remove outdated documentation on old-style packages

comment:15 Changed 5 months ago by mkoeppe

  • Status changed from new to needs_review

comment:16 Changed 5 months ago by mkoeppe

  • Cc vbraun added

comment:17 Changed 5 months ago by mkoeppe

  • Dependencies #30846 deleted

No longer depends on #30846

comment:18 Changed 5 months ago by git

  • Commit changed from 039fed7ec5f15feade1ec8368c106fcbe2c71152 to e263510e5b2056a82d10ede5fcb15ae9a4ec6369

(pushed to wrong ticket)

Last edited 5 months ago by mkoeppe (previous) (diff)

comment:19 Changed 5 months ago by git

  • Commit changed from e263510e5b2056a82d10ede5fcb15ae9a4ec6369 to 039fed7ec5f15feade1ec8368c106fcbe2c71152

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

comment:20 Changed 5 months ago by mkoeppe

Actually best to go through #30865, sage -package download :all:

comment:21 Changed 5 months ago by mkoeppe

  • Authors John Palmieri, Matthias Koeppe deleted
  • Dependencies set to #30865
  • Milestone changed from sage-9.3 to sage-duplicate/invalid/wontfix

comment:22 Changed 5 months ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

comment:23 Changed 4 months ago by chapoton

  • Resolution set to duplicate
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.