Opened 7 years ago
Closed 6 years ago
#15642 closed enhancement (fixed)
Use mirrors for tarballs
Reported by: | vbraun | Owned by: | |
---|---|---|---|
Priority: | blocker | Milestone: | sage-6.7 |
Component: | build | Keywords: | |
Cc: | ohanar, tmonteil, schilly | Merged in: | |
Authors: | Volker Braun | Reviewers: | Volker Braun |
Report Upstream: | N/A | Work issues: | |
Branch: | d79364b (Commits) | Commit: | d79364b9fc5131199eaed14340582715f2f937fb |
Dependencies: | Stopgaps: |
Description (last modified by )
This ticket rewrites the sage-download-file
script in pure Python. It is, first of all, a drop-in replacement of the current script. But it is now also aware of our tarballs, so you can just write e.g.
sage-download-file sympy-0.7.4.tar.gz
and it will download that tarball from the fastest mirror and verify its checksum.
Change History (95)
comment:1 Changed 7 years ago by
- Branch set to u/vbraun/use_mirrors_for_tarballs
comment:2 Changed 7 years ago by
- Commit set to dd4abcb7757c17e5a7bae9cf347c9d34e154503d
comment:3 Changed 7 years ago by
- Component changed from PLEASE CHANGE to build
- Status changed from new to needs_review
- Type changed from PLEASE CHANGE to enhancement
comment:4 Changed 7 years ago by
- Cc ohanar added
- Description modified (diff)
comment:5 Changed 7 years ago by
- Commit changed from dd4abcb7757c17e5a7bae9cf347c9d34e154503d to b1ef0b663cff6fdbcb2ae9cefad7d72c14605b49
Branch pushed to git repo; I updated commit sha1. New commits:
b1ef0b6 | prefer SAGE_DISTFILES for upstream directory
|
comment:6 Changed 7 years ago by
I would replace the regex in _parse_name
with something like
(?P<base>(?:[^-]|-(?!\d))*)-(?P<version>.*)\.(?P<ext>tar.*|zip|tgz)
(We may as well support more general package names, even if our restrictive naming convention doesn't allow it at the moment.)
comment:7 Changed 7 years ago by
- Commit changed from b1ef0b663cff6fdbcb2ae9cefad7d72c14605b49 to cd155cf944bcdede7c6489fc489511703c10c8ad
Branch pushed to git repo; I updated commit sha1. New commits:
cd155cf | more general tarball regex
|
comment:8 Changed 7 years ago by
- Branch changed from u/vbraun/use_mirrors_for_tarballs to u/ohanar/use_mirrors_for_tarballs
- Commit changed from cd155cf944bcdede7c6489fc489511703c10c8ad to 991de268a9fd9bc0ff4bd94295d8b6cfbe78015e
I added a few extra commits to resolve a couple of small bugs I noticed. Pending someone signing off on those, and testing on various versions of python, this looks good to me.
New commits:
f647251 | remove commented out old code
|
905df09 | rework environment stuff a bit
|
991de26 | don't actually write unsorted mirror list to disk
|
comment:9 Changed 7 years ago by
- Milestone changed from sage-6.1 to sage-6.2
comment:10 Changed 7 years ago by
- Status changed from needs_review to needs_info
Why remove the possibility to use the system curl
or wget
programs? That seems like an unneeded regression to me.
comment:11 follow-up: ↓ 12 Changed 7 years ago by
We only need one way to download files. There is no point in having untested (hence broken) code laying around "just in case". There is no use case, just clutter laying around.
comment:12 in reply to: ↑ 11 Changed 7 years ago by
Replying to vbraun:
There is no use case
The use case is building from git on systems without Python installed.
comment:13 follow-ups: ↓ 14 ↓ 15 Changed 7 years ago by
Per the discussion on sage-devel, Python is a requirement. Sooner or later Sage won't build without Python with or without this ticket.
comment:14 in reply to: ↑ 13 Changed 7 years ago by
Replying to vbraun:
Per the discussion on sage-devel, Python is a requirement.
That would require at least a change to the documentation.
comment:15 in reply to: ↑ 13 Changed 7 years ago by
Replying to vbraun:
Sooner or later Sage won't build without Python with or without this ticket.
Sure, but why break building on systems without Python on this ticket? It is absolutely trivial to keep supporting without Python for now (just keep the shell part of the script). It seems you're breaking stuff for no reason.
comment:16 follow-up: ↓ 17 Changed 7 years ago by
So would it make you feel better if we wait with this ticket and merge it only after another change broke building without Python?
comment:17 in reply to: ↑ 16 Changed 7 years ago by
Replying to vbraun:
So would it make you feel better if we wait with this ticket and merge it only after another change broke building without Python?
Essentially yes. I agree with breaking building without Python, but not breaking just for the sake of it.
comment:18 Changed 7 years ago by
- Milestone changed from sage-6.2 to sage-6.3
comment:19 Changed 7 years ago by
- Milestone changed from sage-6.3 to sage-6.4
comment:20 Changed 6 years ago by
- Cc tmonteil added
comment:21 Changed 6 years ago by
I would say this is not breaking building without Python for the sake of breaking, but for the sake of moving the build system to a language that all Sage developers understand (and therefore can make further improvements).
comment:22 Changed 6 years ago by
Currently in Sage, we have a Python script to download stuff with a shell-script wrapper to handle the case where Python in Sage is not yet installed. I would rather keep the shell-script wrapper and only modify the Python script.
comment:23 follow-up: ↓ 24 Changed 6 years ago by
Why have a shell script here-document a fortran program which runs a python script? Just so that we can build Sage without Python for a few more months? The decision is made, Python is a built-time requirement.
comment:24 in reply to: ↑ 23 Changed 6 years ago by
Replying to vbraun:
The decision is made
I don't recall that discussion...
Let me repeat what I said above:
Why break building on systems without Python on this ticket? It is absolutely trivial to keep support for building without Python for now (just keep the shell part of the script). It seems you're breaking stuff for no reason. I don't mind breaking building without Python if there is a good reason, but not breaking just for the sake of it.
comment:25 Changed 6 years ago by
You are going to say the same thing all the time, why rely on a Python script here when you could just as well have an additional shell script to provide a fall back. The answer is: fewer moving parts = less bugs.
comment:26 Changed 6 years ago by
This is not about python as a requirement nor the curl and wget fallbacks, but i see the benefit of being able to set a URL_GRABBER
variable that calls some fetcher with more robust options, as discussed on this sage-devel thread.
comment:27 Changed 6 years ago by
- Branch changed from u/ohanar/use_mirrors_for_tarballs to u/vbraun/use_mirrors_for_tarballs
comment:28 Changed 6 years ago by
- Commit changed from 991de268a9fd9bc0ff4bd94295d8b6cfbe78015e to 74ac9229132583c3636e551e36aa0a53a23dbc8b
- Priority changed from major to blocker
- Status changed from needs_info to needs_work
We must use the mirrors very soon, the centrally-hosted packages will go away.
The downloading script still needs to be called from sage-spkg, when that is finished we must merge this ticket asap.
New commits:
74ac922 | Merge 6.7.beta3 into #15642
|
comment:29 Changed 6 years ago by
- Commit changed from 74ac9229132583c3636e551e36aa0a53a23dbc8b to 33bf83f60c6332bd06d19c177638b00e0e918354
comment:30 Changed 6 years ago by
- Status changed from needs_work to needs_review
comment:31 Changed 6 years ago by
- Commit changed from 33bf83f60c6332bd06d19c177638b00e0e918354 to a7afa48370ccba8b406a3f820b76ff522bea600c
Branch pushed to git repo; I updated commit sha1. New commits:
a7afa48 | Add a SAGE_MIRROR environment variable
|
comment:32 Changed 6 years ago by
The buildbot will set SAGE_MIRROR
to its own so it doesn't have to wait for the mirrors to catch up.
comment:33 Changed 6 years ago by
- Milestone changed from sage-6.4 to sage-6.7
comment:34 Changed 6 years ago by
- Status changed from needs_review to needs_work
"Usage" doesn't match implementation: --fastest-mirror
vs. --print-fastest-mirror
.
comment:35 Changed 6 years ago by
- Commit changed from a7afa48370ccba8b406a3f820b76ff522bea600c to 9a06850aa7e90080cd1841ce7a4aea46da25ccc1
Branch pushed to git repo; I updated commit sha1. New commits:
9a06850 | fix usage doc
|
comment:37 Changed 6 years ago by
Not entirely clear to me why you use .startswith()
though... ;-)
(Ok, it does support --print-fastest-mirror-and-drop-me-a-beer
, but I'm still waiting...)
comment:38 Changed 6 years ago by
Could we please suppress the tracebacks on errors like 404?
comment:39 Changed 6 years ago by
As you said, using startswith() is a cheap emulation of argparse so --quiet=...
works. The point is to be compatible with Python 2.6, of course, so we can't use argparse.
comment:40 Changed 6 years ago by
Mirror layout fixes, part I:
-
src/bin/sage-spkg
diff --git a/src/bin/sage-spkg b/src/bin/sage-spkg index d25542c..0eb2f8a 100755
a b if [ ! -f "$PKG_SRC" ]; then 367 367 echo ">>> Checking online list of $repo packages." 368 368 # File inside DOT_SAGE should be writable 369 369 repolist="${DOT_SAGE}/${repo}.list" 370 sage-download-file "$MIRROR/ $repo/list" $repolist370 sage-download-file "$MIRROR/spkg/$repo/list" $repolist 371 371 if [ $? -ne 0 ]; then 372 echo >&2 "Error: failed to download $MIRROR/ $repo/list, aborting"372 echo >&2 "Error: failed to download $MIRROR/spkg/$repo/list, aborting" 373 373 rm -f $repolist 374 374 exit 1 375 375 fi … … if [ ! -f "$PKG_SRC" ]; then 386 386 387 387 # If INFO is set, try downloading only the .txt file 388 388 if [ $INFO -eq 1 ]; then 389 PKG_URL="$MIRROR/ $repo/$pkg.txt"389 PKG_URL="$MIRROR/spkg/$repo/$pkg.txt" 390 390 sage-download-file "$PKG_URL" && exit 0 391 391 # If the download failed (for whatever reason), 392 392 # fall through and use the .spkg file. … … if [ ! -f "$PKG_SRC" ]; then 416 416 esac 417 417 fi 418 418 fi 419 PKG_URL="$MIRROR/ $repo/$pkg.spkg"419 PKG_URL="$MIRROR/spkg/$repo/$pkg.spkg" 420 420 break 421 421 fi 422 422 done
(Try e.g. ./sage -i foobar
.)
comment:41 Changed 6 years ago by
... we'd better use a variable like MIRROR_PACKAGE_FOLDER
or whatever there.
comment:42 follow-up: ↓ 46 Changed 6 years ago by
how about you just push that to trac, I'm not going to type that in for you
comment:43 Changed 6 years ago by
There are still issues we have to fix w.r.t. MIRROR
vs. SAGE_SERVER
.
comment:44 Changed 6 years ago by
- Commit changed from 9a06850aa7e90080cd1841ce7a4aea46da25ccc1 to d9e66acabedfe0ae55b961482bfa4ec3515257ec
Branch pushed to git repo; I updated commit sha1. New commits:
d9e66ac | no backtrace on IOError
|
comment:45 follow-up: ↓ 47 Changed 6 years ago by
SAGE_SERVER is used elsewhere, its where the docs are hosted etc.
comment:46 in reply to: ↑ 42 Changed 6 years ago by
Replying to vbraun:
how about you just push that to trac, I'm not going to type that in for you
That's not something I expect you to merge; it just shows errors in the current code. As said, we should IMHO use a variable for that. (I'm not doing your work either... ;-) )
comment:47 in reply to: ↑ 45 Changed 6 years ago by
Replying to vbraun:
SAGE_SERVER is used elsewhere, its where the docs are hosted etc.
It's still in an error message in sage-spkg
, although MIRROR
was used.
comment:48 Changed 6 years ago by
- Commit changed from d9e66acabedfe0ae55b961482bfa4ec3515257ec to 30b1647e7b6a527e72daa7835f26a710306a6099
Branch pushed to git repo; I updated commit sha1. New commits:
30b1647 | fix mirror uri
|
comment:49 follow-up: ↓ 50 Changed 6 years ago by
Fixed
comment:50 in reply to: ↑ 49 Changed 6 years ago by
comment:51 follow-up: ↓ 54 Changed 6 years ago by
the official mirror list may or may not have a slash at the end
comment:52 Changed 6 years ago by
FTP doesn't seem to work (hangs after receiving the file).
Try for example
./sage --sh -c 'sage-download-file ftp://ftp.fu-berlin.de/unix/misc/sage/spkg/optional/list'
comment:53 follow-up: ↓ 55 Changed 6 years ago by
Correct, FTP is not allowed
comment:54 in reply to: ↑ 51 Changed 6 years ago by
Replying to vbraun:
the official mirror list may or may not have a slash at the end
A matter of what your script outputs...
So, what about SAGE_SERVER
to specify a mirror? While $SOMEBODY
partially broke that anyway by hardcoding the incompatible packages/
subfolder, we should IMHO still (or again) support that. (At least sage-list-packages
also needs to get fixed, preferably on a separate ticket I'd say.)
comment:55 in reply to: ↑ 53 Changed 6 years ago by
Replying to vbraun:
Correct, FTP is not allowed
The single mandatory argument can be a http:// or ftp:// url or a tarball filename.
if url.startswith('http://') or url.startswith('https://') or url.startswith('ftp://'): http_download(url, destination, progress=progress, ignore_errors=True)
comment:56 follow-up: ↓ 58 Changed 6 years ago by
I don't really like the SAGE_SERVER name (could refer anything, sage cell server), but historically that was where the official mirror list was and thats what I wanted to keep
comment:57 follow-up: ↓ 59 Changed 6 years ago by
Also, IMHO we should just phase out old-style spkgs. As long as we have to drag along that technical debt its very hard to clean up scripts.
comment:58 in reply to: ↑ 56 Changed 6 years ago by
Replying to vbraun:
I don't really like the SAGE_SERVER name (could refer anything, sage cell server), but historically that was where the official mirror list was and thats what I wanted to keep
? Before the breakage, setting SAGE_SERVER
to e.g. http://mirror.switch.ch/mirror/sagemath/
worked, for example with sage --optional
and -- long time ago -- with sage --upgrade
.
Deprecate it, later use SAGE_MIRROR
for package download purposes?
comment:59 in reply to: ↑ 57 ; follow-up: ↓ 61 Changed 6 years ago by
Replying to vbraun:
Also, IMHO we should just phase out old-style spkgs.
I'm all against that. The worst decision was to hardcode package metadata (solely!) into specific Sage releases, while that data doesn't even show whether a package is optional, experimental, or whatever. (It's pretty easy to create "old-style" packages from that though... ;-) )
In my opinion the main reason we should (still) support real spkgs is that anybody can easily create and offer such self-contained packages to others, and installing and using them doesn't artificially depend on the Sage version. (Whether they'll work of course depends on a couple of things.)
They're also often much more convenient for testing.
As long as we have to drag along that technical debt its very hard to clean up scripts.
Perhaps with the exception of left-over old-style standard packages, I don't see a problem.
comment:60 Changed 6 years ago by
- Commit changed from 30b1647e7b6a527e72daa7835f26a710306a6099 to a103ab5dd842e5a6f24970a42649e4df8935a9da
comment:61 in reply to: ↑ 59 Changed 6 years ago by
Replying to leif:
Whether they'll work of course depends on a couple of things.
Even if they work in principle you can't do that much with them as there is no dependency management. Its just an all-out shitty solution for everyone.
comment:62 Changed 6 years ago by
SAGE_SERVER must of course then not be set by default...
comment:63 follow-up: ↓ 66 Changed 6 years ago by
Does any mirror have packages/
? Will they at some point?
Otherwise s|SAGE_SERVER/packages/|SAGE_SERVER/spkg/|
in the docs.
comment:64 Changed 6 years ago by
- Cc schilly added
AFAIK there will only ever be /spkg/ on the mirrors
comment:65 Changed 6 years ago by
- Commit changed from a103ab5dd842e5a6f24970a42649e4df8935a9da to a259f27b95e1b69fbdde1a4b0e8496996cd427fe
Branch pushed to git repo; I updated commit sha1. New commits:
a259f27 | rename SAGE_SERVER/packages to SAGE_SERVER/spkg
|
comment:66 in reply to: ↑ 63 Changed 6 years ago by
Replying to leif:
Does any mirror have
packages/
? Will they at some point?Otherwise
s|SAGE_SERVER/packages/|SAGE_SERVER/spkg/|
in the docs.
Should we fix this here, since we now touch sage-list-packages
anyway?
SAGE_SERVER="http://mirror.switch.ch/mirror/sagemath/" ./sage --optional Using Sage Server http://mirror.switch.ch/mirror/sagemath/packages HTTP Error 404: Not Found ******************************************************************************** Error contacting http://mirror.switch.ch/mirror/sagemath/packages/optional/list. Try using an alternative server. For example, from the bash prompt try typing export SAGE_SERVER=http://sage.scipy.org/sage/ then try again. ********************************************************************************
(http://sage.scipy.org/sage/ is btw. a joke...)
comment:67 Changed 6 years ago by
Concurrent posting (luckily not pushing)...
comment:68 Changed 6 years ago by
- Commit changed from a259f27b95e1b69fbdde1a4b0e8496996cd427fe to d79364b9fc5131199eaed14340582715f2f937fb
Branch pushed to git repo; I updated commit sha1. New commits:
d79364b | fix sage-list-packages
|
comment:69 follow-up: ↓ 71 Changed 6 years ago by
The whole listing of packages is also made hideously complicated by mashing old and new style packages together...
comment:70 Changed 6 years ago by
I thought of just adding a further try: ... except HTTPError: ...
, in case sagemath.org/packages/
remains for a while. (IIRC it had both for a while.)
PKG_SERVER
is superfluous anyway; I'd simply use os.environ['SAGE_SERVER']
there.
comment:71 in reply to: ↑ 69 Changed 6 years ago by
Replying to vbraun:
The whole listing of packages is also made hideously complicated by mashing old and new style packages together...
As I said, in principle only the old standard packages disturb a bit; otherwise it's just a matter of syncing.
I was actually working on that a few days ago (#16759).
comment:72 Changed 6 years ago by
Quoting jdemeyer:
Replying to jhpalmieri:
It also looks like
sage --standard
is broken: it is using old package information.I would say that
sage --standard
is doing the right thing, but somehowhttp://www.sagemath.org/packages/standard/
is no longer updated. The right solution is updating the website script to take into account the new git-style packages (which is certainly a different issue than the one this ticket is about).
comment:73 follow-up: ↓ 74 Changed 6 years ago by
Since I'm CCed: There is currently a move of the whole infrastructure going on. Before that I tried to keep everything compatible with the original system from a long time ago. That's no longer possible. Long story short: the website and the binary-files need to be untangled and that means there are no longer chains of symlink-aliases. Hence, don't use the /mirror
subdirectory of the website and I hope the essence of this ticket is to get rid of this old dependence. The only mirror related aspect is the mirror_list
file for the base URLs.
comment:74 in reply to: ↑ 73 Changed 6 years ago by
Replying to schilly:
Since I'm CCed: There is currently a move of the whole infrastructure going on. Before that I tried to keep everything compatible with the original system from a long time ago. That's no longer possible. Long story short: the website and the binary-files need to be untangled and that means there are no longer chains of symlink-aliases. Hence, don't use the
/mirror
subdirectory of the website and I hope the essence of this ticket is to get rid of this old dependence. The only mirror related aspect is themirror_list
file for the base URLs.
Well, the Sage scripts never used the mirror/
folder. The problematic part here is the structure below the base URL; spkg/
was changed to packages
on sagemath.org, but the mirrors all have the classical structure (plus upstream/
below spkg/
).
The new thing on this ticket is also that we now use http://www.sagemath.org/mirror_list (currently hardcoded) again, which is periodically generated by a script on the server (or at least used to be).
comment:75 Changed 6 years ago by
Harald, also note Jeroen's comment I quoted above.
Meanwhile, it's not just standard packages, but also optional packages, of which (only) some now have the "new style", i.e., the data of newer ones is hardcoded into the Sage releases, while obsolete "old-style" packages remain in spkg/optional/list
(and spkg/optional/
).
./sage --optional
shows some examples of optional packages which became standard:
Optional packages: ... git-1.7.12.2.p0 ........................ installed version: 2.3.0 ... pyzmq-2.1.11.p1 ........................ installed version: 14.3.0 ... zeromq-3.2.0.p0 ........................ installed version: 4.0.5
There are other still optional packages where sage -i ...
would install newer versions than listed by sage --optional
because the "local metadata" ($SAGE_ROOT/build/pkgs/<package>/package-version.txt
) differs from what spkg/optional/list
gives. (Same issue as with our current standard packages.)
A more funny one is TOPCOM:
./sage -i TOPCOM
installs the "legacy" spkg, while ./sage -i topcom
installs the "new-style" spkg. (The package versions are currently still the same though, modulo a .p0
for the "new-style" one.)
comment:76 follow-up: ↓ 77 Changed 6 years ago by
We definitely should not check sagemath.org/packages any more, I've already changed the release management scripts to no longer put the tarballs there.
comment:77 in reply to: ↑ 76 ; follow-up: ↓ 80 Changed 6 years ago by
Replying to vbraun:
We definitely should not check sagemath.org/packages any more, I've already changed the release management scripts to no longer put the tarballs there.
Ok, sagemath.org/spkg/
* gave a 404 for a while, but apparently works again.
comment:78 Changed 6 years ago by
Note for other reviewers: So far I'm ok with the changes/corrections made, but haven't yet tested thoroughly enough (I think).
While I don't like completely changing the previously generic character of sage-download-file
(nor removing the "shell fallback" with e.g. wget
or curl
if Python isn't available or functional), I strongly doubt we could convince the author or the RM to change that back to how it was.
FTP support (probably with Python3) and e.g. usage of a static mirror list (perhaps to be shipped with Sage) is something for follow-up tickets; the broken behaviour of sage-list-packages
is (or will be) dealt with at #16759, which depends on this ticket.
W.r.t. the threat of losing sagemath.org for downloads: According to William, "soon" is not that soon.
comment:79 Changed 6 years ago by
I don't mind if you rewrite the functionality in bash fwiw. I would consider it utterly retarded and backward, but as long as it works and you are committed to maintaining that solution then that would be fine with me.
comment:80 in reply to: ↑ 77 ; follow-up: ↓ 81 Changed 6 years ago by
Replying to leif:
Ok,
sagemath.org/spkg/
* gave a 404 for a while, but apparently works again.
Uhm, to reduce that confusion a little bit: The source of this inconsistency is that right now most of the old infrastructure is still running, including the currently "old" website. I would suggest to wait until the "new" website is operational. Then, any bugs in scripts relying on the website will be obvious (and fixes to the new website visible immediately).
The reason why this is a bit slow is, because there are many links assuming certain subdirectories (for documentation and e.g. /mirror/) and my time is limited. I could report back once the DNS entry has switched to the new page.
PS: "new" just means, that there are minor changes like an additional mirror, publications etc. The current "old" website is an older fork of the website with some other patches. Hence this diverged and there is no need to support the old one any more.
comment:81 in reply to: ↑ 80 ; follow-ups: ↓ 83 ↓ 92 Changed 6 years ago by
Replying to schilly:
I could report back once the DNS entry has switched to the new page.
ETA Hausnummer?
PS: "new" just means, that there are minor changes like an additional mirror, publications etc. The current "old" website is an older fork of the website with some other patches. Hence this diverged and there is no need to support the old one any more.
Could we already test the new one (using its unofficial address) w.r.t. download stuff / package (and mirror) lists?
comment:82 follow-ups: ↓ 84 ↓ 85 Changed 6 years ago by
The attached branch will only use www.sagemath.org to download the mirror list, so once we have merged it there are a lot fewer things that could break.
comment:83 in reply to: ↑ 81 Changed 6 years ago by
Replying to leif:
ETA Hausnummer?
Friday. It's tedious stupid work, so maybe I'm already whining more about it than it will actually take to work on.
Could we already test the new one (using its unofficial address) w.r.t. download stuff / package (and mirror) lists?
No, Github resolves the inofficial URL to the official one and I don't want to mess with this around even more.
comment:84 in reply to: ↑ 82 Changed 6 years ago by
Replying to vbraun:
The attached branch will only use www.sagemath.org to download the mirror list ...
Yes, it's just one file, and as already mentioned: the full list could easily be hardcoded as a backup into Sage to remove that SPOF. Ticket for later please :-)
comment:85 in reply to: ↑ 82 Changed 6 years ago by
Replying to vbraun:
once we have merged it there are a lot fewer things that could break.
Provided the mirrors can still sync...
In principle we could already include a snapshot of mirror_list
into the git tree (in $SAGE_ROOT/
? in more human-readable form?). The dynamic one should rather be located in $SAGE_ROOT/local/var/{lib,tmp}/sage/
, say, not $SAGE_DIST_FILES/
(defaults to $SAGE_ROOT/upstream/
).
comment:86 follow-up: ↓ 87 Changed 6 years ago by
Nothing else needs the mirror list so why put it in a separate file? In any case, this question should be dealt with in a separate ticket.
comment:87 in reply to: ↑ 86 Changed 6 years ago by
Replying to vbraun:
Nothing else needs the mirror list
Human users perhaps?
so why put it in a separate file?
You're right, all package metadata should be moved into sage-spkg
as well.
comment:88 Changed 6 years ago by
There are many packages but only one mirror list. Your argument is invalid.
Not to mention the question of what file format, custom = crap, json = not exactly human-readable, yaml = not necessarily available in the System python, is it supposed to be readable from a shell script, ...
comment:89 follow-up: ↓ 90 Changed 6 years ago by
Plain text, one URL per line, preferably in the top-level folder. What's the problem?
Any human and any kind of script can easily read such. (If you want to add further data such as organization, location etc., just put that after each URL, seperated by whitespace, but I wouldn't even do that. We don't have to process that. Blank lines and comment lines starting with #
at the first position would IMHO be ok, too, but that's not really needed. Order of the mirrors doesn't matter for our purposes either, although we could keep them sorted by continent, say.)
I thought we had postponed that to a follow-up ticket.
comment:90 in reply to: ↑ 89 ; follow-up: ↓ 91 Changed 6 years ago by
Replying to leif:
I thought we had postponed that to a follow-up ticket.
We have, but you keep bringing it up.
comment:91 in reply to: ↑ 90 Changed 6 years ago by
comment:92 in reply to: ↑ 81 ; follow-up: ↓ 94 Changed 6 years ago by
comment:93 Changed 6 years ago by
- Reviewers set to Volker Braun
- Status changed from needs_review to positive_review
comment:94 in reply to: ↑ 92 Changed 6 years ago by
Replying to schilly:
Replying to leif:
Replying to schilly:
I could report back once the DNS entry has switched to the new page.
ETA Hausnummer?
Yay! 2 days ahead of schedule! I fixed all the obviously broken links and there are just two minor details left.
sagemath.org no longer serves packages (especially not the upstream tarballs); is that intentional, or will it again? (Cf. sage-release.)
comment:95 Changed 6 years ago by
- Branch changed from u/vbraun/use_mirrors_for_tarballs to d79364b9fc5131199eaed14340582715f2f937fb
- Resolution set to fixed
- Status changed from positive_review to closed
Branch pushed to git repo; I updated commit sha1. New commits:
Merge tag '6.1.beta4' into 'u/vbraun/use_mirrors_for_tarballs'
py3 fix, also try testing packages