Opened 9 months ago

Closed 8 months ago

#33131 closed defect (fixed)

Installation manual: Add decision tree, remove mention of Sage-mirror-hosted binary distributions

Reported by: Matthias Köppe Owned by:
Priority: critical Milestone: sage-9.6
Component: distribution Keywords:
Cc: Dima Pasechnik, John Palmieri, Volker Braun, Mauricio Collares, Isuru Fernando Merged in:
Authors: Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 0f6ade5 (Commits, GitHub, GitLab) Commit: 0f6ade53850bb5f8ebad86582f54486028261449
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

The binary distributions on the Sage mirrors are unmaintained (#31133) and have been broken for a long time.

We update the manual by including a decision tree. It never leads to the recommendation to use these binary distributions because better alternatives are available for every use case.

Hence, we replace all details regarding binaries by a short notice that they have been discontinued.

PR to also remove mention of the binaries from the website:

Change History (55)

comment:1 Changed 9 months ago by Matthias Köppe

Description: modified (diff)

comment:2 Changed 9 months ago by Matthias Köppe

Branch: u/mkoeppe/installation_manual__remove_mention_of_binary_distributions

comment:3 Changed 9 months ago by Matthias Köppe

Authors: Matthias Koeppe
Commit: 480ec86cca5d5a2ce6acaaabee1a1f5939656a32

New commits:

480ec86src/doc/en/installation: Remove mention of binaries

comment:4 Changed 9 months ago by Matthias Köppe

Cc: Dima Pasechnik John Palmieri Volker Braun added
Status: newneeds_review

comment:5 Changed 9 months ago by Matthias Köppe

Description: modified (diff)

comment:6 Changed 9 months ago by Matthias Köppe

Priority: majorblocker

comment:7 Changed 9 months ago by Matthias Köppe

Description: modified (diff)

comment:8 Changed 9 months ago by Matthias Köppe

Cc: Mauricio Collares Isuru Fernando added

comment:9 Changed 9 months ago by Michael Orlitzky

Huge +1 from me. If you want a binary, you should be getting it with (say) apt-get.

comment:10 Changed 9 months ago by Dima Pasechnik

yes! +100

comment:11 Changed 9 months ago by Emmanuel Charpentier

Binary distributions are a "necessary evil" for platforms where building Sage can be considered as needing "system fu" out of proportion of "normal" using Sage fu.

Cases in point :

  • Windows

Notwithstanding E. Madison Bray and others' gigantic efforts, building the Cygwin port of Sage is difficult ; probably too difficult for casual Windows users. The problem is that this puts Sage above the reach of at least 90% of undergraduate science/math students, i. e. our core target audience for future users.

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

  • Mac

Not approaching a Mac at least than ten yards, I can't possibly knowledgeably comment on this platform ; I am still impressed by the sheer volume of Mac-specific or Mac-related problems that flourish on sage-release at each release...

Again, this platform represents a non-negligible part of our target audience.

These two platforms are a special case, requiring IMHO special consideration...

comment:12 in reply to:  9 Changed 9 months ago by Emmanuel Charpentier

Replying to mjo:

Huge +1 from me. If you want a binary, you should be getting it with (say) apt-get.

Agreed. For Linux (Unix) users.

Which are nowadays a dwindling minority (if an elite one, of course ;-)). See below my remarks on Windows and Macs...

And, BTW, a Debian package is a binary distribution...

comment:13 in reply to:  11 Changed 9 months ago by Dima Pasechnik

Replying to charpent:

Binary distributions are a "necessary evil" for platforms where building Sage can be considered as needing "system fu" out of proportion of "normal" using Sage fu.

Cases in point :

  • Windows

use Docker.

  • Mac

Not approaching a Mac at least than ten yards, I can't possibly knowledgeably comment on this platform ; I am still impressed by the sheer volume of Mac-specific or Mac-related problems that flourish on sage-release at each release...

Again, this platform represents a non-negligible part of our target audience.

for Mac there is a binary installer provided by Snappy group.

comment:14 Changed 9 months ago by Jan Groenewald

The ticket is to remove it from the *installation documentation*. I'm absolutely fine with that if it improves the situation for developers and users.

I would still like *access to easily download* binary distributions.

I recently on Linux (Debian) had problems building a newer version than apt but th ebinary distribution worked fine. I also do have problems with the numpy SIGILL on older CPUs (which the Debian versions don't have; don't know why).

I'd love to have Debian backports and sloppy-backports or third party (preferably sagemath not launchpad) repositories with latest sagemath packages for Debian releases and Ubuntu LTSes at least. Might be a tall order for volunteers to maintain.

comment:15 in reply to:  14 ; Changed 9 months ago by Michael Orlitzky

Replying to pipedream:

The ticket is to remove it from the *installation documentation*. I'm absolutely fine with that if it improves the situation for developers and users.

I would still like *access to easily download* binary distributions.

That's not an option either way:

There are many unfixed problems -- some fundamental -- and to put it bluntly, nobody cares. This ticket just changes the documentation to reflect reality, and perhaps in the future will allow us to not be held back by "what about the binaries?!" arguments.

comment:16 Changed 9 months ago by Thierry Monteil

I think the question is not whether the binaries are maintained or not on our side but whether they are useful to users. I used to build up to 16 binaries at each release some years ago, then stopped since Jan Groenwald at AIMS dedicated some machines for that. Maybe he stopped after many years, but it is not an individual question. If those binaries are useful, then we should put some energy to dedicate some infrastructure for them, not the other way. I have the impression that the same happened with gitlab workers (they were usefull but since they were unmaintained, we stopped them).

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

comment:17 in reply to:  16 Changed 9 months ago by Michael Orlitzky

Replying to tmonteil:

I think the question is not whether the binaries are maintained or not on our side but whether they are useful to users.

It would be useful to our users if they could hop on a zoom call and have a sage developer personally install sage for them. But is that a good use of our time? Is anyone actually going to volunteer to do it?

I would say it's the other way around: if no one is willing to fix all of the bugs and commit to producing and supporting the sage binary releases, then the question of whether or not it would be useful is irrelevant. The source (or distro-based) installations, on the other hand, are actively developed and supported.

comment:18 in reply to:  16 ; Changed 9 months ago by Matthias Köppe

Replying to tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

comment:19 in reply to:  11 Changed 9 months ago by Matthias Köppe

Replying to charpent:

  • Windows

This ticket makes no changes regarding this platform.

comment:20 Changed 9 months ago by Matthias Köppe

Description: modified (diff)

comment:21 Changed 9 months ago by Matthias Köppe

Summary: Installation manual: Remove mention of binary distributionsInstallation manual: Remove mention of Sage-mirror-hosted binary distributions

comment:22 Changed 9 months ago by William Stein

"We direct people instead to: conda-forge, nix, docker, build from source."

It would be nice to include all the known ways to run SageMath online as well, e.g., Sage cell server (https://sagecell.sagemath.org/), CoCalc (https://cocalc.com), etc. Actually, I don't know of any others, but maybe somebody could make a mybinder (https://mybinder.org/) of Sage, if it doesn't exceed the size limits.

Last edited 9 months ago by William Stein (previous) (diff)

comment:23 in reply to:  22 Changed 9 months ago by Samuel Lelièvre

Replying to was:

It would be nice to include all the known ways to run SageMath online as well, e.g., Sage cell server (https://sagecell.sagemath.org/), CoCalc (https://cocalc.com), etc. Actually, I don't know of any others, but maybe somebody could make a mybinder (https://mybinder.org/) of Sage, if it doesn't exceed the size limits.

I once made a list but it was missing actual Binder links:

OpenDreamKit's "try" page

and the "More SageMath tutorials" page

both use the same Binder link, which currently does not work.

Might not be that hard to fix...

comment:24 in reply to:  description ; Changed 9 months ago by Volker Braun

Replying to mkoeppe:

We direct people instead to

  • conda-forge
  • nix
  • docker

That would be nice but the proposed patch mentions neither, nor an distro-specific installation. There are surely issues with the binary packages but they can work for some whereas the current patch isn't helpful at all.

At the very least we should have some eli5 instructions for how to get conda up and running and then install the conda sagemath package. Since conda is really the only way that works without root permissions I'd put that front and center.

comment:25 Changed 9 months ago by Volker Braun

Priority: blockermajor

comment:26 in reply to:  11 ; Changed 9 months ago by Tobias Diez

Replying to charpent:

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

Setting up sage with WSL should be pretty straightforward. First install WSL with Ubuntu using the Windows store (at least on Win 11, https://aka.ms/wslstorepage, otherwise can be done using a few cmds) and then use apt get to install sage in it.

---

+1 for removing binary distributions for linux, cannot say anything about mac.

---

The information for how to install sage is pretty scattered (wiki, website, docs, readme, ...). I think this should be combined in one place, listing the preferred installation method for each system and describe alternative installation methods as subpages.

comment:27 Changed 9 months ago by Dima Pasechnik

our mac binaries are really broken - and there is an alternative.

comment:28 in reply to:  24 Changed 9 months ago by Matthias Köppe

Replying to vbraun:

Replying to mkoeppe:

We direct people instead to

  • conda-forge
  • nix
  • docker

That would be nice but the proposed patch mentions neither

... because all of these are already in the manual.

comment:29 in reply to:  18 ; Changed 9 months ago by Thierry Monteil

Replying to mkoeppe:

Replying to tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

comment:30 in reply to:  26 ; Changed 9 months ago by Thierry Monteil

Replying to gh-tobiasdiez:

Replying to charpent:

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

Setting up sage with WSL should be pretty straightforward. First install WSL with Ubuntu using the Windows store (at least on Win 11, https://aka.ms/wslstorepage, otherwise can be done using a few cmds) and then use apt get to install sage in it.

I had a look some times ago, but i could not find anything (prehaps because i am not user to windows) : it is possible to produce an single installer (.exe), that does the job of activating WSL, installing a GNU/Linux distro, install Sage in it, and add a launcher on the desktop ?

comment:31 Changed 9 months ago by git

Commit: 480ec86cca5d5a2ce6acaaabee1a1f5939656a3293df828b4e2a001ef3b0d9c433d183f6f242d778

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b78260bsrc/doc/en/installation: Remove mention of binaries
93df828src/doc/en/installation/index.rst: Replace section by a decision tree

comment:32 Changed 9 months ago by git

Commit: 93df828b4e2a001ef3b0d9c433d183f6f242d7785af3d04950308af26987dad4a0b7c8fcff7a7e7f

Branch pushed to git repo; I updated commit sha1. New commits:

5af3d04src/doc/en/installation/index.rst: Add header for toc

comment:33 Changed 9 months ago by git

Commit: 5af3d04950308af26987dad4a0b7c8fcff7a7e7f76d2d2a03e5f6cf5f6f74f81b2c2909074514904

Branch pushed to git repo; I updated commit sha1. New commits:

76d2d2asrc/doc/en/installation/index.rst: Typos

comment:34 Changed 9 months ago by Matthias Köppe

Description: modified (diff)
Summary: Installation manual: Remove mention of Sage-mirror-hosted binary distributionsInstallation manual: Add decision tree, remove mention of Sage-mirror-hosted binary distributions

comment:35 Changed 9 months ago by Matthias Köppe

Priority: majorcritical

comment:36 in reply to:  29 ; Changed 9 months ago by Dima Pasechnik

Replying to tmonteil:

Replying to mkoeppe:

Replying to tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

ha! it needs all that Apple codesigning setup noone is willing to duplicate, it also is not free...

comment:37 Changed 9 months ago by Matthias Köppe

https://wiki.sagemath.org/Distribution has some good information, some of which may not be overly outdated.

comment:38 Changed 9 months ago by git

Commit: 76d2d2a03e5f6cf5f6f74f81b2c2909074514904c1f4e468e65a2796923e343543d7093a0c52bfe8

Branch pushed to git repo; I updated commit sha1. New commits:

c1f4e46src/doc/en/installation/index.rst: Link also to https://wiki.sagemath.org/Conda

comment:39 Changed 9 months ago by git

Commit: c1f4e468e65a2796923e343543d7093a0c52bfe80f6ade53850bb5f8ebad86582f54486028261449

Branch pushed to git repo; I updated commit sha1. New commits:

0f6ade5src/doc/en/installation/linux.rst: Link to repology and https://wiki.sagemath.org/Distribution

comment:40 in reply to:  22 Changed 9 months ago by Matthias Köppe

Replying to was:

"We direct people instead to: conda-forge, nix, docker, build from source."

It would be nice to include all the known ways to run SageMath online as well, e.g., Sage cell server (https://sagecell.sagemath.org/), CoCalc (https://cocalc.com)

These 2 options were already mentioned in the manual, still are after the rewrite done in the ticket

comment:41 Changed 9 months ago by Matthias Köppe

Description: modified (diff)

comment:42 Changed 9 months ago by Samuel Lelièvre

For macOS, I would change:

-      You can choose between two options: A 800 MB disk image
-      with the standard Sage distribution,
-      and a 1 GB disk image that adds many optional packages.
+      You can choose between two options: the standard Sage
+      distribution (800 MB disk image, expands to nearly 3 GB),
+      or an extended distribution with many optional packages
+      (1 GB disk image, expands to nearly 3.5 GB).

comment:43 Changed 9 months ago by Matthias Köppe

Please feel free to push improvements to the ticket.

comment:44 in reply to:  30 Changed 9 months ago by Tobias Diez

Replying to tmonteil:

Replying to gh-tobiasdiez:

Replying to charpent:

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

Setting up sage with WSL should be pretty straightforward. First install WSL with Ubuntu using the Windows store (at least on Win 11, https://aka.ms/wslstorepage, otherwise can be done using a few cmds) and then use apt get to install sage in it.

I had a look some times ago, but i could not find anything (prehaps because i am not user to windows) : it is possible to produce an single installer (.exe), that does the job of activating WSL, installing a GNU/Linux distro, install Sage in it, and add a launcher on the desktop ?

I'm not aware of such a possibility. What one could do is to create a custom WSL image (with sage pre-installed), which people could then use. But in order to distribute this in the store, one would need to contact microsoft: https://docs.microsoft.com/en-us/windows/wsl/build-custom-distro#for-linux-distro-maintainers

Another possibility is to install sage as a graphical app in WSLg (https://github.com/microsoft/wslg). For Win11, this then adds a launcher in the Windows start menu.

comment:45 in reply to:  36 ; Changed 9 months ago by Tobias Diez

Replying to dimpase:

Replying to tmonteil:

Replying to mkoeppe:

Replying to tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

ha! it needs all that Apple codesigning setup noone is willing to duplicate, it also is not free...

If you are a non-profit org, you don't have to pay for the developer account (which is needed for the codesigning): https://developer.apple.com/support/membership-fee-waiver/ Not sure if sage officially falls under this, but maybe worth a try?

comment:46 in reply to:  45 Changed 9 months ago by Dima Pasechnik

Replying to gh-tobiasdiez:

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

ha! it needs all that Apple codesigning setup noone is willing to duplicate, it also is not free...

If you are a non-profit org, you don't have to pay for the developer account (which is needed for the codesigning): https://developer.apple.com/support/membership-fee-waiver/ Not sure if sage officially falls under this, but maybe worth a try?

Unless we have a volunteer who's willing to jump through all the hoops to set this all up (we don't have a proper non-profit set up, despite talking about it for years - that's why in particular our donations (GitHub and "direct") are received via OpenCollective, and not in a more direct and cost-efficient way), it's all idle chatter.

Same for a volunteer to actually get on with the Apple developer cerificates, setting the necessary builds up, etc. (as this is all targeting developing via XCode, it's hit and miss with command line).

Last edited 9 months ago by Dima Pasechnik (previous) (diff)

comment:47 in reply to:  15 ; Changed 9 months ago by Karl-Dieter Crisman

Replying to mjo:

Replying to pipedream: There are many unfixed problems -- some fundamental -- and to put it bluntly, nobody cares.

As per usual in these arguments, what this means is, "nobody with the expertise to maintain binary distributions cares enough to put in the nontrivial work it demands", which isn't quite the same thing.

Anyway, if someone with the expertise of pipedream aka Jan G. is asking about Linux binaries, then it should go without saying that we need some sustainable, reliable way to provide Mac and Windows binaries (unless we assume everyone is using CoCalc, which I do not think should be the assumption). If Nathan and the Snappy folks are committed to this on Mac and that is easy to advertise everywhere it needs to be, great. From the discussion above, it sounds like the Windows situation is much less stable, and so comment:11 still stands. How easy is the Docker image to use by someone who is expecting a "download, then double-click" interface? If it's no harder to use than our most recent Cygwin-based one and is sustainable to produce, great. And all this has to be easy to find off the main Sage home page. (An advantage to removing from mirrors is people won't get confused by the mirror selection process.)

Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)

LOL. But interestingly, at UCLA apparently a lot of the latter had used Sage in a lab context (see the book), but I'm not so sure now - see this paper where I only see "Python" not "Sage".

comment:48 Changed 9 months ago by William Stein

LOL. But interestingly, at UCLA apparently a lot of the latter had used Sage in a lab context (see ​the book), but I'm not so sure now - see ​this paper where I only see "Python" not "Sage".

I spent a day with them once. The first few hits for this search

https://www.youtube.com/results?search_query=william+stein+sagemathcloud+ucla

are to some short videos that came out of that.

The answer is that they use CoCalc? (previous called SageMathCloud?) with Sage worksheets and Jupyter notebooks, and have done so since 2014. They do not install Sage locally.

comment:49 in reply to:  47 Changed 9 months ago by Matthias Köppe

Replying to kcrisman:

it sounds like the Windows situation is much less stable, and so comment:11 still stands.

The present ticket improves the Windows situation by giving a clear recommendation by way of the decision tree.

Needs review.

comment:50 in reply to:  47 Changed 9 months ago by Matthias Köppe

Replying to kcrisman:

the Snappy folks are committed to this on Mac and that is easy to advertise everywhere it needs to be, great.

The present ticket improves the macOS situation by doing exactly this, advertising.

Needs review

comment:51 Changed 9 months ago by Dima Pasechnik

Reviewers: Dima Pasechnik
Status: needs_reviewpositive_review

that's fine I think.

comment:52 Changed 9 months ago by Matthias Köppe

For improved usability of Sage-in-WSL, let's continue in #30484.

comment:53 Changed 9 months ago by Samuel Lelièvre

Found this SageMath + Kenzo mybinder.org link

associated with this article:

  • Julián Cuevas-Rozo; Jose Divasón; Miguel Marco-Buzunáriz; Ana Romero. Integration of the Kenzo System within SageMath for New Algebraic Topology Computations. Mathematics 2021, 9(7), 722. DOI:10.3390/math9070722

but trying it failed for me.

comment:54 Changed 8 months ago by Samuel Lelièvre

Milestone: sage-9.5sage-9.6

Setting milestone to 9.6 now that 9.5 is out.

comment:55 Changed 8 months ago by Volker Braun

Branch: u/mkoeppe/installation_manual__remove_mention_of_binary_distributions0f6ade53850bb5f8ebad86582f54486028261449
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.