Opened 7 weeks ago

Last modified 2 weeks ago

#34363 needs_review enhancement

Installation guide: Document Sage installation on Windows via VS Code devcontainers

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-9.8
Component: documentation Keywords:
Cc: Travis Scrimshaw, Kwankyu Lee, Samuel Lelièvre, Darij Grinberg, Tobias Diez, Julian Rüth, William Stein, David Ayotte Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/installation_guide__document_sage_installation_on_windows_via_vs_code_devcontainers (Commits, GitHub, GitLab) Commit: 74448c0e6679e16160a85c79d2362fc1e3a71e89
Dependencies: #33671, #34301 Stopgaps:

Status badges

Change History (49)

comment:1 Changed 7 weeks ago by Matthias Köppe

Branch: u/mkoeppe/installation_guide__document_sage_installation_on_windows_via_vs_code_devcontainers

comment:2 Changed 7 weeks ago by Matthias Köppe

Authors: Matthias Koeppe
Cc: Travis Scrimshaw Kwankyu Lee Samuel Lelièvre added
Commit: 1ca0876539fc36bbc1787d7d5a9bbcb90f9954a5
Description: modified (diff)
Status: newneeds_review

Last 10 new commits:

c5d883csrc/doc/en/developer/portability_testing.rst: Explain the devcontainer name prefix downstream-docker-...
3016ce9build/bin/sage-print-system-package-command (conda): Actually include options in command
84a5e13.devcontainer/downstream-conda-forge-latest/devcontainer.json: Do not use .devcontainer/post_create.sh
193174bMerge #33671
393808csrc/doc/en/installation/binary.rst: No more cygwin binaries
fe98ca1README.md, src/doc/en/installation: Move all Cygwin instructions to one section of the installation manual
550b337README.md, src/doc/en/installation/index.rst: Remove mention of Cygwin
e3af6c9src/doc/en/installation/index.rst: Fix markup
0bd4f7fMerge #34301
1ca0876src/doc/en/installation/index.rst, README.md [Windows]: Recommend VS Code devcontainers

comment:3 Changed 7 weeks ago by Matthias Köppe

Cc: Darij Grinberg added

comment:4 Changed 7 weeks ago by Matthias Köppe

Description: modified (diff)

comment:5 Changed 7 weeks ago by Matthias Köppe

Cc: Tobias Diez added

comment:6 Changed 7 weeks ago by git

Commit: 1ca0876539fc36bbc1787d7d5a9bbcb90f9954a5b442a418167f543f639a7b5fe049c990e3c1853c

Branch pushed to git repo; I updated commit sha1. New commits:

945de29.devcontainer: Reduce parallelism to 4
b29dfa7.devcontainer/downstream-docker-sagemath: Remove
b442a41Merge #33671

comment:7 Changed 7 weeks ago by Matthias Köppe

Cc: Julian Rüth William Stein added

comment:8 Changed 7 weeks ago by David Ayotte

Suggestion: in the alternative installation, I think I would mention that you can also use vscode, but now with extension Remote - WSL which lets you use vscode inside your WSL environment (but this is not mandatory, as someone might want to use an other code editor).

Next, I have a question: is there any performance advantages in using docker containers instead of directly running it inside WSL?

Thank you very much for simplifying the installation tutorial!

comment:9 Changed 7 weeks ago by David Ayotte

Cc: David Ayotte added

comment:10 Changed 7 weeks ago by git

Commit: b442a418167f543f639a7b5fe049c990e3c1853c8231674991b27ceda3d87a80c4869bf198a76e34

Branch pushed to git repo; I updated commit sha1. New commits:

8231674src/doc/en/installation/index.rst (Windows WSL): Link to launching section for VS Code Jupyter

comment:11 in reply to:  8 Changed 7 weeks ago by Matthias Köppe

Replying to gh-DavidAyotte:

Suggestion: in the alternative installation, I think I would mention that you can also use vscode, but now with extension Remote - WSL which lets you use vscode inside your WSL environment (but this is not mandatory, as someone might want to use an other code editor).

Thanks for the feedback!

We already have a section in the manual that explains this. I've linked it.

comment:12 in reply to:  8 ; Changed 7 weeks ago by Matthias Köppe

Replying to gh-DavidAyotte:

is there any performance advantages in using docker containers instead of directly running it inside WSL?

I would think it's the same as running WSL 2 directly

comment:13 Changed 7 weeks ago by git

Commit: 8231674991b27ceda3d87a80c4869bf198a76e3406610b813e606a665708e7a19eb1417484cb5b48

Branch pushed to git repo; I updated commit sha1. New commits:

b09e6d1.devcontainer/downstream-archlinux-latest/devcontainer.json: Remove a 'prefix' symlink if present
f99f09f.devcontainer/downstream-archlinux-latest/devcontainer.json: No need to bootstrap
b72ff17Refinements for downstream-archlinux-latest
9215621Copy instructions for downstream-conda-latest
de05f69Copy instructions for downstream-*
43abc57.devcontainer/develop-docker-cocalc, portability-centos-7-devtoolset-gcc_11-standard: Remove
06610b8Merge #33671

comment:14 in reply to:  12 ; Changed 7 weeks ago by Tobias Diez

Replying to mkoeppe:

Replying to gh-DavidAyotte:

is there any performance advantages in using docker containers instead of directly running it inside WSL?

I would think it's the same as running WSL 2 directly

In general, Docker is slower (for both io and cpu) then WSL2. How much depends on the particular use case and the setup (e.g. whether the files are located in the windows or linux env).

comment:15 Changed 7 weeks ago by Tobias Diez

In general, I'm skeptical if the devcontainer env should be indeed the recommended installation and development method on windows.

For installation, it is not clear to me how you would use the "installed" devcontainer in your own projects (i.e. outside of the vscode instance that hosts the sage source tree). For this, wouldn't it make more sense to recommend that people create a devcontainer file in their own project folder, that references a prebuild sage devcontainer?

For development, I would still favor wsl over docker for the performance. Moreover, I don't see why this should be a windows-specific recommendation in either case.

comment:16 in reply to:  14 ; Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

Replying to mkoeppe:

Replying to gh-DavidAyotte:

is there any performance advantages in using docker containers instead of directly running it inside WSL?

I would think it's the same as running WSL 2 directly

In general, Docker is slower (for both io and cpu) then WSL2. How much depends on the particular use case and the setup (e.g. whether the files are located in the windows or linux env).

Even when using Docker's WSL2 backend?

comment:17 in reply to:  15 ; Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

I don't see why this should be a windows-specific recommendation in either case.

Because on macOS and Linux, it is better to run it natively

comment:18 in reply to:  15 ; Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

For installation, it is not clear to me how you would use the "installed" devcontainer in your own projects (i.e. outside of the vscode instance that hosts the sage source tree). For this, wouldn't it make more sense to recommend that people create a devcontainer file in their own project folder, that references a prebuild sage devcontainer?

Yes, it would also make sense to do that (not instead, but in addition to what is done here). We have a bit of this already in the "portability testing" section added in #33671; see also #34286 (Cookiecutter for Sage user projects with devcontainer).

comment:19 Changed 7 weeks ago by git

Commit: 06610b813e606a665708e7a19eb1417484cb5b48dd5724ec83eaad6b6b4ba8e4349a0bea03b41f0c

Branch pushed to git repo; I updated commit sha1. New commits:

dd5724e.devcontainer/post_create.sh: Install bootstrapping prerequisites

comment:20 Changed 7 weeks ago by git

Commit: dd5724ec83eaad6b6b4ba8e4349a0bea03b41f0c5097583b42b1ea50a5acd836b37f6681497d6f01

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

bad3c16.devcontainer/post_create.sh: Install bootstrapping prerequisites
5097583Merge #33671

comment:21 in reply to:  18 ; Changed 7 weeks ago by Tobias Diez

Replying to mkoeppe:

Replying to gh-tobiasdiez:

For installation, it is not clear to me how you would use the "installed" devcontainer in your own projects (i.e. outside of the vscode instance that hosts the sage source tree). For this, wouldn't it make more sense to recommend that people create a devcontainer file in their own project folder, that references a prebuild sage devcontainer?

Yes, it would also make sense to do that (not instead, but in addition to what is done here). We have a bit of this already in the "portability testing" section added in #33671; see also #34286 (Cookiecutter for Sage user projects with devcontainer).

Can you then please explain and document how to use the sage installation (in the devcontainer) in one's own projects. For example, it is not clear to me how to select the sage python interpreter or juptyer kernel in another instance of vscode.

comment:22 in reply to:  17 ; Changed 7 weeks ago by Tobias Diez

Replying to mkoeppe:

Replying to gh-tobiasdiez:

I don't see why this should be a windows-specific recommendation in either case.

Because on macOS and Linux, it is better to run it natively

Why? The current windows installation is wsl + as you normally install in linux. The thing that you replace now is "normally install in linux" by "use a docker container", while keeping the wsl part. So I don't see why we shouldn't recommend using a docker container also for normal linux dev/install then.

comment:23 in reply to:  16 ; Changed 7 weeks ago by Tobias Diez

Replying to mkoeppe:

Replying to gh-tobiasdiez:

Replying to mkoeppe:

Replying to gh-DavidAyotte:

is there any performance advantages in using docker containers instead of directly running it inside WSL?

I would think it's the same as running WSL 2 directly

In general, Docker is slower (for both io and cpu) then WSL2. How much depends on the particular use case and the setup (e.g. whether the files are located in the windows or linux env).

Even when using Docker's WSL2 backend?

Yes, although the overhead is smaller then with the old legacy hyperv backend.

comment:24 in reply to:  23 Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

Even when using Docker's WSL2 backend?

Yes, although the overhead is smaller then with the old legacy hyperv backend.

Do you have a reference for this at hand so we can add it to the installation guide?

comment:25 in reply to:  22 ; Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

Replying to mkoeppe:

Replying to gh-tobiasdiez:

I don't see why this should be a windows-specific recommendation in either case.

Because on macOS and Linux, it is better to run it natively

Why? The current windows installation is wsl + as you normally install in linux. The thing that you replace now is "normally install in linux" by "use a docker container", while keeping the wsl part. So I don't see why we shouldn't recommend using a docker container also for normal linux dev/install then.

The difference is that neither WSL nor Docker-in-WSL are native Windows.

comment:26 in reply to:  25 Changed 7 weeks ago by Tobias Diez

Replying to mkoeppe:

Replying to gh-tobiasdiez:

Replying to mkoeppe:

Replying to gh-tobiasdiez:

I don't see why this should be a windows-specific recommendation in either case.

Because on macOS and Linux, it is better to run it natively

Why? The current windows installation is wsl + as you normally install in linux. The thing that you replace now is "normally install in linux" by "use a docker container", while keeping the wsl part. So I don't see why we shouldn't recommend using a docker container also for normal linux dev/install then.

The difference is that neither WSL nor Docker-in-WSL are native Windows.

But that should be pretty irrelevant from a users/devs perspective. You will get the same advantages (and disadvantages) from the Docker-in-WSL solution over pure WSL, as you would get Docker in Linux over pure Linux, or not?

comment:27 Changed 7 weeks ago by Matthias Köppe

I see the key benefit of the devcontainer solution for Windows that it isolates the Windows user from having to learn about WSL, Linux distributions etc. So it reduces the learning curve for people who want to get started with Sage.

Of course, devcontainers also have general benefits that make them a good option for consideration on other platforms. But I don't see a reason why we would recommend it to users over what we recommend now - the build from source.

comment:28 Changed 7 weeks ago by git

Commit: 5097583b42b1ea50a5acd836b37f6681497d6f0175c1173df41874fb7427cf23135881e316777d18

Branch pushed to git repo; I updated commit sha1. New commits:

75c1173src/doc/en/installation/index.rst (Windows): Say why and to whom we recommend the devcontainer method

comment:29 Changed 7 weeks ago by Tobias Diez

Status: needs_reviewneeds_work

I would say that reproducibility, not changing the developers system, ease of setting up as well as your point that potential users and devs don't need to learn how to manage packages using the cli are sufficiently convincing to make the devcontainer the recommended installation method on all systems. If they are not sufficient, then it shouldn't be the default recommendation (on neither system).

In either case, comment:21 is still not addressed.

comment:30 in reply to:  29 Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

I would say that [...] are sufficiently convincing to make the devcontainer the recommended installation method on all systems.

I get that you are convinced by it, but it's a long stretch to assume that the users would like that.

Last edited 7 weeks ago by Matthias Köppe (previous) (diff)

comment:31 in reply to:  29 Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

If they are not sufficient, then it shouldn't be the default recommendation (on neither system).

No, this does not follow because of what I have explained in comment:27.

comment:32 in reply to:  21 ; Changed 7 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

Replying to mkoeppe:

Replying to gh-tobiasdiez:

For installation, it is not clear to me how you would use the "installed" devcontainer in your own projects (i.e. outside of the vscode instance that hosts the sage source tree). For this, wouldn't it make more sense to recommend that people create a devcontainer file in their own project folder, that references a prebuild sage devcontainer?

Yes, it would also make sense to do that (not instead, but in addition to what is done here). We have a bit of this already in the "portability testing" section added in #33671; see also #34286 (Cookiecutter for Sage user projects with devcontainer).

Can you then please explain and document how to use the sage installation (in the devcontainer) in one's own projects. For example, it is not clear to me how to select the sage python interpreter or juptyer kernel in another instance of vscode.

Users who just start with Sage do not need any of this. They can just run Sage in the devcontainer of SAGE_ROOT and save their Jupyter notebooks somewhere, maybe in a subdirectory.

Of course, we know that for running the downstream-... devcontainer, one does not really need the whole source tree of Sage. But it gives a convenient transition: If it turns out they do have to make changes to Sage, they can just switch from a downstream-... devcontainer to a develop-... devcontainer without having to change the rest of their workflow.

Creating a "project" using Sage is far beyond what most Sage users do: Note that there are only a few dozens of published user packages ("projects") out there. #34286 is about making this easier.

comment:33 Changed 7 weeks ago by git

Commit: 75c1173df41874fb7427cf23135881e316777d18cec1e84fff22c1cfecc6f795de130b7894d9b77f

Branch pushed to git repo; I updated commit sha1. New commits:

cec1e84src/doc/en/installation/index.rst: Explain when to use manual WSL

comment:34 Changed 7 weeks ago by Matthias Köppe

Status: needs_workneeds_review

How about this?

comment:35 Changed 7 weeks ago by git

Commit: cec1e84fff22c1cfecc6f795de130b7894d9b77f8aa2cc4e7ae23f72212850f455af397a38ab31e2

Branch pushed to git repo; I updated commit sha1. New commits:

8aa2cc4src/doc/en/installation/index.rst (Windows): Change from Recommended/Alternative to Recommended 1/2

comment:36 Changed 7 weeks ago by Kwankyu Lee

The doc structure could be, as for other platforms,

   No development:

       Alternative 1:

       Alternative 2:

   Development:

       Alternative 1:

       Alternative 2:

We may put the recommended in alternative 1."No development alternative 1" would be the recommended approach for most users.

I would put "VS Code + downstream-devcontainer" approach to "No development alternative 1".

I would put "VS Code + WSL" approach to "Development alternative 1". Docker is really not necessary in sage development and conceptually more difficult than WSL, I think.

For "VS Code + downstream-devcontainer" approach, could we remove the git clone part (which is just to get devcontainer.json)?

comment:37 in reply to:  36 ; Changed 7 weeks ago by Kwankyu Lee

Replying to klee:

For "VS Code + downstream-devcontainer" approach, could we remove the git clone part (which is just to get devcontainer.json)?

Providing a zipped folder with ./devcontainer/devcontainer.json for download?

comment:38 in reply to:  37 Changed 7 weeks ago by Matthias Köppe

Replying to klee:

Replying to klee:

For "VS Code + downstream-devcontainer" approach, could we remove the git clone part (which is just to get devcontainer.json)?

Providing a zipped folder with ./devcontainer/devcontainer.json for download?

I would like to keep such things for a follow-up ticket (#34286)

comment:39 in reply to:  32 ; Changed 6 weeks ago by Tobias Diez

Replying to mkoeppe:

Replying to gh-tobiasdiez:

Replying to mkoeppe:

Replying to gh-tobiasdiez:

For installation, it is not clear to me how you would use the "installed" devcontainer in your own projects (i.e. outside of the vscode instance that hosts the sage source tree). For this, wouldn't it make more sense to recommend that people create a devcontainer file in their own project folder, that references a prebuild sage devcontainer?

Yes, it would also make sense to do that (not instead, but in addition to what is done here). We have a bit of this already in the "portability testing" section added in #33671; see also #34286 (Cookiecutter for Sage user projects with devcontainer).

Can you then please explain and document how to use the sage installation (in the devcontainer) in one's own projects. For example, it is not clear to me how to select the sage python interpreter or juptyer kernel in another instance of vscode.

Users who just start with Sage do not need any of this. They can just run Sage in the devcontainer of SAGE_ROOT and save their Jupyter notebooks somewhere, maybe in a subdirectory.

Of course, we know that for running the downstream-... devcontainer, one does not really need the whole source tree of Sage. But it gives a convenient transition: If it turns out they do have to make changes to Sage, they can just switch from a downstream-... devcontainer to a develop-... devcontainer without having to change the rest of their workflow.

Creating a "project" using Sage is far beyond what most Sage users do: Note that there are only a few dozens of published user packages ("projects") out there. #34286 is about making this easier.

Sorry for being imprecise. I meant "project" in the sense of "anything that lives outside of the sagemath folder", e.g. ones own jupyter notebooks or similar stuff. To make it precise: how do I contribute to https://github.com/sagemanifolds/SageManifolds using the devcontainer approach?

comment:40 in reply to:  36 Changed 6 weeks ago by Tobias Diez

Replying to klee:

The doc structure could be, as for other platforms,

   No development:

       Alternative 1:

       Alternative 2:

   Development:

       Alternative 1:

       Alternative 2:

We may put the recommended in alternative 1."No development alternative 1" would be the recommended approach for most users.

I would put "VS Code + downstream-devcontainer" approach to "No development alternative 1".

I would put "VS Code + WSL" approach to "Development alternative 1". Docker is really not necessary in sage development and conceptually more difficult than WSL, I think.

For "VS Code + downstream-devcontainer" approach, could we remove the git clone part (which is just to get devcontainer.json)?

In general, I like the idea. Maybe we can go even one step further and remove the numbering completely, as there is no clear general hierarchy between the options, but the preferred one depends on the situation. So simply something like "Option: Devcontainer" and "Option: Manual compilation from source in WSL".

I would also move the good clarifications by Matthias on when to use which option to the beginning of the decision tree, not after the steps.

comment:41 Changed 6 weeks ago by Matthias Köppe

No objections if you want to make these edits to the structure.

Important point to keep: The devcontainer approach also works on machines where you can run Docker but not WSL. (I do not know how common this scenario is.)

comment:42 in reply to:  39 Changed 6 weeks ago by Matthias Köppe

Replying to gh-tobiasdiez:

how do I contribute to https://github.com/sagemanifolds/SageManifolds using the devcontainer approach?

Just clone it into a subfolder.

comment:43 Changed 6 weeks ago by git

Commit: 8aa2cc4e7ae23f72212850f455af397a38ab31e2986e6760487fd971067669840bd58c75a3c3fa6c

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

79c3bcb.devcontainer/downstream-docker-cocalc/devcontainer.json: Use onCreateCommand, updateContentCommand
fbfab5e.devcontainer/downstream-docker-computop/devcontainer.json: Use onCreateCommand, updateContentCommand
09289e5.devcontainer/downstream-archlinux-latest/devcontainer.json: Use onCreateCommand, updateContentCommand
968ec53.devcontainer/downstream-conda-forge-latest/devcontainer.json: Use onCreateCommand, updateContentCommand
1ab9559.devcontainer/*/devcontainer.json: Use onCreateCommand, updateContentCommand
3ba737bMinor edits
30f3453Small edits
f1f38b1.devcontainer/portability-ubuntu-jammy-standard: Add symlink to portability-Dockerfile
37eb95dRemove the note for starting configured dev container
986e676Merge #33671

comment:44 Changed 6 weeks ago by Matthias Köppe

Merged latest version of #33671.

comment:45 Changed 6 weeks ago by git

Commit: 986e6760487fd971067669840bd58c75a3c3fa6ca16b4770132c9fc5056f3da5ca4dba93b3d2ec7e

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

d5610b1Edit: prefer symlink
179406fsrc/doc/en/developer/portability_testing.rst: Update references to devcontainer ...Command
4affef2.devcontainer/onCreate.sh, portability-updateContent.sh: Rename from post_create.sh, portability-post_start.sh
a16b477Merge #33671

comment:46 Changed 5 weeks ago by git

Commit: a16b4770132c9fc5056f3da5ca4dba93b3d2ec7e74448c0e6679e16160a85c79d2362fc1e3a71e89

Branch pushed to git repo; I updated commit sha1. New commits:

74448c0Merge tag '9.7.rc0' into t/34363/installation_guide__document_sage_installation_on_windows_via_vs_code_devcontainers

comment:47 Changed 3 weeks ago by Emmanuel Briand

For Windows using VS Code, I would find useful to mention also

  • to install git,
  • and that if the Git: Clone command is not recognized, it is because git is not installed.

comment:48 Changed 3 weeks ago by Matthias Köppe

Do you have a VS Code documentation link that explains this well?

comment:49 Changed 2 weeks ago by Matthias Köppe

Milestone: sage-9.7sage-9.8
Note: See TracTickets for help on using tickets.