Opened 3 years ago

Closed 21 months ago

#24842 closed enhancement (duplicate)

Automatically build binder configuration

Reported by: saraedum Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: build Keywords: docker, CI, binder
Cc: roed, embray, nthiery, caruso Merged in:
Authors: Julian Rüth Reviewers:
Report Upstream: N/A Work issues: encode keys base64, support gitlab, squash history once #24854 has ben merged
Branch: u/saraedum/24842 (Commits, GitHub, GitLab) Commit: 9f84f0af6ca867853f4b5862d9e6dcf3fe8f2b86
Dependencies: #24854 Stopgaps:

Status badges

Description

This ticket proposes to add an automatic build for binder to the CI infrastructure from #24655.

Even though there is no support for this in trac yet, people could manually link to the generated binder configuration in their trac tickets. This would make it possible to try the proposed changes out without having to checkout and build the ticket.

If you want to try this out, you can go to the (automatically generated) commit in this github repository.

screenshot

From personal experience, I see a number of reasons why this could be useful:

  • (At Sage Days, new) contributors could start reviewing easy tickets right away without getting frustrated by having to rebuild Sage all the time because they checked out some old branches.
  • Some people use the Sage binaries in their distribution but don't really want to start hacking on Sage on their machines. They still would like to push a certain ticket that is fixing a bug they are experiencing. They are (understandably) reluctant to give things a postitive review without having it tried out themselves.
  • When sharing experimental code with others you usually want to keep the hurdles as low as possible. A link to a Jupyter notebook that just works is certainly much more convenient than complicated instructions on how to git checkout and make a ticket.

Attachments (1)

binder.png (29.2 KB) - added by saraedum 3 years ago.
screenshot

Download all attachments as: .zip

Change History (26)

Changed 3 years ago by saraedum

screenshot

comment:1 Changed 3 years ago by saraedum

  • Status changed from new to needs_review

comment:2 Changed 3 years ago by saraedum

  • Branch set to u/saraedum/binder

comment:3 Changed 3 years ago by git

  • Commit set to 1bcb818963fd4fbbb02995a46222286233fb767a

Branch pushed to git repo; I updated commit sha1. New commits:

d0b3d7fTo make this truly a micro-release
069e722Allow user to override ARTIFACT_BASE on CircleCI
1bcb818Merge branch 'ci' into binder

comment:4 Changed 3 years ago by embray

I could definitely add support for this in the Trac plugin as well, easy peasy.

comment:5 Changed 3 years ago by git

  • Commit changed from 1bcb818963fd4fbbb02995a46222286233fb767a to bcd3692c272db37daba0e3e3315ea5f61ce3fe46

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

be3a0f7The docbuild crashes happen during an os.fork to spawn tachyon
7d85dc7Something related to the sphinxbuild seems to be leaking memory
50898feTry to build-from-scratch on GitLab's underpowered CI machines
95c6275Comment on fork logic in docbuilding
c0574caMerge remote-tracking branch 'trac/develop' into u/saraedum/gitlabci
329fdabfix tags for build-from-latest
048e2d8silence unicode warnings from the patchbot
f9c772eFix filtering logic in sphinx logger
f043163Fixed some minor doctest issues
bcd3692Merge remote-tracking branch 'trac/u/saraedum/gitlabci' into HEAD

comment:6 Changed 3 years ago by saraedum

For this to really make sense we would have to synchronize trac's Git repository to gitlab, say to something like a sage-developers repository and setup CI there.

Version 0, edited 3 years ago by saraedum (next)

comment:7 Changed 3 years ago by saraedum

  • Dependencies changed from #24655 to #24655, #24854

comment:8 Changed 3 years ago by saraedum

  • Status changed from needs_review to needs_work
  • Work issues set to use ssh keys through https://circleci.com/docs/2.0/configuration-reference/#add_ssh_keys

comment:9 Changed 3 years ago by saraedum

  • Branch u/saraedum/binder deleted
  • Commit bcd3692c272db37daba0e3e3315ea5f61ce3fe46 deleted

comment:10 Changed 3 years ago by saraedum

  • Branch set to u/saraedum/24842

comment:11 Changed 3 years ago by saraedum

  • Commit set to f5525204df281522eab6c62b3164f07d36dbd01c
  • Dependencies changed from #24655, #24854 to #24854

Last 10 new commits:

864843cfix syntax error in CircleCI configuration
e1c8536Run short and new doctests on CircleCI for every branch
45579a1Call the right scripts in CircleCI
11f8a6cno port exposure needed for jupyter anymore
1960ba1add openssl/libssl-dev to the docker images
14926cfRevert "add openssl/libssl-dev to the docker images"
f2c641aAdd debug information on patch size
83f71e7Don't pipe "echo" into "rm"
8d3caebMerge branch '24655' into 24854
f552520Merge branch '24854' into HEAD

comment:12 Changed 3 years ago by git

  • Commit changed from f5525204df281522eab6c62b3164f07d36dbd01c to cc3e8c8234b86eb6ba4d2bc2a5233d81ea350f03

Branch pushed to git repo; I updated commit sha1. New commits:

cc3e8c8Fix name of GitLab stage

comment:13 Changed 3 years ago by saraedum

  • Work issues changed from use ssh keys through https://circleci.com/docs/2.0/configuration-reference/#add_ssh_keys to encode keys base64, support gitlab

comment:14 Changed 3 years ago by git

  • Commit changed from cc3e8c8234b86eb6ba4d2bc2a5233d81ea350f03 to 045784dc2f683142f4449da6615075a491c1a0c2

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

4f11757Add --short to options.rst
324808aMerge branch '25270' into 24854
06a0832Merge remote-tracking branch 'trac/develop' into 24854
6e267b6Fix namespacing for GitLab CI
54cfea3Merge remote-tracking branch 'trac/develop' into 24655
b12cd9fDo not use broken docker version on CircleCI
220d948Warn about aufs
c40bb81Add reference to known chown bug in docker/aufs
e501b3fMerge branch '24655' into 24854
045784dMerge branch '24854' into 24842

comment:15 Changed 3 years ago by git

  • Commit changed from 045784dc2f683142f4449da6615075a491c1a0c2 to b8039011a1694445aa4f937e720eb067a1358ca1

Branch pushed to git repo; I updated commit sha1. New commits:

8b10429Merge remote-tracking branch 'trac/develop' into 24655
d313c11Merge remote-tracking branch 'trac/develop' into 24655
8e035deMerge branch '24655' into 24854
1281cbfMerge remote-tracking branch 'trac/develop' into 25270
67f98c6Merge branch '25270' into 24854
fec87aaHonor only_errors option
93d71c2Merge branch '25270' into 24854
b803901Merge branch '24854' into 24842

comment:16 Changed 3 years ago by git

  • Commit changed from b8039011a1694445aa4f937e720eb067a1358ca1 to 4717a31b9407548c79cde70773e65852ad3268ab

Branch pushed to git repo; I updated commit sha1. New commits:

d791779Actually test something in --short runs in CI
829169eMerge remote-tracking branch 'trac/develop' into 25270
b55a1cdMerge branch '25270' into 24854
4717a31Merge branch '24854' into 24842

comment:17 Changed 3 years ago by git

  • Commit changed from 4717a31b9407548c79cde70773e65852ad3268ab to 3333954fb45f4cae933f7af6ceb901eeb6ed3c82

Branch pushed to git repo; I updated commit sha1. New commits:

4152838Merge remote-tracking branch 'trac/develop' into 24655
4cebb4bMerge commit '4152838ef924933eca5ee54447826c803d2dff00' into 24854
a85023cMerge remote-tracking branch 'trac/develop' into 24655
6fa8bc6Builds fail with 8.4.beta0 but the output is truncated and it is not clear what's the issue.
3af7c50Merge remote-tracking branch 'trac/develop' into HEAD
48365edMerge remote-tracking branch 'trac/u/saraedum/24655' into HEAD
3772312Merge remote-tracking branch 'trac/develop' into HEAD
5b3e018Merge branch '25270' into 24854
3333954Merge branch '24854' into 24842

comment:18 Changed 3 years ago by embray

For some reason I thought this was already done but apparently not. Are there any remaining challenges to this? Currently it lists #24854 as a dependency, but I don't know if that's strictly needed. The pipelines are already working for building; being able to launch the built containers in binder is immediately useful, because it also will allow us to run specific tests against each build manually.

Last edited 3 years ago by embray (previous) (diff)

comment:19 Changed 3 years ago by embray

Another idea I had is that if the build was for a merge request on gitlab, we can easily post a comment to the merge request with the link to run the built image on binder.

comment:20 Changed 3 years ago by embray

The branch for this currently has some merge conflict artifacts.

comment:21 Changed 3 years ago by git

  • Commit changed from 3333954fb45f4cae933f7af6ceb901eeb6ed3c82 to 9f84f0af6ca867853f4b5862d9e6dcf3fe8f2b86

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

a8ffbe0Make doctesting work if the docker daemon does not run locally
6a928c3fix permissions of .git directory
b9c7510require the setup to pass
42e6aa3Merge branch '24854' into 24842
8586f9edrop some merge conflict artifacts
c3ea3efMove binder block to the logically correct place
9b3cc0fRun doctests in CI
86c452eMerge remote-tracking branch 'trac/develop' into 24854
b8fe26aMerge remote-tracking branch 'trac/develop' into 24854
9f84f0aMerge branch '24854' into 24842

comment:22 Changed 3 years ago by saraedum

  • Work issues changed from encode keys base64, support gitlab to encode keys base64, support gitlab, squash history once #24854 has ben merged

Last 10 new commits:

a8ffbe0Make doctesting work if the docker daemon does not run locally
6a928c3fix permissions of .git directory
b9c7510require the setup to pass
42e6aa3Merge branch '24854' into 24842
8586f9edrop some merge conflict artifacts
c3ea3efMove binder block to the logically correct place
9b3cc0fRun doctests in CI
86c452eMerge remote-tracking branch 'trac/develop' into 24854
b8fe26aMerge remote-tracking branch 'trac/develop' into 24854
9f84f0aMerge branch '24854' into 24842

comment:23 Changed 2 years ago by embray

  • Milestone changed from sage-8.2 to sage-pending

comment:24 Changed 22 months ago by saraedum

  • Milestone changed from sage-pending to sage-duplicate/invalid/wontfix
  • Status changed from needs_work to positive_review

I am trying a simpler approach in #28457.

comment:25 Changed 21 months ago by chapoton

  • Resolution set to duplicate
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.