Opened 20 months ago

Last modified 4 months ago

#26481 needs_review enhancement

Allow CI to run on old branches

Reported by: saraedum Owned by:
Priority: minor Milestone: sage-duplicate/invalid/wontfix
Component: distribution Keywords: ContinuousIntegration
Cc: embray Merged in:
Authors: Julian Rüth Reviewers:
Report Upstream: N/A Work issues:
Branch: u/saraedum/26481 (Commits) Commit: 28c48df1f2bc5c2c25ccbb9d043b029f4ecf4e94
Dependencies: Stopgaps:

Description

CircleCI and GitLab CI currently both build-from-latest using sagemath/sagemath-dev:develop. This leads to annoying errors when building from older branches as the build process won't find the commit that sagemath-dev:develop was built from in that branch.

Instead, we should build from sagemath-dev:latest-tag-in-branch, so currently we would build from sagemath-dev:8.4.rc0 for new branches but also branches based on older states of the repository would still work. In particular, the build is more reproducible. Rerunning a build later, does not try to build from a different base build.

Change History (7)

comment:1 Changed 20 months ago by saraedum

  • Branch set to u/saraedum/26481

comment:2 Changed 20 months ago by saraedum

  • Commit set to 28c48df1f2bc5c2c25ccbb9d043b029f4ecf4e94

I have not tested whether all the relevant combinations work yet.


New commits:

28c48dfAllow CI to run on old branches

comment:3 Changed 19 months ago by embray

I have been getting bitten by this and it's confusing. That said, I wonder if this is really a good idea. For unmerged tickets we really want to make sure they work against the latest develop branch. Instead I think we should try to apply the ticket on top of the current develop.

Maybe that's what you're doing as well too; I might be missing something. I don't think there's much value in doing builds against old tags.

comment:4 Changed 19 months ago by embray

E.g. If the last commit in the old branch is older than the current develop, then it should try automatically to rebase the branch on the newest develop, or merge it, and if there are merge conflicts then it should refuse to run the rest of the pipeline (on GitHub for example it generally won't run CI services if a PR has merge conflicts with the current master branch IIUC).

comment:5 Changed 4 months ago by saraedum

  • Keywords ContinuousIntegration added

comment:6 Changed 4 months ago by saraedum

In principle I think you are right. But I have a feeling that this is going to cause quite some headaches if I think about typical reruns that we trigger manually. Say, I found out that an old tag did not build and push to docker hub. Now I click "retry" and it will actually merge in the latest develop and build that :(

So, there is ways to find out, and say, do not merge when you are on a tag. But maybe that's just adding too much complexity.

There's a reasonable message in the build log I think. Could not find commit cat docker/.commit in your local Git history. Please merge in the latest built develop branch to fix this: git fetch trac && git merge cat docker/.commit`. I think I'd just leave this alone.

comment:7 Changed 4 months ago by saraedum

  • Milestone changed from sage-8.4 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review
Note: See TracTickets for help on using tickets.