Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup dagger engines with buildkit gc disabled #7376

Closed
wants to merge 1 commit into from

Conversation

matipan
Copy link
Contributor

@matipan matipan commented May 14, 2024

This is a follow up from an internal discussion where @samalba has been investigating the performance of our CI and found that disabling bulidkit's GC has the potential of improving stability of the most heavy jobs. We want to test this in production for a few days and have side by side data to compare the performance and stability of the jobs. I've been running a few isolated tests here: https://github.com/dagger/dagger/actions/runs/9084221778, but we need more data in order to drive a conclusion regarding the GC.

In this PR we are adding two jobs for the engine-and-cli workflow:

  • test-nogc
  • testdev-nogc

This jobs will run in parallel and in a synchronous manner to baseline jobs. To implement it there were two minor changes introduced to _dagger_call:

  • size-dind variable: this variable allows us to customize the runs-on for testdev which we couldn't so far.
  • concurrency: allows the workflow dispatching dagger_call to control the concurrency group. We need this, otherwise nogc and baseline will step over each other's toes and cancel each other out.

Copy link
Member

@gerhard gerhard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few tiny comments. Also, this will need rebasing after these changes went it:

.github/workflows/_hack_make.yml Outdated Show resolved Hide resolved
.github/workflows/engine-and-cli.yml Outdated Show resolved Hide resolved
@matipan matipan force-pushed the ci-no-buildkit-gc branch 4 times, most recently from 781c585 to 4bdb340 Compare May 15, 2024 16:39
Signed-off-by: Matias Pan <matias@dagger.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants