Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move fast tasks from CircleCI to Cirrus #14878

Open
strkkk opened this issue May 15, 2024 · 8 comments
Open

Move fast tasks from CircleCI to Cirrus #14878

strkkk opened this issue May 15, 2024 · 8 comments

Comments

@strkkk
Copy link
Member

strkkk commented May 15, 2024

Reason: Cirrus CI builds are useless now. They are only run for master and all tasks are duplicated by appveyor build.
On the other hand, CircleCI has too many builds + it has restrictions for developers (e.g. I am not allowed to run builds there)

So, workload can be balanced a bit with some jobs moved to Cirrus.
There is a limitation from Cirrus that does not allow us to run some long-running tasks for each PR. However, there are some very fast tasks, that look like a good candidates to be moved to Cirrus.
Those are:

  git-validation:
    jobs:
      - validate-with-script:
          name: "git-no-merge-commits"
          command: "./.ci/validation.sh git-no-merge-commits"
      - validate-with-script:
          name: "git-check-pull-number"
          command: "./.ci/validation.sh git-check-pull-number"

they together take about 1 minute, so it should be more than enough to run them on every PR and respect the limit.
изображение

There are also other good candidates for transfer, such as
изображение

but they can be moved later

@strkkk strkkk added the CI label May 15, 2024
@strkkk
Copy link
Member Author

strkkk commented May 15, 2024

@nrmancuso wdyt?)

@nrmancuso
Copy link
Member

I am not sure I see any point in doing this until we have some issue with CircleCI. We lean heavily on them since they have been the most reliable CI, and allow us to easily use custom images. Additionally, contributors can view their CI runs in CircleCI, which is not always possible in other CI.

@strkkk
Copy link
Member Author

strkkk commented May 15, 2024

The point is that at the moment Cirrus is completely useless. Its builds are waste of time (they are duplcates), but we can use this system to move some of the jobs there to increase parallelisation.

@nrmancuso
Copy link
Member

increase parallelisation.

Are CircleCI jobs being queued?

@strkkk
Copy link
Member Author

strkkk commented May 17, 2024

@nrmancuso Yes, they are. This is an example from my PR today, it was even worse before I took a screenshot

изображение

@nrmancuso
Copy link
Member

nrmancuso commented May 17, 2024

These are all GitHub jobs, and they are typically queued. However, I do not see this happen with CircleCI hardly ever.

Additionally, Cirrus was one of the worst for long queues.

@strkkk
Copy link
Member Author

strkkk commented May 18, 2024

Oh, sorry, it was github.

Anyway, there is queue in Circle too. Most likely, you dont see because you do not open the build itself. All jobs are shown as pending ones.

This is how it looks like if I open Circle. About 10 jobs were waiting for more than a minute (I took a screenshot a bit earlier)
изображение

@nrmancuso
Copy link
Member

Ok let’s move these two jobs over to cirrus and see how they do :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants