You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is an uber issue that we have been tackling in various places, including private Discord threads. It is meant to centralise & summarise the learnings. Expect heavy editing.
The command that we are running:
dagger call --debug --source=".:default" test all 2>&1 \| ts \| tee engine.test.all.txt
# ts is part of moreutils 馃憠 https://manpages.debian.org/testing/moreutils/ts.1.en.html
The root of the problem is inconsistent behaviour when running Engine tests locally:
Sometimes they complete within 15mins, sometimes they timeout after 30mins;
Sometimes they succeed, and sometimes they fail. When they fail, they usually do so at 30mins, due to the timeout;
So far, the following minimum system resources are known to make the tests pass more reliably (more is better):
16 CPUs
32GB RAM
NVMe disk
Note
The above config is what @aluzzardi is able to make it pass locally.
We have a bunch more details here, including system metrics: #7223 (comment)
Some of us - cc @samalba - had success with the following change:
gerhard
changed the title
馃悶 path: ETOOBIG File name length exceeds the maximum supported 255 characters
馃 dagger call --source=.:default test all is timing out locally - inconsistent behaviour
May 14, 2024
What is the issue?
This is an uber issue that we have been tackling in various places, including private Discord threads. It is meant to centralise & summarise the learnings. Expect heavy editing.
The command that we are running:
The root of the problem is inconsistent behaviour when running Engine tests locally:
15mins
, sometimes they timeout after30mins
;30mins
, due to the timeout;So far, the following minimum system resources are known to make the tests pass more reliably (more is better):
Note
The above config is what @aluzzardi is able to make it pass locally.
We have a bunch more details here, including system metrics: #7223 (comment)
Some of us - cc @samalba - had success with the following change:
I applied the same change locally & couldn't see any difference:
The host that I ran this on is a clean Linux 6.8.0 (Pop!_OS 22.04) install with:
I was able to see the same inconsistent behaviour on this fresh local machine, as well as a bunch of all other local machines:
The investigation continues...
The text was updated successfully, but these errors were encountered: