tests(unit): retry failures and upload failure artifacts #15378
Conversation
4604a20
to
a98ba71
Compare
| @@ -56,7 +56,7 @@ | |||
| "unit-viewer": "yarn mocha --testMatch viewer/**/*-test.js", | |||
| "unit-flow": "bash flow-report/test/run-flow-report-tests.sh", | |||
| "unit": "yarn unit-flow && yarn mocha", | |||
| "unit:ci": "NODE_OPTIONS=--max-old-space-size=8192 npm run unit --forbid-only", | |||
There was a problem hiding this comment.
oh. we need to support this in the script. can you not remove ?
There was a problem hiding this comment.
The --forbid-only flag doesn't work when we use it like this. This PR adds it to run-mocha-tests.js and gives it a default value of true in CI instead.
There was a problem hiding this comment.
What do you mean it doesn't work when used like this?
I know it doesn't work at all w/o this PR or #15383 , but I don't understand your meaning here
There was a problem hiding this comment.
Two reasons:
- Before this PR we didn't pipe the
--forbid-onlyoption into mocha - We would need to do
npm run unit -- --forbid-onyto pass the flag all the way down
Instead of messing with the flags in the package scripts, I would prefer to just look at process.env.CI. We already do this for --parallel in run-mocha-tests.js:
lighthouse/core/test/scripts/run-mocha-tests.js
Lines 127 to 132 in 06ebc36
Unit failures are only happening in CI, failure artifacts might give us more insight. Retry unit tests 5 times to unblock CI.
CI failures added to backlog: #15382