Change #1038280 had a related patch set uploaded (by Arthur taylor; author: Arthur taylor):

[mediawiki/core@master] [WIP] Add PHPUnit splitting

https://gerrit.wikimedia.org/r/1038280

Change #1038609 had a related patch set uploaded (by Arthur taylor; author: Arthur taylor):

[mediawiki/core@master] [WIP] Add `phpunit:parallel:extensions` composer command

https://gerrit.wikimedia.org/r/1038609

Change #1039669 had a related patch set uploaded (by Arthur taylor; author: Arthur taylor):

[integration/quibble@master] Add parallel execution for PHPUnit extensions suite

https://gerrit.wikimedia.org/r/1039669

Change #1039675 had a related patch set uploaded (by Arthur taylor; author: Arthur taylor):

[integration/quibble@master] [DNM] Enable parallel runs by setting QUIBBLE_PHPUNIT_PARALLEL

https://gerrit.wikimedia.org/r/1039675

By way of validation, I did a couple of test runs comparing the .phpunit.results.cache files between the linear and parallel execution and found no difference in the tests executed:

Extensions Databaseless

Linear

$ composer phpunit:entrypoint -- --testsuite extensions --exclude-group Broken,ParserFuzz,Stub,Standalone,Database
...
Tests: 30413, Assertions: 86903, Skipped: 163.

$ cat phpunit-result-cache-extensions-databaseless-linear-20240627 | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq | wc -l
30390

Parallel

$ composer run phpunit:prepare-parallel:extensions
$ composer run phpunit:parallel:databaseless
$ cat phpunit-result-cache-extensions-databaseless-parallel-* | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq | wc -l
30403

Note that here 30403 is slightly higher than 30390. I expect this is because we end up repeating some tests in the split mode. The comparison below checks that the test coverage complete - that linear only runs the tests that parallel runs and vice versa - even if some tests are repeated.

Compare

$ cat phpunit-result-cache-extensions-databaseless-parallel-*-20240627 | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq > /tmp/parallel
$ cat phpunit-result-cache-extensions-databaseless-linear-20240627 | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq > /tmp/linear
$ diff /tmp/linear /tmp/parallel

Extensions Database

Linear

$ composer run --timeout=0 phpunit:entrypoint -- --testsuite extensions --exclude-group Broken,ParserFuzz,Stub,Standalone --group Database

ERRORS!
Tests: 16327, Assertions: 70141, Errors: 147, Failures: 14, Skipped: 174.

$ cat phpunit-result-cache-extensions-database-linear-20240627  | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq | wc -l
15102

This particular run had some failures because my environment wasn't set up correctly for the tests to pass, but the comparison should still be valid.

Parallel

$ composer run phpunit:prepare-parallel:extensions
$ composer run phpunit:parallel:database
$ cat phpunit-result-cache-extensions-database-parallel-*-20240627 | jq .times | grep ':' | cut -d':' -f -3 | sort | uniq | wc -l
15102

Compare

$ cat phpunit-result-cache-extensions-database-parallel-*-20240627 | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq > /tmp/parallel
$ cat phpunit-result-cache-extensions-database-linear-20240627 | jq .times | grep ':' | sed -e 's/^\(.*\): .*$/\1/' | sort | uniq > /tmp/linear
$ diff /tmp/linear /tmp/parallel

But I would welcome other people doing similar tests / evaluations, or also alternative approaches.

Change #1039675 abandoned by Arthur taylor:

[integration/quibble@master] [DNM] Enable parallel runs by setting QUIBBLE_PHPUNIT_PARALLEL

Reason:

No longer needed - this is enabled for the relevant CI jobs anyway

https://gerrit.wikimedia.org/r/1039675

Change #1038280 merged by jenkins-bot:

[mediawiki/core@master] Add `phpunit:prepare-parallel:extensions` command

https://gerrit.wikimedia.org/r/1038280

Change #1038609 merged by jenkins-bot:

[mediawiki/core@master] Add `phpunit:parallel:extensions` composer command

https://gerrit.wikimedia.org/r/1038609