T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min sets an ambitious goal for CI job runs, and documents the progress that has been made towards the goal. Since the task was filed in 2019, many steps have been undertaken to optimise the builds including, but not limited to:
- Improving CI caching and generally reducing CI job setup time
- Reducing the runtime of specific long-running tests
- Skipping some tests on gated CI runs
- Fixing bugs that caused unnecessary work during CI
Work has also been started on:
- T50217: Speed up MediaWiki PHPUnit build by running integration tests in parallel
- T226869: Run browser tests in parallel
There is a specific ticket about a slowdown that occurred in 2019 for Wikibase tests: T228739: Jenkins runs Scribunto tests in Wikibase patches. Supposedly the runtime of the tests doubled on July 22nd 2019.
Running the tests today (27. March 2024), the timings break down as follows:
Analysis for change mediawiki/extensions/Wikibase~1015066:
quibble-vendor-mysql-php74-noselenium-docker (165337): 360
Setup: 8
Versions: 1
Ensure dir: '/workspace/log': 0
Zuul clone : 83
Submodule update: /workspace/src: 13
Run npm and composer tests: 0
composer test in /workspace/src/extensions/Wikibase: 28
Revert to git clean -xqdf in /workspace/src/extensions/Wikibase: 0
Install composer dev-requires for vendor.git: 11
Start backends: <MySQL (no socket)>: 4
Run Post-dependency install: 0
Install MediaWiki: 8
npm install in /workspace/src: 1
PHPUnit unit tests: 13
PHPUnit extensions suite (without database or standalone): 181
PostBuildScript: 9
quibble-vendor-mysql-php74-selenium-docker (102918): 775
Setup: 54
Versions: 1
Ensure dir: '/workspace/log': 0
Zuul clone : 81
Submodule update: /workspace/src: 13
Install composer dev-requires for vendor.git: 10
Start backends: <MySQL (no socket)>: 3
Run Post-dependency install: 0
Install MediaWiki: 10
npm install in /workspace/src: 1
Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
Browser tests: mediawiki/extensions/Wikibase: 0
Selenium extensions/Wikibase: 236
Selenium extensions/AbuseFilter: 36
Selenium extensions/CheckUser: 26
Selenium extensions/Cite: 114
Selenium extensions/Echo: 16
Selenium extensions/Popups: 47
Selenium extensions/VisualEditor: 39
Selenium skins/MinervaNeue: 26
PostBuildScript: 62
mwgate-node18-docker (32470): 297
Setup: 248
PostBuildScript: 49
mwext-php74-phan-docker (109326): 108
Setup: 7
Versions: 0
Ensure dir: '/workspace/log': 0
Zuul clone : 33
Submodule update: /workspace/src: 2
Run Post-dependency install: 66
wmf-quibble-vendor-mysql-php74-docker (56161): 1294
Setup: 8
Versions: 0
Ensure dir: '/workspace/log': 0
Zuul clone : 116
Submodule update: /workspace/src: 8
Run npm and composer tests: 0
composer test in /workspace/src/extensions/Wikibase: 24
Revert to git clean -xqdf in /workspace/src/extensions/Wikibase: 1
Install composer dev-requires for vendor.git: 9
Start backends: <MySQL (no socket)>: 4
Run Post-dependency install: 0
Install MediaWiki: 4
npm install in /workspace/src: 3
PHPUnit unit tests: 14
PHPUnit extensions suite (without database or standalone): 157
Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
Run Qunit tests: 33
PHPUnit extensions suite (with database): 872
PostBuildScript: 41
wmf-quibble-selenium-php74-docker (90455): 804
Setup: 17
Versions: 1
Ensure dir: '/workspace/log': 0
Zuul clone : 95
Submodule update: /workspace/src: 9
Install composer dev-requires for vendor.git: 9
Start backends: <MySQL (no socket)>: 4
Run Post-dependency install: 0
Install MediaWiki: 3
npm install in /workspace/src: 6
Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
Browser tests: mediawiki/extensions/Wikibase: 0
Selenium extensions/Wikibase: 225
Selenium extensions/AbuseFilter: 36
Selenium extensions/CheckUser: 26
Selenium extensions/Cite: 43
Selenium extensions/Echo: 14
Selenium extensions/FileImporter: 17
Selenium extensions/GrowthExperiments: 75
Selenium extensions/Math: 12
Selenium extensions/PageTriage: 51
Selenium extensions/ProofreadPage: 31
Selenium extensions/VisualEditor: 40
Selenium skins/MinervaNeue: 26
PostBuildScript: 64
mwext-node18-rundoc-docker (4149): 129
Setup: 116
PostBuildScript: 13
mediawiki-quibble-apitests-vendor-php74-docker (61067): 697
Setup: 42
Versions: 1
Ensure dir: '/workspace/log': 0
Zuul clone : 57
Submodule update: /workspace/src: 12
Install composer dev-requires for vendor.git: 8
Start backends: <MySQL (no socket)>: 4
Run Post-dependency install: 0
Install MediaWiki: 5
npm install in /workspace/src: 1
Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
Run API-Testing: 544
PostBuildScript: 23
wikibase-client-docker (38572): 100
Setup: 8
Versions: 0
Ensure dir: '/workspace/log': 0
Zuul clone : 22
Submodule update: /workspace/src: 3
Create composer.local.json with dependencies ['mediawiki/core': 0
composer update for mediawiki/core: 11
Start backends: <MySQL (no socket)>: 4
Run Post-dependency install: 0
Install MediaWiki: 3
npm install in /workspace/src: 4
Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
User commands: extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh -r client: 38
PostBuildScript: 7
wikibase-repo-docker (38571): 199
Setup: 7
Versions: 1
Ensure dir: '/workspace/log': 0
Zuul clone : 24
Submodule update: /workspace/src: 3
Create composer.local.json with dependencies ['mediawiki/core': 0
composer update for mediawiki/core: 12
Start backends: <quibble.backend.SQLite object at 0x7faf66399b00>: 0
Run Post-dependency install: 0
Install MediaWiki: 3
npm install in /workspace/src: 6
Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
User commands: extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh -r repo: 133
PostBuildScript: 10(analysis courtesy of jenkins-run-analysis)
This places wmf-quibble-vendor-mysql-php74-docker on the critical path with a runtime of 1294 seconds (21.6 minutes) with the longest time being spent in PHPUnit extensions suite (with database) - 872 seconds (14.5 minutes). Any optimisation here would directly reduce the total time before a result comes back from CI.
The scope for this task is to identify and execute steps to significantly reducing the maximum delay between triggering a CI run and getting a full result.