T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min sets an ambitious goal for CI job runs, and documents the progress that has been made towards the goal. Since the task was filed in 2019, many steps have been undertaken to optimise the builds including, but not limited to:
- Improving CI caching and generally reducing CI job setup time
- Reducing the runtime of specific long-running tests
- Skipping some tests on gated CI runs
- Fixing bugs that caused unnecessary work during CI
Work has also been started on:
- T50217: Speed up MediaWiki PHPUnit build by running integration tests in parallel
- T226869: Run browser tests in parallel
There is a specific ticket about a slowdown that occurred in 2019 for Wikibase tests: T228739: Jenkins runs Scribunto tests in Wikibase patches. Supposedly the runtime of the tests doubled on July 22nd 2019.
Running the tests today (27. March 2024), the timings break down as follows:
Analysis for change mediawiki/extensions/Wikibase~1015066: quibble-vendor-mysql-php74-noselenium-docker (165337): 360 Setup: 8 Versions: 1 Ensure dir: '/workspace/log': 0 Zuul clone : 83 Submodule update: /workspace/src: 13 Run npm and composer tests: 0 composer test in /workspace/src/extensions/Wikibase: 28 Revert to git clean -xqdf in /workspace/src/extensions/Wikibase: 0 Install composer dev-requires for vendor.git: 11 Start backends: <MySQL (no socket)>: 4 Run Post-dependency install: 0 Install MediaWiki: 8 npm install in /workspace/src: 1 PHPUnit unit tests: 13 PHPUnit extensions suite (without database or standalone): 181 PostBuildScript: 9 quibble-vendor-mysql-php74-selenium-docker (102918): 775 Setup: 54 Versions: 1 Ensure dir: '/workspace/log': 0 Zuul clone : 81 Submodule update: /workspace/src: 13 Install composer dev-requires for vendor.git: 10 Start backends: <MySQL (no socket)>: 3 Run Post-dependency install: 0 Install MediaWiki: 10 npm install in /workspace/src: 1 Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0 Browser tests: mediawiki/extensions/Wikibase: 0 Selenium extensions/Wikibase: 236 Selenium extensions/AbuseFilter: 36 Selenium extensions/CheckUser: 26 Selenium extensions/Cite: 114 Selenium extensions/Echo: 16 Selenium extensions/Popups: 47 Selenium extensions/VisualEditor: 39 Selenium skins/MinervaNeue: 26 PostBuildScript: 62 mwgate-node18-docker (32470): 297 Setup: 248 PostBuildScript: 49 mwext-php74-phan-docker (109326): 108 Setup: 7 Versions: 0 Ensure dir: '/workspace/log': 0 Zuul clone : 33 Submodule update: /workspace/src: 2 Run Post-dependency install: 66 wmf-quibble-vendor-mysql-php74-docker (56161): 1294 Setup: 8 Versions: 0 Ensure dir: '/workspace/log': 0 Zuul clone : 116 Submodule update: /workspace/src: 8 Run npm and composer tests: 0 composer test in /workspace/src/extensions/Wikibase: 24 Revert to git clean -xqdf in /workspace/src/extensions/Wikibase: 1 Install composer dev-requires for vendor.git: 9 Start backends: <MySQL (no socket)>: 4 Run Post-dependency install: 0 Install MediaWiki: 4 npm install in /workspace/src: 3 PHPUnit unit tests: 14 PHPUnit extensions suite (without database or standalone): 157 Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0 Run Qunit tests: 33 PHPUnit extensions suite (with database): 872 PostBuildScript: 41 wmf-quibble-selenium-php74-docker (90455): 804 Setup: 17 Versions: 1 Ensure dir: '/workspace/log': 0 Zuul clone : 95 Submodule update: /workspace/src: 9 Install composer dev-requires for vendor.git: 9 Start backends: <MySQL (no socket)>: 4 Run Post-dependency install: 0 Install MediaWiki: 3 npm install in /workspace/src: 6 Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0 Browser tests: mediawiki/extensions/Wikibase: 0 Selenium extensions/Wikibase: 225 Selenium extensions/AbuseFilter: 36 Selenium extensions/CheckUser: 26 Selenium extensions/Cite: 43 Selenium extensions/Echo: 14 Selenium extensions/FileImporter: 17 Selenium extensions/GrowthExperiments: 75 Selenium extensions/Math: 12 Selenium extensions/PageTriage: 51 Selenium extensions/ProofreadPage: 31 Selenium extensions/VisualEditor: 40 Selenium skins/MinervaNeue: 26 PostBuildScript: 64 mwext-node18-rundoc-docker (4149): 129 Setup: 116 PostBuildScript: 13 mediawiki-quibble-apitests-vendor-php74-docker (61067): 697 Setup: 42 Versions: 1 Ensure dir: '/workspace/log': 0 Zuul clone : 57 Submodule update: /workspace/src: 12 Install composer dev-requires for vendor.git: 8 Start backends: <MySQL (no socket)>: 4 Run Post-dependency install: 0 Install MediaWiki: 5 npm install in /workspace/src: 1 Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0 Run API-Testing: 544 PostBuildScript: 23 wikibase-client-docker (38572): 100 Setup: 8 Versions: 0 Ensure dir: '/workspace/log': 0 Zuul clone : 22 Submodule update: /workspace/src: 3 Create composer.local.json with dependencies ['mediawiki/core': 0 composer update for mediawiki/core: 11 Start backends: <MySQL (no socket)>: 4 Run Post-dependency install: 0 Install MediaWiki: 3 npm install in /workspace/src: 4 Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0 User commands: extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh -r client: 38 PostBuildScript: 7 wikibase-repo-docker (38571): 199 Setup: 7 Versions: 1 Ensure dir: '/workspace/log': 0 Zuul clone : 24 Submodule update: /workspace/src: 3 Create composer.local.json with dependencies ['mediawiki/core': 0 composer update for mediawiki/core: 12 Start backends: <quibble.backend.SQLite object at 0x7faf66399b00>: 0 Run Post-dependency install: 0 Install MediaWiki: 3 npm install in /workspace/src: 6 Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0 User commands: extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh -r repo: 133 PostBuildScript: 10
(analysis courtesy of jenkins-run-analysis)
This places wmf-quibble-vendor-mysql-php74-docker on the critical path with a runtime of 1294 seconds (21.6 minutes) with the longest time being spent in PHPUnit extensions suite (with database) - 872 seconds (14.5 minutes). Any optimisation here would directly reduce the total time before a result comes back from CI.
The scope for this task is to identify and execute steps to significantly reducing the maximum delay between triggering a CI run and getting a full result.