[go: nahoru, domu]

Page MenuHomePhabricator

[REPO][CLIENT][SW] Make selenium E2E tests run in parallel in CI jobs
Open, Needs TriagePublic

Description

The selenium (quibble-selenium) CI jobs currently take more than 13 minutes to run:

Analysis for change mediawiki/extensions/Wikibase - wmf-quibble-selenium-php74 (17927):
	wmf-quibble-selenium-php74 (17927): 811
		Setup: 15
		Versions: 1
		Ensure dir: '/workspace/log': 0
		Zuul clone : 124
		Submodule update: /workspace/src: 8
		Install composer dev-requires for vendor.git: 9
		Start backends: <MySQL (no socket)>: 3
		Run Post-dependency install: 0
		Install MediaWiki: 4
		npm install in /workspace/src: 5
		Start backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>: 0
		Browser tests: mediawiki/extensions/Wikibase: 0
		Selenium extensions/Wikibase: 205
		Selenium extensions/AbuseFilter: 37
		Selenium extensions/CheckUser: 26
		Selenium extensions/Cite: 80
		Selenium extensions/Echo: 17
		Selenium extensions/FileImporter: 17
		Selenium extensions/GrowthExperiments: 78
		Selenium extensions/Math: 13
		Selenium extensions/PageTriage: 38
		Selenium extensions/ProofreadPage: 37
		Selenium extensions/VisualEditor: 40
		Selenium skins/MinervaNeue: 23
		PostBuildScript: 31

In particular, the Wikibase browser tests take 205 seconds of the total runtime. Investigate ways to reduce the total time spent for this job, by parallelising the execution of the selenium test suites.

Event Timeline

Change #1055185 had a related patch set uploaded (by Arthur taylor; author: Arthur taylor):

[integration/quibble@master] Run Browser tests in parallel if QUIBBLE_SELENIUM_PARALLEL is set

https://gerrit.wikimedia.org/r/1055185

Change #1055186 had a related patch set uploaded (by Arthur taylor; author: Arthur taylor):

[mediawiki/extensions/Wikibase@master] Use fresh user for Repo Selenium tests

https://gerrit.wikimedia.org/r/1055186

The browser tests are run with webdriver.io which already has support for parallelism. We did enabled it for most projects, but there are a few for which the tests end up in race condition. The tracking / epic task is T226869

I also don't think we can run in parallel the already parallelized tests, that is going to drain resource and expose even more race conditions? For Wikibase you can enable parallelization by setting maxInstances:

wdio.conf.js
exports.config = { ...config,
    maxInstances:4 ,
...
}