[go: nahoru, domu]

Page MenuHomePhabricator

EntitySchema conditional namespace registration errors (NS_ENTITYSCHEMA_JSON, NamespaceRegistrationHandler::registerNamespace() TypeError)
Closed, ResolvedPublic5 Estimated Story PointsPRODUCTION ERROR

Description

This error was first observed when I tried to backport two changes to wmf.8 (quickly reverted); however, the error is still reproducible on Test Wikidata, so it’s not specific to the backports and blocks the train.

Steps to reproduce: take a tool that authenticates via OAuth; patch it to contact test.wikidata.org; try to log in. (I don’t know a tool that lets you select the “OAuth wiki” off the top of my head; I used my development version of Wikidata Lexeme Forms. If the broken code reaches Wikidata, then plenty of tools that target Wikidata start to break.)

Error
message
PHP Warning: Use of undefined constant NS_ENTITYSCHEMA_JSON - assumed 'NS_ENTITYSCHEMA_JSON' (this will throw an Error in a future version of PHP)
TypeError: Argument 2 passed to EntitySchema\MediaWiki\Hooks\NamespaceRegistrationHandler::registerNamespace() must be of the type int, string given, called in /srv/mediawiki/php-1.43.0-wmf.9/extensions/EntitySchema/src/MediaWiki/Hooks/NamespaceRegistrationHandler.php on line 52
trace
from /srv/mediawiki/php-1.43.0-wmf.9/extensions/EntitySchema/src/MediaWiki/Hooks/NamespaceRegistrationHandler.php(69)
#0 /srv/mediawiki/php-1.43.0-wmf.9/extensions/EntitySchema/src/MediaWiki/Hooks/NamespaceRegistrationHandler.php(52): EntitySchema\MediaWiki\Hooks\NamespaceRegistrationHandler->registerNamespace(array, string, string, boolean, boolean, string, boolean)
#1 /srv/mediawiki/php-1.43.0-wmf.9/includes/HookContainer/HookContainer.php(159): EntitySchema\MediaWiki\Hooks\NamespaceRegistrationHandler->onCanonicalNamespaces(array)
#2 /srv/mediawiki/php-1.43.0-wmf.9/includes/HookContainer/HookRunner.php(1037): MediaWiki\HookContainer\HookContainer->run(string, array)
#3 /srv/mediawiki/php-1.43.0-wmf.9/includes/title/NamespaceInfo.php(396): MediaWiki\HookContainer\HookRunner->onCanonicalNamespaces(array)
#4 /srv/mediawiki/php-1.43.0-wmf.9/includes/title/NamespaceInfo.php(422): MediaWiki\Title\NamespaceInfo->getCanonicalNamespaces()
#5 /srv/mediawiki/php-1.43.0-wmf.9/includes/language/Language.php(647): MediaWiki\Title\NamespaceInfo->getCanonicalIndex(string)
#6 /srv/mediawiki/php-1.43.0-wmf.9/includes/title/MediaWikiTitleCodec.php(421): Language->getNsIndex(string)
#7 /srv/mediawiki/php-1.43.0-wmf.9/includes/title/Title.php(2751): MediaWiki\Title\MediaWikiTitleCodec->splitTitleString(string, integer)
#8 /srv/mediawiki/php-1.43.0-wmf.9/includes/title/Title.php(456): MediaWiki\Title\Title->secureAndSplit(string, integer)
#9 /srv/mediawiki/php-1.43.0-wmf.9/includes/title/Title.php(404): MediaWiki\Title\Title::newFromTextThrow(string, integer)
#10 /srv/mediawiki/php-1.43.0-wmf.9/extensions/OAuth/src/SessionProvider.php(85): MediaWiki\Title\Title::newFromText(string)
#11 /srv/mediawiki/php-1.43.0-wmf.9/includes/session/SessionManager.php(542): MediaWiki\Extension\OAuth\SessionProvider->provideSessionInfo(MediaWiki\Request\WebRequest)
#12 /srv/mediawiki/php-1.43.0-wmf.9/includes/session/SessionManager.php(248): MediaWiki\Session\SessionManager->getSessionInfoForRequest(MediaWiki\Request\WebRequest)
#13 /srv/mediawiki/php-1.43.0-wmf.9/includes/Request/WebRequest.php(870): MediaWiki\Session\SessionManager->getSessionForRequest(MediaWiki\Request\WebRequest)
#14 /srv/mediawiki/php-1.43.0-wmf.9/includes/session/SessionManager.php(167): MediaWiki\Request\WebRequest->getSession()
#15 /srv/mediawiki/php-1.43.0-wmf.9/includes/Setup.php(488): MediaWiki\Session\SessionManager::getGlobalSession()
#16 /srv/mediawiki/php-1.43.0-wmf.9/includes/WebStart.php(85): require_once(string)
#17 /srv/mediawiki/php-1.43.0-wmf.9/index.php(50): require(string)
#18 /srv/mediawiki/w/index.php(3): require(string)
#19 {main}
Impact

“Pretty bad.” (Exact impact not totally clear to me, but certainly bad enough to block the train, I’m afraid.)

Notes

The error seems to be limited to Special:OAuth/initiate and CentralAuth:

lucaswerkmeister-wmde@mwlog2002:/srv/mw-log$ tail -n 500k error.log | grep NS_ENTITYSCHEMA_JSON | awk '{print $10}' | sort | uniq -c | sort -rn | sed -E 's/[0-9a-f]{39}/redacted/'
   1063 NS_ENTITYSCHEMA_JSON
   1054 /w/index.php?title=Special%3AOAuth%2Finitiate
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org
      1 /w/api.php?centralauthtoken=redacted&origin=https%3A%2F%2Fru.wikipedia.org

There was also a different error during the deployment of those backports: “No such service: EntitySchema.EntitySchemaIsRepo”. That error seemed to go away after a brief spike, and I don’t know how related it is, nor whether it would also affect the train.

Event Timeline

Change #1042308 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/EntitySchema@master] Call NamespaceRegistrationHandler::setConstants() earlier

https://gerrit.wikimedia.org/r/1042308

brennen raised the priority of this task from Medium to Unbreak Now!.Jun 12 2024, 4:32 PM

Raising to UBN per train blocker policy.

I was able to reproduce the OAuth issue locally and the above Gerrit change seems to fix it; I’m just hoping that that’s enough to fix the CentralAuth error as well (I don’t think I can test that locally). It’ll need backporting to wmf.9, and, if we want to try the wmf.8 backports again, to wmf.8 too (but see below).

There was also a different error during the deployment of those backports: “No such service: EntitySchema.EntitySchemaIsRepo”. That error seemed to go away after a brief spike, and I don’t know how related it is, nor whether it would also affect the train.

This one is probably just a race condition – PHP processes with the old code started to see the new extension.json file (which injects the new EntitySchema.EntitySchemaIsRepo service into a bunch of hook handlers) before the php-fpm-restart let them see the new service definition in EntitySchema.ServiceWiring.php. (Edit: I can also confirm by checking the logs that this error seems to have been limited to bare-metal mw[0-9]{4} hosts; in mw-on-k8s, you wouldn’t expect this error to happen, as the new image gets deployed all at once: the new extension.json file is only seen by the new PHP code.) If we want to try backporting these changes to wmf.8 again, we’ll have to keep that in mind; but right now I think I’d rather not do that, and just wait until next week before doing T332157.

Mentioned in SAL (#wikimedia-operations) [2024-06-12T18:01:47Z] <brennen> 1.43.0-wmf.9 train (T361403): currently blocked on T367334, holding at group0 until resolved.

For clarity, is this breakage actually new in some way for wmf.9? I see quite a few of the errors in wmf.8. If nothing changes with the rollout, then this probably shouldn't block.

Change #1042308 merged by jenkins-bot:

[mediawiki/extensions/EntitySchema@master] Call NamespaceRegistrationHandler::setConstants() earlier

https://gerrit.wikimedia.org/r/1042308

Change #1042343 had a related patch set uploaded (by Brennen Bearnes; author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/EntitySchema@wmf/1.43.0-wmf.9] Call NamespaceRegistrationHandler::setConstants() earlier

https://gerrit.wikimedia.org/r/1042343

Change #1042343 merged by jenkins-bot:

[mediawiki/extensions/EntitySchema@wmf/1.43.0-wmf.9] Call NamespaceRegistrationHandler::setConstants() earlier

https://gerrit.wikimedia.org/r/1042343

Mentioned in SAL (#wikimedia-operations) [2024-06-12T19:32:31Z] <brennen@deploy1002> Started scap: Backport for [[gerrit:1042343|Call NamespaceRegistrationHandler::setConstants() earlier (T367334 T363153)]]

Mentioned in SAL (#wikimedia-operations) [2024-06-12T19:35:12Z] <brennen@deploy1002> brennen: Backport for [[gerrit:1042343|Call NamespaceRegistrationHandler::setConstants() earlier (T367334 T363153)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-06-12T19:45:39Z] <brennen@deploy1002> Finished scap: Backport for [[gerrit:1042343|Call NamespaceRegistrationHandler::setConstants() earlier (T367334 T363153)]] (duration: 13m 06s)

brennen lowered the priority of this task from Unbreak Now! to Medium.Jun 12 2024, 7:46 PM

For clarity, is this breakage actually new in some way for wmf.9? I see quite a few of the errors in wmf.8. If nothing changes with the rollout, then this probably shouldn't block.

That would be because I tried to backport the broken patches to wmf.8, then reverted when the errors happened. Or did you see the wmf.8 errors outside of ca. 14:14–15:00 UTC yesterday?

Lucas_Werkmeister_WMDE claimed this task.

I think we can close this: the fix was merged and backported, the train rolled forward (group1 is on wmf.9 now), and the errors didn’t recur as far as I can tell.