Debugging MediaWiki jobs is a pain point, in part because there is no way to trigger detailed debug logging like we can for web requests. This seems fairly easy to fix - MediaWiki just needs to pass the value of X-Wikmedia-Debug to EventBus, and then the job executor service needs to turn it back to an actual header. (That header would affect other jobs in the same batch; that's probably fine.) Alternatively, we can make it part of the job specification like we do with X-Request-ID - that wouldn't allow routing to mwdebug, but would allow logging etc to work.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T291620 Better observability/visualization for MediaWiki jobs | |||
| Open | None | T354042 Forward X-Wikimedia-Debug header to MediaWiki jobs |
Event Timeline
Comment Actions
Note that this isn't as simple as just forwarding the header, as it's parsed by ATS and jobs don't pass through the CDN layer.
Comment Actions
I enhanced one of my bots to check for, and report, errors it encounters.
One error encountered, I was able to recover from my making the bot log back in.
[error] => Array
(
[code] => assertuserfailed
[info] => You are no longer logged in, so the action could not be completed.
[docref] => See https://en.wikipedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes.
)But another error I encountered seems to be a report of an internal problem with the job queue.
[error] => Array
(
[code] => internal_api_error_JobQueueError
[info] => [0f9b54ac-f0e9-4040-9546-b893e162e7b5] Caught exception of type JobQueueError
[errorclass] => JobQueueError
)At the moment, my bot ignores these, and they only are happening sporadically. Most API purge requests, including the ones right after these errors, still work.
Let me know if my bot should do anything special in response to these errors. I'm reporting this here because this Phab relating to MediaWiki jobs is still open, while other Phabs related to my bots API Purge troubles (T351729 and T354460) were resolved and closed. Please do open a new Phab for this, if appropriate.
Comment Actions
It would be more appropriate, but I think it's fine to ignore that error (the exception is "Could not enqueue jobs", caused by a bunch of 503s in EventBus, so probably a server being down or overloaded temporarily).