When a user has loaded a wiki page and hovers over a link, how long does the Hovercard take to show? How does this differ for different connection speeds ( 2G, 3G...WiFi )?
Expected output:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | phuedx | T149801 [EPIC] Refactor Hovercards (née Popups) | |||
| Resolved | ovasileva | T156290 Merge mpga branch into master | |||
| Resolved | Dereckson | T68374 Enable Hovercards on se.wikimedia.org (Swedish chapter wiki) | |||
| Resolved | Jdlrobson | T70860 [GOAL] Graduate Page Previews feature (Popups extension) out of Beta Feature | |||
| Resolved | • Gilles | T70861 Performance review #2 of Hovercards (Popups extension) | |||
| Resolved | ovasileva | T123445 Add support for RESTBase endpoint consumption | |||
| Declined | None | T144544 Spike: How long does a hovercard take to show |
Do note that the "Popups" EventLogging schema (used for the A/B test being analyzed at T139319) already tracks this ("Amount of time (in milliseconds) between the instant the user started dwelling on a link and the instant the preview text rendered visibly, as determined strictly client-side.")
We already have a partial result about this at T139319#2510867: For anonymous users on huwiki, among link interactions where the card was supposed to shown (i.e. which lasted longer then 500ms), delays of more than 200ms happened only for between 1%-2% of cases in the case of anonymous users, and around 0.8% for logged-in users. I can generate a full histogram showing which amount of delay happens how often in the wild, thus complementing the results of the synthetic tests that @Jdlrobson plans to run.
@Tbayer - I think a histogram with details is a great idea. Let's track in https://phabricator.wikimedia.org/T139319, so as to keep analysis in one place. I think mapping this out for various connection speeds could also be of interest. To my knowledge, we're not currently tracking this.
@ovasileva, @Jdlrobson: It looks as if all of the AC were met some time ago as part of T139319: Analyze results of Hovercards A/B test - Hungarian.
As regards comparing 2G, 3G, 4G and WiFi: generally speaking, we don't track the characteristics of a user's connection because there's no hypothesis that requires it and, if there were, no browsers support the API that we could use to determine the device's connection type.
@Jdlrobson: Perhaps we could convert this task into one about graphing the timing information gathered as part of the Popups schema?
@phuedx, @Jdlrobson - I think that makes sense. We could also include some tests on the schema once we merge the mpga branch?
I personally think we should decline this task.
@phuedx I think our responsibility is to ensure the JS we ship is as lean as possible.
This then becomes a problem of how fast RESTBase is.
I think we have graphing information for RESTBase responses already.
Using new Date() to collect metrics knowing that we are introducing timers on the frontend doesn't give me much confidence in that data and I''m not sure we should graph it.
I'm still happy to generate that histogram (I think we were waiting earlier on T145665 and subtasks being resolved, and now that the entire instrumentation has been rewritten, it might make sense to do it for the new data instead).
Per T144544#2973804.
Given the target scale of the Page Previews project, I think it's on us to be aware of how it's performing much like the performance team are with page speed using the NavigationTiming instrumentation. I'll write a task around T144544#2971919.