[go: nahoru, domu]

Page MenuHomePhabricator

Wikifunctions audience research: Baseline quantitative analysis to understand current users
Closed, ResolvedPublic

Description

Goal: Documented baseline understanding of our users including insights into who they are, how they engage with wikifunctions, and what their experience is/was like.

Metrics (to be refined based on discussions with team )
More details provided in the PRD

  • User interface language
    • Proportion of distinct editors and visitors to wikifunctions by language UI.
    • Visualization that highlights areas with gaps between where contributions come from and the languages UI
  • Geographical distribution
    • Distinct editors by country
    • Unique devices by country
    • Visualization that highlights areas with gaps between people who contribute and consume content.
  • Community engagement
    • New editor retention: Out of the users who registered in the month before the previous and made at least one edit in their first 30 days, the proportion who also edited during their second 30 days. Note: The definition aligns with the core metric used by WMF to guide strategic decisions.
    • Active editor retention: Proportion of active editors who are new (registered during the given month) vs returning (registered before the given month).
    • (If retention is low): Investigate editing data for non-retained editors. For example, the edit completion rate for non-retained editors to determine if these editors are struggling to complete an edit.
  • Movement familiarity
    • Overall proportion of wikifunctions editors that contribute to other projects
    • Proportion of wikifunction editors by other supports projects (e.g. X% of wikifunctions editors contribute to “wikidata”, etc. Include breakdowns by editors type (active and new) to highlight differences.
  • How do people use wikifunctions?
    • Contribution ratio.
      • For each function created, assign a contribution ratio to indicate how many distinct users contributed to connecting tests and implementations.
      • Frequency of contribution ratio across all created functions.
    • Activity metrics
      • Daily number of functions, implementations, and tests published by users over time to view changes in rates.
  • What was their experience to become an active contributor (acquisition pathway)
    • Average time from first visit to first edit broken down by experience level and platform.
  • What was their onboarding experience?
    • Average time from account creation to first edit [Conversion]
    • [Nice to have] Proportion of first time contributors by first edit type

Steps:

  • Coordinate with Amin to finalize metrics for analysis needed to inform future work
  • Draft queries to collect data [IN PROGRESS]
  • Analysis and Visualization
  • Share report findings with the team

Event Timeline

MNeisler triaged this task as Medium priority.Jan 24 2024, 6:15 PM
MNeisler added a project: Product-Analytics.
MNeisler moved this task from Triage to Current Quarter on the Product-Analytics board.
MNeisler renamed this task from Wikifunctions audience research: complete quantitative exploratory analysis report of current users to Wikifunctions audience research: complete baseline analysis to understand current users.Jan 24 2024, 6:18 PM
MNeisler claimed this task.
MNeisler updated the task description. (Show Details)
MNeisler renamed this task from Wikifunctions audience research: complete baseline analysis to understand current users to Wikifunctions audience research: Baseline quantitative analysis to understand current users.Feb 23 2024, 4:02 AM
MNeisler updated the task description. (Show Details)

Proposed quantitative metrics for this analysis are summarized in the task description. The Audience PRD for this work includes some additional detail about the purpose of each metric and how it will help inform future strategy.

I've completed an initial analysis exploring user's interface languages, geographical distribution and community engagement. Findings are summarized in a google doc.

@AAlhazwani-WMF - Feel free to review and let me know if you have questions or suggestions on the initial findings.

Next up: I'm working on calculating metrics identified to understand wikifunctions users movement familiarity and how people use wikifunctions. I will summarize those findings by the end of next week.

I've completed the analysis of all the identified metrics, which are summarized in the report.

I'm currently working on creating a summary of high-level insights from the analysis and doing a final read-through and QA of the report prior to sharing more broadly with the team.

The final draft is ready for review.

Please see repo for documentation of queries and methodology.

Resolving this task as I've addressed all comments in the doc but please reach out if you have any questions. Any additional analysis requests about the current user base can be filed under separate tickets.