Wikimedia Research/Research and Data

Group:	Analytics
Team members:	Aaron Halfaker, Oliver Keyes, Erik Zachte, Leila Zia, Ellery Wulczyn
Backlog:	#Research-and-Data
Lead:	Dario Taraborelli

Research and Data

Team

Aaron Halfaker
Principal Research Scientist
Ellery Wulczyn
Data Scientist
Erik Zachte
Data Analyst
Leila Zia
Senior Research Scientist
Dario Taraborelli
Director, Head of Research

Research fellows

Morten Warncke-Wang
University of Minnesota
Robert West
Stanford University

Contacts

Mailing lists: research-wmf (internal team list); research-internal (cross-departmental list); wiki-research-l (public); analytics-l (public)
IRC: #wikimedia-research ^connect; #wikimedia-analytics ^connect

Who we are

We are the Wikimedia Foundation's research team and our mission is to support the organization in using data to make better decisions. The team includes members of the Analytics team as well as analysts and researchers with different departments.

Team responsibilities

The Research and Data team has three main responsibilities:

to support the organization in making research-informed decisions,
to expand our understanding of the editor community and projects, and
to help teams evaluate the impact of new programs and features designed by the Wikimedia Foundation.

Our work

Research methods. We use a variety of methods (both quantitative and qualitative strategies, exploratory analyses, controlled experimentation, and statistical modeling) to answer research questions.

Collaboration. We mostly work by embedding ourselves in the teams that we support. We believe that we work best with customers when we participate in the design process from a very early stage to identify research questions that new products or programs are designed to answer. We also support the teams we work with by advising towards best practices on instrumentation, data modeling, data visualization and the definition of key performance metrics. To determine the best approach and prioritize research questions we typically consult with program and product owners on a quarterly basis, or on a per-project basis as needed.

Transparency. We strive to make the process and output of our work as transparent as possible, by publicly sharing the research reports (see the Research:Index), visualizations, code (see FIXME) and datasets (see our datahub.io group) that we produce. When we publish our work in academic outlets, we make sure that the results are openly accessible to the community.

Product development. We also work with the Analytics Development and the Wikimedia product and engineering teams to identify and prototype new analytics product ideas or ways to use data to improve user experience on Wikimedia sites.

Outreach. We work with community members and academic researchers to spearhead Wikimedia/Wikipedia research and to evangelize open research tools and data sources. We regularly hang out on IRC in the #wikimedia-research ^connect and #wikimedia-analytics ^connect channel, where you can reach us to ask us anything about our work. We are also familiar with the scholarly literature on Wikipedia and online collaboration in general and we strive to bridge the gap between the Foundation, our editor community and the research community by participating in various outreach initiatives.

Grantmaking. We support Wikimedia movement affiliates in making research-driven decisions about how to further our mission and strategic goals through grant funded projects, and how to demonstrate the impact of those projects.

What we are working on

Our current work is tracked via this board. A monthly summary of the team's activity is available via our status page while progress towards quarterly goals is captured in our quarterly reviews.

Work with us

Open positions

We currently have no openings for research positions.

Internships

We currently do not have a paid internship position. If you are interested in an unpaid position, please send an email to leila@wikimedia.org, including your resume, a cover letter, and your availability.

How to get research support

Our work doesn't fit well with the typical structure of a software development process and we strive to remain flexible and adapt to the needs of the teams we support on an ongoing basis. When we work with project teams we're embedded in, we work iteratively with the product owners and team members to provide the best form of support to the team. When we do so, our work is tracked, prioritized and reported by the project team itself. We determine on a quarterly basis what teams need dedicated support with "embedded researchers" (see focus areas below), as a function of our current capacity.
We welcome requests for research support and new research ideas at an exploratory stage. We keep track of these requests via a dedicated board. This tool is not designed as an inbox for raw data requests. You should feel free to create new cards and start submitting ideas at an early stage even if they are not fully articulated, but once you've done that, do get in touch so we can help grow these ideas into well thought-through plans, determine their relative priority and assess their technical feasibility and effort needed.
We also use this tool to keep track of the output of our work and collect links that we need to collect to report our activity on a monthly basis.
The following is a typical workflow of a research request:
- New requests are posted by creating a card under New: that's where people start describing what problem they are trying to solve or research idea they have.
- We review incoming requests and file them under Input needed when more context/information is needed. This will typically be the case for the majority of incoming requests.
- Cards will stay on the Input needed list until they include all the information needed for us to start actually working on them.
- All the other lists (Staged, In progress, Completed) should be self-explanatory and represent the stage of a request.
- Fixed-date requests can be marked by specifying a due date field in the card. We will review them to make sure we can prioritize them based on the team capacity.
- A card filed under Completed will typically link to a research report on Meta, a dataset on the DataHub, code in a code repository, a data visualization etc.
Customers and people outside of the team should feel free to create new cards or comment on existing ones, but not move cards across lists.
- The board is currently readable to anyone in the organization; members of the organization can join and comment; team members are admins.
We host monthly IRC Research Office Hours where you can ask us about what we're doing in general, or get our input on specific questions. The office hours will be announced on the wiki-research-l and analytics-l mailing lists.

Team updates

Recent presentations

Wikimedia traffic update (12.2014)
Wikimedia - New unique visitor data (09.2014)
Trends in Article Creation (Wikimania'14)
Measuring community health (Wikimania'14)
Big in Japan (Wikimania'14)
The Missing Wikipedia Ads (Wikimania'14)
Measuring and presenting value added to Wikipedia (Wikimania'14)
Wikimedia Mobile Trends (07/2014)
The Editor Model: redefining active editors (06/2014)
2014 Mobile Metrics (05/2014)
Cross-wiki deployments: impact and measurement (03/2014)
Metrics standardization and Editor Engagement Vital Signs (03/2014)
Wikipedia: Maintaining production efficiency (03/2014)
Measuring mobile sessions (02/2014)
Wikipedia article creation (02/2014)
2013 traffic trends
On the nature of anonymous editors (12/2013)
Metrics standardization (12/2013)

Quarterly reviews

Q1 2015-16 quarterly review (minutes)
Q4 2014-15 quarterly review (minutes)
Q3-2015 quarterly review (04/2015)
Q2-2015 quarterly review (01/2015)
Q1-2015 quarterly review (09/2014)
Q4-2014 quarterly review (06/2014)
Q3-2014 quarterly review (03/2014)
Q2-2014 quarterly review (10/2013)

What we are working on

Growth: we work with the Growth team to research how to attract and retain new Wikipedia editors.
Mobile editor engagement: we help the mobile team understand the effectiveness of new ways to engage with mobile users (i.e. new users registered on mobile devices as well as existing editors contributing via mobile devices)
Mobile Traffic: we help the mobile team and the foundation understand the breakdown and habits of our mobile readers.

Other stuff we're doing

Monthly research showcase: a monthly showcase of our work.
Labs₂: an initiative to engage with academics and community members interested in Wikimedia research
Research Index: we help maintain Wikimedia's canonical directory of research projects.
Wikimedia Research Newsletter: we contribute to the monthly research newsletter, covering the most recent publications on Wikimedia projects.