Wikipedia:Wikipedia Signpost/2012-02-20/Special report
This week the Wikimedia Foundation reported the results of its New Page Patrol Survey, part of a project to increase understanding of editors who work on new page patrol (NPP). NPP is the first line of defense against articles that do not belong on Wikipedia, be they unsuitable pages, copyright violations, or attack pages.
New page patrolling has long been a prominent problem area for the Wikipedian community. While patrollers complain of being overworked, other editors have raised issues with the patrollers themselves, characterizing them as mostly young and inexperienced volunteers who lack clue, are ignorant of deletion rules, and often mis-tag new articles. The NPP Survey, first suggested by Kudpung, aimed to collect information on current patrollers to base improvements to the NPP process. The results: Most new page patrollers are over 18 years of age, most have at least an undergraduate degree, and most are very much clued into what they are doing. The report – while providing a better view of who the average page patroller is and what the average page patroller does – substantially refutes some stereotypes.
History of the process
Patrolled edits is a software feature that went live on Wikipedia in November 2007, after a proposal to re-enable anonymous page creation (see previous Signpost story). The feature draws on Special:NewPages, which lists newly created articles on Wikipedia, although it can be extended to any other page category by using a drop-down option. The interface lists recently created pages in descending order (the most recent first), allowing users to easily browse recently created pages. New pages are kept on the list for 30 days, after which they disappear from view.
A de facto new pages patrol group has existed since at least March 2004. Its members used the Special:NewPages page directly, a page that has been part of Wikipedia's infrastructure since the introduction of the Phase II software in January 2002; before then, the New topics page was used for this purpose. Problematic new articles were a large part of the votes for deletion process, the unified deletion process that covered all namespaces in earlier days of Wikipedia. (Today, there are separate discussions depending on the type of page to be deleted.)
The introduction of patrolled edits provided a better process for patrolling new pages. Unpatrolled new pages are highlighted in yellow, and patrollers can choose to browse just those unpatrolled new articles. Once viewed, any autoconfirmed editor can mark an article as patrolled by clicking the "[Mark this page as patrolled]" link at the bottom right corner of the page. If the patroller thinks the article is ready for mainspace, or they have added multiple maintenance tags or nominated it for deletion, the article can be marked as patrolled. Articles created by administrators were automatically marked as patrolled by the system; this was unbundled in June 2009 (or, rather, a system of whitelisting was formalised with the creation of the autopatrolled user right), so that now any prolific creator of valid articles can have this privilege. This keeps most of the articles created by established editors (familiar with inclusion criteria) out of the pool of those needing to be patrolled.
Because Special:NewPages only holds onto articles for 30 days, unpatrolled articles that survive that long drop off the list, making them exceedingly difficult to find later. The length of the queue of unpatrolled articles has oscillated for years, but there have been occasions at least as far back as 2009 when there were insufficient numbers to patrol all new articles. Due to the 30-day cutoff there is a constant pressure to keep up with the list, in the face of what many patrollers see as a shortage of reviewers. Coupled with the pressure to "get it right the first time" (because once an article is marked as reviewed, other patrollers no longer see it), the process has been characterized as stressful by some patrollers. The front of the queue can also be stressful if an individual patroller tries to keep pace with the flow of new articles.
The patrollers themselves have come under fire; critics contend that they "deal with new users inappropriately, scaring them off, that they have an unacceptably high error rate when tagging pages for deletion, and that they are more interested in using New Page Patrol as a route to gaining higher userrights [like adminship] than in actually patrolling productively and helping improve new articles" (NPP survey). Many critics consider the underlying issue to be that patrollers were overwhelmingly young and inexperienced editors, much more likely to act immaturely or without regard for new editors or new articles. Others believe that it is the stress of the process that drives the perceived errors among its users.
In response to perceived problems with new page patrolling, on April 3, 2011, a proposal was put forward to reduce the stress in the system by limiting article creation to autoconfirmed users. Snottywong summarized the reasons for this with an analysis that showed that only 17.5% of autoconfirmed users' articles were deleted, compared with 72.5% of new editors' contributions. A second proposal soon after established a "clear consensus for a six-month trial, followed by a one-month period of discussion to determine the trial's effects"; the specifics were finalized at Wikipedia:Autoconfirmed article creation trial.
Yet when the technical solution was submitted for implementation on Bugzilla, it was shot down by the Wikimedia Foundation, being marked as a "RESOLVED WONTFIX". WMF deputy director Erik Möller said that "creating a restriction of this type is a strong a [sic] statement of exclusion, not inclusion, and that it will confuse and deter good faith editors."
Instead, the WMF proposed that there be improvements for the interface that managed the new page backlog, with one goal being to better welcome constructive editors into the community. The refusal to accept community consensus was extremely controversial; many editors considered it damaging to the relationship between the Foundation and Wikipedia's editors. The Foundation focused on a "New Page Triage", an initiative that Möller hoped would "reduce the work involved in patrolling new pages by simplifying and smoothing out the process ... a system that is self-explanatory for newer editors; someone who hasn't done New Page Patrol before can look at it and gain an instinctive understanding of what they're expected to do and how they should work."
The upshot of these developments was the new Zoom interface, a redesign of Special:NewPages. It has a dynamic construction that allows users to mark a page with any of several specific templates, both for maintenance and deletion, before saving their changes and continuing on to the next article. One other change that is currently under consideration by developers is the addition of a patroller user-right to control who can access the interface.
It was quickly found that there was no single way that patrollers did their work, and that they were using a variety of third-party software for their work. The lack of understanding page patrollers led to Kudpung creating a survey to find out more about what an average page patroller did, and who they were.
Survey results
[W]e can confirm that the common stereotype of patrollers as young, poorly educated and ignorant is almost entirely without basis. The vast majority of patrollers are over 18 and have undergraduate degrees or above – in some cases, actually exceeding the average for editors overall ... They are largely familiar with relevant policies, and greatly exceed the expectations set by the stereotype. Indeed, the only major difference between patrollers and any other editor is that patrollers choose to patrol.
Editors were identified as patrollers and surveyed based on three separate sources: 2,504 from a script by Snottywong, the 1,300 editors with a {{User wikipedia/NP Patrol}} template on their userpage, and those 133 with a {{User Newpages with Twinkle}} template. Of the editors asked to fill in the survey, 1,255 did so, but after removing surveys with incomplete answers, errors, and obviously fallacious data (e.g. 10-year-olds from Africa with doctorates), and making adjustments because a number of editors had been mistakenly asked to participate, the total included in the survey was reduced to 309 participants. To supplement the survey, an analysis was done on the top quartile of editors by number of patrol actions. A summary of the results follows.
- Demographics
Demographically, New Page Patrollers were found to indeed be overwhelmingly North American and European, with these regions accounting for 85 percent of those surveyed. Only 8% identify as females, consistent with cross-wiki average found during the April 2011 Editor Survey.
More than 60 percent of patrollers have been editing since 2006 or earlier, the antithesis of the stereotype of inexperienced editors. But this creates a new concern: the absence of new editors, accentuating an emerging wikigeneration gulf and the recently highlighted new editor retention collapse, and strongly supporting the need for a more usable, intuitive interface.
Between 79 and 82 percent of responders were over the age of 18; more than 90 percent had completed secondary schooling, and 63 percent had an undergraduate degree or postgraduate qualifications. In short, new page patrollers are not much different from the rest of those who edit Wikipedia.
- Editing activity
Patrolling distribution was found to have a prominent Long Tail-distribution, with 89% of the work done by 25 percent of the patrollers. The report states that "we clearly need to make involving more users, and involving patrollers to a greater degree, a priority". 64 percent of new page patrollers spend between 1 and 3 hours a day reading and editing Wikipedia. 46 percent of patrollers have made 10,000 edits or more; this is a marked difference from the Editor Survey 2011, in which only 20 percent of editors had reached this count, more evidence against the belief that many patroller problems come from inexperience. In terms of user rights, more than half of patrollers were rollbackers and reviewers, and more than 40 percent had autopatrol rights.
A question on non-patrolling activities found that patrollers did other things: 97 percent were active in anti-vandalism efforts in some form, and 95 percent were adding content to Wikipedia, by creating and editing new articles. As expected, Articles for Deletion, speedy deletion, and similar venues were tied in with the process, and many editors who patrolled new pages also participated in discussions there.
An analysis of tool usage found that a large percentage of page patrollers were aware of and use semi-automated tools like AutoWikiBrowser and especially Twinkle in their work.
- Patrolling
The vast majority of patrollers originally learned about New Page Patrol passively, for instance from a userbox on somebody's userpage that advertised New Page Patrol, or through seeing a new page at Special:RecentChanges and navigating to Special:NewPages from there. Most patrollers also give positive reasons for their motivation: they want to "keep vandalism and bad-faith pages out of Wikipedia" (83 percent) and "watch over the quality of new articles" (80 percent). 35 percent of patrollers were motivated because it "provides experience that may be valuable further down the road".
Activity is unevenly distributed, with almost 40% of patrollers spending 1 hour or less on New Page Patrol per week. How long it took to patrol an article was more evenly distributed, splitting half-and-half at the one-minute line. 28 percent of patrollers found that "trying to decide what should be deleted/if something should be deleted" was the most stressful part of the job, with the rest listing a slew of other reasons. An overwhelming majority have read the various relevant deletion guidelines, and very nearly 100 percent have read the speedy deletion guidelines. About 45 percent of editors reviewed from the front of the Special:NewPages buffer (the newest articles), just under 30% from the back, and 15% chose "Other."
- Improvements
By 53 percent to 45 percent, a majority of patrollers disagreed with the implementation of a patroller user-right. If the right were to be instituted, the largest group of respondents felt that it should be granted automatically at some point, with slightly less support for distribution through Requests for permission. Based on the results, the survey concludes that "it seems clear that some variation on 'X edits and Y months as an editor' is likely to be the most acceptable criteria, but ... any attempt to get firm consensus on this point, whether made by the community or by the Foundation, is likely to be drawn-out and gruelling." Finally, 60 percent of new page patrollers wanted to see technical changes implemented, while 20 percent wanted to see cultural and policy-based decisions. The remaining 20 percent did not comment, or felt that the current system basically works.
The future
What now? The results of the NPP Survey clearly refute many of the views held of patrollers by their critics; nonetheless, it is often the patrollers themselves that are clamoring most loudly for changes.
According to the survey's conclusion, "the next step for the Foundation is to use this data to continue developing the Zoom interface. We have already identified a representative sample of patrollers, and contacted them for detailed interviews and to provide "screencasts" of their patrolling work. With these, developers can examine the process of patrolling, get more details on precisely how patrollers do their work, and try to identify unnecessarily difficult areas that can be simplified to make patrolling easier."
The survey broke new ground by giving everyone a clearer view of a subset of Wikipedian editors; perhaps, in the future, similar surveys will provide detailed information about other specialized groups of editors.
The New Page Patrol survey was conducted by the Wikimedia Foundation's Oliver Keyes, with contributions from staffers Howie Fung and Dario Tarborelli and from Wikipedians Kudpung and Tom Morris. The raw data, sans gender and contact information (per policy), will be made available to anyone willing to sign a non-disclosure agreement (requests can be sent to okeyes
Have a keen interest in or strong feelings on new page patrolling or another issue of relevance to the English Wikipedia community? The Signpost is recruiting reporters and soliciting opinion essay submissions; those interested should apply to wikipediasignpost
Discuss this story
Random Section Break 000
It's ridiculously short-sighted that WMF dismissed the idea of only allowing autoconfirmed users to create articles. This is one of the reasons established editors choose to leave WP. 76.185.111.45 (talk) 03:18, 21 February 2012 (UTC)[reply]
And my rant above assumes that the problem is simply in Wiki markup. Maybe when a given user says editting is "too hard", the user may refer to another part of the experience, such as how to add content so it improves the article, how to best organize an article -- or maybe the problem is (my pet peeve) that they honestly don't know how to research a given topic so they can provide information matched to a reliable source? All I know is that public schools didn't do a good job of teaching kids how to do research, & after reading many articles on Wikipedia I see they still don't do a good job of it more than four decades later. -- llywrch (talk) 19:09, 23 February 2012 (UTC)[reply]
(outdent) Those who complain most vociferously about NPPers and perpetuate this claim that they're mostly young, uneducated, dumb males tend to be extreme inclusionists and people who are mad because their article was deleted. This is how we end up with disruptive breaching experiments like WP:NEWT that end up driving off the most experienced and productive NPPers and result in . - Burpelson AFB ✈ 20:41, 21 February 2012 (UTC)[reply]
First, I do think it's important that the WMF understand the depth of feeling, the lack of confidence and trust, that the ACTRIAL refusal aroused. When I read in the discussion one of the Foundation representatives saying that the sort of crap articles that ACTRIAL would have helped reduce were not a problem because we could "make a policy that marks "crap" articles as being poor quality, then funnel our efforts into make them good", the idea that people so out of touch with the reality of the New Page list could refuse, for what appeared to be ideological reasons, the considered request of en:wp for a change in its internal procedures made me stop reading and bang my head against the desk. When that was followed by the Indian train-wreck, many of us thought that, if the WMF was indeed on the same planet, it was certainly not pulling in the same direction as those who were trying to build an encyclopedia. It is regrettable that the volunteers should see the WMF as the enemy, but it will take a long time to rebuild trust: next time a similar situation arises, think very hard before using your ownership of the servers to over-rule a project's decision about how to run its affairs.
I see that Erik's message you linked above ends "If you don't trust WMF, you can - and probably should - contribute your effort elsewhere, because WMF may - and probably will - do things you won't like." In view of concerns about loss of established editors, that is an unfortunate attitude; it certainly means that WMF should be concerned about losing the trust of the volunteers.
The second reason is that, if I understand the reason for the ACTRIAL refusal, it was a belief, a WMF axiom, that no kind of obstacle must ever for any reason be placed in the way of anyone wishing to write an article.
Now, if you are concerned about editor retention and about newbies finding Wikipedia unfriendly, consider this: many of the new pages that get deleted are not the work of vandals or incompetents, but of good-faith new editors who have not understood what Wikipedia is and what it is not. A policy of "Welcome everyone in, no barriers" means that people sign on who think it is another Myspace or LinkedIn or a free advertising noticeboard, and are here only to post their CV or write about themselves, their garage bands, their self-published books, their companies or their school netball teams. We let them go all the way to writing an article, sometimes putting a great deal of effort into making it look good, before flagging it for deletion and telling them: "Sorry, this is not an encyclopedic subject, no amount of editing will make it acceptable."
Many of the people who complain that Wikipedia is unfriendly, and the "unretained" editors, are of this type. Painting the site pink and decorating it with fluffy kittens will not make these people happy. If you really want to increase percentage editor retention, you should discourage those who are not here to build an encyclopedia from becoming editors at all: replace the "Everyone welcome, come on in and edit!" sign with one that says "This is a project to build an encyclopedia. If you would like to help with that, you are very welcome; but if you looking for somewhere to write about yourself, your friends, your company, your band, or anything you are closely associated with, this is probably not the site for you: Myspace is that way". Stop trying to lure in the maximum number of new editors, and focus on recruiting ones who are able and willing to help build an encyclopedia.
JohnCD (talk) 23:36, 22 February 2012 (UTC)[reply]
Random Section Break 001
And one other question. Why has Sue Gardner said nary a thing on this subject? One would think that ACTRIAL was a sufficiently momentous desire to change that she'd want to be involved, and when Foundation people shot it down one would think she'd want to opine, if for nothing else than to either push the Foundation people to let the trial go or make it appear that it was actually the WMF making the decision rather than a few people injecting their own personal analysis. The Blade of the Northern Lights (話して下さい) 01:15, 24 February 2012 (UTC)[reply]
Random Section Break 002
Bots in April
I don't remember the big group of bot page creations in April. What were they? Nyttend (talk) 20:31, 19 February 2012 (UTC)[reply]
New Page Patrol survey data
The article says "obviously fallacious data" includes, for example, claims to be "10-year-olds from Africa with doctorates".
And what is it about the phrase "from Africa" that makes the data even more improbable? Does no one see anything wrong with such a stereotype? Kablammo (talk) 00:37, 27 February 2012 (UTC)[reply]