Grants:PEG/Aislinn Dewey/Distribute WikiReaders to Schools/Report

Report accepted

This report for a Project and Event grant approved in FY 2013-14 has been reviewed and accepted by the Wikimedia Foundation.

To read the approved grant submission describing the plan for this project, please visit Grants:PEG/Aislinn Dewey/Distribute WikiReaders to Schools.

You may still comment on this report on its discussion page, or visit the discussion page to read the discussion about this report.
You are welcome to Email grants at wikimedia dot org at any time if you have questions or concerns about this report.

Compliance and completion

Did you comply with the requirements specified by WMF in the grant agreement?: Answer YES or NO.; YES

Is your project completed?: Answer YES or NO.; YES

Did you use any of the grant funds?: Answer YES or NO.; YES

Activities and lessons learned

WikiReaders used by students studying in Herminino Salas Gil in Temacunchale, San Luis Potosi, Mexico

This section describes what the grantee did, and what the grantee learned from implementing the project. This section should be useful to others implementing similar projects and is an opportunity for the grantee to reflect on the project's performance.

Activities

Provide a detailed list of activities performed to complete this project, descriptions of these activities, and the amount of time spent on each activity. This section should also include a list of participants, or a link to pictures, blog posts, or videos from the project or event.

The initial phase of this project was preparing surveys to measure impact of this project. Over a few weeks, in collaboration with each school (India, South Africa and Mexico) over email and google docs - three surveys we created to both qualitatively and quantitatively measure success and impact of the project. Once the content was defined, Miguel Balderas, contact in Mexico City connected to the rural Tamanzuchale school Herminino Salas Gil, with help from Ivan Martinez, head of the Mexico Wikipedia chapter translated the surveys. All schools found a way to get the surveys from Google Docs and print them for the students and teachers. The surveys were then scanned in and emailed back using Dropbox or Evernote, except for the South Africa school who mailed back hard copies.

Once the devices arrived from Taiwan, the initial survey was given in all schools. The timing was more delayed than expected as all schools had to pay a large customs fee. India and South Africa's devices were stuck in local customs offices for nearly 3 weeks. On arrival, the principals and teachers discussed the project with students before handing them out. In all schools, both teachers and students for 1-4 classes received devices to use.

Coordinators in each region were:

Miguel Balderas, Graduate student connected to school Herminino Salis Gil, located in Mexico City
Ivan Martinez, Head of Wikipedia chapter, Mexico City
Gustav Sandoval Kingwergs, Volunteer, Wikimedia chapter, Mexico City
Pam Robertson, teacher, Sinenjongo High School, South Africa
Molly Sebastian, non-profit coordinator, GMVS India, St. Anthony's School

Surveys came back roughly every month and a half and were analyzed at the end of the project. Final results were compiled into a spreadsheet here.

Lessons learned

What lessons were learned that may help others succeed in similar projects? Consider the following questions and respond with 1 - 2 paragraphs.

What went well?

The impact of Wikipedia to students, more so than teachers, was profound. Nearly every student reported having learned more from using their WikiReader by searching for words, concepts, places and things they did not know about. The majority of students also felt it was easier to do their homework using this device (2/3 schools asked). The range of topics students looked up was wide-ranging and fascinating to go through as it made me realize how they are so curious for information and answers to so many things most internet-connected people take for granted. As a student in India explains:

Yes. Before taking WikiReader not understand hard word. After taking WikiReader understand hard word, calculator also.

Another interesting success, was the sharing of devices within a family unit. By having the devices to take home in 2 out of the 3 schools, it was common for students to share it with brothers, sisters, parents and extended family living together. The impact, once in a household became greater, as the device was now a resource for 4 or 5 people. Once quote that illustrates this from a South African student:

My mother, she was researching about the party that helps cleaners know their rights.

Another interesting side note and unexpected value of these devices were their calculator function. We were unaware the devices had this capability, and most students figured it out and used the device for all subjects.

What did not go well?

One issue was the cost of import - especially in India and delay in getting the devices out of customs. A second, more significant issue to the sustainability of this project, was a technical malfunction detected in large numbers with the device batteries not holding a charge. The biggest impact of this was in India, as they received the most devices, and reported that 20-30% of their devices by the third month were not holding a charge and could therefore not be used. They sent three back to be inspected by the manufacturer who only found one of the three defective. Its unknown what is causing this issue and if it exists in other devices that are still available for sale. One last issue, was getting surveys from South Africa as it was hard for the teachers to find the time to mail it back.They seemed somewhat less engaged than the Mexico and India schools.

What would you do differently if you planned a similar project?

I would try to connect with someone in the Education ministry of government to try to negotiate a way to get the devices into the country without such high customs fees and taxes given they were being used for a non-profit project. Second, I would want to ensure the manufacturer provided a warranty on the devices for at least a year to avoid any of the defective device issues we encountered. I would also want to line up more volunteers to help analyze the data and coordinate the logistics on the ground as it was difficult to do it on my own.

Project goal and measures of success

This section should reference the project goals and measures of success described in the approved grant submission. See Grants:PEG/Aislinn Dewey/Distribute WikiReaders to Schools to review the goals and metrics listed in the approved submission.

Project goal

Provide the project goal here.: The goal of the project is to run a 3 month pilot of the distribution of 500 WikiReaders that provide offline Wikipedia for kids to learn and do homework with and for their teachers to create lesson plans with. We have raised roughly $3,000 on Indiegogo to start and have selected three project sites - one in a small rural town outside of Mexico City a second in Capetown, South Africa and a third in Uttar Pradesh India. The price of $15 we have negotiated with the manufacturer includes batteries, an 8GB memory card and a custom language install of the latest version Wikipedia (text-only). We have identified three schools that have a huge need to access knowledge without internet in Mexico and India and Capetown, South Africa. The India site, in Uttar Pradesh, has already used WikiReaders that they had purchased full price ($99) a year ago and have had great success using them. We would like to include 3 pilot sites and with more devices, and a second grant upon success, to expand to more schools. Several schools that would like to use WikiReaders in their classrooms by both teachers planning their lessons and students to do their homework have contacted us requesting devices so we know there is a clear need. We are planning to partner with Wikipedia chapters to help coordinate distribution and provide simple training to the schools as is possible.

The WikiReaders sent will be supported by reporting materials and disposable cameras for the teachers and students document their experience and every month to measure the impact of the devices on learning throughout the school year. We will also employ the help of Wikipedia chapters and volunteers, as willing, to help with distribution and occasional project feedback. We will publish and share all reporting to the Foundation and donors on request. If this pilot proves successful at distributing WikiReaders to remote schools in an impactful way, we aim to subsequently buy all remaining WikiReader stock (approx ~20,000 additional devices) to expand distribution to other project sites that are equally in need.

Did you achieve your project goal? How do you know your goal was achieved? Please answer in 1 - 2 paragraphs.: Yes, we successfully raised the funds needed to run this pilot in the budget we estimated and received a wealth of useful feedback around how teachers and students use the devices, the and the value of offline Wikipedia. The pilot ran smoothly with only a few logistical issues to work through. The quality of data we received back was very high and included both quantitative surveys, photos and videos from the schools students and teachers.

Despite achieving the project goal of implementing the 3-month pilot, the overall success of the pilot is mixed. The content the device provided (offline Wikipedia in English or Spanish) was overwhelmingly useful to all participants and the schools were extremely grateful to be given access to it. The device itself, however, had enough flaws in its performance, to make me question whether purchasing more would be worthwhile given the costs of the device and shipping. Powering the device was the biggest issue and the source of the problem has not been uncovered. The quality of batteries the students used, the power button tension turning the device on accidentally or an electrical problem are all possible reasons for this issue.

Measures of success

List the measures of success exactly as provided in the approved grant submission, and evaluate your project according to each measure listed there.: Teachers and school administrators at project sites positively assess of the quality of learning after using WikiReaders for a single school year; Teachers assess their teaching abilities and lesson plans positively with the use of WikiReaders as an educational tool compared to previous methods; Kids in schools where devices are distributed assess positive improvement in their learning experience and homework completion

Provide an overall assessment of how your project went according to these measures.: Through our surveys in each of the schools, we assess that our pilot proved positive results in learning from both students and teachers while using these devices. As one teacher explains:

The students are motivated, because we are far away from the town and also they do not have access to the internet, so the Wikireader is a easy tool.

The pilot has only tracked 3 months of time so it is unknown whether the devices will have continued value over a year. The devices with technical issues most likely will not continue providing value due to the cost of having to replace batteries frequently.

Survey results across these key measures for Students are below. Responses received from 83 South Africa, 60 Mexico and 185 India students. Additional qualitative data is reflected in Pilot Highlights at the end of this report.

Survey Question	India	South Africa	Mexico	Overall
Have you used your WikiReader in the past few weeks?	100% Yes	100% Yes	100% Yes	100% Yes
Has anyone in your family used the WikiReaders?	45% Yes	80% Yes	0% Yes - All kept at school	62% excluding Mexico
Is it useful to you?	93% Yes	95% Yes	98% Yes	95% Yes
Have you had any problems with it working?	83% Yes	48% Yes	22% Yes	51% Yes
Do you think WikiReader made it easier to do your school work?	98% Yes	No Response	100% Yes	99%
Did you find it difficult to use?	7% Yes	6% Yes	14% Yes	9% Yes

Survey results across these key measures for Teachers are below. Responses received from 8 Mexico and 38 India teachers. No South Africa teachers were able to complete surveys.

Survey Question	India	South Africa	Mexico	Overall
Have you used your WikiReader in the past few weeks?	100% Yes	No answer	100% Yes	100% Yes
Have you had any problems with it working?	25% Yes	No answer	33% Yes	29% Yes
Is it useful to you?	95% Yes	No answer	100% Yes	97.5% Yes
Did you find it difficult to use?	5% Yes	No answer	0% Yes	2.5% Yes
Do you notice improvements in students after using WikiReaders?	Not included on survey	No answer	100% Yes	100% Yes

Full survey results spreadsheet is here.

If you were to plan a similar project, would you measure it differently? If yes, please explain how.: Yes, one thing I realized having talked to a Microsoft researcher about running pilots in education is that I should have done a more comparative assessment to show two similar classes performance and improvements with or without a WikiReader. Otherwise, I feel the methods I used to measure success were good.

Impact

This section ties this project to Wikimedia's broader goals, and shows what the project accomplished.

What impact did this project have on WMF's mission and the strategic goals? Please answer in 1 -2 paragraphs and include specific measures of impact such as the number of readers or editors reached by a particular project, or the number of articles edited or improved.

Stabilize infrastructure
Increase participation
Improve quality
Increase reach
Encourage innovation

This project provided Wikipedia access to kids and teachers that are in areas where traditional internet-based access is not possible. By providing offline access in a language appropriate to the schools that are receiving them, we provided a library of knowledge to those most in need of it to learn about the world. Distribution of these devices increased reach to areas often ignored by typical technology solutions. By giving WikiReaders to a single student and seeing that students shared them with their family (80% of students in the South Africa school and 45% of students in the India school that received devices shared them with their family), the web of access quickly stretched to cover and area most wouldn't imagine a single device could. Overall, the reach of Wikipedia went to 480 students and teachers and the larger web of people they exposed to Wikipedia.

During this project, several other schools reached out requesting to get these devices for their schools. This project has, per Wikimedia's mission statement, disseminated educational content effectively and globally. Offline is the best approach to increase reach to those without the technology infrastructure of first-world nations. The project is an example of value that can be leveraged by the community to innovate and develop offline solutions for internet restricted areas of the world.

Reporting and documentation of expenditures

This section describes the grant's use of funds

Documentation of expenditures has been received by WMF.

Documentation

Did you send documentation of all expenses paid with grant funds to grants at wikimedia dot org, according to the guidelines here? Answer "Yes" or "No".: Yes

Expenses

Please list all project expenses in a table here, with descriptions and dates. Review the instructions here.: These expenses should be listed in the same format as the budget table in your approved submission so that anyone reading this report may be able to easily compare budgeted vs. actual expenses.; Note that variances in the project budget over 10% per expense category must be approved in advance by Project and Event Grants program staff. For all other variances, please provide an explanation in the table below.

Number	Category	Item description	Unit	Number of units	Actual cost per unit	Actual total	Budgeted total	Currency	Notes
1	Devices	WikiReader devices with packaging, 8GB memory cards and two AAA batteries	1 device	500	15	7,605	7,500	USD	Additional costs were from $105 of wire transfer fees from my bank to Pegatron Taiwan
2	Shipping	DHL shipping to three countries from Pegatron Taiwan	1 shipment	3	avg 203	610	1000	USD	Shipping costs were under budget
3	Customs	Fee each country had to pay to receive the DHL packages	1 shipment	3	India $1254, South Africa $296, Mexico $473 wire transfers fees $140	2163	1608	USD	Actual costs were over what was budgeted under shipping where we had a buffer we thought would cover customs fees
4	Reporting Materials	Cameras, mailing supplies, postage, surveys, etc for schools	1 packages	1	105	105	1,000	USD	Our contacts were for the most part able to get the surveys from google doc, print, scan and provide photos without us needing to pay for mailing supplies. Cost here was getting Mexico surveys from the city to the school by the Wikimedia Mexico chapter

Approved Project budget

500 WikiReaders at $15.00 USD a piece
$1,000 for shipping to 3 project sites
$1,608 for customs fees (increase requested from additional funds after grant was funded)
$1,000 for reporting materials:

Postage paid packets for the sent to the schools that include:
School worksheets to measure and report out
Disposable cameras to document project

Total cost of project: $11,108

Total amount spent on this project (this total should be the total calculated from the table above): $10,483

Total amount of Project and Event grant funds spent on this project (this total will be the same as the total amount spent if the Project and Event Grant is your only funding source): $8,108

Are there additional sources of revenue that funded any part of this project? List them here.: Indiegogo Campaign

Remaining funds

Are there any grant funds remaining?: Answer YES or NO.; No, all Wikimedia Funds were used. $625 remaining from Indiegogo campaign are being used to re-imburse India school for shipping back defective devices to the manufacturer.

Please list the total amount (specify currency) remaining here. (This is the amount you did not use, or the amount you still have after completing your grant.)

0 remaining of Wikimedia funds, $625 remaining of Indiegogo funds

If funds are remaining they must be returned to WMF, reallocated to mission-aligned activities, or applied to another approved grant.

Please state here if you intend to return unused funds to WMF, submit a request for reallocation, or submit a new grant request, and then follow the instructions on your approved grant submission.

No

Appendix

School Bios

Telesecundaria Herminio Salas Gil, Tamanzunchale, Mexico

Grades 1-3 (12-15yr olds), 213 students total

Number of students who were given a Wikireader to learn and study: 70 Wikireader
Number of teachers to teach using Wikireader: 8 teachers
Computer Equipment for the school: No internet lab
Distance to internet café or library in the area: One hour.
Average cost of internet use in the area: 10 pesos / approximately 1 dollar.
Languages spoke: Spanish
Contact: Aracely Falcon Arellano, chelyfalarae@hotmail.com, Address: Casa Hogar No. 2, Barrio del Carmen, Telephone:48322108

Sinenjongo High School, Joe Slovo Park, South Africa

Grades 8-12, 800 students total

Number of students who were given a WikiReader to learn and study: 95
Number of teachers to teach using WikiReader: 10 teachers
Computer Equipment for the school: Computer lab with internet access for students 1x per week
Distance to internet café or library in the area: One hour
Average cost of internet use in the area: 20 rand / approximately $2 USD
Languages spoke: English, Xhosa
Contact: Pam Robertson, pamrob@futurenet.co.za, Address: 4 Khozi Drive, Joe Slovo Park, Milnerton, South Africa

St.Anthony's, Uttar Pradesh, India

Grades nursery to 9th grade, 700 students total

Number of students who were given a WikiReader to learn and study: 255
Number of teachers to teach using WikiReader: 38
Languages spoke: English
Contact: Molly Sebastian, mollyseb@gmvs.in, Address: St.Anthony's School, H.S.B Inter College,Eidgah, Galsheheed Road, Moradabad, UP,

Indian partner NGO that supports the schools is GMVS

Pilot Highlights

Surveys over the three months showed a few major findings:

Overall both teachers and students found the WikiReaders very useful in and out of school for looking up words, concepts and learning new things
Most students used the WikiReaders to look up both school and non-school related topics, ex. sports figures, places, celebrities, concepts
Teachers felt WikiReaders help them deepen their understanding about topics they haven't the resources to learn more about
Students want the devices to be backlit, especially in India, so they could study with them at home at night
About half of the students experienced technical issues with their device not holding a charge and by the end of the pilot desired a rechargeable battery
Most India and South Africa users shared their devices with family, the Mexico students were told to keep theirs inside the school and therefore did not
Most users were worried about their devices getting lost or stolen

Photos from the Field

Mexico

Wikireaders being distributed in Herminino Salas Gil in Temacunchale, San Luis Potosi, Mexico

Wikireaders in Herminino Salas Gil in Temacunchale, San Luis Potosi, Mexico
Surrounding area of school Herminino Salas Gil in Temacunchale, San Luis Potosi, Mexico
WikiReaders in Herminino Salas Gil in Temacunchale, San Luis Potosi, Mexico
filling out surveys in Herminino Salas Gil in Temacunchale, San Luis Potosi, Mexico