In the 21st century, the proliferation of digital media and technology has fundamentally changed the way we learn. More than ever, children carry computers in their pockets and ever-expanding internet connectivity promises to reach even the most remote classrooms, putting a wealth of information at student’s fingertips. And there are growing demands from parents, educators, governments, and donors to incorporate educational technologies as part of children’s core curricula.
But how does a teacher or administrator decide which technology is a good fit for their classroom? And especially in a global development context, how does a donor know that an investment in technology is the right approach to ensure learning outcomes — that a donation of tablets won’t end up in the corner after a year, collecting dust?
To answer these kinds of common, but challenging questions, MIT researchers have just launched a new decision-making tool for teachers, administrators, governments, global development practitioners, and other stakeholders trying to make smart decisions about incorporating technology in the classroom.
The tool, “A Framework for Evaluating Appropriateness of Educational Technology Use in Global Development Programs,” is an initiative of the Comprehensive Initiative on Technology Evaluation (CITE), a program supported by the U.S. Agency for International Development (USAID) and led by a multidisciplinary team of faculty, staff, and students at MIT. Launched at MIT in 2012, CITE is a pioneering program dedicated to developing methods for product evaluation in global development.
The framework seeks to help stakeholders explore how well a particular technology may fit their educational context by posing straight-forward questions such as: “Does the technology create a burden of extra management for the teacher?” and “Is there evidence that use of this technology aids learning? Is this evidence generalizable to your context?” Questions fall into eight categories: teachers; students; culture; sustainability; community, social, and political; learning; infrastructure; and scalability and market impact.
The framework was developed following an extensive literature review by MIT researchers, and then tested in India by CITE’s partners at the India Institute of Management Ahmedabad, who looked at the deployment of English language learning technologies by NGO and government initiatives.
Why educational technologies?
Despite the enthusiasm and promise of emerging technologies for education, there are a myriad of reasons technologies can fail that have little to do with the technology itself. Variables such as school funding, teacher preparedness, educational philosophy, and technical infrastructure play a major role in determining whether or not, for example, an English-language learning software actually helps children learn English.
“In doing this research, you realize how often adoption of educational technology is done without much forethought,” says Scot Osterweil, CITE Educational Technology Evaluation lead and creative director of the MIT Education Arcade. “Frequently, the decision to use a particular technology is based on who can make the most appealing sales pitch to the buyer, who will not be the user. Then, [the technology] ends up in a classroom where people haven’t prepared for it. There’s always a need to improve the process, even more so in developing countries where these technologies are new.”
MIT research assistant and PhD student Jennifer Groff adds that it's easy to see the positives about a new technology without thinking through potential challenges.
“Naturally, we get excited by the opportunity of something new,” Jennifer explains. “On a deeper level, we have to pause and ask ourselves what the challenges are to implementing something like this meaningfully. The framework is a tool that we hope helps people think through all of the facets of incorporating something new.”
Testing the framework
To test the framework, CITE’s research partners at the Indian Institute of Management Ahmedabad conducted a pilot study which entailed semi-structured interviews and group discussions with various stakeholders in sites where technologies studied were being deployed. The schools involved in the study varied from the most basic to the most modern, including everything from a rural village school in Uttar Pradesh without a building or dedicated classroom to a series of computer labs in public schools and community centers in Mumbai run by an influential NGO, the Pratham Education Foundation.
“One of the most interesting things that became apparent during our fieldwork was the role of continual intermediation by the developers at the sites of implementation,” said local research lead, Professor Ankur Sarin of Indian Institute of Management Ahmedabad. “Technology still remains an external intervention and facilitators play a critical role in determining the efficacy with which the education technology comes to be used. Based on what we learned, the framework was then adapted to account for the role, background and motivations for the facilitators.”
Putting the framework to use
The tool is designed to be useful for many different kinds of stakeholders who work with educational technologies, including developers, adopters, and funders of new technologies. It can be used before the adoption of an intervention, or as an assessment of an intervention as it is being deployed.
The next phase of work for this framework will be turning it into an online, interactive version that would guide the user step-by-step through a series of questions to identify the potential challenges around deploying a certain technology in the classroom.
“We would also like to create a knowledge network of people in different developing countries interested in using the framework who could help us refine it,” Scot says. “We tested our framework in India, but could learn other things in Latin America, Africa, or other parts of Asia if we identify partners elsewhere who could play a role in supporting or advancing the framework.”
In addition, the researchers hope illuminating the complexities of deploying an educational technology in a developing country setting will add to the existing literature in a meaningful way.
“There is a significant body of literature on educational technology and factors for change in developed contexts,” Jennifer explains. “And sometimes in developing countries, there’s an assumption that a technology is just better than what already exists, so a school is pressured to adopt it with questions. But there are many of the same barriers to change in these classrooms. It’s still change management, which isn’t just about bits and bytes, it’s about the way of delivering teaching and learning being much, much different.”
“As someone who’s worked in this area for 25 years, I know that a class could be successful without any technology,” Scot says. “You should only be using technology when you’ve identified a goal it can help you achieve. And we hope to help stakeholders make smart decisions from the very beginning with this new tool.”
CITE’s research is funded by the USAID U.S. Global Development Lab. CITE is led by principal investigator Bishwapriya Sanyal of MIT’s Department of Urban Studies and Planning, and supported by MIT faculty and staff from D-Lab, Priscilla King Gray Public Service Center, Sociotechnical Systems Research Center, the Center for Transportation and Logistics, School of Engineering, and Sloan School of Management.
In addition to Osterweil and Groff, co-authors on the report include Eric Klopfer, Ankur Sarin, Prateek Shah, Stacey Allen, Sai Priya Kodidala, and Ilana Schoenfeld. CITE conducted its research in partnership with the Indian Institute of Management Ahmedabad.
