IDB-based pilot project sheds light on new teaching approaches to improve learning in poor areas
Since Latin America consistently underperforms in international student assessments, governments in the region are looking for pedagogical models to help improve student learning. The region’s students still perform below students in rich nations and East Asian countries, particularly in the areas of mathematics and natural science.
In response to low achievement in science and mathematics in Argentina, the Ministry of Education and the Inter-American Development Bank worked together on a pilot project to learn about more effective pedagogical approaches to teaching natural science and mathematics in primary schools.
The pilot tested the effectiveness of three innovative inquiry-based models introduced in two Argentine provinces during the 2009 academic year. The models constitute a sharp departure from teacher-lead demonstrations and a simple transmission of concepts. They include two different pedagogical models—Science, Technology and Creativity (CTC) and the Scientific Literacy Program (PAC)—and a play-based mathematics approach called Mathematics for All. In all three models, students had the opportunity to pursue rich lines of inquiry under guidance from their teachers.
The project was implemented to gather information about the pedagogical models and their effects on the learning process. The evaluation, carried out by a team of external evaluators from the Catholic University of Uruguay (UCUDAL) and the International Institute for Education Planning (IIPE), was an inherent part of the project and it produced a wealth of material that is being analyzed by the Argentinean authorities.
Measuring the Impact
The evaluation had both quantitative and qualitative components and considered a wide range of parameters including: the effects on achievement; the teaching environment, including classroom dynamics and gender relationships among student groups; the teachers’ subjective representations concerning the students’ learning capacity; the teachers’ subject area and pedagogical knowledge; and the models’ sustainability in terms of the cost of expanding them to the national level and the durability of pedagogical inputs.
The three models were tested through an experimental evaluation in which schools were randomly assigned to either one of the treatment groups (PAC, CTC and Mathematics for All) or to the control group that received the standard program in natural science and mathematics. The pilot covered over 18,100 fourth-grade students in 675 schools in the provinces of Tucumán and Buenos Aires. These two areas were selected based on disadvantaged socioeconomic characteristics and poor educational results.
The quantitative evaluation consisted of a standardized test at the beginning of the school year in March 2009, before the initiation of the pilot, and again at the end of the pilot in December 2009. As the pre-pilot test could not be administered to the entire pilot population, an evaluation sample was formed from 56 randomly-selected schools. The total number of fourth-grade students in the sampled schools was 5,892.
The sample groups were compared along many dimensions, including the repetition rate of the schools in the group, their student-teacher ratio, the seniority of their teachers, the condition and characteristics of their infrastructure, and the availability of teaching materials, equipment, and support staff.
The qualitative evaluation consisted of systematic observation of teaching practices and changes in pedagogy. Extensive information concerning characteristics of the schools, students, teachers, families and community contexts were collected through surveys, interviews and classroom observation.
Before and After
The surveys to establish a baseline for comparison before the project started showed that teachers lacked content knowledge and interest in teaching both subjects. In mathematics, students were perceived to lack interest and to have problems focusing on tasks. In natural science, they were described as interested but unable to focus. Parents were perceived to lack interest in their children’s education.
A year later, new surveys were conducted in the schools participating in the pilot project. The results of the new qualitative evaluation revealed improvements in teachers’ knowledge of curricular concepts, professional self-image, and beliefs about their students’ ability in these subjects. The results of the quantitative evaluation indicate that all beneficiary students improved in their learning more than those in the control group.
The mathematics model had the strongest effect on student learning, with the average test score increasing by 44 points compared to 19 points among students who received the traditional curriculum. The impact was particularly strong in Buenos Aires, where the difference between the two groups was 34 points.
In the case of the two science models the change was significantly different from the control group only in the province of Buenos Aires in the case of the CTC and only in Tucumán in the case of PAC. For all three models, the effect sizes varied by module, providing important information on what dimensions need improvement. For example, in the Mathematics for All program the greatest progress was observed in arithmetic.
The pilot project is providing valuable information on pedagogical approaches and curriculum materials that work in situations where teachers have important content and pedagogical gaps, and students come from socioeconomically disadvantaged backgrounds. Overall, the results indicate that learning increases as pedagogical models move away from the current practice of formula memorization.
The project also showed that many teachers felt uncertain about how to move away from traditional teaching methods. The qualitative evaluation highlights that a key aspect in addressing this issue was the combination of the more traditional teacher training with continuous individual and group tutoring of teachers.
Based on the results of the economic analysis of the pilot, PAC will be brought to a national scale through a Bank-financed operation. Mathematics for All will also be scaled up to benefit students throughout the country. Given the short timeframe of the pilot, the experimental application of the CTC model will be continued in 200 primary schools.
The findings are in the IDB’s 2010 Development Effectiveness Overview, which discusses the development impact of several ongoing IDB projects in Latin America and the Caribbean and reports on the progress the Bank is making to boost transparency and accountability.
(Story updated on April 1st to name institutions that conducted the evaluation.)