Are my children getting a decent education?
It's a question asked by parents everywhere, particularly when they read about international studies that rank countries based on students' answers to reading comprehension questions or math problems. Though controversial, these student achievement surveys can turn a spotlight on deficiencies and galvanize political support for educational reform. In the United States, for example, poor showings on standardized tests recently resulted in major government initiatives to improve mathematics and science education.
So how do students in Latin America and the Caribbean fare in these rankings? Until recently, answers were hard to come by. With few exceptions, the region's countries have declined to participate in international evaluations of educational achievement.
But now, for the first time, citizens in 11 of the region's countries have a rare chance to see how their school systems measure up (see graphs at the left). Over the last five years, through an unprecedented effort coordinated by UNESCO and partly financed by a $750,000 technical cooperation grant from the idb, these countries participated in a massive study of the language and math skills of third and fourth graders. The results of this First International Comparative Study, run by UNESCO's Latin American Laboratory for Assessing Quality in Education, were released last December.
The tests used for the study were designed to reflect the curriculum goals and realities of participating countries. Education officials in each country submitted detailed descriptions of what third and fourth graders are taught and what they are expected to learn by the end of the school year. The language and math questions for the study were developed through a consensual process so that every country was satisfied that the questions were appropriate.
Some 4,000 students in each country took the test. Participating schools were selected to represent three demographic categories: very large cities (one million or more inhabitants), urban centers with less than one million people, and rural areas. Within urban areas, the study also distinguished between public and private schools. Finally, students, teachers and parents at participating schools filled out supplementary questionnaires on a number of issues.
At first blush, the results of the study are discouraging. On average, students correctly answered only 48 percent of the math questions and 62 percent of the language questions. (UNESCO standardized each country's test results and assigned the number 250 to the regional mean, or average, for combined third and fourth grade scores. The graphs on these pages display only fourth grade results relative to the regional mean.)
With the exception of Cuba, which scored far higher than average in every one of the tests, the results were surprisingly homogeneous in view of the wide differences in the development indicators of participating countries. Argentina, Brazil, Chile, Colombia and Mexico performed roughly the same in both math and language, while Bolivia, Honduras, Paraguay, the Dominican Republic and Venezuela performed at or below the average. Peru and Costa Rica participated in the study but have not disclosed their scores.
As expected, students in urban areas and private schools scored hightest. But a country's income levels turned out to have a smaller than expected impact on its education achievement: Cuba, with one of the region's lowest per-capita income levels, did best, while Venezuela, one of the region's highest income countries, was among the poorest performers.
Although the UNESCO test results cannot be precisely compared to other international tests, the anecdotal evidence is not encouraging. Colombia, for example, was one of only two Latin countries to participate in the Third International Mathematics and Science Study (TIMSS) in 1996 (Mexico participated but did not authorize publication of its results). Out of 42 countries participating in the timss, Colombia ranked 41st, and only 4 percent of all Colombian eighth graders scored in the top 50 percent of students worldwide. Yet Colombia was among the better performing countries in the UNESCO study. Also, the math portion of the UNESCO tests covered fewer topics and in less depth than the timss.
The UNESCO results are only a preview of a much more detailed analysis of the test data that will be released later this year. That report will explore the relationships between test results and factors addressed in the supplementary questionnaires, including students' preschool attendance, parents' socio-economic and education level, availability of textbooks and other classroom resources, student/teacher ratios, teachers' level of education, and teachers' attitudes and philosophies. Laurence Wolff, an idb education consultant who worked with UNESCO on the study, said even more valuable information could eventually emerge from a detailed analysis of data on individual countries. "That kind of analysis could let us draw meaningful conclusions about which policies and practices contribute to better results in a specific setting, and which don't," he said.
In short, future analysis should help to answer many of the questions raised by the UNESCO test results, including why Cuba did so much better than every other country. "We really don't have enough information yet to explain that," said Wolff. But participants at a recent seminar on the study at idb headquarters in Washington, D.C., speculated that small student/teacher ratios, high education levels among teachers, and the high rate of preschool education among Cuban children could be part of the answer.
To the extent that it helps to answer these questions, the UNESCO study could turn out to be a goldmine of information for countries that are serious about reforming their schools and bringing them up to internationally competitive levels. Nearly all Latin American and Caribbean countries have created permanent educational assessment agencies over the last decade, and many are receiving financial assistance for these efforts from the idb. Hopefully, a future international student achievement study will show the extent to which today's efforts are bearing fruit.