No magic bullet

gov. patrick, the Boston Globe, MassINC, the Boston Foundation, the business community, and President Obama are all supporting charter schools as a key step toward school improvement, but a careful look at the data suggests that these schools offer no magic bullet for school improvement.

In general, charter schools (and Boston pilot schools) perform no better — and no worse — than other schools, although there is a cluster of Boston middle school charters with very impressive results.

These conclusions are based on an analysis of MCAS scores, averaged across three years (’06 to ’08) to avoid one-year anomalies. Only the charts for English are shown here, but the same analysis was performed for math, with quite similar results.

Charter or no, the major factor determining student performance is demographics. Specifically, African-American and Hispanic students, low-income students, and students with limited English proficiency (LEP) score lower than other students. These factors work separately — all else equal, an African-American student in poverty will score lower than a white student in poverty and lower than an African-American student not in poverty. Any comparison of charter schools with other schools must therefore take student demographics into account.


chart 1 shows performance for elementary schools, with Boston charter schools shown in green, other charters in orange, Boston pilot schools in lavender, and all other schools in blue. (Download the PDF of this article for a larger view.) Each circle represents one school; the size of the circle is proportional to the size of the school, and every elementary school in the state is included.

Student performance is shown by vertical position (the higher the circle for each school, the better its score). The measure of performance — “net percent pass” — is the percent of students proficient (a score of 240 or higher) minus the percent failing (less than 220). Student demographics are shown by the horizontal position (the farther to the right, the greater the percent of minority, low-income, and/or LEP students). The “disadvantage percentage” is the sum of the minority, poverty, and LEP totals, where each student can be counted more than once. An all-minority, all-poor school where half the students don’t speak English would be at 250 percent.

The curve flows from upper left to lower right, reflecting the fact that the greater the disadvantage percentage, the lower the school’s score. To my mind, the most successful schools are not those with the highest absolute MCAS scores, but rather those that are the farthest above the trend line (shown in light blue). That is, they are the ones that have most outperformed schools with similar demographics. High performers in the upper right — schools that have done the best with disadvantaged populations — are of particular interest, and I’ve therefore identified them.

There’s simply no evidence that charter elementary schools outperform other schools; most of the charter schools are very close to the trend line, and only one is any significant distance above.

The three strongest elementary schools in the state are all regular schools in Springfield: the Talmadge, the Glenwood, and the Washington. I ran a similar analysis five years ago, and the Talmadge and the Glenwood showed up then as well. These schools have outperformed their demographics by a wide margin, and have done so consistently over a very long period of time. I’ve visited these schools; their results are no accident. They reflect very strong leadership by their principals and such steps as ample common planning time for teachers and moving LEP students into regular classrooms as quickly as possible.

Middle school performance is shown in Chart 2, omitting exam schools such as Boston Latin. Again, the vast majority of charter schools are close to the trend line. However, there’s a cluster of high-performing, high-poverty charter schools, including Excel Academy, Edward Brooke, and Roxbury Prep in Boston and the Family Development in Lawrence. Although the math chart is not shown here, these same three charter schools, along with Boston Prep and the KIPP Academy in Lynn, are even farther above the trend line in math. These charter schools are well ahead of virtually all non-charter schools.

As with middle schools, most charter high schools are close to the trend line (Chart 3). A few charters — and virtually no regular high schools — are well above the trend. But they’re tiny — the largest of these high-scoring charters tested only 29 10th-graders in 2008. Fully 96 percent of all high school students in the state are in larger schools than this; the average high school tested 79 students.

Most charter schools — elementary, middle, and high school — are on or very close to the trend line; that is, their performance is roughly similar to that of regular schools with similar demographics. Simply attaining charter status is no guarantee of success. Indeed, those three Springfield elementary schools succeed year after year without charter status, without merit pay, and without change to the union contract. Clearly, factors other than union rules, new governance, and pay incentives are critical for success; such factors include the use of data, helping teachers improve their teaching, strong leadership, and research-based pedagogy. In my own experience, strong leadership from the central office — something charter status eliminates — makes it easier for a school to improve itself.

if these other factors are the keys to success, what reason (other than ideology) is there for pushing so hard for more charters and taking on the bruising (and unnecessary) fights with unions, administrators, and community opponents?

Charter or not, successful elementary schools will need to work on pedagogy and leadership. Why not proceed directly there and skip past the battle over charter status?

The cluster of successful Boston middle school charters illustrates the useful role that a few highly successful charters can play in developing promising new approaches. What pedagogy and leadership style sets them apart from all the less successful middle schools (charter and otherwise)? Only one or two non-charter middle schools even come close to their results. Is there something about the independence and flexibility of charter status that is a necessary condition for their success?

It would also be worth knowing why the successful charter high schools do so well, and whether their techniques would work in a larger school. But until someone starts a full-size charter high school and demonstrates success, it’s not clear that charter status is a workable answer to the dilemma of achieving high school improvement.

Meet the Author
There’s enormous political, administrative, and financial cost involved in establishing a charter school. As charter status is no guarantee of ultimate success, wouldn’t those resources be better spent dealing directly with the issues of pedagogy, leadership, expectations, and teacher development that are ultimately what determines whether any school — charter or not — is successful? I hope this data will spark a more pragmatic look at charter schools and at the broader question of what it takes to help low-income and minority youngsters perform at high levels.

Edward Moscovitch is president of Cape Ann Economics and chairman of the Bay State Reading Institute. These charts are taken from a forthcoming study for Mass Partners, a coalition of Massachusetts teachers’ unions, principals, parents, superintendents, and school committees.