It’s springtime, which has come to mean not only balmy weather and blooming flowers but MCAS testing in our public schools. In April, fourth-, eighth-, and 10th-graders took their tests in English composition. In May, these students will sharpen their Number 2 pencils for subject-matter tests in four areas: English Language Arts and Literature; Mathematics; Science and Technology; and History and Social Sciences. In all, the Massachusetts Comprehensive Assessment System will consume more than two weeks of schooling, not counting the weeks of practice questions and test-prep beforehand.
As I know from my own daughter’s experience — she took MCAS in eighth grade in 1998, its first administration, and is now agonizing over the 10th-grade version — MCAS is an endurance test. State officials regularly refer to the diagnostic instrument as “challenging,” but students find challenge enough simply in making it to the end.
For teachers, administrators, state officials, and the public itself, education reform is an endurance test of a different type. Over the past seven years, the state has nearly doubled its annual funding of public education. Some officials — notably Gov. Paul Cellucci and House Speaker Thomas Finneran — are starting to ask, not unreasonably, Where are the results? The time, they say, has come for accountability.
So far, when it comes to accountability, MCAS is it. Kids are the canaries in the mineshaft of education reform. If they pass, education reform is succeeding. If they fail, education reform has failed. In some important ways, this exercise in brinksmanship is working. High-stakes testing has focused the attentions and the energies of the educational establishment in a way that no past reform measure has ever done. Schools are changing, and that is no small achievement. But as we approach 2003 — the first year students who are unable to pass the 10th-grade MCAS tests in English and math will be refused diplomas — we should take a hard look at what MCAS is, and isn’t, measuring, in our schools and our kids.
It may be too soon to panic, but from this distance 2003 is looking pretty grim. The first two years of test scores were, as they say in polite company, disappointing. Even as the state Board of Education set the passing grade — appropriately — at the lowest point it could possibly justify (one point above the category the board had already designated as “failing”), one third of 10th-grade students statewide failed the English test last year; half couldn’t pass the math test.
But statewide averages don’t tell the full story. Affluent communities have little to fear from the coming high stakes: In Weston, for example, 92 percent of 10th-graders passed the English test last year, 87 percent the math exam. But in New Bedford, 44 percent of sophomores failed English, 60 percent math; Springfield’s failure rates were 48 percent in English, 67 percent in math. In Boston, where the highest performing students are concentrated in three competitive-admission high schools, nearly 90 percent of kids in some non-exam schools seem headed for failure. It’s these schools that the 1993 Education Reform Act was especially supposed to improve, and these students it was intended to help.
Some urban educators will cite the poor results as proof that their schools need even more resources to do the job of lifting up the Commonwealth’s neediest students. But low scores will also give ammunition to critics who say that our cities’ schools have proven themselves irredeemable, and that the state’s investment of billions of dollars in them has been an expensive exercise in futility. Either way, the most visible product of education reform on its 10-year anniversary could be thousands of students denied diplomas on Graduation Day — that is, if the prospect of failure hasn’t driven them out of school already.
Still, what proponents of standards-based reform fear most is that the political cost of carrying out high stakes will cause a retreat from high standards altogether. When politicians — and the public — start to get cold feet, the standards standard-bearers will issue the rallying cry, “Stay the course.” But for that slogan to win the day, the course we are on had better be the right one.
Our current heading is not exactly the direction in which we set sail in 1993. The education reform law called for curriculum “frameworks” to set broad parameters for subject matter, explicitly rejecting a mandated statewide curriculum. Yet something very close to a full-blown — if not overblown — curriculum is what we’ve ended up with. As the educational culture wars raged, the debate came to be dominated by subject specialists and university professors grinding their professional axes. As a result, the frameworks and the MCAS now reflect narrow academic disciplines more than knowledge and skills of general applicability to the world of adult work and study.
Take the 1999 10th-grade English test. Rather than testing literacy and communication skills, it focuses on literary analysis and linguistic devices. In the first 10 questions, two hinge on knowing the difference between metaphor and simile, one on recognizing alliteration. Even the grammar questions seem to go out of their way to avoid any demonstration of applied knowledge, concentrating instead on parts of speech. Many standardized tests make the taker correct errant punctuation, but MCAS asks whether a particular comma in a particular sentence punctuates: a) a noun in apposition, b) a noun in direct address, c) an interrupting expression, or d) an introductory clause.
The writing topics are relentlessly literary in nature, focusing on craft rather than meaning. After reading Jack London’s account of the 1906 San Francisco earthquake, students are asked to give two examples of “vivid language” and explain how they “furthered the author’s purpose.” Test-takers are asked to write not about the content of Bruce Catton’s comparison of Civil War generals Grant and Lee, but rather whether the essayist wrote “an effective conclusion.”
Now, reading with understanding and writing clear, thoughtful, and grammatical prose are among the most important skills our schools can teach. And if MCAS, with its open-ended questions, has accomplished anything, it is the emphasis on writing now incorporated into all subjects. But the English test seems to be measuring not so much higher skills but more particular skills. The techniques of figurative speech can be very valuable indeed, but writing about writing is pretty esoteric stuff. For prospective English literature majors at liberal arts colleges, these would be appropriate — and challenging — questions. But is “Every Child a Literary Critic” really what’s meant by high standards?
The 10th-grade math test, which seems to give more students fits than the English test, is a bit more straightforward, though it’s not easy, and it covers a wide range of mathematical skills. To try to make sense of what this test represents, I reviewed two student answer sheets, provided (minus names and certain unreleased test items) by the state Department of Education. One student barely passed the test, with a score of 220. The other narrowly failed, with a 218.
Among other shortcomings, these students were both suckers for testing trickery. Some problems were crafted in such a way that failing to think through the question carefully led directly to an easy, but incorrect, answer. One problem presented a rectangle made up of eight identical squares. The area of one square is given. What is the perimeter of the rectangle? Both students chose the answer that represented the area, rather than the perimeter, of the rectangle. Another question asked about cleaning up an oil spill: If 10 percent of the remaining oil can be removed from the harbor each day, how much oil is left after three days, and how many days will it take to clean up half the spill? Rather than recognizing the question as a compound-interest problem in reverse, both students said 30 percent of the oil would be removed by day three, and half in five days. (The question gave no explanation for why the amount of oil cleaned up would vary day by day. Perhaps the test writers thought that would have given away the game.)
Neither of these questions is particularly daunting. Both students answered more difficult questions correctly (just as they screwed up others of similar or lesser difficulty — such was the uneven nature of their performance). But tripping up the mentally lazy — and fatigued — is one of the classic ways tests like this separate the student wheat from the chaff. Being on the lookout for traps like these is, as much as content knowledge, what separates people who test well from those who don’t. Such teasers of mental acuity — just how closely are you paying attention? — have their place. But is their place a high-stakes test determining high-school graduation?
It’s turning out that high standards, while a vital educational goal, are not so easy to define — or to test for. So far, Massachusetts has tried to solve that conundrum by giving every student a difficult test, then allowing a low score — 40 to 50 percent of answers correct — to count as passing. But do students understand that? A key factor in whether MCAS motivates hard work or discourages any work at all lies in whether students think they can do better next time. Spending hour after hour, day after day, slogging away at a test on which you have a clue on less than half of the questions is hardly calculated to inspire redoubled effort.
Since the state releases most MCAS questions after each administration, the test has to be substantially rewritten every year. State officials should take that opportunity to focus future MCAS tests on core knowledge and skills in each subject area, avoiding both academic esoterica and testing-industry mind games. If that makes MCAS more of a basic skills test, so be it. Better that than telling kids they can’t be high-school graduates because they can’t tell alliteration from assonance.
Then the state should put some effort into determining what represents a meaningful standard of performance that justifies the high stakes attached to it. Trial runs of MCAS should be administered to a range of adults in various fields, to make sure that the tested material bears some relationship to skills and knowledge of value in the real world, not just whatever happens to be in the curriculum. Among the groups the test should be tried out on is high-school graduates who have successfully completed at least one year of post-secondary education.
And the state should track a sample of students who got various scores on the first 10th-grade MCAS in 1998 through high school and beyond — in 2002, they’ll be two years out of high school — to see how they’ve fared in college and the job market. Is there a minimum MCAS score that truly correlates with post-graduate competence? There had better be, if we’re going to withhold diplomas from students solely because of it.
But at some point, the state is going to have to come to terms with the limitations of MCAS as its chosen mechanism of individual and institutional accountability. Based on reporting by CommonWealth and others, it seems that urban school districts that have answered the education-reform bell — Everett, Worcester, and many others — are using the rigors of high standards to improve instruction and get students learning. But that progress is not showing up in significantly rising MCAS scores, and may not for years to come. Let’s not snatch defeat from the jaws of education-reform victory by mislabeling their successes — and their students — as failures.