Education reform for adults

Meet young Dale Hollingsworth and don’t be surprised if Matt Damon’s Good Will Hunting comes to mind. Like the hit film’s title character, the attractive, charismatic 25-year-old spends his days cleaning buildings at MIT and his nights reading whatever he can find. Whenever he can take a break from scrubbing toilets and mopping floors, he turns his attention to great literature, mathematics, history, and science.

But that’s where his life stops imitating art. Hollingsworth, who is from Barbados and lives in the Dorchester section of Boston, is no undiscovered prodigy. He never finished high school because he was, in his words, “a slow learner” who’d rather “play than study.” It’s not calculus or existential philosophy he’s studying now, but basic algebra and eighth-grade English, with a bit of social studies and earth science on the side. He’ll probably never be recruited for a six-figure salary in a high-tech start-up or a major corporation; he just wants to do something more with his life than cleaning up after other people. And with even most entry-level jobs at decent wages requiring a mastery of reading, basic math, and computer skills, Hollingsworth knows he needs at least a high school equivalency degree if he’s ever going to build a better life for himself and–someday, perhaps–a family.

But that’s all down the road. His main mission today? To better understand what he reads. He is a top student in his twice-weekly night class run by the nonprofit human service agency Action for Boston Community Development (ABCD) because he always shows up and he always tries hard.

“It works out good. I enjoy it,” says Hollingsworth, a native English speaker with a soft, Caribbean lilt to his speech. “Everyone has fun. Everyone learns together.”

Indeed, ask adults struggling to read aloud from Richard Wright’s classic novel Black Boy in ABCD’s fifth-floor classroom near Boston Common what they think of the literacy program and they offer high praise. They love their teacher, Liz Zachry, a patient, enthusiastic Harvard graduate student who explains how to make an inference from a reading passage and leads them through science vocabulary, writing words like “condense” and “contract” on a white board next to a map of the world. They come at night, so school won’t interfere with their jobs, and they don’t have to pay a dime. For the first time, some say, they’re starting to feel confident in their abilities.

In the past, such anecdotal evidence of customer satisfaction was a sufficient sign of success for the 179 government-funded adult basic education programs across Massachusetts. Whether the subject was basic skills, English as a second language, or preparation for high school equivalency exams, the only result the state measured was how many students came to class. Now with big increases in funding but few answers to fundamental questions such as how much students are learning, the $43 million Massachusetts adult basic education system is attracting unprecedented government scrutiny–to find out whether the programs work and, ultimately, whether taxpayer money is well spent. While education officials and business leaders say the new rules will help make the classes dramatically more effective, many in the field worry that the changes will gut their programs and end up hurting the very students they’re trying to help.

Sound familiar? It should. It’s education reform, Massachusetts-style, all over again, with many of the same controversies that have plagued efforts to improve the state’s public schools, but this time for adults. So far, it’s been happening with remarkably little fanfare–or even public attention–outside the world of adult education. A fraction of the size of the million-student, $3 billion K-12 system, with most classes taught by part-time staff, adult basic education has never been taken as seriously. But the stakes could be just as high. For the programs, failure to show sufficient student progress could mean loss of funding and possible shutdown. For the students, failure to learn new skills could leave them unable to support their families, help their children succeed in school, or meet important personal goals. And for the rest of us, failure to maintain a skilled workforce could slow the state’s economy and lower everyone’s standard of living.

It’s education reform, Massachusetts-style, all over again–but now for adults.

“We believe students deserve a better deal than they’ve traditionally got[ten] in adult education classes,” says Robert Bickerton, administrator of the Adult and Community Learning Services division of the state Department of Education, which oversees and funds the programs. For more than a decade, Bickerton has singlehandedly pushed for improvements throughout the sprawling, decentralized adult education system he oversees. The result is a system of adult learning that Bickerton contends is “much better than the average nationally.”

But not good enough. Thirty percent of the students drop out each year, he notes, while others languish for years without making progress. “What students have always received is well over 100 percent of the commitment and sweat and effort of people who work in adult education centers,” Bickerton says. “But we need to get better at getting them the results they seek.”

No avoiding accountability

The pressure for reform is bearing down on an adult basic education system already under tremendous strain. This year, about 24,000 adults are learning to read, speak English, or perform skills typically expected of high school graduates in a broad array of programs run by local schools, community colleges, and social service agencies across the state. But there are thousands more people on waiting lists–at least 8,000, by the most recent state estimate–hoping for a seat. And there are hundreds of thousands more who need the services but may not even know where to seek help. In fact, the Department of Education estimates that more than two million people–almost 45 percent of the state’s adult population–are under-educated or can’t speak English proficiently. That includes about 877,000 adults (over the age of 18) who are functionally illiterate and another 1,162,000 who don’t have the skills expected of a high school graduate. And that’s almost 85 times as many people in need as the number served.

The demand is only going to grow. In just over two years, when the final phase of the state’s K-12 education reform plan kicks in, passing 10th-grade tests in English and math will be a requirement for high school graduation. While state and local education officials are offering remedial programs to help kids who fail, it is widely expected that thousands of teenagers will simply give up on school. For those who realize the limitations of the dropout lifestyle, the state’s adult basic education system will be the place to go.

But, ironically, the higher standards they escape by quitting the K-12 will catch up to them in the world of adult education. The state is creating a comprehensive “performance accountability” system for adult basic education based on the same fundamental ideas underlying its efforts at K-12 education reform. The Department of Education is introducing guidelines called “curriculum frameworks” that outline what every adult student should know and be able to do; all students will be evaluated on a regular basis to determine how much they have learned. Those seeking a high school credential will have to prove their skills, though it’s still unclear what form that evaluation will take. (See “MCAS for adults?” page 40) And the programs that teach adults will be asked to demonstrate certain levels of performance. In addition to measuring academic gains, the state also will consider whether students achieve their personal goals, such as getting a job or becoming a US citizen, and will continue to look at attendance.

Here are some of the key changes, which take effect this fiscal year:

  • Programs must measure each student’s academic progress three times a year with a standardized procedure. For now, they can use any one of several standardized tests that private companies sell or–if they can prove the scores correlate with standardized test scores–an alternative assessment they create themselves. However, within two years the state is likely to narrow the range of acceptable assessments. In the past, it was up to programs to decide how often, and by what means, to assess student progress. Many did not report learning gains to the state. Of those that did, it was difficult to compare gains in one program to another because they were measured in different ways. Often teachers simply estimated based on observation.
  • Programs must ask students to identify a primary and a secondary goal for each year and follow up at the end of the year (or when they leave) to determine whether they have met them. For students whose goals are to get a new job or to keep their current job, programs will have to track them down several months after they leave. In the past, it was up to programs to decide if, when, and how to determine whether students had attained their goals. Many did little formal follow-up at all.
  • Programs must report learning gains and goal achievement for each student, along with a host of demographic information, to the state Department of Education through a database on the World Wide Web. Programs have reported other information through SMARTT (System for Managing Accountability and Results Through Technology) for several years, but now much more data is required.
  • And finally, the state is encouraging programs to start incorporating the curriculum frameworks that outline what’s expected of students in six subjects: English Language Arts, History and Social Sciences, Mathematics, Science and Technology, Health, and English for Speakers of Other Languages. Using the frameworks is not required, but Bickerton says that, in two years, all of the state-approved academic assessments will be tied to these guidelines, so programs that don’t use them will be at a disadvantage.

    Under Bickerton’s direction, the state has been working since the late 1980s to hold adult basic education providers more accountable. The 1993 Education Reform Act, which called for improving the adult system along with the public schools, endorsed that effort but did not outline how to proceed. Forcing the issue now is the federal government–specifically new rules in the 1998 Workforce Investment Act, which collapsed more than 50 federal grant programs in employment, training, and literacy into three block grants. The law requires each state to justify the federal investment–about $8.7 million in Massachusetts this year–by showing that its adult literacy programs are effective and improving each year. Without proof of progress, Congress, which is demanding similar results from other grant programs, may cut future funding. On the other hand, if the state meets its goals in adult literacy and other areas, it will be eligible for up to $3 million in incentive grants.

    Under new federal rules that kicked in July 1, all states are required to do a few basic things, such as measure student learning and set minimum program performance levels. But states have a great deal of freedom to decide the details. Massachusetts has many critical questions left to answer. Among them: Which tests or other measures of learning gains will the state ultimately accept? And what are the performance levels that individual programs will have to meet? Complicating matters is the fact that the curriculum frameworks–which the federal government does not require, but the state wants to use as the basis for future tests–are still not finished after five years of work.

    Bickerton says he expects the frameworks to be “in good shape” by early next year. Most of the remaining questions, he says, will be answered by a group of adult educators and assessment experts he plans to convene in January to help design the accountability system. Their mission is to determine what will yield “valid and reliable” data–precisely what has been missing from past attempts to evaluate the system–in each of the state’s three measures of effectiveness: learning gains, goal achievement, and attendance. However, he acknowledges that it will take at least two years for the group to do its job. In the meantime, the state will meet the federal requirements using the best data it has available. “But by the third year, we need to have something more substantial,” Bickerton says.

    And not just to satisfy federal bureaucrats. Although the tight labor market is making it easier for some poorly educated people to find jobs, employers say they are having a terrible time finding employees with sufficient skills. Not knowing whether the adult basic education programs that job seekers list on their applications have been worthwhile compounds the problem. It’s critically important to bring all these programs, which have varied widely in quality, up to a basic level of performance, says Andre Mayer, who tracks education-and-training policy and economic trends for Associated Industries of Massachusetts. Without that guarantee, employers will remain uncertain about whom they’re hiring–and will often regret their decisions.

    “After all, we’re talking about…the group of people about whom the least information is available, because they’re not college grads, they don’t have a high school diploma from a school you may know something about. You don’t know what you’re getting,” Mayer says. “You’re talking about a field about which there’s been very little to go on…. Higher standards will make a difference here.”

    Adult literacy experts agree, saying the programs have suffered from low expectations. “There’s absolutely no question we need more accountability because there’s been almost none,” says John Strucker, a research associate at the National Center for the Study of Adult Learning and Literacy, based at Harvard’s Graduate School of Education. The field “has been ignored for so long it’s developed a lot of bad habits.”

    “There’s no question we need more accountability because there’s been almost none.”

    The students tend to suffer from the same low expectations, so they rarely pressure the programs to improve, according to Strucker, who taught adults in Cambridge for 11 years. “They tend to be grateful for whatever they’re getting,” he says. That makes it all the more important, he adds, for someone to push for improvements on their behalf.

    The lone ranger

    No one in Massachusetts has been pushing harder than Bob Bickerton. A former teacher and program director, he became the first head of the new state office of adult basic education when it was created in 1988, and set out to improve the field with a zeal rarely associated with government bureaucrats. Until his arrival, the state had been providing only $100 per student in funding and let programs do pretty much whatever they wanted. The results were poor: While the average class met for about 200 hours, the average student was present for only 55 of them.

    By all accounts, Bickerton ushered in a new era. He helped secure more money, and he demanded better results. He also gave adult educators something they had never had before–guidance. For the first time, the state outlined the conditions needed to make an effective adult basic education program and required every one receiving state or federal funds to meet a slew of new rules. The state’s 32 “guidelines” mandate, among other things, at least 12 hours a year of teacher training and a maximum class size of 15 to 20 students (depending on the level). Bickerton’s office also established the System of Adult Basic Education Support (known as SABES) to provide training and technical assistance from community colleges around the state.

    Christine Taylor, who started the Framingham ESL program in the mid-1980s, credits Bickerton and his staff with changing the field’s second-class status. “When we started 16 years ago, adult education was really kind of an informal little extra part-time thing people did because they wanted to feel good. It just wasn’t taken very seriously–not only by the field itself, but by other educators,” Taylor says. “They’ve worked very hard to pull the whole field up by the bootstraps.”

    Though relatively low on the Department of Education totem pole, Bickerton appears to have pull not by virtue of his position, but by the force of his personality. Known as a workaholic, he is tough and demanding–on programs funded by his office, on his staff, and on himself. Some complain that he is difficult to deal with, that he is rigid, a micromanager. Colleagues who consider him a friend say his motivation is that he cares deeply about his mission. “He is very committed to seeing poor and working-class people get a fair shake,” says David Rosen, director of the Adult Literacy Resource Institute, the SABES center for Greater Boston.

    And adult educators know that he knows what he’s talking about. Bickerton started his career 30 years ago coordinating volunteer tutors and went on to be a teacher, trainer, and director at programs run by local nonprofits, a school district, a community college, and the city of Cambridge. “Even his most severe critics would agree the field has advanced under his leadership in ways it has never advanced before,” Rosen says.

    Finding a fair test

    Still, Bickerton’s critics have a lot to complain about these days. The push for accountability has set off a mad scramble in adult-ed circles, as educators try to figure out what the new rules mean. While everyone likes “accountability,”–or so they say–at least in the abstract, many program directors and instructors strenuously object to the state’s data-driven approach to it.

    “The accountability is narrowly defined in a way that doesn’t reflect what I think are the most important aspects of what we do,” says Linda Faria Braun, who runs the Brockton Adult Learning Center and is co-chair of the Massachusetts Adult Basic Education Directors’ Council. What matters under the new rules, she says, is only what can be measured. “While I certainly understand the need for accountability when you’re giving out millions of dollars to programs, it’s a very one-dimensional requirement,” she says.

    One of the biggest complaints is reminiscent of recent debates over the state’s K-12 education reform plan: the use of standardized tests to measure student progress. Simply put, adult educators say, the handful of standardized tests available on the market are not up to the task of measuring what an adult has learned. “There is a lot of question about their validity,” says Mina Reddy, director of the Community Learning Center in Cambridge, one of the largest programs in the state, serving about 1,200 students a year.

    “The lack of good tests is a huge problem,” agrees Sally Waldron of World Education, the SABES central resource center in Boston. “They don’t exist yet.” For example, she says, the widely used BEST test, which measures English language abilities, was designed for recent immigrants, and can’t capture the learning gains of more advanced speakers. Also, most commercial tests measure broad abilities, not progress on specific skills. Intended for placement, they are not designed to be repeated every few months, instructors say, and don’t show the small gains that are typically made during that time. Plus, the tests are not tied to the program’s curriculum, so they can’t be used to check how much a student has picked up from the course or to improve instruction.

    For the time being, the Department of Education will allow adult programs to use home-grown alternatives–but only if they can prove the results correlate to standardized tests. That may not be possible, say some adult educators, because their assessments are designed to be different, identifying specific skills rather than a broad “grade-level equivalent.” Susan Riley, who coordinates basic-skills classes for SCALE, the Somerville Center for Adult Learning, says it would be tough to claim equivalence for the tests her agency has developed for beginning readers. “But it works,” she says bluntly. “I’ve been in this field for many years. I think I know how to tell when people are making progress…. Some people who have never seen these students are coming up with these [standardized] tools.”

    Nor do many of these students shine on tests. Many adult learners struggled–and failed–when they were in traditional public schools, often because of learning disabilities. Adult basic education programs work hard to make their students feel “safe to expose their own lack of skills, safe to make mistakes,” says Clare Shepherd, who oversees education and job training at ABCD’s LearningWorks program. “Standardized testing is just about the worst thing you can do to someone in that position. Just the word ‘test’ is enough to make some of our students break out in a cold sweat and not show up again.”

    ABCD student Dale Hollingsworth says much the same thing about himself. “Oh, I get shivers, I get really nervous,” he says. “That’s why I always mess up sometimes.”

    Scoring Goals

    In his 18 years working in adult education programs in Massachusetts, David Rosen says, he has never met anyone whose motivation for returning to school was, “I’d like to improve my score on a standardized test.” Adult students, who range in age from 16 to 66, have many different reasons for taking classes. Many want a high school credential so they can get a better job, but others simply want to communicate with colleagues, find a doctor, or help their children with schoolwork, he says.

    It is precisely because of such concerns that Massachusetts education officials have decided to incorporate many student goals into the accountability system. While the federal government requires follow up on only a few education and employment goals, the state is requiring programs to track whichever of 19 separate goals adults say they want to accomplish when they sign up for classes.

    Many adult educators are relieved that the state will recognize the validity of these personal goals, such as earning US citizenship, registering to vote, or getting off welfare. But gathering information on whether their students achieved them is easier said than done.

    Anne Serino, director of Operation Bootstrap, an adult learning center in Lynn, says her program has found it impossible to track down many students once they leave. “Most of our students are apartment dwellers. They move often, their phone numbers get disconnected, letters come back [marked] ‘addressee unknown,'” she says. Each spring, the Community Learning Center in Cambridge–one of the few programs that has even tried to track outcomes in any systematic way–calls people who have completed classes in the past five years. Reddy, the director, says they’re able to reach only 10 percent of the students. Now, the federal government wants every program to get answers from 50 percent of its students each year.

    “Programs aren’t really staffed to do that level of follow up,” says Waldron, of World Education. “Sometimes more is being asked than there’s capacity to provide.”

    But ultimately, say some adult educators, no list of goals or test scores could capture much of what they do. What would still be missing, says Mary D. Mello, adult education coordinator for ABCD LearningWorks, are many of the “qualitative changes” adults go through, things like raising self-esteem or being able to tell their landlord–in English–what repairs are needed in their apartment.

    “How do you measure students going to Shakespeare in the Park for the first time, and before that they read Shakespeare…and really analyzed what the universal lessons are from Shakespeare?” Mello says. “We believe in our program, that it enhances people’s lives.”

    Educators also worry that students will be put off, if not driven away, by state demands for personal information. A six-page form asks not only each student’s name, address, and phone number but also whether he or she is on welfare and the number, ages, and grades of their children. (Inquiring about income is still optional.) Of particular concern is the possibility that the state may ask for social security numbers, which some policy makers say is the only valid way to track goals such as finding employment or passing the GED exam. Immigrants who are just learning to speak English or who had negative experiences with government in their native countries are particularly wary of such inquiries, educators say.

    “We spend a whole lot of time trying to assuage their fears,” says Susan Riley, of the SCALE program. “People have concerns that information about them is going into some big computer and God knows where it goes after that. The term ‘Big Brother’ has been used more than once.”

    Setting standards

    It is hard for critics not to see in these complaints signs of resistance to accountability of any kind. But some program administrators do see value in setting learning standards in the form of curriculum guidelines for adult education. Staff turnover is high and new instructors come to their jobs without any common background or training. But now the state is outlining, for the first time, what skills and content matter most for adult learners. That kind of uniformity has “long been needed,” says Meg Murphy, director of an adult education partnership between Bristol Community College and the Taunton Public Schools.

    Anne Serino of Operation Bootstrap, one of the few programs that has tried to use the frameworks, which are still in draft form, says it’s “really important” to incorporate the curriculum guidelines into classes, even if it seems “overwhelming” to do so. “People have kind of done their own thing for a long time here and it’s been, let’s say, inconsistent in quality,” says Serino.

    But even some supporters of these new standards worry that the adult-ed curriculum frameworks will suffer the same fate as their K-12 counterparts. The content of the K-12 frameworks, which form the basis for the new statewide tests every student must pass to graduate from high school, has been one of the most contentious parts of the education-reform plan. Educators from different backgrounds wanted the frameworks to include very different content, and when they got to the state Board of Education for approval, several were substantially rewritten.

    It’s unclear whether the adult frameworks will be subject to similar political wrangling. So far the guidelines focus more on skills development than knowledge of specific facts, a balance that most adult educators agree is appropriate for adult learners. But the frameworks have already been revised several times over the past five years. And in the case of the history document, the first group’s work was tossed out by Department of Education staff, says Clare Shepherd of ABCD, who helped write the original.

    However, the adult-ed frameworks have not yet reached the Board of Education and may never do so. Bickerton says that the 1993 Education Reform Act did not require curriculum frameworks for adult education, and so “we did not need to go through the same very highly public process as [with] K-12.” James Peyser, the board chairman, was not aware that adult basic education frameworks were even being developed until told so during an interview with CommonWealth.

    Ultimately, of course, program administrators are most afraid their programs will lose funding and be forced to cut back or close down, if they can’t show the progress the state and federal governments demand. If recent history is any indication, they have reason to worry. Thirty of the 134 programs competing this spring for five-year grants distributed by the state (a combination of state and federal funds) were turned down, according to Bickerton, and 24 of those had been previously funded. About 12 percent of each proposal’s score from a team of reviewers was based on past performance. “If we had a higher degree of confidence [in the performance measures], we would have made it a bigger part,” Bickerton adds.

    Which is exactly what has program directors worried. But Bickerton says his search for “valid and reliable” data–which will continue next year with his group of adult educators and assessment experts–will cover not only traditional standardized tests, which are usually the most cost-effective, but also portfolios and so-called performance assessments, which ask students to perform certain tasks to demonstrate they have mastered a skill. However, he says, “The highest likelihood is that there will continue to be paper-and-pencil tests.”

    Changing rules, changing lives

    So can the requirements of the new accountability era be worked out in a way that satisfies government funders, adult educators, and adult students alike? “That’s the million-dollar question,” says Steve Reuys, staff development coordinator of the Adult Literacy Resource Institute. His answer is no. “I don’t see at this point a way in which they can be made compatible.”

    But others have a more hopeful, if not optimistic, view. Anne Serino of Operation Bootstrap says if the information is used in a way that helps to improve all programs, not just punish “bad” ones, then today’s agonizing will be worth it. “We always need to be thinking about how we can do our job better,” she says. “I happen to think it’s all possible, but I also know it makes people very anxious. For a long time we’ve had to do nothing but count heads in our program. This is a big change.”

    “We’ve had nothing to do but count heads…This is a big change.”

    Most adult educators say additional funding–from both the state and federal governments–would do a lot to help programs meet all of the new requirements. And some say the state, in particular, needs to help programs navigate the rules and regulations of the new accountability era.

    “What’s missing now is a sense of mutual accountability,” says Mina Reddy of the Community Learning Center. “Programs are being asked to produce data and produce results, but we’re not being given adequate support to really improve the field.”

    The biggest push should come in professionalizing the teaching corps, adds John Strucker of the National Center for the Study of Adult Learning and Literacy. Without pay to attract full-time–and long-term–instructors, and without a commitment to train them, all of the accountability measures in the world will show nothing more than the need for improvement. “Until something is done about that,” Strucker says, “we’re not going to get anywhere.”

    It’s not that the state hasn’t put money into adult education. In 1987, the state provided only about $4 million for adult education; today, its contribution is more than $30 million. That’s almost an eight-fold jump in a little more than a decade. (A Board of Education-appointed commission in the mid-’90s recommended even more–up to about $40 million by fiscal year 2000.) The per-pupil expenditure is now almost $1,800, dramatically better than the $100 spent on each student in the 1980s, though still far less than the $6,700 per student for K-12 education. But there is no additional funding to support the new assessment and reporting requirements. Indeed, this year’s state budget is the first to contain no additional money for adult education in five years. While the Senate and the House approved an extra $4 million, Gov. Cellucci vetoed the increase. Meanwhile, the federal government’s contribution has risen only slightly–to $8.7 million–after holding steady at roughly $7 million for the past three years.

    Meet the Author
    Back at the downtown adult literacy program run by ABCD, Dale Hollingsworth doesn’t know much about the state’s new accountability rules, except that more tests are coming his way. But on his personal educational journey, nothing’s holding him back. Having advanced to a GED-preparation math class in August, he hoped to move out of Liz Zachry’s pre-GED English class this fall. “I don’t want to stay as long as some of [the students] … because some are there a very long time,” says Hollingsworth, who started his studies not quite a year ago. “I try my hardest.”

    For now, Hollingsworth is still cleaning up after MIT students. But ask him if he thinks the education he’s getting is going to change his life, and he doesn’t have to ponder the question: “It’s already started,” he says.