Network News

X My Profile
View More Activity

Experience Corps: Tutoring That Works

I get a lot of telephone calls and e-mails. Everyone seems to have a foolproof way to save our kids from ignorance and sloth. They all sound wonderful -- new Web sites, reading curricula, school designs, math tests, music lessons, textbook series, art lessons, charter school networks, after-school programs, teacher training routines and even a few breathing exercises.

I thank them for their information. I blame my editors, my favorite fall guys, for my not being able to write immediately about their amazing ideas. “I can’t get anything into the paper, or our Web site, until you have some results,” I say.

Even when they have data points, however, they are often disappointing. Some research organizations, and methods, are better than others. Take for instance the report I have been examining about Experience Corps, a 14-year-old program that is new to me but has produced gains for kids that cannot be ignored.

Experience Corps was founded in 1995 as a way to engage people over 55 in their communities’ social challenges. Today it has 2,000 members in 23 cities tutoring about 20,000 public school students. Tutoring programs are common these days. But the fact that Experience Corps has persuaded the Atlantic Philanthropies, one of the savviest of foundations, to fund a $2 million study of their work impressed me, as did the organizations that did the research: the Center for Social Development at the Brown School of Social Work of Washington University in St. Louis and Mathematica Policy Research Inc. of Princeton, N.J.

The results are remarkable. Over a single school year, students with Experience Corps tutors made more than 60 percent more progress in learning two particular reading skills — sounding out new words and reading comprehension — than similar students not served by the program.

Two aspects of this study strike me as worth remembering whenever we judge educational enterprises. First, Experience Corps turns out to be different from the make-the-participants-feel-good tutoring stints that many of us have volunteered for. In my experience, you read with a child, or maybe two or three children together, a couple of hours a week and that’s it. Experience Corps, on the other hand, screens, interviews and trains its tutors in teaching reading and relationship building. Most of them work about 15 hours a week and receive small stipends. They are supervised by Experience Corps staff members who coordinate what the tutors are doing with the classroom teachers and evaluate the tutors’ performance.

Second, the Washington University and Mathematica researchers used a randomized study plan in which the learning gains of students tutored by Experience Corps were compared to the gains of students who also qualified for the program, with similar backgrounds and achievement deficiencies. Teachers were encouraged to recommend as many children as they thought could use the help, with the understanding that there would be more children that needed the service than Experience Corps could provide for. A random lottery decided who would be tutored by the organization and who would not be. This seems a cruel way to conduct research, but it is done routinely in medicine, and provides more dependable results than the usual method: comparing how one school’s kids did with another school’s kids, with no careful assessment of the differences between those two populations.

The research findings, “Evaluation of Experience Corps: Student Reading Outcomes” available at this site , provide the details of this rigorous — and expensive — study method. More than 1,000 students in three cities — Boston, New York and Port Arthur, Tex. — were assigned to the research sample. Parents of 81 percent gave permission for them to participate. They were then randomly assigned to an Experience Corps tutor for one academic year, or to a control group. Students were tested at the beginning and end of the academic year, using three assessments—the Woodcock Johnson word attack subscale, the Woodcock Johnson passage comprehension subscale and the Peabody Picture Vocabulary test.

Experience Corps tutored 430 of these students on a one-to-one basis, and 451 were in the control group. Of the 881 students, 822 were included in the final data set, including 332 first graders, 304 second graders and 186 third graders. There were 420 males and 402 females. Many control group members joined other reading programs, although about 30 percent received no additional help outside their usual classroom instruction.

I asked some Experience Corps members to tell me what they did. Ron Williams, 61, who worked at an elementary school in Oakland, said among other things he gave students a chance to gain confidence by reading passages “several times with correction given when appropriate, and by eliminating distractions, competition and having a clear objective.” Sarah Seaver, 56, read with students in Tempe, Ariz., and conducted reading and vocabulary activities like sight word bingo, syllable puzzles and sound- and word-matching games.

The effect was significant, particularly given the reduced cost. The Experience Corps stipends amount to less than $5 an hour. The researchers noted that programs using one-to-one tutoring with certified teachers, such as Reading Recovery, had better effects, but were much more expensive.

Experience Corps had similar impact no matter what the gender, ethnicity, grade, English proficiency or classroom behavior of the students. The study did find, however, that students with diagnosed learning disabilities made less improvement than those without such designations, suggesting they need special help.

Experience Corps chief executive officer Lester Strong said the program’s potential is great. “Nearly 10,000 baby boomers turn 60 every day now,” he said. “Millions want the opportunity to be part of something bigger.” He said he wants to expand, but only in ways that will maintain the program’s quality.

It takes more time and money to study learning improvements in the way Washington University and Mathematica have done it. But the results are more trustworthy and can give policymakers — and taxpayers — greater confidence that money spent on such programs will have the desired effect. That increases the chances of getting the funds Experience Corps needs to grow, and investing in more children, including those consigned to the control group that was so important in figuring out what was going on.

By Washington Post editors  | June 5, 2009; 5:00 AM ET
Categories:  Trends  | Tags:  experience corps tutoring  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Extra Credit: There's a Place for Cut-and-Paste Learning, And It's Not Fourth Grade
Next: Is AP for All A Formula For Failure?

Comments

Mr. Matthews,

I don't think that you are using the medical analogy appropriately.

In high quality medical studies, it is important to understand what you are controlling for when you have a control group. It is important to be quite clear what you are comparing to what.

That is why placebo's are so important. They control for the ritual and discipline of taking the pills, and then attention/importance of being in a study.

What were the controls in this case? I understand that there is a comparison group, but is it a good comparison? Yes, on some factors, but what about others?

As an educational study, what you describe sounds pretty good. As a medical study, it does not. There really is not a control group, as medical researchers would describe it.

I appreciate the advance in educational research that these lottery winner/loser comparisons can represent, but you should not treat the technique as though it brings a study up to the highest standards of medical research. It does not, and it it might not even be appropriate to do so with children or educational interventions. Due of the ethics of workings with children -- something that quite very very few medical experiments do -- we simply cannot expect educational research to be able to give us the kinds bold answers that medical experiments often do.

Portraying a good educational study as though it's rigor rises to the quality of the best medical experimental standards does your readers a disservice, as it also does to educational researchers everywhere.

Posted by: ceolaf | June 5, 2009 12:33 PM | Report abuse

All good points, ceolaf. I didn't say it was the same quality as medical research, i just said randomized control groups seem cruel, but are routine in medical research.

Posted by: Jay_Mathews | June 5, 2009 2:58 PM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company