Network News

X My Profile
View More Activity


Posted at 12:10 PM ET, 04/ 5/2010

Willingham: What NAEP reading scores really show

By Valerie Strauss

My guest is cognitive scientist Daniel Willingham, professor at the University of Virginia, and author of “Why Don’t Students Like School?”


By Daniel Willingham
As Chad Aldeman pointed out at the Quick and the Ed, many major newspapers missed the story on the recent National Assessment of Educational Progress scores. The New York Times bemoaned that fourth-grade reading scores have barely increased since the early 1990s.

Aldeman pointed out that reading scores look somewhat better if you separate the data by race, as shown here.

willingham 2.jpg


The percentage of Hispanic test-takers increased between 1992 and 2008 and Hispanics, on average, score lower than Whites. Hence the overall mean score was pulled down by the shift in demographics. In fact, the increase in the mean score is smaller than the increase in any of the subgroups.

Aldeman is definitely right that the picture looks better with the data disaggregated.

But I think the more important part of this story is the utter lack of progress in 8th grade and 12th grade reading, which the Times mentions only briefly and Aldeman not at all.

Unlike the fourth-grade scores, things don’t look much different when the data are disaggregated, as shown for eighth graders:

willingham 3.jpg



And for twelfth graders:


willingham 1.jpg

If fourth grade scores have improved at least somewhat in the last 15 years, why don’t these same fourth graders show better scores when they get to eighth and twelfth grade?

Fourth grade scores have been improving because we’ve gotten better at teaching kids how to decode--that is, how to translate letters into sounds. In the fourth grade some kids are good decoders and some are not, so differences in reading scores are largely differences in decoding.

By upper elementary school, most kids are fluent decoders so reading score differences are not due to differences in decoding. They are due to differences in comprehension.

Comprehension differences are mostly due to differences in background knowledge, a point I’ve made here before. Thus, the problem is not, as Susan Pimentel, a member of the NAEP governing board describes it that “We’re not asking them to read nearly enough.”

In fact, Americans are reading more text than they ever have before. And kids in lower elementary already spend half their time on language arts, and less than ten percent of their time on social studies and science, combined.

The belief that kids will be better readers if we simply get them to read more is rooted in the belief that reading comprehension is a transferable skill that, once mastered, applies to any text. That’s true of decoding, but not of comprehension.

What’s needed is a substantial knowledge base. Knowledge of the content they are likely to encounter when reading the sorts of materials we expect them to read confidently: newspapers, magazines, and serious books.

That knowledge should be accumulated beginning in Pre-K, with read-alouds, activities, field trips, and the like. It should continue throughout their education.

Until we start paying more attention to content, expect flat reading scores.

-0-

Follow Valerie’s blog all day, every day by bookmarking washingtonpost.com/answersheet And for admissions advice, college news and links to campus papers, please check out our new Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | April 5, 2010; 12:10 PM ET
Categories:  Daniel Willingham, Guest Bloggers, Reading  | Tags:  Daniel Willingham, NAEP, guest bloggers, reading  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Negotiating the price of college -- Flagel, Part 4
Next: College Tour '10: Brandeis University

Comments

I posted an analysis of Reading MSA scores in Montgomery County, on March 11, 2010 in the DC Examiner (please see http://www.examiner.com/x-29782-DC-Gifted-Education-Examiner~y2010m3d11-How-well-does-the-Maryland-School-Assessment-MSA-measure-proficiency). Isn't this analysis supportive of the conclusion therein? Aren't we arguing for a change in the existing teaching paradigm, from one geared towards passing the NCLB mandated State assessment to a rigorous course of instruction aimed at content comprehension?

Posted by: DC_Gifted_Education_Examiner | April 5, 2010 1:53 PM | Report abuse

I also wonder if this increased focus on "reading" per se is counterproductive, because it is limited to a single type of "reading": the ability to read, interpret, and deconstruct fiction. While that is certainly a good skill, it is not the sole determinant of "reading" ability -- nor even the most important one for kids to learn.

I have to admit, as an old English major myself, this has been difficult for me to get my mind around. But then I had DD. And I've spent this year wondering: how can a girl who the school tested as reading 3+ grade levels ahead be pulling straight Cs in Reading? Her teacher thinks she's an underachiever who does as little as she can get away with. Somehow, the kid who can spin the most amazing fanciful tales in her head morphed into the girl who described the Greek myth about the judgment of Paris as "he gave the apple to the girl."

Then I saw two assignments that she did. For the first one, her reading teacher asked her to explain how a story would have been different had it been written in the first person instead of the third person. Instead of writing a paragraph explaining how the reader's knowledge would have been different, imperfect narrator, etc., DD wrote one sentence: she explained that the narrator would have said "I" instead of "she." Which is both absolutely correct and absolutely literal -- and not at all what the teacher wanted.

For the second one, she was asked to write a similar paragraph in science explaining what she knew about the interrelationship between people and the environment. Boy, was this one night and day: she went way over the allotted lines; she chose language that brought her excitement right through the page; she supported her ideas with specific details -- in short, all the things that her reading teacher has been begging her to do all year.

And I realized: I have a very literal kid. She enjoys reading stories to find out what happened, but she doesn't get -- or care about -- abstract stuff, the "art" part of fiction writing. She wants to know facts, and then know more facts, and then to be praised for how many facts she knows. She is actually an excellent reader, and a pretty good writer (well, considering that she's 8) -- but just not in the areas that her "reading" class focuses on.

And yet, thanks to the way the school teaches -- and emphasizes -- reading, she thinks she's dumb and has learned to "hate" writing. Argh.

Posted by: laura33 | April 5, 2010 2:34 PM | Report abuse

I understand with and agree with many of laura33's points, but would like so add a few unkind comments about comprehension.

The elementary school my kids attend caps reading assessment at one grade level above actual. In other words, the highest reading level a 5th grader can earn at year's end is a 60. The argument is the lag between decoding and comprehension in most readers. There is always room for improvement in comprehension. You could probably reread "Mike Mulligan and His Steam Shovel" and posit that it covertly addresses the labor movements of the 1930s.

But I don't think being able to truly understand a fiction or non-fiction text automatically converts into the ability to construct considered short responses about character motivation, plotting elements, and the other bookclubby exegesis that appears on evaluations and tests. I think some readers find these "insights" self-evident, falling below the threshold of something they would consider important enough to pass along to another person. In other words, they understand what's key in the text, but don't know why it would be notable. The best solution, evaluation-wise--is the development of a rote response which prompts these strong readers to plug in the right information. Boring, I know. Bring me the hoops and let's get jumping.

I took a lot of literature seminars, and I wish I had understood that the professors were looking simply for an organized paper that had a workable argument and some supporting citations. I was always trying to break completely new ground in analysis. I got a few incompletes. Breaking new ground is exciting, but it's a fickle plow.

Posted by: gardyloo | April 5, 2010 5:29 PM | Report abuse

@laura33

I had to laugh when I read what your daughter wrote. My kids do the same thing. It is as if the person writing the question is thinking about the reading in a whole different way. My son especially does not get into elaborating about fiction. He does remember facts and details well when wriitng about non-fiction.

Posted by: celestun100 | April 5, 2010 6:03 PM | Report abuse

The argument about building up background knowledge is interesting because that was big back in the 80's at least where I went to college.

Recently I was dismayed to hear my own children say there wouldn't be anymore "Pioneer Days" at their school for fifth graders, because there was "no learning" going on. I took that to mean no learning that was tested on the Maryland test. But, in this program the kids got to make a covered wagon and walk 5 miles and eat stew, in short, pretend they were pioneers. I know it will be easier for them to understand history texts about that period. They have a much better idea of what that period was about.
I hope they don't cancel that program.

Posted by: celestun100 | April 5, 2010 6:08 PM | Report abuse

With all the emphasis on phonics and decoding, someone forgot that these skills have a PURPOSE, which is to aid comprehension. There's no point in learning how to decode if you don't put it to use by reading a rich variety of fiction and non-fiction; otherwise, it serves no purpose. The non-teachers who authored NCLB don't understand that, or else don't care because phonics materials have been more profitable.

Posted by: aed3 | April 5, 2010 8:05 PM | Report abuse

I think phonics are easy and don't take much critical thinking, therefore they give good results early on. Obviously, you need to be able to read different genres and have some vocabulary instruction, at the very least. But, I think the NCLB thing is inexpensive when compared to field trips, science labs and so forth. Much cheaper and orderly to have everyone quietly bubbling in those circles on the tests.

Posted by: celestun100 | April 5, 2010 8:32 PM | Report abuse

I agree that the phonics materials make it appear that many students are learning to read early, at least it appears that way to the inexperienced. These programs focus on phonics and reading fluency at the expense of developing reading comprehension.

I work with gifted students, and I get really sick of teachers sending me kids who are in 1st grade, but they supposedly "read on a 5th grade level" according to whatever phonics measure is being used. Usually, these kids are great at calling words, but their comprehension is within the normal range for 1st graders.

I once asked a teacher about one of her alleged 5th grade level readers "how is her comprehension?" The teacher replied, "Oh, she has some of that, too!"

The phonics programs have been very expensive. Districts have been compelled to purchase programs such as Reading First and measuring instruments like DIBELS, as well as other programs that are not cheap. Many of these programs were never intended to be used as a definitive measure of reading ability, but districts like data, and they provide it.

Posted by: aed3 | April 5, 2010 11:31 PM | Report abuse

Many of the students I have worked with have needed a heavy dose of decoding help -- that's just where they have been developmentally. Jan Richardson offers a developmental approach to reading instruction that strategically guides readers along the reading continuum. A key component of her approach for helping coach readers to move beyond the "transitional stage" of the literacy continuum is guided writing. Students can benefit from coaching in small group settings to learn how to use the elements of a quality response in a written response. Many assessments of comprehension such as the DRA and QRI evaluate certain elements and skills using rubrics. Students need lots of strategy instruction and closely monitored practice to develop these skills and strategies, i.e., regular small group instruction opportunities. Too much whole group instruction tends to lack sufficient differentiation.

Posted by: dannykurland1 | April 6, 2010 6:24 AM | Report abuse

As a Special Ed. teacher, I see the value in teaching decoding. Basically, if you can't decode, you can't read. The problem, as is pointed out well in this article, is that reading tests mostly assess comprehension as you go higher in the grades.

Here in Philadelphia, we have introduced a supplementary reading program called Corrective Reading. In the primary grades, it's Reading Mastery. These programs are heavily geared to decoding and I've found them useful for some of my students with disabilties. The problem is that the PSSA (standard state test) emphasizes comprehension. Yes, the students do need to be able to decode words and it's true that many of our students are several years below grade level. However, their greatest challenge on the state tests will be on comprehension questions. Many teachers have become very discouraged because they know that their students will still face an uphill battle on the state tests because the intervention programs don't build comprehensiion skills.

Posted by: Nikki1231 | April 6, 2010 6:32 AM | Report abuse

Well. . .The best way to improve (and test)comprehension is to ask questions about the reading and require students to WRITE their answers in full sentences. Teachers who want their students to succeed will READ all of these answers and RESPOND to the individual students' questions and/or comments. Yes--it's a LOT of work, but what could be better than opening the world to a new learner??

PARENTS--If your child's teacher is NOT doing this. . .YOU do it. If your child's teacher IS doing this. . .you do MORE!!

Teachers and parents are not supposed to leave children alone. We're supposed to "bother" them with new challenges.

What could be better than that?

(P.S. I have two grown children and three
grandchildren and am a retired teacher.)

Posted by: wwIIbaby | April 6, 2010 9:00 AM | Report abuse

laura33: Thank you! I, too, am literal-minded, and I always wondered if all the symbolism we were supposed to find in novels was really there. One high-school teacher told us the purple dress worn by the female character foreshadowed the suffering to come. First, being unfamiliar with the use of purple vestments in some churches during Lent, I thought it was pretty inconsiderate of the author to chose a symbol a lot of his readers would be unfamiliar with, and I also wondered if he didn't just put her in purple because he needed some description and just liked the color purple. Needless to say, the teacher didn't like my take on the dress. (Don't forget, Baum named his fictional land OZ simply because as he was making up the story he happened to be sitting near a filing cabinet and the drawer on his level was labelled "O-Z.")

Posted by: sideswiththekids | April 6, 2010 9:45 AM | Report abuse

Decoding is important. It's just that it is not the only part of reading. Different methods have to be used and as the author mentions, building up background knowledge for difficult concepts is important. Decoding alone would be terrible for limited English proficient kids, since they wouldn't know what the words mean once they sounded them out. That's not to say they don't need to learn the phonics system, just that relying heavily on that wouldn't help.

Posted by: celestun100 | April 6, 2010 10:11 AM | Report abuse

While the facts in the article are clear and many of the comments are right on, one issue has not been addressed here. As in much thought about testing and the whole attempt to find out how schools and students are doing, the difference between two little words is often overlooked. As a high school teacher for 39 years, I often noticed a marked difference between what students CAN do and what they WILL do. Especially on tests such as NAEP, where the results have NO consequence for the individual or school, many students will hurry through them and, at most, just guess when confronted with any very difficult passages/questions.
Accountability is a necessary component of any education system but to ignore the difference between what students are capable of doing and what they choose to do on any given day is another example of forgetting that complex issues have complex causes and effects.

Posted by: aspnh | April 6, 2010 11:22 AM | Report abuse

As it has been said so many times before, don't ignore the facts. Are we gaining on the literacy front, no? Is the answer a simple one, no?

When viewing NAEP statistics it is easy to dismiss the whole for the individual. That is the escape pod many of us ride in regularly. The individual student is not a statistic, but we must look at the whole first. We need to dig down, but that requires we start at the top.

RTI has given us a method of monitoring we can apply to individual need. It helps us start at the top and work down. Gone, can be, are the days of not knowing who needs what. We are far more able to ameliorate the issues of poor progress than ever before. Here is where we need to place our attention:

1. Set Standards of Scores on a valid reliable measure in each of our districts (we do not need to do the same one).

2. Create District teams to peel through data -- then connect that team to school level teams that peels through just that school’s data.

3. Use RtI to find children in need, or identify core curricular problems.

4. Individualize instruction; (differentiate) for the areas of highest concern.

5. Monitor utilizing initial assessment tool (or other). Remember Fidelity, oh yea and Reliability!

6. Bring to the classroom instructional help, not evaluation or assessment. Thus, change or aid in changing what instructional alterations need be applied.

7. Monitor and share across grades.

Dr. Willingham said what we need to hear. There IS a difference in what we do at 4th grade and at 8th grade, for reading. We need to instructionally meet our demons of change and hear the facts; we cannot afford to lay comfortably on the cot of what has gone before. Starting with deepening knowledge across the school year, increasing content awareness, and beginning in 5th grade, how can we miss?

Posted by: sjohnson9 | April 6, 2010 11:47 AM | Report abuse

As it has been said so many times before, don't ignore the facts. Are we gaining on the literacy front, no? Is the answer a simple one, no?

When viewing NAEP statistics it is easy to dismiss the whole for the individual. That is the escape pod many of us ride in regularly. The individual student is not a statistic, but we must look at the whole first. We need to dig down, but that requires we start at the top.

RTI has given us a method of monitoring we can apply to individual need. It helps us start at the top and work down. Gone, can be, are the days of not knowing who needs what. We are far more able to ameliorate the issues of poor progress than ever before. Here is where we need to place our attention:

1. Set Standards of Scores on a valid reliable measure in each of our districts (we do not need to do the same one).

2. Create District teams to peel through data -- then connect that team to school level teams that peels through just that school’s data.

3. Use RtI to find children in need, or identify core curricular problems.

4. Individualize instruction; (differentiate) for the areas of highest concern.

5. Monitor utilizing initial assessment tool (or other). Remember Fidelity, oh yea and Reliability!

6. Bring to the classroom instructional help, not evaluation or assessment. Thus, change or aid in changing what instructional alterations need be applied.

7. Monitor and share across grades.

Dr. Willingham said what we need to hear. There IS a difference in what we do at 4th grade and at 8th grade, for reading. We need to instructionally meet our demons of change and hear the facts; we cannot afford to lay comfortably on the cot of what has gone before. Starting with deepening knowledge across the school year, increasing content awareness, and beginning in 5th grade, how can we miss?

Posted by: sjohnson9 | April 6, 2010 12:10 PM | Report abuse

As it has been said so many times before, don't ignore the facts. Are we gaining on the literacy front, no? Is the answer a simple one, no?

When viewing NAEP statistics it is easy to dismiss the whole for the individual. That is the escape pod many of us ride in regularly. The individual student is not a statistic, but we must look at the whole first. We need to dig down, but that requires we start at the top.

RTI has given us a method of monitoring we can apply to individual need. It helps us start at the top and work down. Gone, can be, are the days of not knowing who needs what. We are far more able to ameliorate the issues of poor progress than ever before. Here is where we need to place our attention:

1. Set Standards of Scores on a valid reliable measure in each of our districts (we do not need to do the same one).

2. Create District teams to peel through data -- then connect that team to school level teams that peels through just that school’s data.

3. Use RtI to find children in need, or identify core curricular problems.

4. Individualize instruction; (differentiate) for the areas of highest concern.

5. Monitor utilizing initial assessment tool (or other). Remember Fidelity, oh yea and Reliability!

6. Bring to the classroom instructional help, not evaluation or assessment. Thus, change or aid in changing what instructional alterations need be applied.

7. Monitor and share across grades.

Dr. Willingham said what we need to hear. There IS a difference in what we do at 4th grade and at 8th grade, for reading. We need to instructionally meet our demons of change and hear the facts; we cannot afford to lay comfortably on the cot of what has gone before. Starting with deepening knowledge across the school year, increasing content awareness, and beginning in 5th grade, how can we miss?

Posted by: sjohnson9 | April 6, 2010 12:14 PM | Report abuse

Caution should be used when interpreting NAEP 12th grade results. As aspnh noted, NAEP has no consequences for students. 12th graders are savvy enough to know this.

Posted by: mgribben | April 6, 2010 12:54 PM | Report abuse

Perhaps a lot of the brighter 8th and 12th graders have left the public school system and therefore aren't being included in the results?

Regardless, you clearly have the right idea in mind: at 4th grade, reading scores are IQ+phonics; at 12th grade, they're IQ.

I'm glad to see people involved in education are finally looking at the data this way. Next maybe someone will apply this keen analysis to realize that what we're seeing in NAEP scores now is what we're going to see in America's economic productivity in the future.

Posted by: qaz1231 | April 6, 2010 1:29 PM | Report abuse

This articles totally ignores the fact that by the fourth grade, test scores can indicate the children that are already reading on a 12th grade level.

Reading tests at the fourth grade are tests of comprehension and not simply an identification of the word "blue" from the word "red".

Research of the children that read at the 12th grade level in these tests would probably find that these children knew how to read before entering the first grade.

Reading is a skill that a child either has or does not have and it should be no surprise that children that do not have this skill by the fourth grade do not have it by the 8th or 12th grade.

The focus of public school education has to be almost exclusively on children learning to read by the fourth grade, instead of the pretense that children that fail to learn to read by the fourth grade will later learn to read.

Posted by: bsallamack | April 6, 2010 2:15 PM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company