Network News

X My Profile
View More Activity

Posted at 3:00 PM ET, 02/18/2010

Data-driven gobbledygook from Ed Dept

By Valerie Strauss

Help. There is something about the “major findings” of a new report released by the Education Department that I’m not grasping. The conclusions seem to be so self-evident that I’m not sure why a study had to be done to reach them, though, to be fair, this is hardly the only education report that has crossed my computer screen and left me thinking the same thing.

According to a recent press release now promoted on the homepage of the department’s website, researchers surveyed officials from 529 school districts, conducted in-depth site visits at 36 schools in 12 districts that lead the way in data usage, and analyzed secondary data from a survey of more 6,000 teachers--all to obtain a national picture of current data use practices at the local level.

They produced a report called “Use of Education Data at the Local Level: From Accountability to Instructional Improvement.” Here are the five “major findings” highlighted in the report:

*Data-driven decision making is an ongoing process rather than a one-time event centered on the acquisition of a data system. Districts will get more out of their investments in electronic data systems if they think about data-driven decision making as a system-wide innovation and develop a long-term strategy for its implementation as part of a continuous improvement process.

*To influence teachers’ day-to-day instruction, data systems must provide teachers with information that is both timely and relevant to their instructional decisions. To be useful to teachers, systems need to provide data from recently given assessments that provide diagnostic information on students’ learning needs.

*Human and organizational supports for data use are just as important as the technical quality of the data system. Professional development around data use is widespread, but only a small minority of districts and schools have made data use a regular part of teachers’ practice.

*Districts can promote data-driven decision making in schools by providing time for teachers to meet with colleagues to discuss and use data, funding positions for instructional coaches who help teachers connect data to alternative instructional approaches and by modeling data-driven decision making for continuous improvement in their own operations.

*Districts’ greatest perceived area of need with respect to data-driven decision making is for models of how to connect student data to instructional practices. Among teachers, there is a need to enhance their assessment interpretation and data use skills.

So what I am missing? Did we need to survey 6,000 teachers to figure this out?


Follow my blog all day, every day by bookmarking And for admissions advice, college news and links to campus papers, please check out our new Higher Education page at it!

By Valerie Strauss  | February 18, 2010; 3:00 PM ET
Categories:  Education Secretary Duncan, Research  | Tags:  education research  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   StumbleUpon   Technorati   Google Buzz   Previous: Should 10-year-old train hard for Olympics?
Next: Colleges admit freshmen for winter term


*To influence teachers’ day-to-day instruction, data systems must provide teachers with information that is both timely and relevant to their instructional decisions.
Yes it is important that teachers know that for the current day they should teach 2+7 instead of 2+4.


From a nation that decides the aim of public school education is that every student is proficient and which is in reality to teach to the lowest common denominator, to the new policy that data systems are needed to inform teachers daily to what is the lowest common denominator.

Posted by: bsallamack | February 18, 2010 3:23 PM | Report abuse

"So what I am missing? Did we need to survey 6,000 teachers to figure this out?"

Did they contract out this survey?
Either way, a great waste of time and money.

Posted by: edlharris | February 18, 2010 3:23 PM | Report abuse

Teachers have known this for years. It's called "getting feedback" and teachers are taught to get it as frequently as possible so as to inform subsequent instruction. Good teachers have been doing it for ages. Even when I was a child, the teacher would present a lesson and then check for understanding. Most teachers kept careful track of student progress by recording information from frequent informal assessments.

We have been in a "dumb phase" in education for over ten years. I'm still counting on President Obama to use his brilliant mind to lead us out of it.

Posted by: Linda/RetiredTeacher | February 18, 2010 3:43 PM | Report abuse

It is over a year since the only real standardized tests of public schools were given in 2009 by the government and the results for the 4th grade and 8th grade tests in Reading are still not available.

Perhaps the government would better spend their time and effort on speeding up getting the results of tests that really show the state of public education in this country.

Posted by: bsallamack | February 18, 2010 4:36 PM | Report abuse

Looks to me to be a complete waste of money. Maybe there's an appendix lsiting districts that are actually collecting useful data? And then changing instgructional practices? But if studies like this one don't give examples of successful approaches so that those interested can talk to them and avoid some of their implementation mistakes and adopt some of their successful approaches, we'd be better off using the money to pay down the deficit (or buy more Predator unmanned planes for Afganistan).

Posted by: mct210 | February 18, 2010 4:41 PM | Report abuse

We have been in a "dumb phase" in education for over ten years. I'm still counting on President Obama to use his brilliant mind to lead us out of it.

Posted by: Linda/RetiredTeacher
Yes from his hand picked Secretary of Education who believes the problem of disruptive children is because of teachers who can not do class management.

Of course there is also the "test them until they drop" and "teach to the test" of the Secretary of Education.

One grows tired of the brilliant mind of the President who has so far simply shown the mind of Chicago politics in governing the country.

Posted by: bsallamack | February 18, 2010 4:43 PM | Report abuse

I have to admit that I am not impressed by what the Obama administration has done so far. But I'm still hoping he'll come through for us.

Posted by: Linda/RetiredTeacher | February 18, 2010 5:21 PM | Report abuse

I'm agreeing with Linda on both points. If we'd elected President Obama as School Board Chair and he continued the absurdity of the last ten years, that would be one thing. But I'm sticking with the president through thick and thin.

Posted by: johnt4853 | February 18, 2010 5:37 PM | Report abuse

Well...while people are "waiting"

hopefully those who need to get step up are doing so and prepared because this bus is coming. Either prepare to get on or prepared to get off because students have been thrown under them (busses) far too long.

And as far as Data compilations. It's the waive of the future and industries nationally/internationally are embracing data driven formats, and using this information to determine trends, weaknesses, strengths, overhead, budgeting to include forcasting best assessment while forecasting industry projection compilations.

Posted by: TwoSons | February 18, 2010 5:39 PM | Report abuse

"Data driven" means teachers are required to post electronic grades far more often than needed, twice a week per subject in my district. It's complete grading overkill. It produces mass quantities of data with little useful information.

Linda is exactly right: good teachers constantly gather feedback about how students are doing. But the current requirement to quantify every academic burp is a huge waste of everyone's time and energy.

Posted by: aed3 | February 18, 2010 6:25 PM | Report abuse

....and then there's the whole issue of interpreting the data. It's used to make all kinds of crazy assumptions by fools who think children are like computer punch cards.

Posted by: aed3 | February 18, 2010 6:28 PM | Report abuse

As one of these teachers in these data driven schools, I find I have less information on students. My county thinks the only way to test student’s knowledge is through multiple choice tests. I teach social studies and all I see is students learn how to pick a, b, or c. I have no view into their thought process, so I never see why they choose their answer. I have no time to delve into the topic because I have to move at super speed so they can answer more multiple choice questions.
We begin our day late on Friday so we can work together to access the data. Would you tell me how I am suppose to use data to help me try and figure out why the student picked Catherine the Great instead of Louis XIV as the absolute ruler of France. Plus, if I presented the lesson in four different ways and they still don’t get the information, how do I get them to learn it.
Does anyone know how to remediate children when you taught the information five different ways and they still don’t get it? By the way louder and slower doesn’t work.

Posted by: ktksmom | February 18, 2010 7:07 PM | Report abuse

I am so tired of this Horse Hockey!! It's passed time for us teachers to unite nationwide and take back the profession of teaching from the policy wonks and bean counters.

Let's have a national strike and shut down all schools until the administrivial types let go of the reigns so we can get back to teaching again!

We could set a date, May 15th. National teacher walk out day!

Human beings have been successful teachers for hundreds of thousands of years. We don't need computers and politicians to tell us what we already know how to do instinctively.

We need start this revolution before our children/or society become drones! Start passing this on to all of your teacher colleagues. Time to unite!

Posted by: tazmodious | February 18, 2010 10:49 PM | Report abuse

As they say, if we knew what we were doing, it wouldn't be called research.

No one could say in advance whether teachers had anything penetrating and useful to say about the use of data.

Turns out they didn't.

Posted by: nunyo555 | February 19, 2010 6:54 AM | Report abuse

I disagree with the premise of the column entirely. I admit the writing in the report seems a little jargony, but it highlights important points about the use of data. Too often government collects data and does not put it to good use. This report seems to be making the point that we need to do more to make data systems useful. Something really revolutionary has happened in our education data systems over the last 10 years. We finally have access to individual student level data. This is an opportunity to actually engage in "data based decision making" on a scale not seen in US history. The potential is powerful, but we need to take a mindful approach to our use of data. Research like this is the type of work that needs to be done. I think the writers criticism is weak, and an attempt to score easy points. Unfortunately it is also unelightened and counterproductive.

Posted by: poppycock | February 19, 2010 9:19 AM | Report abuse

Possible reasons for doing this research:

> They really knew the answer but needed data to back it up.

>there were people at DOE who really didn’t believe this, so it had to be proved to them.

>They had a big pot of money that had to be spent on research, and this is the least controversial study they could think of. And/or it was the pet project of someone with limited judgment but lots of power

They got the data in order to misuse it somehow, while saying “research confirms” (as Melinda Gates does in the 2/19/10 editorial*) but not citing the research. Or even if they do cite it, not that many people will check it out, so however it’s misconstrued will become an urban legend.

*In the online version of that editorial, there are links to the gates Foundation, KIPP, RttT and the State applications to RttT, but there are no links to the “data” she cites about teacher effectiveness.

This is reminiscent of Michael Petrilli of Education Next and the Fordham Foundation who, when praising Rhee for raising NAEP scores, linked to everything but the data that showed the scores had been rising long before Rhee came. To his credit, he added a link when readers provided the actual data and commented on his lapse.

Posted by: efavorite | February 19, 2010 9:23 AM | Report abuse

tazmodious- It would be great for teachers to "take bake" education. To do that, you have to start acting like the AMA and not the UAW. The teacher's unions have too long concentrated only on lunchbox issues and so their message as heard by the public has been only about pay scales and benefits. Teachers need to be at the forefront of educational policy. And to do so are going to need data such as this to make their case.

Posted by: senigma | February 19, 2010 10:18 AM | Report abuse

It's not for educators that this study was necessary.

It's for the general public who question school systems when they spend money on data systems. It's for the legistlators who want to cut,cut,cut, foolishly thinking that their cuts won't make any difference.

Effective use of data is indeed common sense. But it also costs money - to buy, maintain, and operate the system. And it takes additional planning time - to evaluate the results the data system is giving us.

Policy-makers (and voters) don't often realize what REALLY needs to happen in a school, even if it makes commone sense. So, we need to have a report stating the obvious. It helps us refute those who think education should always be done "on the cheap."

Posted by: mdennis74 | February 19, 2010 1:04 PM | Report abuse

If they didn't do / commission studies about data...and then do analysis...then what else would people work on? Heaven forbid they actually visit the schools and talk to the students / teachers / administrators about what they really need.
I'm sure this document employed numerous people, what with the writing, editing, revising, editing, re-writing, data "analysis", editing, get the idea.

Posted by: robjdisc | February 19, 2010 1:41 PM | Report abuse

Because nobody listens to teachers, everything in education tends to swing to the extremes one way or the other. Data collection is the new swing and we try to collect too much of it, because nobody trusts teachers either.
When you focus too much on data, you spend all of your time looking behind you and not looking ahead. It is also the reason why CPA's don't necessarily make the best presidents of companies.

Posted by: benathornton1 | February 19, 2010 2:21 PM | Report abuse

Posted by senigma: "It would be great for teachers to "take bake" education. To do that, you have to start acting like the AMA and not the UAW. The teacher's unions have too long concentrated only on lunchbox issues and so their message as heard by the public has been only about pay scales and benefits."

Typical. Blame the teachers and the unions. The unions are run by the teachers by the way.

"Teachers need to be at the forefront of educational policy. And to do so are going to need data such as this to make their case."

No. You don't need reams of data to select the most effective teachers and in-school administrators to become part of a cohort that can make the best decisions for their community and for education as a whole.

All of this data collection for the bean counters and policy wonks is a huge waste of money and time. The public has been duped into believing this data is in any way important.

If we just give the folks who know education and the needs of the students the chance to make the decisions, the public will see positive results and completely disregard this data driven fallacy for what it is.

Just another dog and pony show for the bean counters and policy wonks.

Posted by: tazmodious | February 19, 2010 2:48 PM | Report abuse

Teachers across this country have come to live everyday with this absurdity. Incessant testing with no relation to the real world, the mindless collection of trivia classified as data, forcing a "business model" like Enron or Lehman Brothers or General Motors on the public schools, driving the arts and the social sciences out of the curriculum, and watching every Chancellor, Superintendent, Commissioner, and Secretary of Education promote charter schools over their own public schools at every turn. Absurd! But again the product of a hidden agenda is always absurdity.

Because we are bombarded with it by the corporate media, there is the temptation to believe the global economy will enjoy a "recovery" and the US will visit even greater heights of material prosperity. This is a delusion that is being foisted on the American people. It's part of a scam. There is no rational reason for this system to be revived and there are oligarchs, and people at Goldman Sachs, and people in the US government and military that know this. They have left behind some functionaries in the public schools, "dead-enders" like Michelle Rhee in Washington D.C. and Joel Klein in NYC to soldier on with the corporate catechism. They have not bothered to demobilize the cults created to undermine the public schools; Teach For America, Green Dot and KIPP charter schools, but the true believers and their cults are no longer a credible threat.

The new danger appears in the rise of the seamless melding of the corporation and the state in the US. The corporate-state was certified as constitutional by the US Supreme Court in its recent decision on corporate campaign financing. The new reality is reflected in the unprecedented amount of money Secretary of Education Arne Duncan suddenly has at his disposal to undermine the public schools. Duncan has put the 50 states in a competition he calls the Race To The Top, to become the most effective at destroying public education and advancing the charter school movement. Duncan will spread over $4-billion among the "winning" states. The denial of funds is expected to finish off public education in the "losing" states.

Posted by: natturner | February 19, 2010 3:20 PM | Report abuse

Valerie, Maybe you didn't need a survey of 6000 teachers to tell you what is common sense, but you have obviously not met many of the dimwits who run our school systems. Believe me, when these people make a cake, the first thing they do is put an empty pan in the oven. Then they pour 2 cups of milk into the flour jar and scoop out the mushy paste, smash 3 eggs into it, put the whole mass into the microwave with a few sticks of bakers chocolate and some sugar. Then they burn their hands getting the now heated pan out of the oven to fill with the goop from the microwave ... In other words, they have real difficulty with systematic and logical thinking. They believe the miracle of technology will magically turn itself on and start spitting out teacher tips for the day. In reality, the old IT maxim "garbage in, garbage out" is the order of the day.

Posted by: dz159 | February 19, 2010 9:14 PM | Report abuse

I'm sorry, but the data issue isn't as simple as it seems. Garbage in, garbage out. What factors might skew the data? Skewed data can lead to misleading conclusions -- just look at the recent controversy over global warming for examples.

What samples are we pulling? At what frequency are we collecting the data? Do we include ESOL and special ed in the data? What are the baseline understandings of the children we are evaluating?

How are the assessments designed? Are we merely testing what John A. Van De Walle calls Procedural Knowledge, or are we have a good procedure for evaluating problem solving ability, which depends on understanding connections between conceptual and procedural knowledge. This is no trivial matter. Van De Walle looks at mathematical knowledge on a continuum, with "relational" understanding on one end, and "instrumental" knowledge on the other end. Students who have not developed relational knowledge frequently misapply strategies they have been taught, and might add when they should multiply, get confused when subtracting across the zeroes, etc.

Students who lack a conceptual schema might have a surface level understanding of 7 and a counting procedure without automatically knowing that 5+2, 3+4, 9-2 relate; might not know it is prime; might not know that it is large compared to .7, or small compared to 70; might not know that 26x7 can be computing mentally by thinking of 25x7, which can be seen as (25x4) + (25x3) + 7 (Van De Walle).

Are we advancing a generation of students through the curriculum without taking care of individual developmental needs? Does pacing guide pressure leave sufficient time to develop unit level understandings? How do we know whether a student has truly "constructed" relational knowledge versus merely learned how to perform an unconnected jumble of mechanical answer finding tricks?

I remember how it became fashionable to believe that real estate is always a good bet. Then, out of nowhere, we had a banking crisis. The truth was out there, but few people were looking.

Posted by: dannykurland1 | February 20, 2010 9:29 AM | Report abuse

The comments to this entry are closed.

RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company