Network News

X My Profile
View More Activity


Posted at 8:00 AM ET, 02/18/2011

Dawn of the dumbest school data

By Valerie Strauss

Mr. Teachbad's Blog of Teacher Disgruntlement has some very funny, telling posts (though some have language that would not be allowed on a Post blog) about what teachers face today. He writes, obviously, under a pseudonym in order to, obviously, keep his job.

By Mr. Teachbad
Dawn of the dumbest data ... data-driven dementia... data: It keeps teachers busy. Take your pick. But these cats at my school really have to be stopped.

As you may suspect, we here at my school are “data-driven.” That’s right. There is no substitute for data. And the best thing about it, from an administrator’s point of view, is that you don’t have to worry about how long it takes teachers to collect the data or if it is really of any value in the first place. Just collect that data and tell everybody that you are collecting it and using it to make data-driven decisions ... for the kids. The rest, my friend, will fall into place. No worries.

Here is our scenario:

At the beginning of every course we give a “diagnostic” exam which covers the content of the course. About 30 or so multiple choice questions. Each question is to be matched to a standard. There may be more than one question per standard. After the exams are graded, each answer to each question from each student is put into the “standards mastery tracker” spreadsheet. This is a “tool”, if you know what I mean. (Can you just imagine how excited they were when they found this? It must have been something to see.)

Over the course of the course we are to “track” each students’ mastery of each standard and create reteaching “action plans” and all manner of whatnot, driven by the data, to ensure student mastery yadda yadda yadda....

I will comment on the general stupidity of this in a moment. But first I want to mention this semester’s addition, which is sure to close the achievement gap very soon: It is that we must now not simply code answers to the questions as, for instance, 1= correct; 0= incorrect. We must now also indicate which of the three possible wrong answers each child chose for each question.

Now the general critique.

1) When I test students on the content of a course they have not yet taken, and then I find that they score poorly, I am not sure what I have learned. For example, the last time I did this, the overall number of correct answers was 30.2 percent. On a multiple choice exam with four choices, that is pretty much exactly what you would expect if people were just guessing. So I have learned that the students do not yet know the content of the course they have not yet taken. Is that about it? And all I had to do was enter 2,500 data points.

2) Then I was required to create an “action plan” based on this data. Seriously. OK ... you dummy, my action plan is to now teach the course....

3) Why does it matter how every student does on every standard? Isn’t that what quizzes, midterms, projects and finals are for? ... The point of looking at all of these assessments as a group is that at the end of the class we can look at a student’s work and see if he or she basically got it or not, how well, etc. If there is a systemic problem, like 80 percent of the class thinks folic acid comes from farts and methane is needed to produce red blood cells ... THEN we need an action plan. Not before.

4) These questions from which we glean our precious data may or may not be good questions. They are just questions pulled from websites, written on the train on the way to work or after a couple glasses of wine at dinner. There is no quality control or testing of these questions that would give us any certainty that answering them correctly would constitute “mastery” of anything.

I have worked for three years at a university survey research center and I bet I have taken more college and graduate courses in statistics and research methodology than all of my administrators combined. What we do to and in the name of data here might be legal, but I’m sure it’s a sin.

-0-

Follow my blog every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | February 18, 2011; 8:00 AM ET
Categories:  Guest Bloggers, Laugh and cry, Teachers  | Tags:  data-driven reform, data-driven school reform, guest bloggers, mr. teachbad, school data, school reform, teachers, test-driven data  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Texas may reopen social studies standards for rewrite
Next: D.C. vouchers and the arrogance of Congress

Comments

Great blog. One other aspect about collecting data like this, is what about those questions answered correctly, when that standard hasn't been taught? Should the teacher not teach it?
If all students knew or guessed correctly on a standard that had only one question, it would show a 100% mastery, despite that aspect not being taught. Even an administrator would acknowledge this is a problem with this type of data collection. They don't see it when a question is answered incorrectly even though the conclusion is just as erroneous.

Posted by: researcher2 | February 18, 2011 8:47 AM | Report abuse

Everyone in our district is excited to have a new Data Team that's driving us even more to the data! I'm sure I'll do much better as a teacher now.

http://speakingofeducation.blogspot.com/2011/02/3rd-quarter-of-year.html

http://speakingofeducation.blogspot.com/2011/01/mothering-tigers-crouching-standardized.html

http://speakingofeducation.blogspot.com/2011/01/problem-with-public-schools.html

Posted by: speakingofeducation | February 18, 2011 8:51 AM | Report abuse

The course pre-test IS a waste of time, granted. One has to presume most students are going to test out negatively.

The test at the end of the course (the final) is the worthwhile data. What test items proved most troublesome for students? Did the teacher spend enough time on that standard or was the material simply too difficult for the class? Would an alternative strategy produced better results?

One alternative to collecting this data and amending one's way based on the results would be to ignore everything. That's right. Heck, we could probably even eliminate the achievement gap overnight if we ignored this information (Rod Paige).

That's the way it was prior to NCLB. There was little or no quantitative information on students/teachers anywhere (other than perhaps the SATs) that showed whether students were actually learning anything for the enormous sums of money spent of public education.

Don't get me wrong. There is too much testing and we need to minimize it whenever possible. However, you cannot eliminate tests all together. There must be information produced to demonstrate whether learning has or has not occurred. Instead of testing students every year grades 3-8, maybe every other year or every third year would suffice, especially for the kids who perform well.

Posted by: paulhoss | February 18, 2011 9:50 AM | Report abuse

paulhoss,
Teachers do give quizzes and test to assess learning, and where the gaps are. Teachers have always used these tests (and briefer on-spot assessments..are kids expressions confused as the lesson is being taught).

The issue being addressed here is looking at multiple choice tests, created by those not in that classroom, to assess "what has been learned" via these data points.

The issue I brought up, one question for one standard (which does occur) where everyone gets it right due to a combination of guessing and prior knowledge. Does that really equate to a 100% mastery? No. It doesn't. Just as questions that show kids didn't "master it" at whatever percentage mastery is stated to be demonstrated on that particular test. Some questions are poorly worded. Some questions don't really assess the standard they are supposedly assessing. Classes and teachers have been known to discover the key has wrong answers, but the data has been compiled prior to that discovery.

Assessing kids yearly in grades 3-8, or quarterly, or every two years may provide worthwhile information, but it isn't the information that folks currently state it is. It doesn't address teacher quality. The only way you can assess teacher quality is perhaps a growth model of testing (which would include pre-testing. Pre-testing can show you if the students have the necessary background knowledge. Not testing on what will be taught, but what they should already know to learn what will be taught, i.e. do they know their multiplication tables? If not..hmm, maybe Algebra in 7th grade isn't the appropriate starting place) But again, the teacher teaching the course should create the pre-test. Throughout the course teacher created tests are given..and at the end of the course a test is given. Have the students learned what was taught? Then the teacher taught well. Did the kids with lots of background knowledge learn more? Then the teacher taught well; and that is key to many people's dismay. Students who are bright and inquisitive, with lots of background knowledge aren't making the extra progress they should make due to the focus on NCLB.
We should be assessing each child's growth. Whether they start way below or way ahead, a year's worth of growth should be evidenced. Looking at SAT type of tests (standardized with norm references) does not address teacher quality. It is this erroneous assumption that is destroying education and the teaching profession. Creating norm referenced, standardized tests was never meant to address teacher quality..due to the nature of the test; i.e they all have natural bell curves. Questions that result in too many correct answers are thrown out etc. etc.

Posted by: researcher2 | February 18, 2011 10:09 AM | Report abuse

Because sinister intentions must be kept on the down low, data driven education is packaged and sold as economical but revolutionary pedagogy come to the classroom. Absurdity is the inevitable result. And so it is that our system of universal public education is now trapped in a scene from Woody Allen’s farce Bananas. The new leader has decreed that, “From this day on the official language of San Marcos will be Swedish. In addition to that, all citizens will be required to change their underwear every half hour. Underwear will be worn on the outside so we can check. Furthermore, all children under 16-years-old are now…16-years-old!”

Promoting insanity in the classroom has proven to be quite an effective weapon of public school destruction. Teachers are being broken down and driven away at an unprecedented rate. One in five new teachers will not make it through their first year. Half of them will be gone inside of five years.

My fellow dutiful educators, its time to quit banging our heads against a word wall. Under layer upon layer of silly directives you work yourselves to a state of physical ruin and mental exhaustion termed “burnout” by observers. You do it desperately trying to prepare the young for a future our economic system does not intend to provide. The world you grew up in will not exist for these children.

Bill Gates, the prime profiteer for trivia (data) collection, likes to call the high school obsolete. He’s half right. Our schools are obsolete because the system that has enriched him beyond all reason is doomed. His obscene wealth is a symptom of a deadly economic malady. His world is unsustainable. It is not viable. It has begun to breakdown and will in time collapse.

Posted by: natturner | February 18, 2011 11:33 AM | Report abuse

I am sorry that this teacher is dealing with a weak administration that can't answer his simple questions and has possibly designed a weak measure. I think the concept is great, but the execution here sounds poor. It's ridiculous that you have to do any data entry- this is a minimal amount of work to fully automate.

" For example, the last time I did this, the overall number of correct answers was 30.2 percent. On a multiple choice exam with four choices, that is pretty much exactly what you would expect if people were just guessing."
That's why you are being asked to look at the data on a topic by topic and student by student basis. This is what intelligent people would do. Looking at the overall score is obviously meaningless.

"Then I was required to create an “action plan” based on this data. Seriously. OK ... you dummy, my action plan is to now teach the course...."
If the measure was well designed, you may have learned that some course sections are already well understood. You may have also learned the different levels of understanding across the class so that you can provide differentiated instruction.

"Why does it matter how every student does on every standard? Isn’t that what quizzes, midterms, projects and finals are for"
Because there are too many classes that claim to be teaching A but are actually teaching B or not teaching much of anything at all.

"These questions from which we glean our precious data may or may not be good questions."
Just like the material on your "quizzes, midterms, projects and finals". This is a real issue though, and the questions themselves need to be analyzed carefully.

"The course pre-test IS a waste of time, granted. One has to presume most students are going to test out negatively."
No way, the course pre-test is not a waste of time. It is a great device to avoid wasting time. Another key is for the test to include the material from the year before to ensure that basis material is in place.

Posted by: staticvars | February 18, 2011 12:05 PM | Report abuse

Absolutely perfect description of the utter nonsense behind "data-driven instruction." Now follow up with a piece on just how much your school district spends on this nonsense. It's absolutely criminal and parents, teachers, students, and citizens should be up in arms! There is something very sinister to the agenda of these often for-profit companies pushing this meaningless assessment and that meaningless assessment. We must stop the insanity now or we will have a whole generation of cynics.

Posted by: Jennifer88 | February 18, 2011 1:57 PM | Report abuse

After working on test preparation materials for publishers (and after taking all too many standardized tests), I am totally opposed to standardized tests. But I can see the value of tracking which wrong answer is selected. If the majority of the students select a specific incorrect answer, obviously the question has more than one interpretation. Years ago, test-scorers were startled when almost all first-grade students in a school failed to correctly circle the picture of the first thing they used in the morning. Instead of the toothbrush, every one of them circled the bucket; in their rural Appalachian area at this time, few houses had indoor plumbing and a first-graders' traditional morning chore was to carry water from the outside pump before anything else got done.

In my junior-high history class, the teacher asked us to name a principle "peculiar" to a certain political movement. He meant "specific," but every student in the class assumed the word meant "strange" and selected the item that was not associated with the movement. If the officials had not looked at the specific answers, only at the number correct, no one would have looked further into the reason for the answer.

Posted by: sideswiththekids | February 18, 2011 3:37 PM | Report abuse

If you want a good laugh about what teachers have to go through in today's climate of pacing guides and standards based curriculum, I highly recommend this youtube video:

http://www.youtube.com/watch?v=nVXhA_hs2J8&feature=youtube_gdata_player

Posted by: chicogal | February 18, 2011 10:56 PM | Report abuse

Education should be data informed, not data driven.

Posted by: educationlover54 | February 19, 2011 4:04 PM | Report abuse

Here's a great video:

http://www.youtube.com/watch?v=BpmQZ5MXs8c&feature=related

Posted by: educationlover54 | February 19, 2011 4:06 PM | Report abuse

Educationlover54: thanks for providing the youtube link

http://www.youtube.com/watch?v=BpmQZ5MXs8c&feature=related

I think there are a lot of classroom teachers who are being "grilled" by coaches who spew "the top down" education party line - all the pseudo lingo included! I am sure they can relate (albeit tragically).

Posted by: teachermd | February 19, 2011 5:55 PM | Report abuse

Post a Comment

We encourage users to analyze, comment on and even challenge washingtonpost.com's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.




characters remaining

 
 
RSS Feed
Subscribe to The Post

© 2011 The Washington Post Company