Network News

X My Profile
View More Activity

Posted at 5:00 AM ET, 02/ 8/2011

How can we fairly assess teachers, students, schools?

By Valerie Strauss

This was written by educator Anthony Cody, who taught science for 18 years in inner-city Oakland and now works with a team of science teacher-coaches that supports novice teachers. He is a National Board-certified teacher and an active member of the Teacher Leaders Network. This post appeared on his Education Week Teacher blog, Living in Dialogue.

By Anthony Cody
A colleague at the Teacher Leaders Network raised a provocative pair of questions.

1. In an era where numbers are currency, what alternative set of metrics and numbers (beyond assessment) can we suggest that reformers and policymakers consider when weighing teacher/school effectiveness? (i.e: parent/student satisfaction surveys, levels of funding, graduation rate, rate of enrollment in AP classes, rate of employment or enrollment in college after graduation)

2. Given the limits of numerical accountability, what alternatives can we offer to reformers who are open to considering results that cannot be accounted for by a number? What are the softer variables that cannot be easily measured? (i.e: student engagement, attitudes towards school, divergent thinking)

I have heard different forms of this conversation several times over the past few weeks. On the one side we have people, largely from the world of business, who have developed what seems to them a perfect way to improve our work. This method amounts to a four step cycle. First, set some measurable goals for ourselves. Then do our best to meet our goals. Then review our outcomes and see where we fell short and where we succeeded. Use this data to guide a revision to our methods so that our outcomes will improve.

This logic, coupled with the accountability mechanisms built into No Child Left Behind, have amounted to an almost irresistible set of pressures on teachers to become "data driven." Some have succumbed, but many of us still resist, clinging to quaint ideals about the value of the whole child, the need for critical thinking and curiosity, and other things which are difficult to measure on standardized tests. But then we face a challenge, which my colleague has captured in these two questions.

What could possibly be wrong with the improvement model offered above? Lots of things. First of all, the data most readily available for measuring outcomes is usually standardized test scores. This leads us into the test preparation sinkhole most of our high needs schools find themselves, where instruction is continually narrowed to focus on improving those scores - to the detriment of many other learning goals that we value.

So then we get to the next question, which my colleague posed above. If we do not accept the test scores as an adequate marker of our effectiveness, what do we wish to offer in their place, since we must be accountable for student learning in some concrete and measurable way?

I believe that any answer to this must encompass the complexity of learning, and of our goals as educators.

The way I get my mind around this is to think about the ways that I have seen teachers take responsibility for student learning in meaningful ways. I cannot discuss this in the abstract. So here are some real models of authentic assessment.

Lesson Study: In this process teachers begin by discussing what it is that they desire for their students. What do they value most? What do they want to see from their students at year's end? But this is a truly open-ended question. It is not "which standards do we want to choose to emphasize." If the teachers are most concerned about how their students are treating one another, this would be a perfectly acceptable focus for their lesson study. Once they select the focus for their work, then they collaborate to create a set of lessons that will result in students learning this. The lessons are taught, and carefully observed, with close attention being paid to evidence of student learning. This, to me, is an example of teachers taking responsibility for student outcomes.

National Board certification
likewise asks candidates to gather solid evidence of the impact their instruction has had on students, and document this with student work samples. Candidates must show concretely how student work reflects growth over time, and how their instruction made a difference. Videotapes of student-teacher interactions also shed light on this.

Oakland history teachers have been working for more than a decade on an assessment system where teachers district-wide give their students a common writing task, to respond to a question while drawing on evidence from a selection of primary historical documents. Students are given editorial cartoons, photographs, and written documents from the period in question, and asked to apply what they have learned about the events as they answer the question. Teachers then bring samples of their students' work to district-wide scoring sessions, which allows them to compare the work their students are doing to work being done elsewhere in the district. This has helped to create a rich environment for collaboration and the sharing of strategies, as teachers whose students' work is especially strong, can share the techniques they found effective.

In the mentoring program I help direct, TeamScience, we use the Formative Assessment Tools associated with the Beginning Teacher Support and Assessment program (BTSA). Central to this process is a protocol called Assessment of Student Work, in which we collect all the student work from a given assignment, sort it into different levels of accomplishment, then work with our mentee to figure out how to move students at each level forward, based on the evidence we see.

All four of these are meaningful ways that teachers are learning about their teaching from looking closely at the work their students are producing. This is raw data come to life, as we delve into what our students are producing, and seek to overcome the obstacles we uncover.

From my point of view as an educator, the best reason to look at data is in order to get useful feedback to guide us in becoming better as teachers. We want to know, if we have a goal that our students are able to write a coherent analysis of a historic event, citing evidence, what is it they are actually able to do? Where are they falling short? How can we build these skills so they are successful?

The entire structure of No Child Left Behind has created a whole other purpose for gathering and looking at data, and that is to hold teachers and schools "accountable" for student test scores. Thus we have high stakes consequences - and ever more of them - for student achievement. This is a different purpose than we have as teachers, and unfortunately, when accountability drives assessment, we get a whole host of unintended consequences that we have become all too familiar with.

Assessment for accountability is, by necessity, going to look very different from assessment for the improvement of instruction. It must be standardized, it must be taken by large numbers of students at the same time in order to allow "fair" comparisons, and it must be cheap to score. Teachers are far more interested in the more authentic assessments I describe here, because they actually help us improve and better serve our students. But we are deeply concerned with data that shows how our students are learning, and our best professional growth often revolves around collaborative reflection on our instruction and the student work that results from it.

This does lead us in an improvement cycle similar to the one offered by the business model. But in the test-score driven cycle above, the question is almost always the same: "How can we boost these scores?" In the inquiry cycle that is represented by the examples I offer, the questions really vary, according to the challenges we have identified as teachers. The collection of data remains a critical step, but the data is more varied, and sometimes more qualitative. The teachers' role as an active agent of change is much stronger, as they must play an active role in determining the focus of their inquiry, and figuring out the strategies they will pursue in order to improve their outcomes. We must look at student outcomes, but we cannot let the constraints of assessment for accountability purposes determine the nature of those outcomes.

And what might an evaluation look like connected to this? How about one that asked, as National Board portfolios do, for a teacher to share a collection of student work that demonstrates growth over time? How about one that took into account evidence that a teacher is engaged in the reflective processes described above? How about an evaluation where the evaluator spent time in the teacher's class to see how he was applying the lessons he learned from examining last year's student work?


Follow my blog every day by bookmarking And for admissions advice, college news and links to campus papers, please check out our Higher Education page at Bookmark it!

By Valerie Strauss  | February 8, 2011; 5:00 AM ET
Categories:  Anthony Cody, Guest Bloggers, Standardized Tests, Teacher assessment  | Tags:  evauation systems, lesson study, national board certification, standardized tests, student assessment, teacher assessment, teacher qualifications  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   StumbleUpon   Technorati   Google Buzz   Previous: New study: How L.A. Times teachers data is flawed
Next: School board conferees get tough with Duncan


Just a comment on the national certification process...there is a huge problem with teachers gathering their own data on where they have helped Students learn. Having seen a teacher gain national certification because the video she submitted showed her finally, finally teaching students on those days leads me to know the national certification cannot be trusted. I do know other teachers who are nationally certified and they are wonderful. However, please question a teacher's ability by asking prior or current students and parents. Never take a title or initials after a person's name as meaning anything other than they passed a test or procedure. The person needs to demonstrate mastery.

Posted by: knoxelcomcastnet | February 8, 2011 7:12 AM | Report abuse

Anthony Cody's suggestions for better ways to measure student progress are great (especially the final two; I share knoxelcomsastnet's reservations about national certification, and lesson study while wonderful does not focus on outcomes but inputs). I hope he and similar teachers can come up with a way to take these types of practices to scale. At the same time, we will still need some external measure of student learning, and it will probably involve standardized tests, though one would hope fewer of them.

Posted by: jane100000 | February 8, 2011 8:33 AM | Report abuse

Adult programs require objectives. Objectives are what is expected of the student at the end of a lesson, chapter, semester, etc. These are testable items, and they are the only items tested.

Kids are not much different in that they want to be told, to some extent, what they need to know. What does the teacher, school, system, or other program require of them? If it is addition, tell them what they will be expected to know, what is the acceptable practice, or the required activities.

Objectives help keep everyone on track, give guidance to both teacher, student, parents, and system, and provide measures by which acceptance is governed. What is wrong with those points?

It is easily measured then by meeting acceptable criteria. I like the Formative Testing. That is a guide for students also, but only if provided to the student as a guide. Putting that information in a file cabinet does little to serve the student.

Posted by: jbeeler | February 8, 2011 10:25 AM | Report abuse

Lesson Study: "The lessons are taught, and carefully observed, with close attention being paid to evidence of student learning."
How is that going to work? What is the output of this close attention- students appeared to be learning? Seems clearly inferior to a test or quiz as a measure of learning.

Certification: "Candidates must show concretely how student work reflects growth over time, and how their instruction made a difference."
So teacher's complain about time spent preparing for tests, but they are willing to come up with their own evaluation frameworks and criteria (which may or may not be related to basic objectives to measure things such as reading comprehension? Why not standardize the test?

Oakland: "give their students a common writing task" "district-wide scoring sessions, which allows them to compare the work their students are doing to work being done elsewhere in the district"
Sounds quite like the AP test grading process- plus, the test is standardized.

As I read the constant laments against standardized tests, I see two primary issues- one is separating student performance from teacher performance. This is a hard problem, and requires careful consideration of the history of the student. This also requires that everyone admit that all students do not have equal ability, capability, and work habits.

The real value of the tests though is for the students. only becomes clear with the pre-test, post-test approach. Through use of pre-tests, I can accurately assess where a student's starting line is located.

The secondary motivation is that teachers want more autonomy. I certainly think that effective teaching goes way beyond teaching to a test, but creative, inspired teaching should ultimately improve test scores, since it provides more memorable ways for students to learn things.

I would also suggest that teachers unhappy with the standardized test frameworks get involved in the whole process. Provide input to the standards. Provide feedback on the tests. If you are designing your own evaluation metrics, please share them and don't force me to come up with my own.

Posted by: staticvars | February 8, 2011 1:00 PM | Report abuse

Some common public school problems include no "BOOKS" for the students. Yes, that is right. Teachers lecture, might use power point or the chalk board, there may be books in the room that students can look at for a few minutes in the classroom. Here is the kicker, if the student's parent is on the school board or is a powerful person in the community, the student receives their very own new copy of the text for each course. Thus, parents believe that all students have books. Nice Touch! Then no one knows why the average student has low reading scores. Answer; students do not read their assignments. If a student says, I never have homework, that means that the students don't have books. They might have handouts if the xerox machine is working. Reading a handout is not the same as reading a textbook assignment. Also, without a book, one cannot reread the assignment and the parents can't read the material either. As for 'evolution' well, teachers shy away from that subject because they can get fired for mentioning it. Most teachers try to avoid teaching Biology because of the Evolution issue and opt for Chemistry or Physics. One parent asked to see his son's 3rd grade math book and was told the books must remain in the class room. The parent offered to buy one for all students and the school refused. He contacted the publisher and was told that publishers have an agreement with schools not to sell directly to parents because it isn't fair for some students to have a book and not others unless your parent is on the school board. The parent enrolled his son in a private catholic school that had a take home text for each course. Berkman and Plutzer from Penn State who conducted a survey know all about these wierd problems as many of the above problems are common to Pennsylvania. P.S. Replace the director of the department of education for not taking action on such problems. Blab schools went out of vogue about 200 years ago.

Read more:

Posted by: windmill3 | February 8, 2011 3:44 PM | Report abuse

Great job, Anthony, of illustrating the complexity of the task if there are to be meaningful results. Current high-stakes tests sample a tiny representation of stated standards and drive the entire school program down a very narrow road. Any claims that these tests actually measure a students true learning or a teacher's effectiveness are pure rubbish. The examples you provide are a starting point if we wish to really improve teaching and learning.

Posted by: rvaliant | February 9, 2011 9:34 AM | Report abuse

Post a Comment

We encourage users to analyze, comment on and even challenge's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.

characters remaining

RSS Feed
Subscribe to The Post

© 2011 The Washington Post Company