# Accountability in DCPS: Details from teacher's IMPACT report

*My guest is Aaron Pallas, professor of sociology and education at Teachers College, Columbia University. He wrote a piece last week about the IMPACT teacher evaluation in D.C. public schools, and this is a follow-up.
*

*Pallas writes the Sociological Eye on Education blog for The Hechinger Report, a nonprofit, non-partisan education-news outlet affiliated with the Hechinger Institute on Education and the Media. Pallas has also taught at Johns Hopkins University, Michigan State University, and Northwestern University, and served as a statistician at the National Center for Education Statistics in the U.S. Department of Education.*

By Aaron Pallas

My post last week on the recent firing of 241 teachers in the D.C. public schools elicited some strong reactions.

I had argued that school districts such as those in Washington D.C. and New York City, which are usin “value-added” measures for high-stakes personnel decisions (such as deciding which teachers to grant tenure, lay off or fire), have an obligation to make the technical features of these measures available for public scrutiny.

“Value-added” measures attempt to isolate how much individual teachers are contributing to a student’s current achievement from other relevant factors, such as a student’s poverty status or her achievement in the preceding year. The goal is to determine whether students are learning more or less from a particular teacher than statistical models would predict they’d learn from a typical teacher, and then to base teacher evaluations in part on the results.

In many states and school districts, value-added measures are limited to teachers in grades four through eight, because all states currently test students in grades three through eight in reading and math, and two years of data are needed to estimate a teacher’s influence on student achievement.

I also noted that the procedure described in the DCPS IMPACT Guidebook for calculating a teacher’s value-added score, which involves subtracting students’ scores on the DC CAS from 2009 from their scores in 2010, was seriously flawed, because the scores for one grade are on a different scale from the scores for the adjacent grade.

I concluded that if the district were following the description it gave in the Guidebook, it had botched the calculation of value-added scores for teachers – and that these flawed calculations may have been used to justify firing 26 teachers and placing hundreds more at risk of being terminated next year.

Frederick M. Hess of the American Enterprise Institute, in a blog post entitled “Professor Pallas’s Inept, Irresponsible Attack on DCPS,” raised two major objections to my piece. The first was that my analysis rested on a document I found on the DCPS website rather than on phone calls or emails to DCPS. Second, Hess said I misrepresented the complex procedures that DCPS and its contractor, Mathematica Policy Research, used to calculate its value-added measures.

Hess and I both agree that “the simple subtraction exercise” I described in my first post wouldn’t result in accurate value-added scores. And that’s precisely my point: the procedures DCPS described in the Guidebook are seriously flawed.

But whereas Hess is willing to take on faith the validity of whatever DCPS and Mathematica actually did – simply because “there’s a growing industry that specializes in doing precisely this” – I am more skeptical, because the description provided in the DCPS IMPACT Guidebook is so off-base.

Below, I present evidence from a current teacher’s IMPACT report to show that what DCPS provided to teachers two weeks ago states that a simple subtraction exercise is used to calculate value-added scores.

I worked from the Guidebook because that is what is publicly available. It doesn’t take a leap of logic or faith to expect that the Guidebook should accurately describe how IMPACT works.

It is the only document that was made available to DCPS teachers to explain the new system by which they were to be evaluated – and possibly fired – in the most recent school year. I was also careful to qualify my statements – noting that “I cannot be sure that this is what happened” – for the simple reason that there’s no technical report.

Such a report would detail the methodology used to make the calculations, allowing outside experts to confirm or dispute IMPACT’s validity. It would also help teachers better understand their evaluations.

A colleague of mine who sought a copy of the technical report, Justin Snider of The Hechinger Report, managed to reach the DCPS Chief of Data and Accountability, Erin McGoldrick.

McGoldrick explained that the technical report for the IMPACT system is currently being finalized by Mathematica. “DCPS is considering releasing the document publicly once it is finalized,” she said.

Meanwhile, Mathematica has now posted a brief description on its website of the value-added procedures it developed for DCPS.

Stanford economist Eric Hanushek, who serves as a technical advisor to IMPACT, told me that IMPACT uses a fairly standard regression model, which predicts current-year achievement by taking into account prior achievement and a variety of student characteristics like age, gender and socioeconomic status.

This is reassuring, although I still have many questions about the details of the model, such as its reliance on a teacher’s performance solely in the 2009-2010 school year rather than multiple years. Hanushek also told me that he’s never looked at what DCPS reports about the value-added measures.

I’m happy to hear that DCPS seems not to have botched the calculation of the value-added scores by subtracting last year’s score from this year’s score in the calculation of actual or predicted student performance, even if this is what DCPS is telling teachers it’s doing.

And I still contend that teachers whose careers were placed in jeopardy by the results should have been notified of the methodology in advance, not after the fact. Also, I reserve the right to be critical of the procedures if and when they are made public. The weight of expert scholarly opinion is that many technical and practical issues must still be worked out before value-added measures can be fairly used in high-stakes personnel decisions.

So why has DCPS misrepresented the value-added methodology to teachers and the public?

I didn’t invent the procedures described in the Guidebook – it’s an official DCPS publication. Is it really plausible that these procedures were only intended to illustrate the logic of the value-added approach to a lay audience? Other districts, such as New York City, seem able to represent the actual methods used. Do DCPS officials believe that teachers don’t need to understand the system that’s being used to evaluate their performance, or, worse, that they aren’t capable of understanding it?

Here’s an example of how the misrepresentation in D.C. continues.

A teacher in IMPACT Group 1 – the group for whom 50 percent of the IMPACT score is based on Individual Value-Added (IVA) – generously provided me a copy of her actual IMPACT report for 2009-10. (Details have been changed to protect the teacher’s identity.) This teacher received an overall impact score that was in the “Effective” range. Pages 4-8 of her report make clear that the calculation of the value-added score is based on a measure of “growth” that involves subtracting a student’s score from 2009 from his or her score in 2010.

For example, in the section entitled “Your IVA Score for Reading,” Step 1 reads as follows: “We calculated the average reading growth of your students over the past school year. On the spring 2009 DC CAS (before they entered your class), your students’ average scale score was 443.8. On the spring 2010 DC CAS (after being in your class), your students’ average score was 547.2. Therefore, on average, your students grew 103.4 DC CAS points this past school year.”

Step 2 is: “We calculated the average reading growth of students like yours over the past school year. On the spring 2009 DC CAS, the average scale score of students like your 2009-10 students was 443.8. On the spring 2010 DC CAS, the average scale score of students like your 2009-10 students was 539.8. Therefore, on average, students like yours grew 96.0 DC CAS points.”

Finally, in Step 3: “We compared the average reading growth of your own students with the average reading growth of students like yours. Recall that your own students grew 103.4 DC CAS points and students like yours grew 96.0 DC CAS points. Thus, your students grew 7.4 DC CAS points more than similar students. Your raw value-added score, then, is +7.4.”

In its reporting to teachers, then, DCPS appears to be saying one thing – which is clearly incorrect – but doing another. The big question, then, is why?

McGoldrick of DCPS provided this explanation: “The full technical description of the value-added model designed for us by Mathematica Policy Research includes sophisticated statistical techniques that require specific expertise to understand fully.We wanted to make the model as accessible as possible to teachers so they could understand how they were being evaluated. Therefore, we focused our materials on conveying the core concepts of value-added to teachers, leaving the full description to our technical advisors who have specific expertise in value-added.”

Okay, but as we learned from the financial meltdown, there are serious risks in relying on technologies so complicated that hardly anyone – and perhaps no one setting policy for DCPS – truly understands them. Involving outside experts can diffuse responsibility so that no individual is solely in charge of decisions with real consequences.

-0-

* Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!*

By
Valerie Strauss
| August 5, 2010; 6:00 AM ET

Categories:
D.C. Schools, Guest Bloggers, Research, Standardized Tests, Teachers
| Tags:
IMPACT and d.c., d.c. teacher evaluations, d.c. teachers, dc schools and teachers, how to evaluate teachers, value added evaluations

Save & Share:
Previous: Ravitch: Mayoral control means zero accountability

Next: Principal: Not the change I had in mind

Posted by: adcteacher1 | August 5, 2010 7:53 AM | Report abuse

Prof. Pallas may have inadvertently painted a picture where -- and this is by deduction -- only he, apart from Mathematica, understands the methodology released so far. He has provided no evidence that no DCPS decisionmakers have good and sufficient understanding of it--unless he hangs his hat on one phone interview. Mathematica is reputable, as Pallas knows from his own business as a consultant. But, ya know, as an outside body with offices close to an Ivy League school and with some well paid people it must be beaten up and found to be part of the larger Gates-Walton-Duncan, yada, yada plot to crush American Public Education, and specifically bust the DC teachers union, while permanently disadvantaging youth in the District. That's a mandatory charge that must be brought by we professional victims in DC. This plot is essential to understanding us and our vast progress in public education until recently.

Posted by: axolotl | August 5, 2010 8:08 AM | Report abuse

DC teachers need a good lawyer.

Posted by: Linda/RetiredTeacher | August 5, 2010 8:58 AM | Report abuse

IMHO, it is time for axolotl to either cite clear data to support remarks made on this site or be ignored.

Posted by: lacy41 | August 5, 2010 9:09 AM | Report abuse

-- This has now devolved into a "Just Trust Me" situation. We can't explain it but our hired gun experts assure us that "all is well".

-- Trust is something that is earned by performance and accountability. Currently there is no way to verify either the actual calculations made or the validity of the methodology. I, for one, need this verification to earn my trust. (There are others that don't seem to need this level of close inspection to trust the current administration.)

-- It's still NOT a "vertical scale" so DC CAS scaled scores cannot be compared from one grade to the next. (Yes, the numeric values are 300-399, 400-499, etc. but the actual test questions are not designed to measure learning progress across grades but rather are designed to measure proficiency on grade specific criterion-based standards.) See the McGraw-Hill technical papers at OSSE.

-- Since the Technical Support Document has not been published, no one at DCPS knows how the actual calculations are done. This implies that DCPS must have spent a fortune to have the calculations done for them at Mathematica. And that they just trusted them to do it "right".

-- Current research suggests that such growth measures are fraught with very high error volatility (that regression analysis cannot smooth out satisfactorily.) See: http://voices.washingtonpost.com/answer-sheet/teachers/study-error-rates-high-when-st.html#more Perhaps the Technical Document speaks to these very real issues.

-- What is the definition of "students just like yours'? Does it take into account the number of students that have been or should have been retained? Does it take into account the number of divorces or separations that occurred during the school year? Does it take into account loss of school time due to injury or illness? Does it take into account discipline issues this year versus last year versus the "students just like yours"? Does it take into account students participating in extended hours or Saturday classes last year versus this year (ie: programs cut from the budget this year versus last year)?

-- Ms. McGoldrick educational background doesn't seem to be too strong on "statistical student growth models" From the DCPS website: "Ms. McGoldrick earned a bachelor’s degree in the classics from the University of Notre Dame and a master’s degree in public policy from UCLA’s School of Public Policy and Social Research." http://dcps.dc.gov/DCPS/About+DCPS/Who+We+Are/Leadership+Team/Erin+McGoldrick

-- If it's just "a fairly standard regression model" then it should not be very hard to explain. It's harder to prove that it really works by citing applicable current research and studies. In particular, DC CAS has been used by DCPS since 2006 so now we have 5 years of consistent data. How has this model from Mathematical worked over those 4 years?

-- To me, the IMPACT growth model still feels like an experiment in the early stages but is already being used!

Posted by: interested8 | August 5, 2010 11:38 AM | Report abuse

You will also notice that Mathematica's discription of this is filled with hedges. From their website.

"For example, as with any statistical model, there is uncertainty in the estimates produced; therefore, two teachers with similar value-added estimates are said to be “statistically indistinguishable” from one another. We quantified the precision with which the measures are estimated by reporting the upper and lower bounds of a confidence interval of performance for each teacher."

Really. DCPS teachers never saw any confidence intervals, just a single number. I would be very interested to know how wide those confidence intervals are. Given the tiny sample sizes for Elem. school teachers, I wouldn't be surprised if it were 15 or so scale points, which suggests some level of doubt to teacher scores.

"In addition, because value-added estimates measure not only the effectiveness of the teacher but also the combined effect of all factors that affect student achievement in the classroom, some caution should be applied when comparing teachers across schools. Finally, if student assignment to teachers was based on unobservable factors—for example, pairing difficult-to-teach students with teachers who have succeeded with similar students in the past—a value-added model might unfairly penalize these teachers because it cannot statistically account for factors that cannot be measured."

So teachers might be unfairly punished for being good at their jobs and being given more challenging students? I can't think of anyone who has ever brought that up EXCEPT FOR EVERY TEACHER ON THESE BOARDS.

Given these flaws, are we sure this system is even close to working. I asked DCPS for the whole model, and was told that I wasn't entitled to see it, despite the fact that I am being evaluated on it.

I'm afraid that Rhee burned trust a long time ago, so telling me to trust that the system is fair isn't going to fly with me or a lot of teachers. (Note: I graded highly effective, so it isn't sour grapes here...)

Posted by: Wyrm1 | August 5, 2010 12:33 PM | Report abuse

"-- This has now devolved into a "Just Trust Me" situation. We can't explain it but our hired gun experts assure us that "all is well"."

Trust Michelle Rhee who lied on her resume about her Baltimore Miracle (TM).

Trust Jason Kamras, who appears to have made false claims about his success at Sousa Middle School.

http://gfbrandenburg.wordpress.com/2009/11/21/what-is-the-value-of-having-a-superstar-teacher/

Posted by: phillipmarlowe | August 5, 2010 12:54 PM | Report abuse

"McGoldrick explained that the technical report for the IMPACT system is currently being finalized by Mathematica. “DCPS is considering releasing the document publicly once it is finalized,” she said."

Umm...WHAT? They're "considering" releasing it *once it's finalized*? 200+ people have lost their jobs, another 700 or so are on watch and may lose their jobs next year, and it's not even finalized? They're only just now *considering* if they'll let people know how they're being evaluated???

And what about the IES study--conducted by two Mathematica researchers-- that found that current value-added models have error rates upwards of 25% (when using 3 years of data), and 35% when using just one?

How can they just implement a system that has runaway problems like this, refuse to inform people about it, and call it reform?

Teachers dig into our own pockets to pay for basic supplies. Yet there's always money for central office staff and pricey Data consultants. And more tests.

These poor children.

Posted by: TeacherSabrina | August 5, 2010 1:25 PM | Report abuse

So Pallas is saying that the difference between being a Level 3 teacher (effective) and a Level 2 teacher (minimally effective) is about 5-10 points? And that is acquired how? Oh, I know… They used the cosmological constant! Great, a teacher’s job rests upon a fudge factor…

The attempt to control for so many social descriptors with numerical values inserted into a "formula" takes this further and further from any kind of classroom reality, and frankly, credibility. It does belong in outer space.

And speaking of ineptitude and irresponsibility, where does the American Enterprise Institute, a neocon think tank, fall on that continuum? Of course Hess would "take on faith" whatever bogus math was done. It is his job to conclude that we must trust the "growing industry" dedicated to such nonsense. If there's a "business" in charge, than all is well, and if it's a BIG business, than whoop-ee!

"The goal is to determine whether students are learning more or less from a particular teacher than statistical models would predict they’d learn from a typical teacher, and then to base teacher evaluations in part on the results."

And who or what is a "typical teacher"? Where is the model for that? Looks like we’re back to the fudge factor… and it’s obvious that the people factoring it are NOT as smart as Einstein, or Mr. Spock.

Posted by: Incidentally | August 5, 2010 2:42 PM | Report abuse

Professor Pallas – did you know that this is not the first time that Rhee and Company has been dodgy about the numbers?

As I’ve said before, it took considerable badgering on my part to get the Post to stop repeating Rhee’s line that Shaw middle school’s DC-CAS scores stayed about the same, when fact they decreased. But the Post finally did it,* and in future articles, the Post acknowledged that the principal’s efforts in his first year of a turnaround had resulted in lower scores. PBS aired the same false information about Shaw in the summer of 2009, but they were much quicker to correct their error,* no badgering needed. In both cases there were official stats to back up the claims. Thank you for your inquiry into IMPACT and please stay on the trail.

* references in the order mentioned. Please check them and see for yourself

http://voices.washingtonpost.com/class-struggle/2009/10/one_of_the_struggles_most.html

http://learningmatters.tv/blog/on-the-newshour/michelle-rhee-in-dc-episode-10-testing-michelle-rhee/2476/comment-page-1/#comment-322

For detailed info on numerous DCPS statistical claims, see: http://gfbrandenburg.wordpress.com/

Posted by: efavorite | August 5, 2010 3:05 PM | Report abuse

value-added model designed for us by Mathematica Policy Research includes sophisticated statistical techniques that require specific expertise to understand fully.

.................................

Time for Americans to understand what is a mathematical model.

Mathematical models were used to convince investors that the bundles of the worthless mortgages that were being offered for sale, would be great investments.

These models of Mathematica Policy Research are as worthless as the models selling bundles of worthless mortgages.

Statistically it is totally meaningless and impossible to derive anything from the limited number of classes per grade.

There are only being generous 140 classes of 25 students per grade.

44,000 students in D.C. public schools, divided by 13 grades, divided by 25 students per grade.

The 140 classes are too small a sample to account for the random composition of classes with 25 students of different skill levels.

Imagine 4 different levels of skills, 1-4.

With 4 levels and 25 students there are 390,625 possible random classroom compositions. 2 student in a class with 4 levels is 16 possible classroom compositions. 2 raised to the power of 4.

The sample of 140 classes are too small and very little probability of finding two classrooms with the same class composition if class composition is done randomly with 4 levels and 25 student.

Okay well we attempt to make all the compositions of the classrooms the same.

Yes but now you do have the randomness of statistics, and the validity of 4 levels are no longer valid since you are not looking at random class composition where a very large random sampling set would be appropriate.

Let us say that level 1 is failure. Level 1 can not be used to simply equalize the composition of classes. On a test where less than 200 is failure there are large differences in a student that failed with a score of 50 and a student that failed with a score of 190.

Since you are attempting to equalize the composition of classes you have to equalize the average previous scores of students. But with only 25 students this can not be done in a random fashion.

Does a student with a high scores of 450 and an student with very low scores of 50 equate to two student that each have a score of 250?

Time to recognize that test scores in D.C. can not used in D.C. to evaluate teachers.

The students are too different in skills in this school system to have compositions of classes where a small sample can be used.

It has been seen that you can create mathematical models to sell worthless mortgages. Time to recognize that when you hear "includes sophisticated statistical techniques that require specific expertise to understand fully" it probably means that the model has very little validly with reality and even with mathematics.

Posted by: bsallamack | August 5, 2010 4:01 PM | Report abuse

Such a report would detail the methodology used to make the calculations, allowing outside experts to confirm or dispute IMPACT’s validity. It would also help teachers better understand their evaluations.

.............................

It is time for Mathematica Policy Research to release full details of the model for using test scores in evaluating teachers.

It is also time for Mathematica Policy Research and the DCPS to release models and methods for equalizing the class composition of students based upon test results.

Once these are released and reviewed by mathematicians I believe that it will be found that the model is very flawed mathematically.

Posted by: bsallamack | August 5, 2010 4:43 PM | Report abuse

If the education of children and the livelihoods of teachers weren't on the line, this would be high comedy...

"We calculated the average reading growth of your students over the past school year." - NO, you calculated the average growth in OBSERVED SCORES (which have a margin of error relative to what's called a "true" score), and those scores come from a test that has a debatable correlation to a subset of the larger set of skills involved in reading.

“We calculated the average reading growth of students like yours over the past school year." - NO, you calculated the average change in SCORES among some students who MIGHT resemble other students in a small number of ways for which you have data, and you ASSUMED that these were relevant factors. Furthermore, you ASSUMED that where you had NO information about the students, that the information you lacked would have no effect in distinguishing one group of students from another. Apparently, you did not consider how long students had been in the class or in the school system; whether or not the student had access to books and computers at home; whether or not the student participated in any tutoring or mentorship activities; whether or not the student lived with two parents, one parent, or another arrangement; how much school the student missed this year; whether or not the student's attendance record this year was comparable to prior years; whether or not the student or a family member experienced serious illness, loss of employment, loss of a home.

By evaluating teachers comparatively, you ASSUME that any changes in the school or curriculum affected all teachers equally, and that all factors in the school, like tutoring, administration, and the impact of other teachers, will affect all teachers equally. By comparing teachers to the group based on such simple data and attributing differences to the teachers themselves, you ASSUME that the teachers have equal access to and benefits from training and support. By making high-stakes decisions based on scores from year to year, you IGNORE or DON'T CARE about significant variables from year to year in the teacher's work or life, such as changes in teaching assignments; extended absence or illness; bereavement.

Good luck, D.C. teachers. I feel for you.

Posted by: DavidBCohen | August 5, 2010 7:37 PM | Report abuse

Mathematica Policy Research announces new method to evaluate teachers based on test results.

Those at Mathematica Policy Research are celebrating with champagne.

They are eagerly awaiting for the Federal government to provide the first batch of clone children that have been raised in government facilities to prove the effectiveness of evaluating teachers by test results.

Meanwhile others at Mathematica Policy Research, that have been working on a Federal contract, will shortly be ready to show the Federal government their model that predicts that 10 years or more of massive unemployment is an indication of a healthy economy.

Posted by: bsallamack | August 5, 2010 8:14 PM | Report abuse

2010 DC-CAS individual scores are up

Really Great job by Hardy MS 8th graders:

They went up above their 7th grade scores and last years 8th grade

29.41% Advanced in reading (83.33% prof and Adv)

18.45% Advanced Math (89.32% Prof and Adv)

Posted by: edlharris | August 5, 2010 11:25 PM | Report abuse

DavidBCohen,

Unfortunately, with Race to the Top, all teachers in this country will be evaluated in this way.

Posted by: tutucker | August 5, 2010 11:41 PM | Report abuse

If anyone has read the Bridging Differences blog, Richard Allington posted this. . .

"I'm in agreement with Diane on this one. RttT is but another "grab the money and go" scam. John Papay, Harvard University, has a paper that will appear in the American Educational Research Journal soon on pay for performance that is a must read. He finds that 40% of the top teachers using scores fom reading test were among the lowest performing teachers using a different reading test. And vice-versa. It's not just that we have bad tests but, as I have been telling my students all along, but we have bad tests that provide bad advice. We also have no tests that measure the real goal of reading instruction, creating kids (then adults) who read for information and entertainment. I'd love a system that rewarded our best teachers financially but we don't have any tests currently that I'd trust in that system. And if my experiences are valid, then most teachers are not in teaching for the money but for everything else that goes along with teaching."

If you are not a teacher, please know that Dr. Richard Allington is a highly respected literacy researcher. So not only is IMPACT bad but so is the reading tests we're giving our students.

Posted by: tutucker | August 5, 2010 11:45 PM | Report abuse

When I graduated from college, I spent a year subbing. The big thing in school at that time was behavior modification.

In a first grade class, I was so busy running around trying to figure out who got a star for what that I didn't teach at all that day. Fortunately, the kids knew the system and were able to tell me when they got stars for their behavior etc. Unfortunately, most of their learning was all focused on this reward program.

In the same building I subbed for a 5th grade teacher. After panicking when I didn't see a behavior plan, I asked a student what his teacher did when a student misbehaved. He told me, "She tells us to stop."

Can you believe that worked? :-)

Just let us teach in ways that kids can learn.

I'm so tired of data. Data rich, information poor. yuckkkkkkkkkkkkkk!!!

Posted by: tutucker | August 5, 2010 11:51 PM | Report abuse

(Bsallamack, you're starting to rub off on me. :-) )

I'm tutoring a student this summer and have been front loading him with the concepts he'll be working on for next year.

As we started working on energy, I got a little caught up in the learning and started thinking out loud.

"So electricity is one form of energy and a source of energy is the sun, but it also says that kinetic and potential energy are forms of energy. But I think kinetic and potential energy are present in all forms of energy.

I've always wondered how solar energy works. Let's look it up on Google images. It talks about DC and AC power. I have no idea what that means, but I'll ask my hubby later as that's his job."

I looked over at my tutoree and smiled. But then I realized this is what learning is.

During the school year, I try to instruct in this way. Kids research and learn about content area through investigation, questions, connections etc. I remember studying about the nervous system this past year and a student stating that she was like the associative neuron because she passed messages between two different groups during recess.

I also work hard at getting kids to think at a different level in regards to all other academic areas too.

I expect I could be scored effective by just having kids fill in the blanks, but I'm not interested in that. But the above learning that I attempt to teach can also be captured through portfolio assessments etc. Performance based assessments. Even an outside person can come in and talk to the kids about the critical learning happening in their academic areas.

I want kids to be curious, and to pursue their learning. I have also noticed that the skills come much easier when kids are active learners and not passively working on skill and drill in isolation.

And yet, we are continuing on the road that narrows the instruction and thus dumbs down our children by holding teachers accountable to a fill in the blank or multiple choice test.

Posted by: tutucker | August 6, 2010 12:09 AM | Report abuse

I want kids to be curious, and to pursue their learning. I have also noticed that the skills come much easier when kids are active learners and not passively working on skill and drill in isolation.

And yet, we are continuing on the road that narrows the instruction and thus dumbs down our children by holding teachers accountable to a fill in the blank or multiple choice test.

Posted by: tutucker

.................................

You are right. Really the multiple choice tests simply are a measure but the politicians have confused this totally.

On a reading comprehension test it is impossible for a teacher to drill children or prep them for the test. The test measures reading ability and usually this ability comes solely from reading. The only way you could drill students is where each student works by them self on a computer with a computer program where children read on their own and then answer questions to show that they have understood the material. You can not do this with a teacher in the classroom.

In reality this is the same with math. Teacher teaches the concepts to get them started and a computer program where each student is drilled by the program. Drilling 2+2 by a teacher over and over is senseless when some students know it and some do not.

The politicians and even educators do not understand that a teacher can use repetition only to a certain point. The idea is that the child sees the pattern from the repetition.

DC and AC. Direct curent is from a battery where the atoms are traveling in one direction. Alternate current is from a electrical generator with a magnet moving around coils of wiring. The atoms alternate movement in one direction and then the reverse direction because of the spin of the magnetic field. Originally when transmission of electricity in this country started, there had to be a decision on how to transmit energy. Alternate current was selected as the method for transmission of current. ( Even with the magnet and coil generator you could convert this into direct energy for transmission but this is more complex.)

Never did like kinetic energy versus potential energy. Since everything is composed of atoms that are constantly in motion seem like everything is kinetic energy. We seem to build constructs that are really chains on thought.

Posted by: bsallamack | August 6, 2010 1:25 AM | Report abuse

2010 DC-CAS individual scores are up

Really Great job by Hardy MS 8th graders:

They went up above their 7th grade scores and last years 8th grade

29.41% Advanced in reading (83.33% prof and Adv)

18.45% Advanced Math (89.32% Prof and Adv)

Posted by: edlharris

...............................

I would not be too impressed with this since the national tests of D.C. indicate that the DC-CAS.

Actually this year many D.C. are getting bonuses simply because of easy test.

Teachers should not be fired because of test results and also they should not be given bonuses. The test results really are based upon the composition of students in the class.

Time to revert back to a student getting X on test only means the student got a X on the test. It does not tell you anything about the teacher and it certainly does not tell you whether the test was too easy.

Posted by: bsallamack | August 6, 2010 1:33 AM | Report abuse

Good luck, D.C. teachers. I feel for you.

Posted by: DavidBCohen

.........................

Yes you are right about the secondary issues.

But the real issues is that you can not use tests to evaluate teachers unless each class has the same composition of students based upon the tests of previous grades.

Example of composing classes in grade A with 25 students to have same composition.

10 students failed previous grade test

8 students basic previous grade test

5 students proficient previous grade test

2 students advanced previous grade test

Even this presents a problem.

Student a failed with a 50 on the previous grade test.

Student b failed with a 190 on the previous grade test.

Having student a in your class is not equivalent to student b in a different class.

And of course you are not even considering:

Student a failed the two previous grade tests.

Student b only failed the previous grade teat and passed the test in the grade prior to the previous grade.

Having student a in your class is not equivalent to student b in a different class.

It is impossible to use test results to fairly evaluate teachers.

Posted by: bsallamack | August 6, 2010 2:02 AM | Report abuse

bsallamack,

I'm starting to like you. :-) Thanks for the info on energy. I plan to investigate it more. Your response alone has increased my interest.

On your point in regards to reading comprehension. I had a student who had incredibly high levels of higher level thinking around his reading. He increased the level of thinking in our classroom in his responses to our class reading. He could talk about author's intent, evaluate the characters' actions etc. But he stunk at multiple choice tests. he was not strong in the skill aspect of reading, and his thinking was so diverse.

I would argue that a multiple choice test can't tell us much in regards to a child's reading. I can listen to him read, talk to him about his reading etc., and if I know what I'm doing, I can evaluate him and move him to another place. I can do much more for him than a multiple choice test can.

Posted by: tutucker | August 6, 2010 10:51 AM | Report abuse

bsallamack,

I'm starting to like you. :-) Thanks for the info on energy. I plan to investigate it more. Your response alone has increased my interest.

On your point in regards to reading comprehension. I had a student who had incredibly high levels of higher level thinking around his reading. He increased the level of thinking in our classroom in his responses to our class reading. He could talk about author's intent, evaluate the characters' actions etc. But he stunk at multiple choice tests. he was not strong in the skill aspect of reading, and his thinking was so diverse.

I would argue that a multiple choice test can't tell us much in regards to a child's reading. I can listen to him read, talk to him about his reading etc., and if I know what I'm doing, I can evaluate him and move him to another place. I can do much more for him than a multiple choice test can.

Posted by: tutucker | August 6, 2010 10:53 AM | Report abuse

I would argue that a multiple choice test can't tell us much in regards to a child's reading.

Posted by: tutucker

....................................

I saw a problem in the math tests as there are trick questions which do not belong on a math test. Math is being tested and not the ability to see tricks. Trick questions on a math test are not appropriate.

You may be right about reading tests as I have really not looked at the reading tests in a while and they may have changed them to make them invalid.

Reading tests have been around for over 50 years and if valid are effective in providing information. The score is information.

As a teacher you should be able to make the determination whether the reading test is fair or not.

Maybe the child has made a mistake as an offset in filling in the blanks. If you have the test you can check this where you see correct answers that have been offset.

Give the child a test where the child circles the answers on the page instead of filling in blanks.

"I would argue that a multiple choice test can't tell us much in regards to a child's reading."

I disagree with the above. The idea of correctly structured multiple choice tests are the only fair method of evaluating all students.

You have to make the determination whether the specific test has been correctly structured.

Posted by: bsallamack | August 6, 2010 12:44 PM | Report abuse

Posted by: tutucker

.............................

Other possibilities for your student.

Reading too fast because it is a test and making mistakes. A fast runner can lose in a race because of poor pacing.

Child may be displaying memory skills and not reading skills in the classroom. Simply repeating from memory. This skill would not help on a test. Being able to memorize symbols does not imply reading ability and really is not very different from someone that can memorize complex formulas but has no understanding of applying them to solve a given problem.

Posted by: bsallamack | August 6, 2010 3:15 PM | Report abuse

Bsallamack and tutucker, I like you both. Let's talk about energy some, and I'll come back to education standards from that.

The idea that the phenomenon of energy is one "thing" which can be recognised in many forms depends on being able to put a number on it - "quantify" it - when it is approached in different manifestations. In our current International System of scientific measurement, as long as we use the seven base units to start our calculations, we can measure kinetic (and potential!) energy from any source, in any manifestation, in joules. Kinetic energy is 1/2 the mass of a moving object x its velocity squared, in joules. The energy released by a nuclear explosion is the mass destroyed x the speed of light squared, also in joules. An electron volt is one joule of energy per coulomb of charge. The energy in a photon of light is plancks constant times the frequency of the light. In joules. Energy is work, which equals force x distance, in joules. Heat is also energy, the statistical average of the kinetic energy of individual molecules and atoms: the energy needed to raise the temperature of a gram of water by one degree celsius is 4.118 joules. We express the potential energy stored in chicken fat, gasoline, glucose and ethanol in the form of kilojoules per mole. When we burn them or metabolise them, they release that much heat, or do that much work.

None of that is in the current Massachusetts framework for high school chemistry! And if it was, it would be broken down into bullets, and we would be pressured and "professionally developed" to follow test preparation teaching methods to turn it into disconnected gibberish. I know: I do still teach it, with all my strength, in a low income public school where all our resources are being stolen by these lying, cheating data-driven profiteers.

Posted by: mport84 | August 7, 2010 9:25 AM | Report abuse

None of that is in the current Massachusetts framework for high school chemistry! And if it was, it would be broken down into bullets, and we would be pressured and "professionally developed" to follow test preparation teaching methods to turn it into disconnected gibberish. I know: I do still teach it, with all my strength, in a low income public school where all our resources are being stolen by these lying, cheating data-driven profiteers.

Posted by: mport84

................................

Yes the measurement are there for describing potential and kinetic energy but how do these correlate to the force field of a magnet or the actions of movement of atoms. The lump of coal where energy could be be measured by heating the coal, and in the same way that the energy of a piece of wood could be measured. Yet there will be differences and simply stating this is potential energy hides these differences.

Is electricity that is moving in transmission lines only potential energy until the electricity is used by a device?

Moving can be changed to poised to move if one is speaking of AC.

Agree with you that the standardized testing is mostly a waste of money.

I can see having basic standardized tests for reading and arithmetic but question the money spent for science, writing, and various other subjects for standardized tests.

Instead of spending money on these standardized tests it would be better to spend the money for more equipment in science labs for experiments, and computers in writing classes.

Let teachers grade students in these classes and use the expense of standardized tests for other purposes.

My physic class in my last year of high school was an elective and I dropped it quickly when I saw that it would be doing pound foot measurements and that I could get out of school at 2:15 instead of 3.

I could see teaching basic logic in the public schools with a standardized test, but the politicians would never allow this since the voters later would recognize the false premises of the politicians. It is very interesting that everyone wants students to think, yet no one is willing to allow logic into the public schools with the basic laws in regard to thinking logically. Think for yourself but we will not teach you the basic rules of logic.

Posted by: bsallamack | August 8, 2010 3:35 PM | Report abuse

The comments to this entry are closed.

We who are evaluated by the methods used for IMPACT simply call this the shell game. Given how IMPACT worked at my own school it is hard to believe that this is actually expected to identify and keep effective teachers at a school. As it has been used at my school IMPACT has proved to be a subjective tool with mumbo jumbo math thrown in to try and make it look objective. In the end, due to the way math is used for the scoring from tests and the fact that Group 1 teachers'test scores account for 55% of their score (not the 50% that is constantly reported in your paper), very few teachers will really receive incentive bonuses. This works out great for a cash strapped school system that could use the foundation money elsewhere - for instance, for the highly paid consultants who designed IMPACT and who will be designing the tests to be used to evaluate pre-K and Kindergarten students next year. This is a sham and your paper's refusal to investigate fully the paid consultant scandal that is Rhee's tenure is shameful.