Dry Runs And Dress Rehearsals
A house oversight panel held an interesting hearing last week about the troubled $1.2 billion worth of contracts at the Department of Homeland Security to buy radiation detectors known as Advanced Spectroscopic Portals.
The focus of the hearing by the House Energy and Commerce investigations subcommittee was tests conducted earlier this year to determine the effectiveness of the contractors' machines, cutting edge but largely untried devices that cost hundreds of thousands of dollars. The department's Domestic Nuclear Detection Office, or DNDO, wants to deploy hundreds of them at border crossings and ports to improve the detection of radioactive materials in trucks, cargo containers and such.
The machines currently in place work but have high false alarm rates that officials say the new machines would improve upon.
Regular readers of Government Inc. have heard about this project before. It has been beset by questions from lawmakers and the Government Accountability Office. The tests conducted in February and March were conducted after the GAO questioned whether the DNDO really knew whether the machines would work as billed and were worth the expense.
What stands out from the House hearing last week is this: The government allowed the contractors to practice repeatedly detecting the radioactive material before the actual tests. The DNDO called these practice sessions "dry runs" and "dress rehearsals."
In testimony, GAO officials could not have been clearer about what that meant to them: "DNDO used biased test methods that enhanced the performance of the ASPs. DNDO conducted numerous preliminary runs of almost all of the materials, and combinations of materials, that were used in the formal tests and then allowed ASP contractors to collect test data and adjust their systems to identify these materials."
Under tough questioning, Vayl Oxford, director of DNDO, defended his office's efforts, saying the testing has been "comprehensive and rigorous."
"DNDO and [Customs and Border Protection] believe that tests performed to date have shown that ASP systems provide enhanced detection and identification capabilities," he said in his prepared statement.
When asked about their participation in the process, the contractors -- Raytheon, Canberra and Thermo Fisher -- deferred to the DNDO to comment. They in turn deferred to a spokesman for the homeland security department, who said that "dry runs are a smart and common practice for testing programs and technologies prior to live use."
"ASP systems will have been subjected to the most rigorous and comprehensive evaluation of any radiation detection equipment ever deployed by the U.S. government," the spokesman said. "The department has put forward a prudent approach to testing a system that is very much needed, and is already showing promise of considerable improvements over current capabilities."
What does all this mean to you, readers? Is it appropriate to give contractors such a test ahead of time before the results are actually counted?
In other news:
GovernmentExecutive has a story about an effort by Democrats to create a wartime commission to review contracting fraud, waste and abuse.
The LA Times writes about the State Department's intervention in a congressional investigation of Blackwater's activity in Iraq.
By Robert O'Harrow |
September 27, 2007; 6:09 AM ET
homeland security
Previous: Calling Telecom: Let's Begin The Overhaul |
Next: Private Armies, Public Debate Redux
Posted by: Michael Lent | September 27, 2007 8:31 AM
These tests are a good practice and can produce useful data that can be re-used to improve system performance, and can help identify system vulnerabilities.
Posted by: Michael Reilly | September 28, 2007 10:28 AM
Testing is multi-purpose in Defense contracting especially for new technology.
First, to reach funding milestones, it is often necessary to show progress beyond theory has been pursued and prototyped.
Second, it may signal a secret has been compromised and full scale assesments in civilian and intelligence community threat elimination should be expected by all covert operatives as activity peaks in all agencies foreign and domestic, clandestine and clearly transparent.
Sometimes, for example, the Chinese anti-satellite deomonstration in January 2007 is an exhibition for a sales campaign on the equipment market. If an imminent purchase of a foreign product is delayed because a new defense is demonstrated it could cause a multi-billion dollar deal to fail for a Chinese competitor.
One never really knows what testing has done to secret plans and projects.
Posted by: Michael E. Huang | September 28, 2007 12:18 PM
The comments to this entry are closed.
Dry runs for tests to qualify, certify, or accept devices have been used repeatedly in some DoD weapons systems development program. Several of the anti-missile ballistic defense platforms and guidance systems had the benefit of those rehearsals.
Familiarization is fine, but rehearsals shouldn't be. If we want DHS to lemming-like emulate the worst-practices of Defense, the overseers should just back off.
Question to Robert O'Harrow: from your detailed familiarity with this case, which party--government or the contractors--is at fault for this arrangement? Or, is it another (lame and questionable) example of "partnering"?