U.S. troops in Afghanistan first to get new device; ‘red’ means you're lying
By Bill Dedman
Investigative reporter
updated 6:00 a.m. ET April 9, 2008
FORT JACKSON, S.C. - The Pentagon will issue hand-held lie detectors this month to U.S. Army soldiers in Afghanistan, pushing to the battlefront a century-old debate over the accuracy of the polygraph.
The Defense Department says the portable device isn't perfect, but is accurate enough to save American lives by screening local police officers, interpreters and allied forces for access to U.S. military bases, and by helping narrow the list of suspects after a roadside bombing. The device has already been tried in Iraq and is expected to be deployed there as well. “We're not promising perfection — we've been very careful in that,” said Donald Krapohl, special assistant to the director at the Defense Academy for Credibility Assessment, the midwife for the new device. “What we are promising is that, if it's properly used, it will improve over what they are currently doing.”
But the lead author of a national study of the polygraph says that American military men and women will be put at risk by an untested technology. "I don't understand how anybody could think that this is ready for deployment," said statistics professor Stephen E. Fienberg, who headed a 2003 study by the National Academy of Sciences that found insufficient scientific evidence to support using polygraphs for national security. "Sending these instruments into the field in Iraq and Afghanistan without serious scientific assessment, and for use by untrained personnel, is a mockery of what we advocated in our report."
The new device, known by the acronym PCASS, for Preliminary Credibility Assessment Screening System, uses a commercial TDS Ranger hand-held personal digital assistant with three wires connected to sensors attached to the hand. An interpreter will ask a series of 20 or so questions in Persian, Arabic or Pashto: "Do you intend to answer my questions truthfully?" "Are the lights on in this room" "Are you a member of the Taliban?" The operator will punch in each answer and, after a delay of a minute or so for processing, the screen will display the results: "Green," if it thinks the person has told the truth, "Red" for deception, and "Yellow" if it can't decide.
The PCASS cannot be used on U.S. personnel, according to a memo authorizing its use, signed in October by the undersecretary of defense for intelligence, James R. Clapper Jr.
The Army has bought 94 of the $7,500 PCASS machines, which are sold by Lafayette Instrument Co. of Lafayette, Ind. The algorithm, or computer program that makes the decisions, was written by the Advanced Physics Lab at Johns Hopkins University. Besides the Army, other branches of the U.S. military have seen the device and may order their own. The total cost of the project so far is about $2.5 million.
Congress has not held any hearings on the PCASS device. Myron Young, a spokesman for the Pentagon’s Counterintelligence Field Activity agency, which sponsored development of PCASS, said it informed congressional committees in a memo in November that the device had been approved for use. But five months later, no hearing has been scheduled. Congress has already scaled back its oversight of the polygraph. Five years ago it eliminated a requirement that the Defense Department produce an annual report on polygraph use.
Less accurate than a polygraph
Polygraphs have sparked a fierce debate for at least a century. While supporters claim the devices are reliable, the National Academy of Sciences allows only that they're "well above chance, though well below perfection." Polygraphs are not allowed as evidence in most U.S. courts, but they're routinely used in police investigations, and the Defense Department relies heavily on them for security screening.
Both the proponents and critics agree on one thing: This new device is likely to be less accurate than a polygraph, because it gathers less physiological information.
Like a polygraph, the PCASS uses two electrodes to attempt to measure stress through changes in electrical conductivity of the skin. It also gauges cardiovascular activity, though with a pulse oximeter clipped to a fingertip, rather than a polygraph's arm cuff, which has the advantage of measuring both pulse rate and blood pressure. Unlike the polygraph, the PCASS does not measure changes in the rate of breathing, and it has no way to detect countermeasures, or efforts to fool the machine, such as by making unusual movements.
The training is different, too. While polygraph examiners for the Defense Department must have four-year college degrees and experience in law enforcement, the PCASS operators are mostly mid-level enlisted personnel and warrant officers, some as young as 20 years old. While polygraph examiners take a 13-week course and a six-month internship, PCASS operators undergo only one week of training, though most have military training as interrogators. The Defense Department says PCASS is simple to operate, because judgment of truthfulness is left to the computer.
Discarding 'inconclusives'
The debate over the device’s usefulness boils down to a disagreement over its accuracy.
The Pentagon, in a PowerPoint presentation released to msnbc.com through a Freedom of Information Act request, says the PCASS is 82 to 90 percent accurate. Those are the only accuracy numbers that were sent up the chain of command at the Pentagon before the device was approved.
But Pentagon studies obtained by msnbc.com show a more complicated picture: In calculating its accuracy, the scientists conducting the tests discarded the yellow screens, or inconclusive readings.
That practice was criticized in the 2003 National Academy study, which said the "inconclusives" have to be included to measure accuracy. If you take into account the yellow screens, the PCASS accuracy rate in the three Pentagon-funded tests drops to the level of 63 to 79 percent.
Even if you accept the lower accuracy rates, the Pentagon officials say, the device is still better than relying on human intuition.
"Let's take a worst-case scenario here, and let's say PCASS really is 60 percent accurate," said Krapohl, who heads the project for the Defense Academy for Credibility Assessment at Fort Jackson, S.C. "So let's get rid of the PCASS because it makes errors, and go back to the approach we're currently using, which has less accuracy? As you can see, that's really quite untenable if we're interested in saving American lives and serving the interests of our commanders overseas.”
Msnbc.com asked Fienberg, a professor of statistics and social sciences at Carnegie Mellon University in Pittsburgh, to review the unclassified Pentagon studies of the PCASS. He said he was not impressed.
"I, like everyone else I know, want the troops in Iraq, in Afghanistan, elsewhere in the world, to be protected," Fienberg said. "I want terrorists to be detected. I do not want to be living in a threatened world, and I want to give my government the best possible advice.
"They need devices that work. And if they rely on things that really don't work, and act as if they do, we will have a greater disaster on our hands than we already do in the field in Iraq. We simply do not know what a device like this hand-held device will produce in that kind of setting, except for the fact that there's scant evidence that it will produce anything of value."
Only 'a triage device'
Pentagon officials say that the new device will not be used to make final decisions, that the rules forbid it. They say it is intended only to pare down a large group of people to a smaller group that will receive further scrutiny through traditional means, such as interviews or a full polygraph exam.
"The PCASS is envisioned more or less like a triage device," Krapohl said. "That is, it's not used as a standalone technology to make final decisions regarding a person's truthfulness. ... There are locals, there are Iraqis and third-country nationals who apply for access to U.S. military bases to provide support functions. And as you might well imagine, there's not really a good way to do a background investigation of these individuals. And so decisions whether to allow them to come on post and to take these jobs are being based on some pretty imprecise methods, primarily interviews and whatever record checks are available. ... So the idea of adding the PCASS is to incrementally improve the decisions that are made so that we protect our forces."
The term "triage" normally means deciding who gets attention first. But if PCASS is used to pare down a large group to a smaller one, wouldn't a person who's red-lighted be denied access to the military post? Krapohl acknowledges that possibility, saying it's not dissimilar from the ways colleges choose which students to admit.
"Let's say that they have 10 positions that are open, and they have 100 people who apply, which is very realistic,” Krapohl said. “Whatever tools you use to make that assessment, 90 people are not going to get that job anyway. So your role as a decision-maker is to help improve your decision process by making sure that those 10 are flawless, that those individuals have nothing in their background to raise your attention. Therefore a commander might be inclined if he tests or she tests 100 people, and you get 50 green lights, ... to restrict the decisions to just those 50."
No harm in that, Krapohl said. "That's how we make decisions for hiring people everywhere, or making decisions on college applications or so forth. Most people are not going to get in. How do you improve the likelihood that those who do get in are going to be good candidates?”
One floor down from Krapohl's office, the lead instructor for the PCASS project, team chief James Waller, said he's confident that the science behind the device is solid.
"Spent 20 years in the military. If I didn't think it would work, I wouldn't put it in the hands of a soldier, 'cause I was a soldier,” Waller said. “So it's not perfect, but they don't have a device at all over there to do this kind of work."
Testing far from a battlefield
Fueling the debate over the PCASS is the fact that the tests before deployment were all conducted on Americans, in English, far from a battlefield. The Pentagon tested the device on Army recruits at Fort Jackson, S.C., and on civilians in Columbus, Ohio, who answered classified ads. In all, only 226 people were tested in the scientific experiments before the machine was approved for use by soldiers in the field.
Although the device was also tried in the field last year in Iraq and Qatar on allied forces, Iraqi police officers, translators and prison guards, the Defense Department says these were only "field vetting," not scientific tests. The goal was to make sure the device was easy to use and would hold up to battlefront conditions.
"It's virtually impossible to do a validity study in a war zone," Krapohl said.
The desire for a portable lie detector at the war front is described in a memo from Camp Cropper, a U.S. Army detention camp on the outskirts of Baghdad. The unclassified memo from April of last year was written by David Thompson, the team leader of the counterintelligence human intelligence support team at Cropper. In screening Iraqis to work as corrections officers and interpreters, Thompson said, the HUMINT collectors (or human intelligence interrogators) are trying to discover who might be involved in militia or insurgent activities, or who might break the rules by passing information from prisoners to people outside the base.
To make these decisions, Thompson wrote, the interrogators often have little more than gut instinct to go on.
"The ability of the collector to filter truth from fiction is highly dependent on the individual collectors training and real world experience," Thompson wrote. "As the demand for HUMINT personnel has risen, the standards we use to select applicants to the HUMINT field has dropped, yielding a higher percentage of young and inexperienced personnel."
So what's needed, Thompson explains, is a machine.
"While not a perfect technological solution that would completely remove the 'human factor,' it appears to be an invaluable tool," Thompson writes. "PCASS is not a system that will remove the need for experienced collectors but rather will enhance their capabilities. ... If the PCASS system gives a 'red' reading to one of these questions, the collector can focus his efforts on exploitation, and use the traditional skills of a collector to uncover deception or information."
'What's new here?'
Lie detectors sometimes work because people believe they work, deterring the wrong people from applying for jobs in the first place, or prompting admissions of guilt during interrogations. Thompson explains that may be the most important use of PCASS. "Once this capability is introduced and word spreads of its use, militia or insurgent related individuals will be less likely to attempt to gain admittance" as correctional officers.
Fienberg, the leader of the National Academy of Sciences study, said this memo demonstrates one of his concerns, because the interrogator shifts all of his attention to those who got a red light. The professor said the danger to soldiers doesn't come from the false positive, the person who fails the lie detector test with a red light despite being truthful. Those people just get further scrutiny, or barred from base. The danger, he said, lies in the false negative, the liar who tests "green" and escapes further scrutiny.
"They've been testing polygraph-like devices for a long long time, and this one doesn't look like it works as well as other ones that were discredited," Fienberg said. "I believe we need a different way to think about interrogation and information gathering, instead of using the polygraph as a magic bullet."
The designers of the PCASS said they attempted to make it lean toward detecting deceptive people, because of the disproportionate consequences of green-lighting a liar in a war zone. The Johns Hopkins researchers said they tweaked the algorithm so it takes only a little evidence of deception to turn the lights red. They also tried to minimize the yellow lights, at the Pentagon's request.
But they acknowledged that this was no easy task. They use the word "non-trivial," which in scientific lexicon means a problem is difficult, even unsolvable.
"Determining these decision rules," the researchers wrote, "is both non-trivial and subjective."
Fienberg, the professor of statistics, said the Pentagon's pre-deployment studies have several weaknesses: They're conducted and paid for by the same people promoting the device. They're small studies, involving few people. The mock crime scenarios don't match the stresses of a war. And no allowance was made for differences in language and culture. Finally, they assume that about half the population is telling the truth, and half are lying; in the real world, where the number of liars may be very small, the number of false positives (truthtellers who get a red light) can overwhelm the resources of investigators.
"Ask the question, what's new here?" Fienberg said of the PCASS. "They went to a group that developed a computer-based algorithm that produces green, yellow and red lights. That's the new technology. It's still based on the same noisy psycho-physiological channels for detecting deception. So we have no new way to detect deception. What we've got is a streamlined version of the polygraph. Therefore you would expect this not to be as accurate as the polygraph.
"Almost everything that has gone on since our report was published is obfuscation," Fienberg said, "including the use of the euphemism 'credibility assessment' to describe the DoD activities."
Krapohl and the Defense Department say they hope to develop better tools for detecting deception. Meanwhile, by being portable and better than human intuition, PCASS fills a niche in a war zone.
"That's the best that we can do right now," Krapohl said. "But if you come back next year or in five years, I might have a different answer for you."
The PCASS is being developed only for military use. Krapohl said he doesn't foresee it being sold at the pharmacy for use by parents of teenagers, or by the local police.
"I don't want to see a state trooper with one in the back seat asking you what you have in the trunk."
© 2008 MSNBC Interactive
URL: http://www.msnbc.msn.com/id/23926278/
