TechnologyBuilding a better lie detector

The Intelligence Advanced Research Projects Activity (IARPA), within the Office of the Director of National Intelligence (ODNI), announced the other day the winner of its first public challenge contest, Investigating Novel Statistical Techniques to Identify Neurophysiological Correlates of Trustworthiness (INSTINCT). Troy Lau and Scott Kuzdeba’s winning solution, “JEDI MIND,” could drive further high-risk, high-payoff research on using one’s own neural, physiological, and behavioral signals to help anticipate other people’s intentions or behavior.

A DNI release reports that in JEDI MIND — Joint Estimation of Deception Intent via Multisource Integration of Neuropsychological Discriminators — Lau and Kuzdeba used a combination of innovative statistical techniques to improve predictions approximately 15 percent over the baseline analysis. They found that someone’s heart rate and reaction time were among the most useful signals for predicting how likely their partner was to keep a promise. The team’s combination of focused expertise with broader interdisciplinary interests helped them to address the complexities of the challenge — while both have experience with computational neuroscience, Lau is a Ph.D. physicist with a background in data mining and finance, and Kuzdeba is a research engineer with experience in statistical learning, various engineering applications, and economics. Both team members work in BAE Systems’ Adaptive Reasoning Technologies Group, located in Burlington, Massachusetts. 

Predicting one person’s trustworthiness from another’s signals is a difficult task, and IARPA chose the challenge format to solicit a wide range of statistical approaches to address it. “The overall structure of the challenge was a really positive and fun experience,” said Lau. “The crowdsourcing competition aspect was nice, but the way it was structured with five submissions per week and a real-time leaderboard was the best part. It made for some unique ‘instant gratification’ that is rare in research.”

Over the seventy days that the INSTINCT challenge was open, thirty-nine solvers developed algorithms using the training set and then submitted algorithms to be scored against the test set. Seven of these exceeded baseline performance for the test data set, and their authors were invited to submit their “best and final” algorithm for independent evaluation against a third, non-released data set. Lower performance with new datasets is common, but JEDI MIND’s algorithms performed well on the new data set used for evaluation.

”We’re delighted with Lau and Kuzdeba’s insight into the data,” said Adam Russell, the TRUST program manager. “Their performance under the rigorous evaluation process of the INSTINCT Challenge provides additional evidence in support of one of the TRUST program’s basic hypotheses:  that the self’s own, often non-conscious signals — if they can be detected and leveraged appropriately — may provide additional valuable information in trying to anticipate the intentions of others.”

Creating analysis techniques that generalize well is a major issue for many kinds of multimodal data sets and unsurprisingly was one of the largest hurdles for INSTINCT solvers. Although only one solver successfully overcame this barrier, all of the finalists produced innovative work and are to be commended. IARPA is currently assessing next steps for potential new research in this area based on lessons learned from the INSTINCT Challenge.

The release notes that IARPA invests in high-risk, high-payoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage over our future adversaries. Additional information on IARPA and its research may be found on iarpa.gov