Researchers at Hightouch tested whether lightweight decision models could solve identity resolution—the process of mapping data records to real people—and found that pairing Jev with Claude Opus achieves frontier-level accuracy at over 4x lower cost than using Opus alone.
Identity resolution is foundational for customer analytics and personalization. The challenge is determining whether multiple records (differing addresses, emails, names with typos) refer to the same person or different people. Manual review is slow—humans check about 50 identities per hour—and estimating error rates precisely requires sampling over 1,000 identities.
Hightouch built a benchmark using pseudopeople, a tool from the University of Washington that simulates a fictional town with realistic data errors. They tested 1,000 identities, half of which were single people with noisy records and half intentional over-merges (multiple people incorrectly grouped as one).
Standalone performance showed Opus at 99.0% accuracy, while Jev and Luna—decision models that return confidence scores—trailed at 95.6% and 97.3% respectively. However, Jev’s confidence calibration proved superior. Most of Jev’s errors came with low confidence, while Luna’s errors carried high confidence, making Jev better suited for hybrid deployment.
In the hybrid approach, Jev handles initial screening at high confidence, referring only uncertain cases to Opus. This strategy achieved frontier-level accuracy while dramatically reducing costs: decision models cost $0.05–$0.10 per 1,000 identities versus $3.86–$4.48 for Opus. Processing speed also improved—decision models answered in 0.15 seconds compared to 2–4 seconds for frontier models.
The researchers tested various confidence thresholds, finding that Jev’s performance improved with adjusted cutoffs. A 60% confidence threshold for Jev maximized accuracy in the hybrid model. According to the findings, Jev’s calibration advantage enabled it to outperform Luna in the hybrid setup, even though Luna performed better out of the box on general cases. The gap between decision and frontier models narrowed significantly on harder cases.
Key facts
- Jev + Opus hybrid model achieves frontier accuracy at over 4x lower cost than Opus alone
- Jev’s mistakes come with low confidence signals, enabling better hybrid model performance than Luna
- Decision models cost $0.05–$0.10 per 1,000 identities versus $3.86–$4.48 for frontier models
- Jev achieves 95.6% accuracy out of the box; Luna 97.3%; Opus 99.0%