The intended point was not that smaller models are universally better, but that performance depends on the job they are trained to do.
“The frontier models are amazing generalists,” Armstrong wrote. “But increasingly you’ll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.”
Coinbase’s Owning Intelligence series described how that approach evolved, with the company first adding an AI risk agent to its existing machine-learning model and rules engine for Onramp, which lets users buy crypto through partner applications, including guest checkout without a Coinbase account.
An online experiment found 30% fewer fraudulent transactions and 22% less fraud value in the group using the additional AI review. The agent examined recent transaction sequences to identify patterns that might not be apparent from a single payment. It could block a transaction that passed the initial checks, but could not reverse an upstream block.
Coinbase then tested newer versions of closed-source models against a fixed benchmark of 16,140 transactions across 7,293 users, including 813 confirmed fraudulent transactions. It reported that newer Opus, Sonnet, and GPT versions performed worse on several fraud-detection measures than their predecessors.
You may also like:
- AI Agents Take on Bigger Role in Cybersecurity, CertiK Says
- Apple Patches iOS Flaw That Could Let Attackers Run Malicious Code on iPhones
- Coinbase Faces $25M Loss Claim From Crypto Investor
GPT-5.6 improved precision but recorded lower recall, meaning it identified a smaller share of fraudulent transactions. That result motivated the move to an open-weight model, whose parameters could be adapted directly rather than accessed only through a provider’s API.
Issue Flagged Over AI Autonomy
Coinbase’s approach fits into a wider discussion about AI agents taking on financial crime and security work, with Certik warning last week that more autonomous security agents could bring risks from incorrect outputs and reduced human scrutiny. The firm recommended clear limits on autonomous decisions and a named owner for every agent.
However, the crypto exchange’s agent works within such limits, returning a risk score and having application codes that apply the decision policy. It can also block a transaction that earlier checks allowed, but it cannot override a block that’s already been made.
