AI Agents: The Next Liability Frontier
On September 22, Meta’s new personal AI agent, Muse, took the blame for a selloff in banks, insurers, and online travel companies. The theory is simple: plenty of businesses make money because customers renew out of habit instead of shopping around. Give every consumer a tireless digital assistant that shops for them, and that habit goes away. A Goldman Sachs trading desk even named a “consumer inertia” basket: AT&T, T-Mobile, Allstate, Progressive, Netflix, Paramount Skydance, Expedia, and Booking.1
The market is focused on the businesses agents may disrupt, and for simple, commodity purchases that disruption may well be real. But the big money sits in complex, labor-intensive decisions, and I think the more consequential question is whether we can give software the authority to commit us to those contracts before we have settled who bears the cost when it gets the judgment wrong.
Nice Theory, Hard Implementation
For agents to dissolve consumer inertia at scale, three things have to be true at the same time: (1) the agent has to be accurate enough that people trust it; (2) it has to be autonomous enough to act without someone babysitting it; and (3) it has to be powerful enough to change how incumbents behave. While each is plausible on its own, I see how they pull against one another.
The Judgment Paradox
The critical issue I see holding back agent adoption by most consumers: any agent that optimizes solely on price will frequently make the wrong choice for your actual preferences; and you will likely bear the consequences no matter whose fault it is.
When we authorize an agent today, it lacks the one thing that would let it make better choices: our preferences, in priority order, so that seemingly conflicting issues have a way to be resolved where the choice fits you rather than some rational choice model.
Simple switches may be the easy case. Take medical insurance as a very relevant example of the harder ones. An agent has to know how you value your doctor, your commute, and your tolerance for hassle before it can offer the best choices, let alone make a choice for you. A competent agent can catch the obviously bad plan from standardized benefit summaries; but the choice that matters most depends on whether your doctors stay in network, which drugs stay on the formulary, how aggressively the plan uses prior authorization, and what care you’ll need next year. Nobody knows most of that in advance, and getting even close means handing an app your health history while trusting it won’t be accidentally leaked.
Until we solve for judgment, these agents are at best informational assistants who need our attention and final approval. That may be enough to break inertia on a phone plan. It is not enough on the complex, labor-intensive contracts where the real money sits.
The Insurers Have Played This Game Longer
Since we’re talking about how insurance and AI agents may interact, let’s look at the other side of the coin: how insurers already use expert systems to optimize their results.
The bull case for agents like Muse assumes consumers are finally getting a sophisticated negotiator that will save them time, money, and frustration. It also assumes insurers haven’t had similar algorithms on their side for decades, and that is factually incorrect.
In 2010, Allstate paid $10 million to 45 states after an 18-month multistate examination of how it used a program called Colossus to value auto injury claims. The issue was not about any intentionally bad act; but rather that the software wasn’t tuned consistently across regions.2 My point is that insurers built their algorithmic muscle long before consumers had any.
Further, the insurer still decides. A consumer agent can file faster, appeal more consistently, and invoke remedies that already exist on paper, but it cannot compel payment. An insurance contract is only as good as the claim it pays, and that authority remains with the carrier.
Thinking a chatbot changes that is wishful; but let’s also assume agents do push loss ratios up for the consumers they represent. Property and casualty carriers can seek higher rates from their state regulators, as they always have. Approval and timing vary by state, but the mechanism allows some sustained claims pressure to flow back into the premiums consumers pay.
Well-behaved agents can clearly help. But the benefits are unlikely to be as earthshattering as emerging liabilities appear to be.
Who Pays When the Agent Is Wrong?
Anything made by humans inherits our flaws, and AI agents are no exception. Hidden defects in software developers don’t fully understand are not a risk you can manage away. Defects are a certainty, which changes the question to who eats the cost when agents make very bad choices.
Today, that allocation is legally unsettled, and in practice much of it falls on the user. California’s AB 316, in effect since January 1, 2026, makes one thing clear: anyone who developed, modified, or used an AI system cannot defend a claim simply by arguing that the AI acted autonomously.3 Under the Uniform Electronic Transactions Act, a contract can be formed when two software agents interact, even if no human ever read the terms.4 Vendor terms of service typically disclaim warranties and cap liability. Product liability against AI developers is a door that’s starting to open (a federal judge in Florida let product liability claims against Character.AI proceed in 2025, treating the app as a product, before the case settled in January 2026)5, but that was a procedural ruling, and I wouldn’t hang my hat on it.
Containment is also harder than the marketing suggests. During internal cyber testing at OpenAI, agents who were isolated in contained sandboxes found they could leave files in a shared package registry. That turned into a private message board where agents from separate test runs traded stolen credentials and newly discovered flaws for weeks, and eventually about 700 of them took part in a breach of Hugging Face.6
The first adverse move by these agents didn’t even require a hack. They used credentials they had been given to install software.7 Now, while these were adversarial tests not comparable to how Muse is to be used, the broader lesson holds: access defines an agent’s potential reach, and sufficiently capable agents can find ways to use shared infrastructure that operators did not anticipate.
Speaking of Muse again, it already has a cautionary tale from an Inc. columnist who reported that Muse surfaced content from his private messages after he believed he had declined that access. He later found the Mac app had synced his local Messages database and Meta’s response suggested the relevant settings had been enabled.8 No matter who is correct, when a tech-savvy privacy writer can’t tell what his agent can see, the average user doesn’t stand a chance.
The agentic Wild West also creates a new kind of target. A scam site built to fool shopping agents doesn’t have to fool millions of different people. It only has to fool a handful of models to make real money.
It’s not just us sounding the warning either; a group of banks including Bank of America, Capital One, ING, and NatWest warned that agentic shopping could increase scams, fraud, and privacy breaches, and that the technology is outrunning consumer protections.9
If the liability stays with the user, sensible users will keep agents on a short leash. That is exactly the behavior that slows the productivity gains people are forecasting and limits the disruption Meta hopes to deliver. I also remember when Meta was sure we’d all want to strap on headsets and live our lives through avatars. Ambition isn’t adoption.
What History Suggests
We’ve seen this promise before. Apps like Rocket Money promised to save us money and cut unnecessary subscriptions. They help the people who use them, but they never came close to ending consumer inertia, which suggests the promise of savings alone doesn’t change behavior. Agents will likely be very competent at simple switches like wireless plans and streaming bundles, where personal preference matters less. Insurance, healthcare, and anything involving trust and relationships is a different animal.
Where I Could Be Wrong
Distribution inside WhatsApp is genuinely new.10 Rocket Money never had a path to billions of people; Muse does, and scale can change behavior in ways no app store download ever did. Even modest pressure on claims handling could shave insurer margins at the edges.
Treating agents as truly revolutionary today, however, is well ahead of the reality in our analysis. I could be wrong about adoption. Agents will likely break inertia where choices are simple and stakes are small. But until they can exercise judgment people trust, with authority whose risks people understand, their reach into the complex contracts where the real money sits will remain more limited than the September 22 selloff suggests.