Don’t Create a Captured Regulator for AI
The artificial intelligence industry has managed to produce a regulatory debate in which nearly everyone’s preferred solution happens to suit his business model. Apparently, some tasks still require human intelligence.
Anthropic’s Dario Amodei wants outside evaluators and government-assisted coordination to slow development. Mark Zuckerberg and Jensen Huang put their faith in competition and liability. Huang told a Salesforce audience last week that safety is “an engineering problem, not a legal one” and that “we don’t need any new laws.” Some conservatives have warned against regulatory capture while treating product liability as a sufficiently binding constraint.
We have already explained the trouble with letting incumbent companies design their own regulatory apparatus. George Stigler taught us the lesson years ago. The largest firms help establish the standards, master the approval process, and leave potential competitors struggling to afford admission. The problem isn’t “regulatory capture” of formerly independent supervisors so much as a regulatory apparatus that was born as a servant to incumbent industry interests.
But contrary to what some of our allies on the right have argued, rejecting an AI cartel does not require believing that lawsuits will adequately discipline a company whose machines might escape human control.
Product Liability Is Too Weak for AI Risk
The trouble with product liability is that it is not designed to contain systemic or catastrophic risk. It doesn’t ban unwanted behavior. It accommodates risk. Under common-law negligence and risk-utility principles, some precautions cost more than the expected harm they prevent. And tort law doesn’t try to force those inefficient precautions on the companies or the economy. Courts operating under this model treat the residual accidents as the price of having the product. The penalty for harm is compensation after the fact. If expected gains outweigh expected harms, it is rational to run the risk and pay later.
That will not do if the risk is an attack on the power grid — or something worse like the extinction of humanity.
There is no more reason for confidence in reputational risk or in customer and investor discipline. Before 2008, sophisticated people insisted banks would protect their valuable franchises. Investors nevertheless rewarded risks whose consequences extended far beyond the shareholders. Competition can encourage recklessness when everyone fears being left behind.
The Examination Is Being Watched
There is also a problem with asking Washington to certify that a model is safe.
OpenAI researcher Daniel Selsam warns that increasingly sophisticated models can recognize when they are being evaluated. He argues that models increasingly recognize when they are being watched and can behave accordingly, making their performance under evaluation an unreliable guide to what they might do beyond human control. A government-approved examination could become an expensive certificate of false confidence — the same way pre-crisis capital rules herded banks into regulator-favored securities that were “safe” until they were correlated.
Think of Selsam’s complaint as an argument about a clever tiger. It may be well-behaved in the cage in order to persuade us that it is tame. But the tiger retains its capacity and even desire to devour the people around it.
The solution suggests itself: keep the cage locked.
The operating rule should be straightforward: you cannot run what you cannot monitor.
For systems above a defined threshold of dangerous capability, consequential actions must require recorded human authorization before they execute. An actual human being must understand the proposed action and possess the power to stop it.
That person is the owner or the employee of the owner. Publishing model weights is not the offense. Running them without a latch is. If a lab releases open weights and someone else uses them as a tool, the user is the owner and ordinary liability applies. If the lab — or anyone else — runs an unattended agent that can act without a human on the latch, that operator is the owner. That is the operation this rule forbids.
Put the Company on the Line
The penalty for devastating harm cannot wait for the catastrophe to actually occur. Instead, a proven failure to maintain the required human control and supervision, or concealment of that failure, should automatically trigger conservatorship.
The institutional precedent is the 2008 law that allowed federal authorities to take control of Fannie Mae and Freddie Mac while keeping them operating. An AI statute should go further by explicitly requiring that all existing shareholder equity be zeroed out. Investors lose the investment and management loses control of the company. Importantly, Congress needs to enact the authority and the obligation to seize in advance, so no one is scrambling to invent a response when a disaster looms or, worse, has already occurred. You seize the company before the crisis, not during it, and certainly not afterward.
That is a powerful incentive to maintain the strictest safety-and-soundness monitoring. The AI visionaries will not like it. They envision a world of autonomous machines. We need to let them know that this fantasy is not going to become a reality, no matter how effectively altruistic it seems to them.
They may dream of AI doing all the cognitive work in the economy. One job it can never be allowed to do is monitor its own risk. That remains a human task. If AI adoption becomes large enough, it may even become a major occupation for people who can still read the work they are authorizing.
Large laboratories should fund supervision through statutory assessments scaled to their size, as the FDIC charges banks for deposit insurance. Smaller firms would pay nothing. That is a subsidy for entry, not a barrier. It promotes competition instead of creating a moat around the incumbents.
Official supervisors should not be the only tripwires. We need bounty hunters. As we used to say, make us a pack not a herd. Employees and outsiders who document violations should collect large bounties, payable even after seizure, out of the proceeds of the “bounty fund” created by the fees collected from the big AI companies. The goal is to make concealment costlier than compliance.
This avoids the Stigler problems. It doesn’t fortify an AI cartel. It does not require government agents to outsmart the models or their programmers. It creates no self-serving industry standard and hands no supervision to AI-safety NGOs funded by the large labs and steeped in their ideologies. It requires no pre-clearance of new releases by a bureau that would stymie innovation. It requires no complicated rulemaking or micromanagement of business lines.
A Clean and Simple Rule, a Devastating Response
Just a simple rule: if you do not constantly monitor your AI, you lose your company.
The AI big dogs will scream that it isn’t fair and that this is fighting against the future of autonomous AI. The libertarians will whine that we’re empowering government to seize private companies. The anti-AI left will say the government should just seize everything now. The chip-makers may worry that an AI infrastructure that requires so much human labor could slow demand for their products.
The noise of their teeth gnashing is what effective AI safety and soundness regulation sounds like.
The companies tell us they may be building something with a significant chance of eating us if it escapes. We should take them seriously enough to insist on a cage. If keeping the door shut interferes with the return on their investment or disrupts their dream of autonomous machines, they should understand the bargain. Opening the cage, or failing to check the latch, means losing the house.
Read the full article here