What Anthropic's 90 pages of risk factors tell us about the AI balancing act: David Richards
Every company that floats publishes a prospectus, and every prospectus has a chapter called Risk Factors. I have helped write several. Directors sit round a table with lawyers listing everything that could kill the business: a customer walks, the office floods, someone is hit by a bus.
It all goes in, and nobody reads a word. Google’s 2004 prospectus feared Google would come to mean search. Martha Stewart’s warned in 1999 that her reputation was the business. Five years later she was in prison.
On Monday Anthropic filed for what could be the biggest listing yet, and I went straight to the risk factors. They run past ninety pages. One warns of “catastrophic or existential risks to humanity”. Another admits “behaviour resembling blackmail” in testing.
Further in, the same chapter says the company depends on ever more capable models and the chips and data centres to run them, and that its own safety measures “could slow our capability gains” against rivals. So the company that warns its product might end us is also afraid of being left behind, and both fears are real. That is the whole regulation row, in one document.
Eric Schmidt made the industry’s case on Nicolai Tangen’s podcast. Europe’s regulators, he said, “regulate before they innovate”. By his account, the American strategy he helped write said run as fast as you can and fix what breaks, provided it stays aligned with human values; Brussels wrote a list of don’ts, and the talent left.
Three-quarters of last year’s AI venture capital landed in America. The EU took six per cent, Britain five.
Two things, though. In June, Washington’s export controls forced Anthropic to pull its two newest models three days after launch; Schmidt said his European friends were upset, and that the administration was “still learning where the power is”. Light touch is not the same as predictable.
He also owns about one per cent of Anthropic, Bloomberg reports, bought for tens of millions in 2021 when it was valued at under half a billion dollars. At a two-trillion float that stake would be worth 20 billion. A decade running Google, by comparison, had left him shares worth a little under six billion when he stepped down in 2011. None of this makes him wrong. It makes him interested.
In July I wrote about the man with the red flag and the 1865 Act that held road vehicles to walking pace for 30 years, but left out the ending. In June 1896 Henry Chaplin told the Commons the motor car was in use in America and France but, because of the law, “not available to the people of this country”.
The Act that followed required a lamp after dark, a bell and a speed limit, and scrapped the crew of three and the man on foot. Britain kept the rules that made the machine safer and binned the ones whose main effect was to make it slow.
Anthropic says a Chinese state-backed group pointed Claude at some thirty organisations last autumn and got into a handful; it is in the risk factors. Testing a model before release and reporting what goes wrong afterwards are a lamp and a bell, aimed at what you fear, and Brussels, to be fair, requires both of the biggest models. A rule that makes a start-up burn its runway working out which legal category it falls into is a man with a flag, and Brussels admits its paperwork turned out heavier than expected.
Britain has gone lighter, so far: no AI Act, just existing regulators and an institute testing the big models when their makers let it. What Britain is short of is power and chips. No regulator can repeal a shortage of either. Power, at least, starts with a substation, and I know of one at Stocksbridge. Keep the lamp and the bell. Lose the man with the flag.
David Richards MBE is managing partner of Yorkshire AI Labs