Anthropic’s cofounders, including CEO Dario Amodei, left OpenAI because they believed AI safety features should be developed alongside model capabilities from the very start of the development process. This mission has shaped the company, and has carried through to the way it’s framing its IPO, which could happen before Thanksgiving. 

In its S-1 filing, Anthropic tells the story of a company that wants to benefit all of the world’s people, but can do so only if it goes big and leads the market. That smacks of effective altruism, a twenty-first-century philosophical approach that’s caught on with young Silicon Valley founders. Several Anthropic researchers told Wired in June that it’s important Anthropic stay in the lead among frontier artificial intelligence labs so that mass market AI will be built safely. Anthropic’s leaders believe that if the lab can reach major research milestones first, they can authoritatively define the safety regime needed to guard against the risk of the new capabilities.

But developing such safety regimes costs time and money. Challenged to maintain its lead, Anthropic might be tempted to make compromises on safety. Releasing a model quickly may improve Anthropic’s competitive position among AI labs but leave less time for safety testing. Or, restricting a model’s capabilities could reduce safety risks but limit its commercial value. 

Competitive pressure could encourage faster model development even when safety questions remain unresolved. The company has already shown a vulnerability to such pressure. In Anthropic’s original Responsible Scaling Policy, the company pledged that it would not develop or deploy certain advanced systems if it could not implement adequate safety measures.

Source link