On June 14, Satya Nadella published an article on X about corporate AI that left me completely impressed: his main point, that the enterprise’s durable asset was not the foundation model but the learning loop, was precisely the idea my company and I had been developing for the last three years. I responded with an article on Fast Company, not expecting an answer (tech CEOs are more notoriously one-sided and difficult to reach than kings or presidents), but trying to elevate the discussion a bit more: the central challenge of enterprise AI is not intelligence itself — it is architecture.

Now, Nadella has written another article, in which he goes one step further: not only is the model not the durable asset, but additionally, it must be treated as a completely untrusted component. We must assume that all AI models could be compromised, and design our systems to contain them from the very beginning. 

This idea has been captured by the press as advocating for some sort of “emergency brake”: an authorized person should always be able to interrupt a model’s activity at any given time. However, that’s definitely not the most interesting or even the most important part in Nadella’s article. The truly important part is not the theoretical stop button, but the architecture that makes it possible. A model should neither be able to control its own permissions, nor decide by itself what it can access; execute directly all of its actions or produce the only available evidence on the actions it performed.

Source link