I’m talking to many companies these days that are trying to incorporate AI into their business practices. Most of them are still in what I call “the administrative phase,” when they just try modest automations that allow some workers to draft documents or presentations, answer emails, or do research on certain topics, but given my recent articles arguing for companies to go much farther, I’m starting to get more and more questions from enterprises looking to go way beyond that, and get AI to become more “strategic.”

For these companies, the main concern is expressed as a question: Will AI be smart enough? But I think a better and more useful question would be: How does the AI know whether what it did actually improved the business?

The examples are well known and not theoretical: from optimizing a salesperson for revenue, margin, or retention, to a factory in which we can improve throughput, but we could also improve quality, or reduce defects. In business, “optimization” rarely means just one thing: improving one metric can easily damage another. So, in the same way management already works through objectives and feedback, AI agents need to do exactly that. In fact, OpenAI now explicitly frames enterprise evaluation as “specify → measure → improve, and in doing so, it can turn what would be vague business goals into actionable and measurable expectations.

Current AI metrics often measure variables such as answer quality, task completion, latency, cost, etc. However, what businesses actually care about are things like customer churn, margin, conversion, claims accuracy, delivery times, or customer lifetime value.

Source link