Agentic AI has set off a firestorm in the world of CPUs over the past six months, yet it’s an area of performance that still doesn’t have solid, repeatable benchmarks to evaluate where the latest hardware falls. We’ve been working on an internal set of tests to evaluate at least one aspect of contentious agentic loops end-to-end, isolating CPU and memory performance from the accelerator to understand how these components contribute to the agentic loop.

This is still a work in progress, but we wanted to bring Tom’s Hardware Premium readers behind the scenes to understand how we’re going about tackling this problem, our methodology for testing, and detail what we’ve learned about agentic loops along the way. We've prototyped the test and settled the central design, but there's still work to do before the numbers it produces hold up to scrutiny.

Given that this area of computing is bleeding-edge and rapidly evolving, we undoubtedly have blind spots here. That’s OK. The testing we’re doing is built to look at how agentic performance scales across the CPU as sessions increase, providing an ideal throughput for each chip that provides as many agent sessions as possible while remaining interactive.

Toward the end of this article, we've included a section on some of the shortfalls of this approach, as well as some ideas we’re toying with to tackle things like multi-turn agentic flows. And keep in mind that these tests aren't meant to replace our other testing of local AI, but to supplement it.

Source link