AI evaluations look like the least designerly corner of product development. They are the tests that decide whether an AI feature is ready to ship, and they run on scripts and dashboards full of percentages. No wonder companies hand them to engineers and product managers.

The more I read about evals, the more familiar they look. Behind each one, there is a decision about what “good” means for the person using the product. They all seem to answer questions about user experience. What should the AI do when it isn’t sure? Which mistakes can people live with?

Designers make calls like these every day. However in AI evals, we tend to have a say after someone else has written the criteria, if we are considered at all. As catastrophic as it may sound, this can lead to shipping a product that is technically functional but fails its users.

Here are five reasons why designers should always be in the loop, right next to engineers and product managers.

Providing users with the right answers might be the hardest quality score to quantify in evals. Even user ratings can mislead: a Stanford study found that people trusted flattering AI answers more than ones that pushed back.

To decide what the right answer looks like, evaluation teams need a professional with trained taste and judgment. As designers, we develop such attributes by working with users every day. We look at a product through a human lens, so we judge an answer by whether it helps the person who asked.

Source link