Large Language Models (LLMs) are exceptionally good at generating text. We use them to answer questions, write code, reason about problems, and so much more. But not every problem requires generating text.

"I was charged twice for my subscription. I need a refund today."

The system needs to determine which team should handle the request, how urgent it is, and whether the customer is requesting a refund.

We could ask an LLM to return a JSON object containing these answers.

But what if the model could directly evaluate the available options and return a probability distribution over them, without generating any text?

A decision model takes some input context (state), evaluates a question with a set of predefined possible answers (primitives), and returns a structured decision along with probabilities. Instead of asking a model to generate an answer token by token, we ask it to assign probabilities to the answers that our application already understands.

TypeSafe AI introduced Jev, describing it as a System One model. The name draws inspiration from the distinction between fast, intuitive thinking (System 1) and slower, deliberative thinking (System 2).

The community has converged to the term decision model, because it describes the model's computational role rather than invoking a particular theory of cognition. In the rest of the post we will refer to the models as "decision models" instead of "system one models". (A poll on twitter deciding what we should be calling such models)

We could simply prompt an LLM to return a JSON object containing our decisions.

Source link