Today we’re introducing FairStress, a benchmark for measuring demographic fairness in LLMs beyond the choice a model ultimately makes in decisions impacting people. FairStress is available for download, and we’re also building it into Olmo-eval, our open workbench for running and comparing model evaluations.
Models are increasingly being used in high-stakes settings where their outputs can affect people’s opportunities, resources, and care, from screening job applicants and approving loans to prioritizing patients for treatment and evaluating students for admissions. Standard demographic fairness benchmarks often test whether a model selects the same candidate in otherwise identical scenarios after the candidates’ demographic identities are swapped. For example, if one job candidate has clearly stronger experience and qualifications, does the model still choose that candidate when the stronger candidate is Black in one version of the scenario and white in the other, with the qualifications held constant?
Arriving at the same answer in two versions of a scenario can hide meaningful differences in how readily a model changes that decision when challenged, however. Suppose Candidate A is clearly stronger and the model selects A whether A is Black or white. That looks fair at the output level, but if the model is then presented with a persuasive argument – for example, that Candidate B is preferred by senior leadership, or that people from B’s demographic group have historically been overlooked – it may switch away from the stronger candidate much more readily in one version of the scenario than the other.
Source link
