Bias in AI evaluation refers to systematic differences in behavior or outcomes across groups, topics, languages, dialects, or contexts. Bias can appear in model outputs, retrieval results, ranking systems, labeling data, or evaluator judgments.
Evaluating bias requires representative datasets and careful rubrics. It is not enough to run a generic safety check. Teams need to define the specific harms and populations relevant to the product.