Log probabilities, or logprobs, are the natural logarithms of the probabilities a language model assigns to tokens. In AI engineering, they help you spot where a model was uncertain, score fixed answer choices, and route doubtful outputs for review, as long as you remember that likely is not the same as correct.
For example, suppose a model is completing “The capital of France is”. It gives the token ” Paris” a logprob of -0.08, which converts to a probability of about 0.92. The alternative ” Lyon” has a logprob of -3.10, or a probability of about 0.045. The model strongly expected ” Paris”, but that number alone does not prove the answer is right.
Models report logprobs per token, during generation or when scoring existing text.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why log probabilities matter
A logprob is always zero or negative. Values closer to zero mean the model found that token more likely in context.
That signal is useful for:
- Finding uncertain spans: Locate the tokens in an output where the model was least confident.
- Scoring fixed choices: Compare the likelihood of a small set of labels instead of parsing free text.
- Routing and abstaining: Send low-likelihood outputs to review or to a fallback path.
- Debugging regressions: Compare token likelihoods across prompt or model versions on traces.
Token likelihood is a property of the model’s distribution. It is not a calibrated probability that the task answer is right.
How log probabilities work
At each generation step, the model produces a score called a logit for every token in its vocabulary. A function called softmax turns those logits into a probability distribution that sums to 1.
The model then samples or selects a token from that distribution. This loop is described under autoregressive models. The logprob of the chosen token is the natural log of its probability at that step.
Logs are used because probabilities multiply across tokens and quickly become too small for a computer to represent. This problem is called underflow. In log space, the values add instead of multiply.
To recover a probability from a logprob, exponentiate it.
Formula:
logprob = ln p(token | context)
p = e^logprob
Where:
- p(token | context) is the probability the model assigns to the token, given the text before it.
- ln is the natural logarithm.
- e is Euler’s number, about 2.718.
Sequence logprobs and length
The logprob of a whole sequence is the sum of its token logprobs. Because every term is negative, longer sequences have lower totals.
To compare sequences of different lengths, use the average logprob per token.
Formula:
log p(x_1, …, x_T) = Σ log p(x_t | x_<t) summed over t = 1 to T
average logprob = (1 / T) × Σ log p(x_t | x_<t) summed over t = 1 to T
Where:
- x_1 through x_T are the tokens in the sequence.
- T is the number of tokens.
- p(x_t | x_<t) is the probability the model assigns to token x_t given all tokens before it.
The negative average logprob is the per-token cross-entropy. Exponentiating that value gives perplexity: perplexity = e^(−average logprob).
The Hugging Face Transformers text generation documentation shows how generation and scoring APIs expose the underlying scores these values come from.
A token is a model-specific unit. The same word may be one token for one model and three for another, which changes per-token values. For more on how text is split, see tokenization.
How to read log probabilities
The following Python code converts the logprobs from the opening example into probabilities. It then computes the sum, average, and perplexity for the four-token sequence ” Paris is a city.”, using its token logprobs of -0.08, -0.40, -1.20, and -0.30.
from math import exp
alternatives = {" Paris": -0.08, " Lyon": -3.10}
for token, logprob in alternatives.items():
print(f"{token!r}: logprob {logprob:.2f} -> p = {exp(logprob):.3f}")
token_logprobs = [-0.08, -0.40, -1.20, -0.30]
total = sum(token_logprobs)
average = total / len(token_logprobs)
perplexity = exp(-average)
print(f"sum = {total:.2f}, average = {average:.3f}, perplexity = {perplexity:.2f}")
Output:
' Paris': logprob -0.08 -> p = 0.923
' Lyon': logprob -3.10 -> p = 0.045
sum = -1.98, average = -0.495, perplexity = 1.64
Some providers return alternative tokens at the same step, like ” Lyon” here. Others return only the chosen token, and some return no logprobs at all.
Here is what these numbers do and do not tell you. The model strongly expected ” Paris” after that context.
Nothing in the -0.08 establishes that Paris is the right answer to the user’s actual question. It does not show that the question was interpreted as intended. It also does not show that a longer answer containing this span is well supported.
How to use log probabilities to inspect and debug
- Spotting unlikely spans: Sort a trace’s tokens by logprob to find where the model was uncertain. Read the surrounding text too, because rare names and numbers are legitimately low-probability.
- Scoring fixed candidates: For classification-style prompts, compare the logprobs of a small set of label tokens rather than parsing free text. State whether you summed or averaged them.
- Setting a routing threshold: Route cases with a low average logprob for review. Then validate that threshold against labeled outcomes.
- Watching judge consistency: Treat model-derived confidence signals as measurements to validate, not verdicts.
Scoring label tokens is one of the approaches discussed under score models.
A routing threshold is a hypothesis until you check it with LLM evaluation. Arize’s post on LLM judge consistency gives useful context on validating model-derived signals.
Log these values on traces so you can inspect regressions later. The Arize AX documentation covers instrumenting LLM calls, and AI observability covers reviewing them in production.
Log probabilities vs. logits
Logits are the raw, unnormalized scores before softmax. Logprobs are the log of the normalized probabilities after softmax. The two are not interchangeable.
Logits have an arbitrary scale and offset. Only the differences between logits at the same step matter.
The following Python code applies a log softmax to three hypothetical logits. It then adds 5 to every logit and repeats the calculation.
from math import exp, log
logits = {"A": 2.0, "B": 1.0, "C": 0.1}
shifted = {t: v + 5.0 for t, v in logits.items()}
def log_softmax(scores):
log_total = log(sum(exp(v) for v in scores.values()))
return {t: v - log_total for t, v in scores.items()}
for name, scores in (("logits", logits), ("logits + 5", shifted)):
logprobs = log_softmax(scores)
print(name, {t: round(lp, 3) for t, lp in logprobs.items()})
Output:
logits {'A': -0.417, 'B': -1.417, 'C': -2.317}
logits + 5 {'A': -0.417, 'B': -1.417, 'C': -2.317}
Shifting every logit by the same amount leaves the logprobs unchanged. This is why raw logit values are not comparable across steps, while logprobs correspond to probabilities at each step.
Log probabilities vs. calibrated confidence
Calibrated confidence is a third, separate idea. It is a statement like “cases scored 0.8 are right about 80% of the time.” You can only establish it empirically, by checking scores against real outcomes.
A token logprob can feed a confidence estimate. Raw likelihood, though, mixes fluency, frequency, and phrasing with correctness.
A common, confidently repeated falsehood can carry high probability. A correct but unusually worded answer can carry low probability.
Limitations of log probabilities
- Length penalty: Sequence sums fall as length grows, so compare averages or candidate sets of similar length.
- Split probability: Probability mass spreads across paraphrases such as “NYC” and “New York.” Any single phrasing can look unlikely even when the model “knows” the answer.
- Limited exposure: Logprobs say nothing about tokens the provider does not return.
- Sampling settings: Settings such as temperature reshape the distribution tokens are drawn from. Providers differ on whether returned logprobs reflect those settings or the unmodified model distribution.
- Weak where it matters: Likelihood is least reliable on novel, contested, or adversarial inputs.
Use logprobs as a diagnostic and ranking signal with validated thresholds, not as proof of correctness. For more on how sampling settings change outputs, see LLM temperature.
Frequently asked questions
How do I turn a logprob back into a probability?
Exponentiate it: p = e^logprob. A logprob of -0.08 is about 0.92, -3 is about 0.05, and -10 is about 0.000045.
Is a higher average logprob a better answer?
Only in the narrow sense that the model found the wording more expected. It does not establish factual correctness, instruction following, or usefulness. Validate it against labeled outcomes before you use it as a quality gate.
What is the difference between logits and logprobs?
Logits are unnormalized scores per token at one step, and they become logprobs after softmax and a logarithm. Logit magnitudes are not comparable across steps. Logprobs correspond to probabilities at each step.
Why do long answers always have lower total logprob?
Each token adds a negative term, so the sum grows more negative with length, even for fluent text. Compare the average logprob per token, or score fixed-length candidate labels.
Putting log probabilities to work
Before you build on logprobs, check what your provider returns. Confirm whether you get alternative tokens and whether the values reflect sampling settings.
Record the token logprobs on each trace, along with the model, tokenizer, and sampling settings. When you use an average logprob to route or abstain, write down the threshold and validate it against labeled outcomes before relying on it.
