> ## Documentation Index
> Fetch the complete documentation index at: https://arizeai-433a7140.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Code Evaluator Output Shapes
This page documents every return shape a code evaluator can produce and how Phoenix maps each one to an `EvaluationResult` with `label`, `score`, and `explanation` fields.
This page covers the **server-side code evaluators** that run in the Phoenix UI (Sandbox evaluators). For client-side `create_evaluator` / `createEvaluator` SDK evaluators, see [Code Evaluators](/docs/phoenix/evaluation/how-to-evals/code-evaluators).
## The Triple-Collapse Model
Every return value from a code evaluator is normalized to a **triple**: `(label, score, explanation)`. Phoenix applies this in two stages:
1. **Stage 1 — Extract**: The raw return value is mapped to a `Triple` based on its shape (bare scalar or dict-by-key).
2. **Stage 2 — Validate**: The triple is checked against the evaluator's output config (categorical, continuous, or none).
Any value that cannot be cleanly mapped raises a `ValueError` whose message enumerates all accepted shapes for the configured output type.
## Accepted Shapes by Output Config
### Categorical Output Config
A categorical config defines a fixed set of `{label, score}` pairs. The evaluator must return one of the configured labels; Phoenix looks up the associated score automatically.
**Bare string (recommended):**
```python theme={null}
return "pass"
```
```typescript theme={null}
return "pass";
```
**Dict with label and optional explanation:**
```python theme={null}
return {"label": "pass", "explanation": "The output matched the expected format."}
```
```typescript theme={null}
return {"label": "pass", "explanation": "The output matched the expected format."};
```
Notes:
* The label must exactly match one of the configured values; unrecognized labels raise `ValueError`.
* Including a `score` key in the dict that conflicts with the config's lookup value raises `ValueError`.
* Free-form `explanation` strings are always accepted and passed through to `EvaluationResult.explanation`.
* Tuple shorthand (`return ("pass", 1.0)`) is **not** accepted; use the dict form if you need to supply additional fields.
### Continuous Output Config
A continuous config validates that the returned value is a finite number within optional `lower_bound` / `upper_bound` bounds. Labels are optional and free-form.
**Bare number (recommended):**
```python theme={null}
return 0.85
```
```typescript theme={null}
return 0.85;
```
**Dict with score and optional explanation:**
```python theme={null}
# score in range 0.0 - 1.0
return {"score": 0.85, "explanation": "High confidence based on keyword match."}
```
```typescript theme={null}
// score in range 0.0 - 1.0
return {"score": 0.85, "explanation": "High confidence based on keyword match."};
```
Notes:
* `bool` values are **not** treated as numeric and raise `ValueError`.
* `NaN` and `Infinity` are rejected.
* Free-form string labels are allowed in the dict form alongside a numeric score.
* Tuple shorthand is **not** accepted.
### No Output Config
When no output config is specified, Phoenix applies a permissive bare passthrough:
| Return value | Result |
| -------------------------------------------------- | ----------------------------------------------- |
| `str` | `label=` |
| `int` or `float` | `score=` |
| `bool` | `label="True"` or `label="False"` (not numeric) |
| `None` | `(label=None, score=None)` |
| `{"label": ..., "score": ..., "explanation": ...}` | triple by key |
**Lists and arbitrary nested objects are rejected** — they previously silently stringified into labels, which masked misconfiguration. Return a recognized shape instead.
## The `explanation` Field
Any accepted shape may include an `explanation` string. Phoenix passes it through to `EvaluationResult.explanation` unchanged:
```python theme={null}
return {"label": "fail", "explanation": "Response contained prohibited content."}
```
```typescript theme={null}
return {"label": "fail", "explanation": "Response contained prohibited content."};
```
The explanation appears in the Phoenix UI alongside the label and score and is available in the evaluation results API.
## Multi-Output Evaluators
When an evaluator has **multiple output configs** (e.g., one for toxicity and one for safety), Phoenix supports two routing modes:
### Shared value (default)
Return a single value — Phoenix applies the same return value to each output config independently:
```python theme={null}
return "pass" # applied to every output config
```
```typescript theme={null}
return "pass"; // applied to every output config
```
### Per-config routing dict
Return a dict whose keys match every output config name. Phoenix routes each value to the corresponding config:
```python theme={null}
return {
"toxicity": 0.1,
"safety": "pass",
"explanation": "Content appears safe.", # shared fallback
}
```
```typescript theme={null}
return {
"toxicity": 0.1,
"safety": "pass",
"explanation": "Content appears safe.", // shared fallback
};
```
Routing rules:
* The dict must contain a key for **every** output config name; a partial match is treated as a shared value, not a routing dict.
* A top-level `"explanation"` key acts as a **shared fallback**: if a per-config sub-value omits explanation, the top-level value fills it in.
* Per-config sub-values may themselves be dicts with their own `"explanation"` key — per-config explanation takes precedence over the shared fallback.
**Per-config explanation example:**
```python theme={null}
return {
"toxicity": {"score": 0.9, "explanation": "Contains slurs."},
"safety": "fail",
"explanation": "Overall content is unsafe.", # only used for safety
}
```
```typescript theme={null}
return {
"toxicity": {"score": 0.9, "explanation": "Contains slurs."},
"safety": "fail",
"explanation": "Overall content is unsafe.", // only used for safety
};
```
### Multi-output naming convention
Each output config produces a separate `EvaluationResult` named `{evaluator_name}.{config_name}`. For example, an evaluator named `content-check` with configs `toxicity` and `safety` produces two results: `content-check.toxicity` and `content-check.safety`.
## Error Messages
When a return value does not match the accepted shapes, the `ValueError` message enumerates all valid shapes for the configured output type in the evaluator's language. For example, a categorical config with values `["pass", "fail"]` in Python would produce:
```
Label 'unknown' not in categorical output config values ['pass', 'fail'].
Valid shapes:
return "pass"
return {"label": "pass", "explanation": "..."}
```
This makes it straightforward to identify and fix mismatches without consulting documentation.