> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Evaluate Receipt Agents with an Image Judge
> Trace receipt-image extraction in Arize AX and use an image-aware LLM judge to evaluate whether structured output is visually grounded.
Run the companion app with 14 fictional receipt images and URL-based trace inputs.
## Overview
In this tutorial, you will trace a receipt-extraction agent that takes an image of a receipt, and produces structured JSON with details. You will then create a span-level LLM-as-a-judge evaluator that decides whether the extracted JSON is supported by the image.
This tutorial assumes you have basic knowledge of:
* [tracing](/docs/ax/observe/tracing)
- [LLM-as-a-judge evaluators](/docs/ax/evaluate/evaluators/llm-as-a-judge)
By the end, you will be able to:
* Trace an image reference, extraction output, and image metadata together.
* Map a receipt image and extracted JSON to a visual-groundedness evaluator.
* Distinguish supported outputs, unsupported assertions, and images that require human review.
## Why visual groundedness needs its own evaluator
A JSON-shaped response from a receipt-extraction agent can be valid while still inventing a total, merchant, or line item. Text-only correctness checks cannot see the evidence. A visual-groundedness evaluator receives both the receipt image and the extraction, so it can ask one focused question: **does the output say only what the image supports?**
The companion app uses `gpt-5.4-mini` to extract receipt data. The tested evaluator configuration uses `gpt-5.6-luna`, a stronger image-aware model, to judge it. If your AX AI integration exposes a different judge model, use that model consistently in the evaluator and app metadata.
## Before you start
You will need:
* An OpenAI API key with access to `gpt-5.4-mini` for extraction and an AX AI provider integration with `gpt-5.6-luna` for judging.
* Python 3.10 or later.
* The example receipt app from [the Arize tutorials repo](https://github.com/Arize-ai/tutorials/tree/main/python/cookbooks/receipt_image_evals)
## Run the receipt intake app
Clone [Arize-ai/tutorials](https://github.com/Arize-ai/tutorials), then install and configure the companion app:
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
cd tutorials/python/cookbooks/receipt_image_evals
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
Copy the `.env.example` file to `.env`, then set the relevant API keys and Arize AX space Id. Then run the app:
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
python receipt_app.py
```
You will see a local URL printed to the terminal. Open this to see the receipt parser.
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
❯ python receipt_app.py
* Running on local URL: http://127.0.0.1:7860
```
From the app, select some receipts, then click **Process expense** to process the image. From Arize AX, open the `receipt-image-evals` project, and look at the trace.
## Create the image-aware judge
Create a categorical [LLM-as-a-judge evaluator](/docs/ax/evaluate/evaluators/llm-as-a-judge) for visual groundedness.
In the left sidebar, select **Evaluators**. Click **New Evaluator**, select **LLM-as-a-Judge**, then choose **Create from blank**. Name it `Receipt Visual Groundedness`.
Set the evaluator scope to **Span**. Each `receipt.extract` span contains one self-contained receipt image input and its extracted JSON output, so the judge does not need the rest of the trace or session.
For **Judge Model**, select your OpenAI provider integration and `gpt-5.6-luna`. Then use the following template. It gives the judge the visual-grounding rubric and separates the image-bearing input from the JSON it must assess. You will map `receipt_image` and `extraction` to span fields in a later step.
```text theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
Assess whether the structured extraction is visually grounded in the receipt
image.
{receipt_image}
{extraction}
Choose grounded only when every asserted merchant, item, amount, currency,
and total is visibly supported. Choose not_grounded for invented or
contradicted values. Choose needs_review when the image is degraded or
ambiguous enough that a reliable decision cannot be made.
Select the classification label using the configured choices. In the AX
evaluator's `explanation` field, return only valid JSON, without Markdown
fences or any other text, with exactly these fields:
{
"evaluated_extraction": ,
"judge_result": {
"label": ,
"reason":
}
}
```
Add these three classification choices and scores. Set the optimization direction to **maximize**, so the aggregate score represents the share of receipts with safely grounded extractions.
| Label | Score | Meaning |
| -------------- | ----: | ----------------------------------------------------------------------------------------- |
| `grounded` | `1` | The merchant, line items, values, and review decision are supported by the receipt image. |
| `not_grounded` | `0` | The result asserts content that the image does not support. |
| `needs_review` | `0` | The receipt is too degraded or ambiguous to evaluate with confidence. |
In **Configuration**, set **Data Source** to the `receipt-image-evals` project. Under **Filter Spans**, filter to `name = "receipt.extract"` so the evaluator receives only the extraction spans that contain the receipt image and structured output. Confirm that **Matching Spans** shows processed `receipt.extract` spans before continuing.
Under **Map Variables to Data**, map `receipt_image` to `attributes.input.value` and `extraction` to `attributes.output.value`. Check the preview for each mapping: the first should contain the receipt image URL and the second the extracted JSON.
Leave **Run Continuously** enabled so new matching extraction spans are evaluated as they arrive. Turn on **One-Time Backfill** to also evaluate the processed receipt spans that already exist in the project.
Click **Create and Run Evaluator** to save the evaluator and start the backfill and continuous evaluation task.
## Review the eval results
Now you have the evaluator running, you can review the evaluator results for the existing receipts that have been processed, and for any new traces.
Select a trace in the project, then select its `receipt.extract` span. In the **Evaluations** section, open **Receipt Visual Groundedness** to see the label, score, and explanation. The explanation contains the JSON extraction evaluated by the judge and its visual-grounding review.
## Summary
In this tutorial, you:
* Traced receipt-image extraction with an image URL and structured JSON output.
* Created a span-level LLM judge to assess whether each extraction is visually grounded.
* Configured continuous evaluation and a backfill, then reviewed the label, score, and JSON explanation on the extraction span.
## Next steps
Calibrate the judge against reviewed examples before relying on it for production decisions.
Set sampling, filters, and monitoring for evaluators running on live traces.
Send ambiguous receipt results to a review queue and use the labels to improve the evaluator.