> ## Documentation Index
> Fetch the complete documentation index at: https://arizeai-433a7140.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Get Started: Evaluations

Now that you have Phoenix up and running, and sent traces to your first project, the next step you can take is running **evaluations** of your Python application. Evaluations let you measure and monitor the quality of your application by scoring traces against metrics like accuracy, relevance, or custom checks.

<Steps>
  <Step title={<span className="step-title">Launch Phoenix</span>}>
    Before running evals, make sure Phoenix is running & you have sent traces in your project. For more step by step instructions, check out this [Get Started guide](/docs/phoenix/get-started) & [Get Started with Tracing guide](/docs/phoenix/get-started/get-started-tracing).

    <Card>
      <Tabs>
        <Tab title="Self-Host">
          Run Phoenix on your own infrastructure, backed by PostgreSQL so traces persist beyond a single process. This is the option to reach for once Phoenix is shared across a team or environment.

          The [self-hosting guide](/docs/phoenix/self-hosting) covers [Kubernetes](/docs/phoenix/self-hosting/deployment-options/kubernetes), [Helm](/docs/phoenix/self-hosting/deployment-options/kubernetes-helm), [Railway](/docs/phoenix/self-hosting/deployment-options/railway), [AWS CloudFormation](/docs/phoenix/self-hosting/deployment-options/aws-with-cloudformation), [Google Cloud Run](/docs/phoenix/self-hosting/deployment-options/google-cloud-run), [Azure](/docs/phoenix/self-hosting/deployment-options/azure), and [Render](/docs/phoenix/self-hosting/deployment-options/render), plus authentication and configuration.
        </Tab>

        <Tab title="Local">
          ```bash theme={null}
          uvx arize-phoenix serve
          ```

          No [uv](https://docs.astral.sh/uv/)? `pip install arize-phoenix && phoenix serve` does the same thing. See [Terminal setup](/docs/phoenix/environments#terminal) for customization.
        </Tab>

        <Tab title="Container">
          ```bash theme={null}
          docker run -p 6006:6006 -p 4317:4317 arizephoenix/phoenix:latest
          ```

          Images are published to [Docker Hub](https://hub.docker.com/r/arizephoenix/phoenix). See [Docker](/docs/phoenix/self-hosting/deployment-options/docker) for volumes, PostgreSQL, and other options.
        </Tab>
      </Tabs>

      Phoenix serves its UI and OTLP HTTP on port **6006**, and OTLP gRPC on port **4317**. For a local instance that's [http://localhost:6006](http://localhost:6006) — leave it running while you work.
    </Card>

    Next, tell the Phoenix client where that instance lives. Run this alongside the rest of the code in this guide (in your notebook or script):

    Point your code at the Phoenix instance you started. The endpoint below is the default for a local `phoenix serve`; for a deployment running elsewhere, use its hostname instead.

    ```python theme={null}
    import os

    os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"

    # Only if the deployment has authentication enabled
    # os.environ["PHOENIX_API_KEY"] = "your-api-key"
    ```
  </Step>

  <Step title={<span className="step-title">Install Phoenix Evals</span>}>
    You'll need to install the evals library that's apart of Phoenix.

    ```bash theme={null}
    pip install -q "arize-phoenix-evals>=2"
    pip install -q "arize-phoenix-client"
    ```
  </Step>

  <Step title={<span className="step-title">Pull down your Trace Data</span>}>
    Since, we are running our evaluations on our trace data from our first project, we'll need to pull that data into our code.

    ```python theme={null}
    from phoenix.client import Client

    px_client = Client()
    primary_df = px_client.spans.get_spans_dataframe(project_identifier="crewai-tracing-quickstart")
    ```
  </Step>

  <Step title={<span className="step-title">Set Up Evaluations</span>}>
    In this example, we will define, create, and run our own evaluator. There's a number of different evaluators you can run, but this quick start will go through an LLM as a Judge Model.

    **1) Define your LLM Judge Model**

    We'll use OpenAI as our evaluation model for this example, but Phoenix also supports a number of [other models](/docs/phoenix/evaluation/how-to-evals/configuring-the-llm/).

    If you haven't yet defined your OpenAI API Key from the previous step, let's first add it to our environment.

    ```python theme={null}
    import os
    from getpass import getpass

    if not (openai_api_key := os.getenv("OPENAI_API_KEY")):
        openai_api_key = getpass("🔑 Enter your OpenAI API key: ")

    os.environ["OPENAI_API_KEY"] = openai_api_key

    from phoenix.evals.llm import LLM
    llm = LLM(model="gpt-4o", provider="openai")
    ```

    **2) Define your Evaluators**

    We will set up a Q\&A correctness Evaluator with the LLM of choice. I want to first define my LLM-as-a-Judge prompt template. Most LLM-as-a-judge evaluations can be framed as a classification task where the output is one of two or more categorical labels.

    ```python theme={null}
    CORRECTNESS_TEMPLATE = """
    You are given a question and an answer. Decide if the answer is fully correct.
    Rules: The answer must be factually accurate, complete, and directly address the question.
    If it is, respond with "correct". Otherwise respond with "incorrect".
    [BEGIN DATA]
        ************
        [Question]: {attributes.llm.input_messages}
        ************
        [Answer]: {attributes.llm.output_messages}
    [END DATA]

    Your response must be a single word, either "correct" or "incorrect",
    and should not contain any text or characters aside from that word.
    "correct" means that the question is correctly and fully answered by the answer.
    "incorrect" means that the question is not correctly or only partially answered by the
    answer.
    """
    ```

    Now we want to define our Classification Evaluator

    ```python theme={null}
    from phoenix.evals import ClassificationEvaluator

    correctness_evaluator = ClassificationEvaluator(
        name="correctness",
        prompt_template=CORRECTNESS_TEMPLATE,
        llm=llm,
        choices={"correct": 1.0, "incorrect": 0.0},
    )
    ```
  </Step>

  <Step title={<span className="step-title">Run Evaluation</span>}>
    Now that we have defined our evaluator, we're ready to evaluate our traces.

    ```python theme={null}
    from phoenix.evals import async_evaluate_dataframe

    results_df = await async_evaluate_dataframe(
        dataframe=primary_df,
        evaluators=[correctness_evaluator],
        concurrency=10,
    )
    ```
  </Step>

  <Step title={<span className="step-title">Log results to Visualize in Phoenix</span>}>
    You'll now be able to log your evaluations in your project view.

    First, format the evaluation results for logging using the `to_annotation_dataframe` utility:

    ```python theme={null}
    from phoenix.evals.utils import to_annotation_dataframe

    # Format evaluation results for logging
    annotations_df = to_annotation_dataframe(results_df)
    ```

    Then log the annotations to Phoenix:

    ```python theme={null}
    px_client.spans.log_span_annotations_dataframe(dataframe=annotations_df)
    ```
  </Step>
</Steps>

## Learn More:

<Columns cols={2}>
  <Card title="Evaluation Concepts" href="/docs/phoenix/evaluation/concepts-evals/llm-as-a-judge" icon="book-open" description="Evaluation fundamentals" />

  <Card title="Evals in Phoenix" href="/docs/phoenix/evaluation/llm-evals" icon="chart-line" description="Phoenix evals overview" />
</Columns>