> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluator and task mutations

> Create evaluators and run online eval, experiment and prompt optimization tasks. Arguments, return types and a validated example for each of the 14 mutations.

Create evaluators and run online eval, experiment and prompt optimization tasks.

For task-oriented walkthroughs of these operations, see the [Evaluator and task guide](/docs/ax/graphql-reference/guides/evaluators-and-tasks). Every mutation below is sent as a `POST` to `https://app.arize.com/graphql` with an `x-api-key` header; see [Forming calls](/docs/ax/graphql-reference/overview/how-to-use-graphql/forming-calls).

## Mutations in this page

* [`createEvaluator`](#createevaluator): Create a new evaluator
* [`createEvaluatorVersion`](#createevaluatorversion): Create a new evaluator version
* [`editEvaluator`](#editevaluator): Edit an existing evaluator
* [`deleteEvaluator`](#deleteevaluator): Delete an existing evaluator
* [`createEvalTask`](#createevaltask): Create a new online eval task
* [`patchEvalTask`](#patchevaltask): Patch eval task.
* [`patchEvalTaskWithEvalHub`](#patchevaltaskwithevalhub): Patch eval task with eval hub.
* [`migrateTaskToEvalHub`](#migratetasktoevalhub): migrates a task's evaluators to eval hub
* [`runOnlineTask`](#runonlinetask): Run an online task
* [`cancelOnlineTaskRun`](#cancelonlinetaskrun): Cancel an online task run
* [`deleteOnlineTask`](#deleteonlinetask): Delete online task.
* [`runExperimentTask`](#runexperimenttask): Run an experiment task with configurable LLM interactions
* [`createPromptOptimizationTask`](#createpromptoptimizationtask): Create a new prompt optimization task
* [`patchPromptOptimizationTask`](#patchpromptoptimizationtask): Patch prompt optimization task.

## Reference

### createEvaluator

Create a new evaluator

`createEvaluator(input: CreateEvaluatorMutationInput!): CreateEvaluatorMutationPayload`

#### Arguments

<ParamField body="input" type="CreateEvaluatorMutationInput!" required>
  <Expandable title="CreateEvaluatorMutationInput fields">
    <ParamField body="spaceId" type="ID!" required>
      The space id for the evaluator
    </ParamField>

    <ParamField body="name" type="String!" required>
      The name of the evaluator
    </ParamField>

    <ParamField body="description" type="String">
      The description of the saved evaluator
    </ParamField>

    <ParamField body="commitMessage" type="String!" required>
      The commit message describing the changes in this version
    </ParamField>

    <ParamField body="templateEvaluator" type="TemplateEvaluationConfigInput">
      <Expandable title="TemplateEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the template evaluation config
        </ParamField>

        <ParamField body="rails" type="[String!]">
          The rails associated with the config
        </ParamField>

        <ParamField body="template" type="String!" required>
          The template associated with the config
        </ParamField>

        <ParamField body="turnDefinition" type="String">
          Optional turn-definition template for session-granularity evaluators. When non-empty, the evaluator operates in custom-turn mode: each session trace yields one turn rendered from this template. Uses the same \{variable} syntax as the main template.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="includeExplanations" type="Boolean!" required>
          Whether the config includes explanations
        </ParamField>

        <ParamField body="useFunctionCallingIfAvailable" type="Boolean!" required>
          Whether the config uses function calling if available
        </ParamField>

        <ParamField body="useStructuredOutputIfAvailable" type="Boolean">
          Whether the config uses structured output if available
        </ParamField>

        <ParamField body="classificationChoices" type="JSONObject">
          The classification choices for the config (Record\<string, number>)
        </ParamField>

        <ParamField body="direction" type="TemplateEvaluationConfigDirection">
          The direction for the config One of: `maximize`, `minimize`, `none`.
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          Data granularity (span, trace, or session). Defaults to span when omitted. One of: `span`, `trace`, `session`.
        </ParamField>

        <ParamField body="llmConfig" type="TemplateEvaluationLlmConfigInput">
          Llm Config for this evaluator

          <Expandable title="TemplateEvaluationLlmConfigInput fields">
            <ParamField body="integrationId" type="ID!" required>
              The selected named LLM integration id (relay global id). If provided, server will use its configuration.
            </ParamField>

            <ParamField body="modelName" type="String!" required>
              The LLM model
            </ParamField>

            <ParamField body="invocationParameters" type="JSONObject!" required>
              parameters used when running the Llm
            </ParamField>

            <ParamField body="providerParameters" type="JSONObject!" required>
              parameters used to initialize the Llm
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="codeEvaluator" type="CodeEvaluationConfigInput">
      <Expandable title="CodeEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the evaluation config
        </ParamField>

        <ParamField body="evaluationClass" type="String">
          The evaluation type associated with the config
        </ParamField>

        <ParamField body="evalClassCodeBlock" type="String">
          The code block for the evaluation class
        </ParamField>

        <ParamField body="evaluationInputParams" type="JSON">
          The evaluation input params
        </ParamField>

        <ParamField body="packageImports" type="String">
          The package imports for the evaluation class
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="spanAttributes" type="[String!]">
          The data column associated with the config
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          The data granularity associated with the config (span, trace, or session) One of: `span`, `trace`, `session`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="harnessEvaluator" type="HarnessEvaluatorInput">
      <Expandable title="HarnessEvaluatorInput fields">
        <ParamField body="llmIntegrationId" type="ID!" required>
          Relay GID of the llm\_integrations row. Provider must be 'anthropic'.
        </ParamField>

        <ParamField body="sandboxIntegrationIds" type="[ID!]">
          Optional relay GIDs of sandbox\_integrations to link to the new sandbox config at creation time.
        </ParamField>

        <ParamField body="name" type="EvalColumnName!" required>
          Eval column prefix name. Harness output is written under eval.\<name>.\*.
        </ParamField>

        <ParamField body="template" type="String!" required>
          Natural-language eval template. \{column\_name} markers are substituted at run time.
        </ParamField>

        <ParamField body="classificationChoices" type="JSON">
          Optional fixed label set. Null means the agent discovers labels freely.
        </ParamField>

        <ParamField body="strictChoices" type="Boolean">
          When true the agent must only emit labels from classificationChoices.
        </ParamField>

        <ParamField body="llmModelName" type="String">
          Optional model name hint forwarded to the pod as ANTHROPIC\_MODEL.
        </ParamField>

        <ParamField body="direction" type="HarnessEvaluationConfigDirection">
          Optimization direction for scored harness eval output. One of: `maximize`, `minimize`, `none`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="remoteEvaluator" type="RemoteEvaluatorInput">
      <Expandable title="RemoteEvaluatorInput fields">
        <ParamField body="endpoint" type="String!" required>
          HTTP URL of the remote evaluation endpoint.
        </ParamField>

        <ParamField body="headers" type="JSON">
          Optional custom request headers sent to the endpoint (encrypted at rest).
        </ParamField>

        <ParamField body="inputSchema" type="JSON">
          Optional JSON schema describing the expected request body.
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          Data granularity (span, trace, or session). Defaults to span when omitted. Fixed at creation; later versions inherit it. One of: `span`, `trace`, `session`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="systemOneEvaluator" type="SystemOneEvaluatorInput">
      <Expandable title="SystemOneEvaluatorInput fields">
        <ParamField body="llmIntegrationId" type="String!" required>
          Relay global ID of the TypeSafe AI integration this evaluator calls.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          Provider model route, for example `jev-latest`. Required: the column has no default, so which route was asked for is always recorded.
        </ParamField>

        <ParamField body="stateTemplate" type="String!" required>
          The author's JSON-shaped state template, stored verbatim, carrying \{variables} resolved per record.
        </ParamField>

        <ParamField body="questions" type="[SystemOneQuestionInput!]!" required>
          At least one. Every question sees the same state and is evaluated independently in one request.

          <Expandable title="SystemOneQuestionInput fields">
            <ParamField body="questionName" type="String!" required>
              Becomes the provider's question key and the published eval column suffix, so it is validated against the eval-name rules.
            </ParamField>

            <ParamField body="questionType" type="SystemOneQuestionType!" required>
              One of: `boolean`, `choice`, `score`.
            </ParamField>

            <ParamField body="instructions" type="String!" required>
              The complete judgment. Question names are not sent to the provider as context.
            </ParamField>

            <ParamField body="criteria" type="JSON">
              Required for choice and score; optional for boolean. Shape is validated per type.
            </ParamField>

            <ParamField body="direction" type="OptimizationDirection">
              Read for score questions only, where it defaults to maximize. Normalized to `none` for every other type. One of: `minimize`, `maximize`, `none`.
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          Data granularity (span, trace, or session). Defaults to span when omitted. Fixed at creation; later versions inherit it. One of: `span`, `trace`, `session`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="evaluator" type="Evaluator!">
  <Expandable title="Evaluator fields">
    Scalar fields: `id`, `name`, `description`, `taskType`, `commitHash`, `commitMessage`, `createdAt`, `updatedAt`.

    Object fields (select subfields): `versionHistory`, `webhookSubscriptions`, `config`, `harnessEvaluationConfig`, `remoteEvaluationConfig`, `systemOneEvaluationConfig`, `createdBy`, `tags`, `onlineTasks`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation CreateEvaluator($input: CreateEvaluatorMutationInput!) {
    createEvaluator(input: $input) {
      evaluator { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "spaceId": "<ID>",
      "name": "<string>",
      "commitMessage": "<string>"
    }
  }
  ```
</CodeGroup>

### createEvaluatorVersion

Create a new evaluator version

`createEvaluatorVersion(input: CreateEvaluatorVersionMutationInput!): CreateEvaluatorVersionMutationPayload`

#### Arguments

<ParamField body="input" type="CreateEvaluatorVersionMutationInput!" required>
  <Expandable title="CreateEvaluatorVersionMutationInput fields">
    <ParamField body="evaluatorId" type="ID!" required>
      The evaluator ID in which the version will be created
    </ParamField>

    <ParamField body="commitMessage" type="String!" required>
      The commit message describing the changes in this version
    </ParamField>

    <ParamField body="name" type="String">
      Optional evaluator name to update
    </ParamField>

    <ParamField body="description" type="String">
      Optional evaluator description to update
    </ParamField>

    <ParamField body="templateEvaluator" type="TemplateEvaluationConfigInput">
      <Expandable title="TemplateEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the template evaluation config
        </ParamField>

        <ParamField body="rails" type="[String!]">
          The rails associated with the config
        </ParamField>

        <ParamField body="template" type="String!" required>
          The template associated with the config
        </ParamField>

        <ParamField body="turnDefinition" type="String">
          Optional turn-definition template for session-granularity evaluators. When non-empty, the evaluator operates in custom-turn mode: each session trace yields one turn rendered from this template. Uses the same \{variable} syntax as the main template.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="includeExplanations" type="Boolean!" required>
          Whether the config includes explanations
        </ParamField>

        <ParamField body="useFunctionCallingIfAvailable" type="Boolean!" required>
          Whether the config uses function calling if available
        </ParamField>

        <ParamField body="useStructuredOutputIfAvailable" type="Boolean">
          Whether the config uses structured output if available
        </ParamField>

        <ParamField body="classificationChoices" type="JSONObject">
          The classification choices for the config (Record\<string, number>)
        </ParamField>

        <ParamField body="direction" type="TemplateEvaluationConfigDirection">
          The direction for the config One of: `maximize`, `minimize`, `none`.
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          Data granularity (span, trace, or session). Defaults to span when omitted. One of: `span`, `trace`, `session`.
        </ParamField>

        <ParamField body="llmConfig" type="TemplateEvaluationLlmConfigInput">
          Llm Config for this evaluator

          <Expandable title="TemplateEvaluationLlmConfigInput fields">
            <ParamField body="integrationId" type="ID!" required>
              The selected named LLM integration id (relay global id). If provided, server will use its configuration.
            </ParamField>

            <ParamField body="modelName" type="String!" required>
              The LLM model
            </ParamField>

            <ParamField body="invocationParameters" type="JSONObject!" required>
              parameters used when running the Llm
            </ParamField>

            <ParamField body="providerParameters" type="JSONObject!" required>
              parameters used to initialize the Llm
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="codeEvaluator" type="CodeEvaluationConfigInput">
      <Expandable title="CodeEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the evaluation config
        </ParamField>

        <ParamField body="evaluationClass" type="String">
          The evaluation type associated with the config
        </ParamField>

        <ParamField body="evalClassCodeBlock" type="String">
          The code block for the evaluation class
        </ParamField>

        <ParamField body="evaluationInputParams" type="JSON">
          The evaluation input params
        </ParamField>

        <ParamField body="packageImports" type="String">
          The package imports for the evaluation class
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="spanAttributes" type="[String!]">
          The data column associated with the config
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          The data granularity associated with the config (span, trace, or session) One of: `span`, `trace`, `session`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="harnessEvaluator" type="HarnessEvaluatorVersionInput">
      <Expandable title="HarnessEvaluatorVersionInput fields">
        <ParamField body="llmIntegrationId" type="ID">
          Relay GID of the llm\_integrations row. Provide to create a new sandbox; omit to inherit.
        </ParamField>

        <ParamField body="name" type="EvalColumnName!" required>
          Eval column prefix name. Harness output is written under eval.\<name>.\*.
        </ParamField>

        <ParamField body="template" type="String!" required>
          Natural-language eval template. \{column\_name} markers are substituted at run time.
        </ParamField>

        <ParamField body="classificationChoices" type="JSON">
          Optional fixed label set. Null means the agent discovers labels freely.
        </ParamField>

        <ParamField body="strictChoices" type="Boolean">
          When true the agent must only emit labels from classificationChoices.
        </ParamField>

        <ParamField body="llmModelName" type="String">
          Optional model name hint forwarded to the pod as ANTHROPIC\_MODEL.
        </ParamField>

        <ParamField body="direction" type="HarnessEvaluationConfigDirection">
          Optimization direction for scored harness eval output. One of: `maximize`, `minimize`, `none`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="systemOneEvaluator" type="SystemOneEvaluatorVersionInput">
      <Expandable title="SystemOneEvaluatorVersionInput fields">
        <ParamField body="llmIntegrationId" type="String!" required>
          Relay global ID of the TypeSafe AI integration this evaluator calls.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          Provider model route, for example `jev-latest`. Required: the column has no default, so which route was asked for is always recorded.
        </ParamField>

        <ParamField body="stateTemplate" type="String!" required>
          The author's JSON-shaped state template, stored verbatim, carrying \{variables} resolved per record.
        </ParamField>

        <ParamField body="questions" type="[SystemOneQuestionInput!]!" required>
          At least one. Every question sees the same state and is evaluated independently in one request.

          <Expandable title="SystemOneQuestionInput fields">
            <ParamField body="questionName" type="String!" required>
              Becomes the provider's question key and the published eval column suffix, so it is validated against the eval-name rules.
            </ParamField>

            <ParamField body="questionType" type="SystemOneQuestionType!" required>
              One of: `boolean`, `choice`, `score`.
            </ParamField>

            <ParamField body="instructions" type="String!" required>
              The complete judgment. Question names are not sent to the provider as context.
            </ParamField>

            <ParamField body="criteria" type="JSON">
              Required for choice and score; optional for boolean. Shape is validated per type.
            </ParamField>

            <ParamField body="direction" type="OptimizationDirection">
              Read for score questions only, where it defaults to maximize. Normalized to `none` for every other type. One of: `minimize`, `maximize`, `none`.
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="remoteEvaluator" type="RemoteEvaluatorVersionInput">
      <Expandable title="RemoteEvaluatorVersionInput fields">
        <ParamField body="endpoint" type="String">
          HTTP URL of the remote evaluation endpoint.
        </ParamField>

        <ParamField body="headers" type="JSON">
          Optional custom request headers sent to the endpoint (encrypted at rest).
        </ParamField>

        <ParamField body="inputSchema" type="JSON">
          Optional JSON schema describing the expected request body.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="evaluator" type="Evaluator!">
  <Expandable title="Evaluator fields">
    Scalar fields: `id`, `name`, `description`, `taskType`, `commitHash`, `commitMessage`, `createdAt`, `updatedAt`.

    Object fields (select subfields): `versionHistory`, `webhookSubscriptions`, `config`, `harnessEvaluationConfig`, `remoteEvaluationConfig`, `systemOneEvaluationConfig`, `createdBy`, `tags`, `onlineTasks`.
  </Expandable>
</ResponseField>

<ResponseField name="evaluatorVersion" type="EvaluatorVersion!">
  <Expandable title="EvaluatorVersion fields">
    Scalar fields: `id`, `evaluatorId`, `commitHash`, `commitMessage`, `createdAt`, `updatedAt`, `versionNumber`.

    Object fields (select subfields): `evaluator`, `harnessEvaluationConfig`, `remoteEvaluationConfig`, `systemOneEvaluationConfig`, `createdBy`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation CreateEvaluatorVersion($input: CreateEvaluatorVersionMutationInput!) {
    createEvaluatorVersion(input: $input) {
      evaluator { id name }
      evaluatorVersion { id }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "evaluatorId": "<ID>",
      "commitMessage": "<string>"
    }
  }
  ```
</CodeGroup>

### editEvaluator

Edit an existing evaluator

`editEvaluator(input: EditEvaluatorMutationInput!): EditEvaluatorMutationPayload`

#### Arguments

<ParamField body="input" type="EditEvaluatorMutationInput!" required>
  <Expandable title="EditEvaluatorMutationInput fields">
    <ParamField body="evaluatorId" type="ID!" required>
      The evaluator id to edit
    </ParamField>

    <ParamField body="name" type="String">
      The new name to use for the evaluator
    </ParamField>

    <ParamField body="description" type="String">
      The new description for the evaluator
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="evaluator" type="Evaluator!">
  <Expandable title="Evaluator fields">
    Scalar fields: `id`, `name`, `description`, `taskType`, `commitHash`, `commitMessage`, `createdAt`, `updatedAt`.

    Object fields (select subfields): `versionHistory`, `webhookSubscriptions`, `config`, `harnessEvaluationConfig`, `remoteEvaluationConfig`, `systemOneEvaluationConfig`, `createdBy`, `tags`, `onlineTasks`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation EditEvaluator($input: EditEvaluatorMutationInput!) {
    editEvaluator(input: $input) {
      evaluator { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "evaluatorId": "<ID>"
    }
  }
  ```
</CodeGroup>

### deleteEvaluator

Delete an existing evaluator

`deleteEvaluator(input: DeleteEvaluatorMutationInput!): DeleteEvaluatorMutationPayload`

#### Arguments

<ParamField body="input" type="DeleteEvaluatorMutationInput!" required>
  <Expandable title="DeleteEvaluatorMutationInput fields">
    <ParamField body="evaluatorId" type="ID!" required>
      The evaluator id to delete
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="success" type="Boolean!">
  Indicates whether the evaluator was deleted
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation DeleteEvaluator($input: DeleteEvaluatorMutationInput!) {
    deleteEvaluator(input: $input) {
      success
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "evaluatorId": "<ID>"
    }
  }
  ```
</CodeGroup>

### createEvalTask

Create a new online eval task

`createEvalTask(input: CreateEvalTaskMutationInput!): CreateEvalTaskMutationPayload`

#### Arguments

<ParamField body="input" type="CreateEvalTaskMutationInput!" required>
  <Expandable title="CreateEvalTaskMutationInput fields">
    <ParamField body="modelId" type="ID">
      The model id for the model to be evaluated
    </ParamField>

    <ParamField body="datasetId" type="ID">
      The dataset id for the dataset to be evaluated
    </ParamField>

    <ParamField body="name" type="String!" required>
      The name of the online task
    </ParamField>

    <ParamField body="filters" type="[FilterItemInputType!]">
      The filters associated with the online task

      <Expandable title="FilterItemInputType fields">
        <ParamField body="filterType" type="FilterRowType!" required>
          The type of filter. One of: `featureLabel`, `tagLabel`, `predictionValue`, `actuals`, `modelVersion`, `batchId`, `predictionClass`, `predictionScore`, `actualClass`, `actualScore`, `topKPercentile`, `spanProperty`, `llmEval`, `annotation`, `userAnnotation`.
        </ParamField>

        <ParamField body="operator" type="ComparisonOperator!" required>
          The operator of the filter. One of: `greaterThan`, `lessThan`, `equals`, `notEquals`, `greaterThanOrEqual`, `lessThanOrEqual`, `topN`, `contains`, `containsString`, `similarTo`.
        </ParamField>

        <ParamField body="dimension" type="DimensionInput">
          <Expandable title="DimensionInput fields">
            <ParamField body="id" type="ID!" required />

            <ParamField body="name" type="String!" required>
              The label of this dimension, e.g. 'Bank Name' or 'age'
            </ParamField>

            <ParamField body="dataType" type="DimensionDataType!" required>
              The data type of the values in dimensionName, ex. 'FLOAT', 'STRING', etc. One of: `STRING`, `LONG`, `FLOAT`, `DOUBLE`, `EMBEDDING`, `STRING_LIST`, `DICTIONARY`.
            </ParamField>

            <ParamField body="category" type="DimensionCategory">
              The category of the dimension, e.g. 'tag' or 'featureLabel', etc. One of: `featureLabel`, `prediction`, `actuals`, `actualScore`, `actualClass`, `predictionClass`, `predictionScore`, `tag`, `spanProperty`, `llmEval`, `annotation`, `userAnnotation`, `modelVersion`, `batchId`.
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="dimensionValues" type="[DimensionValueInput!]">
          The dimension values of the filter.

          <Expandable title="DimensionValueInput fields">
            <ParamField body="id" type="ID!" required />

            <ParamField body="value" type="String!" required>
              The value of this dimension value, e.g. 'Wells Fargo' or '60'
            </ParamField>

            <ParamField body="similarityReference" type="DimensionValueSimilarityReference">
              The similarity references for this dimension value Nested `DimensionValueSimilarityReference` (same shape as above).
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="queryFilter" type="String">
      Filters to apply to the dataset. Compatible with existing filters, joined with AND.
    </ParamField>

    <ParamField body="multiSpanQuery" type="MultiSpanQueryDefinitionInput">
      Multi-span query definition for admission. Mutually exclusive with queryFilter.

      <Expandable title="MultiSpanQueryDefinitionInput fields">
        <ParamField body="subqueries" type="[MultiSpanSubqueryDefinitionInput!]!" required>
          <Expandable title="MultiSpanSubqueryDefinitionInput fields">
            <ParamField body="name" type="String!" required>
              Subquery name (A-E)
            </ParamField>

            <ParamField body="query" type="String!" required>
              Span-selection filter for this subquery
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="expression" type="String!" required>
          Boolean expression over the subquery names
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="samplingRate" type="Float!" required>
      The sampling rate of the online task
    </ParamField>

    <ParamField body="runContinuously" type="Boolean!" required>
      Whether the online task runs continuously
    </ParamField>

    <ParamField body="llmConfig" type="OnlineTaskLLMConfigInput">
      The LLM config for the online task

      <Expandable title="OnlineTaskLLMConfigInput fields">
        <ParamField body="integrationId" type="ID!" required>
          The selected named LLM integration id (relay global id). If provided, server will use its configuration.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          The LLM model
        </ParamField>

        <ParamField body="invocationParameters" type="JSONObject!" required>
          parameters used when running the Llm
        </ParamField>

        <ParamField body="providerParameters" type="JSONObject!" required>
          parameters used to initialize the Llm
        </ParamField>

        <ParamField body="temperature" type="Float">
          The temperature of the LLM config
        </ParamField>

        <ParamField body="provider" type="LLMIntegrationProvider">
          The LLM provider One of: `openAI`, `anthropic`, `awsBedrock`, `azureOpenAI`, `vertexAI`, `custom`, `nvidiaNim`, `gemini`, `litellm`, `fireworks`, `togetherAi`, `cursor`, `typeSafeAi`.
        </ParamField>

        <ParamField body="deploymentName" type="String">
          The deployment name of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiVersion" type="String">
          The api version of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiEndpoint" type="String">
          The api endpoint of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="region" type="String">
          The region of the LLM config
        </ParamField>

        <ParamField body="maxTokens" type="Float">
          The max tokens of the LLM config
        </ParamField>

        <ParamField body="maxCompletionTokens" type="Float">
          The max completion tokens of the LLM config
        </ParamField>

        <ParamField body="topP" type="Float">
          The top P of the LLM config
        </ParamField>

        <ParamField body="topK" type="Float">
          The top K of the LLM config
        </ParamField>

        <ParamField body="stop" type="[String!]">
          The stop associated with the config
        </ParamField>

        <ParamField body="customModelEndpointId" type="String">
          The id of the Custom Model Endpoint. Required if provider is custom.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="templateEvaluators" type="[TemplateEvaluationConfigInput!]">
      <Expandable title="TemplateEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the template evaluation config
        </ParamField>

        <ParamField body="rails" type="[String!]">
          The rails associated with the config
        </ParamField>

        <ParamField body="template" type="String!" required>
          The template associated with the config
        </ParamField>

        <ParamField body="turnDefinition" type="String">
          Optional turn-definition template for session-granularity evaluators. When non-empty, the evaluator operates in custom-turn mode: each session trace yields one turn rendered from this template. Uses the same \{variable} syntax as the main template.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="includeExplanations" type="Boolean!" required>
          Whether the config includes explanations
        </ParamField>

        <ParamField body="useFunctionCallingIfAvailable" type="Boolean!" required>
          Whether the config uses function calling if available
        </ParamField>

        <ParamField body="useStructuredOutputIfAvailable" type="Boolean">
          Whether the config uses structured output if available
        </ParamField>

        <ParamField body="classificationChoices" type="JSONObject">
          The classification choices for the config (Record\<string, number>)
        </ParamField>

        <ParamField body="direction" type="TemplateEvaluationConfigDirection">
          The direction for the config One of: `maximize`, `minimize`, `none`.
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          Data granularity (span, trace, or session). Defaults to span when omitted. One of: `span`, `trace`, `session`.
        </ParamField>

        <ParamField body="llmConfig" type="TemplateEvaluationLlmConfigInput">
          Llm Config for this evaluator

          <Expandable title="TemplateEvaluationLlmConfigInput fields">
            <ParamField body="integrationId" type="ID!" required>
              The selected named LLM integration id (relay global id). If provided, server will use its configuration.
            </ParamField>

            <ParamField body="modelName" type="String!" required>
              The LLM model
            </ParamField>

            <ParamField body="invocationParameters" type="JSONObject!" required>
              parameters used when running the Llm
            </ParamField>

            <ParamField body="providerParameters" type="JSONObject!" required>
              parameters used to initialize the Llm
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="codeEvaluators" type="[CodeEvaluationConfigInput!]">
      <Expandable title="CodeEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the evaluation config
        </ParamField>

        <ParamField body="evaluationClass" type="String">
          The evaluation type associated with the config
        </ParamField>

        <ParamField body="evalClassCodeBlock" type="String">
          The code block for the evaluation class
        </ParamField>

        <ParamField body="evaluationInputParams" type="JSON">
          The evaluation input params
        </ParamField>

        <ParamField body="packageImports" type="String">
          The package imports for the evaluation class
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="spanAttributes" type="[String!]">
          The data column associated with the config
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          The data granularity associated with the config (span, trace, or session) One of: `span`, `trace`, `session`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="experiments" type="[ID!]" />

    <ParamField body="postProcessingSteps" type="[OnlineTaskPostProcessingStepInput!]">
      <Expandable title="OnlineTaskPostProcessingStepInput fields">
        <ParamField body="targetType" type="String!" required>
          The type of the target (e.g. dataset, annotation queue)
        </ParamField>

        <ParamField body="targetId" type="ID!" required>
          The ID of the specific target (e.g. dataset ID)
        </ParamField>

        <ParamField body="position" type="Int!" required>
          Ordering position of the post-processing step
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional filter expression for selecting which data is added to target
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="evaluators" type="[OnlineTaskEvaluatorInput!]">
      <Expandable title="OnlineTaskEvaluatorInput fields">
        <ParamField body="evaluatorId" type="ID!" required>
          id of evaluator to attach to online task
        </ParamField>

        <ParamField body="evaluatorVersionId" type="ID">
          Optional id of the evaluator version to pin. Omit (or null) to always run the latest version.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          position of evaluator in ui
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="columnMappings" type="JSON">
          Mapping of evaluator variable names to dataset column names
        </ParamField>

        <ParamField body="subqueryColumnMappings" type="[SubqueryColumnMappingInput!]">
          Subquery-aware variable mappings. Populated instead of columnMappings on tasks that have a multiSpanQuery.

          <Expandable title="SubqueryColumnMappingInput fields">
            <ParamField body="variableName" type="String!" required>
              Evaluator variable name
            </ParamField>

            <ParamField body="subqueries" type="[String!]!" required>
              Source subquery names (A-E) from the task's multi\_span\_query that may populate this variable. When multiple listed subqueries match for a unit, their resolved values are concatenated. The list must contain at least one subquery; an empty list matches nothing and is rejected.
            </ParamField>

            <ParamField body="attributePath" type="String!" required>
              Span attribute path to resolve within each subquery
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="tracingEnabled" type="Boolean">
      Whether tracing is enabled for the online task
    </ParamField>

    <ParamField body="arizeIntegrationId" type="ID">
      Arize sandbox integration ID (BYO path for harness tasks). When provided for a harness task the integration is linked to this task so the sandbox pod receives the Arize API key. Omit to use auto-provisioning (phase 2).
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="evalTask" type="OnlineTask!">
  <Expandable title="OnlineTask fields">
    Scalar fields: `id`, `taskType`, `name`, `samplingRate`, `lastRunAt`, `queryFilter`, `createdAt`, `updatedAt`, `deletedAt`, `runContinuously`, `tracingEnabled`, `experimentIds`, `hasUnavailableEvaluatorIntegration`.

    Object fields (select subfields): `model`, `multiSpanQuery`, `createdBy`, `llmConfig`, `taskConfig`, `evaluators`, `onlineTaskEvaluators`, `allEvaluatorConfigs`, `taskRuns`, `experiments`, `dataset`, `postProcessingSteps`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation CreateEvalTask($input: CreateEvalTaskMutationInput!) {
    createEvalTask(input: $input) {
      evalTask { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "name": "<string>",
      "samplingRate": 1,
      "runContinuously": true
    }
  }
  ```
</CodeGroup>

### patchEvalTask

`patchEvalTask(input: PatchEvalTaskMutationInput!): PatchEvalTaskMutationPayload`

#### Arguments

<ParamField body="input" type="PatchEvalTaskMutationInput!" required>
  <Expandable title="PatchEvalTaskMutationInput fields">
    <ParamField body="onlineTaskId" type="ID!" required>
      The online task id for the eval task to be patched
    </ParamField>

    <ParamField body="modelId" type="ID">
      The model id for the model to be evaluated
    </ParamField>

    <ParamField body="datasetId" type="ID">
      The dataset id for the dataset to be evaluated
    </ParamField>

    <ParamField body="name" type="String">
      The name of the online task
    </ParamField>

    <ParamField body="filters" type="[FilterItemInputType!]">
      The filters associated with the online task

      <Expandable title="FilterItemInputType fields">
        <ParamField body="filterType" type="FilterRowType!" required>
          The type of filter. One of: `featureLabel`, `tagLabel`, `predictionValue`, `actuals`, `modelVersion`, `batchId`, `predictionClass`, `predictionScore`, `actualClass`, `actualScore`, `topKPercentile`, `spanProperty`, `llmEval`, `annotation`, `userAnnotation`.
        </ParamField>

        <ParamField body="operator" type="ComparisonOperator!" required>
          The operator of the filter. One of: `greaterThan`, `lessThan`, `equals`, `notEquals`, `greaterThanOrEqual`, `lessThanOrEqual`, `topN`, `contains`, `containsString`, `similarTo`.
        </ParamField>

        <ParamField body="dimension" type="DimensionInput">
          <Expandable title="DimensionInput fields">
            <ParamField body="id" type="ID!" required />

            <ParamField body="name" type="String!" required>
              The label of this dimension, e.g. 'Bank Name' or 'age'
            </ParamField>

            <ParamField body="dataType" type="DimensionDataType!" required>
              The data type of the values in dimensionName, ex. 'FLOAT', 'STRING', etc. One of: `STRING`, `LONG`, `FLOAT`, `DOUBLE`, `EMBEDDING`, `STRING_LIST`, `DICTIONARY`.
            </ParamField>

            <ParamField body="category" type="DimensionCategory">
              The category of the dimension, e.g. 'tag' or 'featureLabel', etc. One of: `featureLabel`, `prediction`, `actuals`, `actualScore`, `actualClass`, `predictionClass`, `predictionScore`, `tag`, `spanProperty`, `llmEval`, `annotation`, `userAnnotation`, `modelVersion`, `batchId`.
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="dimensionValues" type="[DimensionValueInput!]">
          The dimension values of the filter.

          <Expandable title="DimensionValueInput fields">
            <ParamField body="id" type="ID!" required />

            <ParamField body="value" type="String!" required>
              The value of this dimension value, e.g. 'Wells Fargo' or '60'
            </ParamField>

            <ParamField body="similarityReference" type="DimensionValueSimilarityReference">
              The similarity references for this dimension value Nested `DimensionValueSimilarityReference` (same shape as above).
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="queryFilter" type="String">
      Filters to apply to the dataset. Compatible with existing filters, joined with AND.
    </ParamField>

    <ParamField body="multiSpanQuery" type="MultiSpanQueryDefinitionInput">
      Multi-span query definition for admission. Mutually exclusive with queryFilter.

      <Expandable title="MultiSpanQueryDefinitionInput fields">
        <ParamField body="subqueries" type="[MultiSpanSubqueryDefinitionInput!]!" required>
          <Expandable title="MultiSpanSubqueryDefinitionInput fields">
            <ParamField body="name" type="String!" required>
              Subquery name (A-E)
            </ParamField>

            <ParamField body="query" type="String!" required>
              Span-selection filter for this subquery
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="expression" type="String!" required>
          Boolean expression over the subquery names
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="samplingRate" type="Float">
      The sampling rate of the online task
    </ParamField>

    <ParamField body="runContinuously" type="Boolean">
      Whether the online task runs continuously
    </ParamField>

    <ParamField body="llmConfig" type="OnlineTaskLLMConfigInput">
      The LLM config for the online task

      <Expandable title="OnlineTaskLLMConfigInput fields">
        <ParamField body="integrationId" type="ID!" required>
          The selected named LLM integration id (relay global id). If provided, server will use its configuration.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          The LLM model
        </ParamField>

        <ParamField body="invocationParameters" type="JSONObject!" required>
          parameters used when running the Llm
        </ParamField>

        <ParamField body="providerParameters" type="JSONObject!" required>
          parameters used to initialize the Llm
        </ParamField>

        <ParamField body="temperature" type="Float">
          The temperature of the LLM config
        </ParamField>

        <ParamField body="provider" type="LLMIntegrationProvider">
          The LLM provider One of: `openAI`, `anthropic`, `awsBedrock`, `azureOpenAI`, `vertexAI`, `custom`, `nvidiaNim`, `gemini`, `litellm`, `fireworks`, `togetherAi`, `cursor`, `typeSafeAi`.
        </ParamField>

        <ParamField body="deploymentName" type="String">
          The deployment name of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiVersion" type="String">
          The api version of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiEndpoint" type="String">
          The api endpoint of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="region" type="String">
          The region of the LLM config
        </ParamField>

        <ParamField body="maxTokens" type="Float">
          The max tokens of the LLM config
        </ParamField>

        <ParamField body="maxCompletionTokens" type="Float">
          The max completion tokens of the LLM config
        </ParamField>

        <ParamField body="topP" type="Float">
          The top P of the LLM config
        </ParamField>

        <ParamField body="topK" type="Float">
          The top K of the LLM config
        </ParamField>

        <ParamField body="stop" type="[String!]">
          The stop associated with the config
        </ParamField>

        <ParamField body="customModelEndpointId" type="String">
          The id of the Custom Model Endpoint. Required if provider is custom.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="templateEvaluators" type="[TemplateEvaluationConfigInput!]">
      <Expandable title="TemplateEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the template evaluation config
        </ParamField>

        <ParamField body="rails" type="[String!]">
          The rails associated with the config
        </ParamField>

        <ParamField body="template" type="String!" required>
          The template associated with the config
        </ParamField>

        <ParamField body="turnDefinition" type="String">
          Optional turn-definition template for session-granularity evaluators. When non-empty, the evaluator operates in custom-turn mode: each session trace yields one turn rendered from this template. Uses the same \{variable} syntax as the main template.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="includeExplanations" type="Boolean!" required>
          Whether the config includes explanations
        </ParamField>

        <ParamField body="useFunctionCallingIfAvailable" type="Boolean!" required>
          Whether the config uses function calling if available
        </ParamField>

        <ParamField body="useStructuredOutputIfAvailable" type="Boolean">
          Whether the config uses structured output if available
        </ParamField>

        <ParamField body="classificationChoices" type="JSONObject">
          The classification choices for the config (Record\<string, number>)
        </ParamField>

        <ParamField body="direction" type="TemplateEvaluationConfigDirection">
          The direction for the config One of: `maximize`, `minimize`, `none`.
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          Data granularity (span, trace, or session). Defaults to span when omitted. One of: `span`, `trace`, `session`.
        </ParamField>

        <ParamField body="llmConfig" type="TemplateEvaluationLlmConfigInput">
          Llm Config for this evaluator

          <Expandable title="TemplateEvaluationLlmConfigInput fields">
            <ParamField body="integrationId" type="ID!" required>
              The selected named LLM integration id (relay global id). If provided, server will use its configuration.
            </ParamField>

            <ParamField body="modelName" type="String!" required>
              The LLM model
            </ParamField>

            <ParamField body="invocationParameters" type="JSONObject!" required>
              parameters used when running the Llm
            </ParamField>

            <ParamField body="providerParameters" type="JSONObject!" required>
              parameters used to initialize the Llm
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="codeEvaluators" type="[CodeEvaluationConfigInput!]">
      <Expandable title="CodeEvaluationConfigInput fields">
        <ParamField body="name" type="EvalColumnName!" required>
          The name of the evaluation config
        </ParamField>

        <ParamField body="evaluationClass" type="String">
          The evaluation type associated with the config
        </ParamField>

        <ParamField body="evalClassCodeBlock" type="String">
          The code block for the evaluation class
        </ParamField>

        <ParamField body="evaluationInputParams" type="JSON">
          The evaluation input params
        </ParamField>

        <ParamField body="packageImports" type="String">
          The package imports for the evaluation class
        </ParamField>

        <ParamField body="position" type="Int!" required>
          The position of the config
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="spanAttributes" type="[String!]">
          The data column associated with the config
        </ParamField>

        <ParamField body="dataGranularityType" type="EvalDataGranularityType">
          The data granularity associated with the config (span, trace, or session) One of: `span`, `trace`, `session`.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="experiments" type="[ID!]" />

    <ParamField body="postProcessingSteps" type="[OnlineTaskPostProcessingStepInput!]">
      <Expandable title="OnlineTaskPostProcessingStepInput fields">
        <ParamField body="targetType" type="String!" required>
          The type of the target (e.g. dataset, annotation queue)
        </ParamField>

        <ParamField body="targetId" type="ID!" required>
          The ID of the specific target (e.g. dataset ID)
        </ParamField>

        <ParamField body="position" type="Int!" required>
          Ordering position of the post-processing step
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional filter expression for selecting which data is added to target
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="evaluators" type="[OnlineTaskEvaluatorInput!]">
      <Expandable title="OnlineTaskEvaluatorInput fields">
        <ParamField body="evaluatorId" type="ID!" required>
          id of evaluator to attach to online task
        </ParamField>

        <ParamField body="evaluatorVersionId" type="ID">
          Optional id of the evaluator version to pin. Omit (or null) to always run the latest version.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          position of evaluator in ui
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="columnMappings" type="JSON">
          Mapping of evaluator variable names to dataset column names
        </ParamField>

        <ParamField body="subqueryColumnMappings" type="[SubqueryColumnMappingInput!]">
          Subquery-aware variable mappings. Populated instead of columnMappings on tasks that have a multiSpanQuery.

          <Expandable title="SubqueryColumnMappingInput fields">
            <ParamField body="variableName" type="String!" required>
              Evaluator variable name
            </ParamField>

            <ParamField body="subqueries" type="[String!]!" required>
              Source subquery names (A-E) from the task's multi\_span\_query that may populate this variable. When multiple listed subqueries match for a unit, their resolved values are concatenated. The list must contain at least one subquery; an empty list matches nothing and is rejected.
            </ParamField>

            <ParamField body="attributePath" type="String!" required>
              Span attribute path to resolve within each subquery
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="tracingEnabled" type="Boolean">
      Whether tracing is enabled for the online task
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="evalTask" type="OnlineTask!">
  <Expandable title="OnlineTask fields">
    Scalar fields: `id`, `taskType`, `name`, `samplingRate`, `lastRunAt`, `queryFilter`, `createdAt`, `updatedAt`, `deletedAt`, `runContinuously`, `tracingEnabled`, `experimentIds`, `hasUnavailableEvaluatorIntegration`.

    Object fields (select subfields): `model`, `multiSpanQuery`, `createdBy`, `llmConfig`, `taskConfig`, `evaluators`, `onlineTaskEvaluators`, `allEvaluatorConfigs`, `taskRuns`, `experiments`, `dataset`, `postProcessingSteps`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation PatchEvalTask($input: PatchEvalTaskMutationInput!) {
    patchEvalTask(input: $input) {
      evalTask { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "onlineTaskId": "<ID>"
    }
  }
  ```
</CodeGroup>

### patchEvalTaskWithEvalHub

`patchEvalTaskWithEvalHub(input: PatchEvalTaskWithEvalHubMutationInput!): PatchEvalTaskWithEvalHubMutationPayload`

#### Arguments

<ParamField body="input" type="PatchEvalTaskWithEvalHubMutationInput!" required>
  <Expandable title="PatchEvalTaskWithEvalHubMutationInput fields">
    <ParamField body="onlineTaskId" type="ID!" required>
      The online task id for the eval task to be patched
    </ParamField>

    <ParamField body="modelId" type="ID">
      The model id for the model to be evaluated
    </ParamField>

    <ParamField body="datasetId" type="ID">
      The dataset id for the dataset to be evaluated
    </ParamField>

    <ParamField body="name" type="String">
      The name of the online task
    </ParamField>

    <ParamField body="filters" type="[FilterItemInputType!]">
      The filters associated with the online task

      <Expandable title="FilterItemInputType fields">
        <ParamField body="filterType" type="FilterRowType!" required>
          The type of filter. One of: `featureLabel`, `tagLabel`, `predictionValue`, `actuals`, `modelVersion`, `batchId`, `predictionClass`, `predictionScore`, `actualClass`, `actualScore`, `topKPercentile`, `spanProperty`, `llmEval`, `annotation`, `userAnnotation`.
        </ParamField>

        <ParamField body="operator" type="ComparisonOperator!" required>
          The operator of the filter. One of: `greaterThan`, `lessThan`, `equals`, `notEquals`, `greaterThanOrEqual`, `lessThanOrEqual`, `topN`, `contains`, `containsString`, `similarTo`.
        </ParamField>

        <ParamField body="dimension" type="DimensionInput">
          <Expandable title="DimensionInput fields">
            <ParamField body="id" type="ID!" required />

            <ParamField body="name" type="String!" required>
              The label of this dimension, e.g. 'Bank Name' or 'age'
            </ParamField>

            <ParamField body="dataType" type="DimensionDataType!" required>
              The data type of the values in dimensionName, ex. 'FLOAT', 'STRING', etc. One of: `STRING`, `LONG`, `FLOAT`, `DOUBLE`, `EMBEDDING`, `STRING_LIST`, `DICTIONARY`.
            </ParamField>

            <ParamField body="category" type="DimensionCategory">
              The category of the dimension, e.g. 'tag' or 'featureLabel', etc. One of: `featureLabel`, `prediction`, `actuals`, `actualScore`, `actualClass`, `predictionClass`, `predictionScore`, `tag`, `spanProperty`, `llmEval`, `annotation`, `userAnnotation`, `modelVersion`, `batchId`.
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="dimensionValues" type="[DimensionValueInput!]">
          The dimension values of the filter.

          <Expandable title="DimensionValueInput fields">
            <ParamField body="id" type="ID!" required />

            <ParamField body="value" type="String!" required>
              The value of this dimension value, e.g. 'Wells Fargo' or '60'
            </ParamField>

            <ParamField body="similarityReference" type="DimensionValueSimilarityReference">
              The similarity references for this dimension value Nested `DimensionValueSimilarityReference` (same shape as above).
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="queryFilter" type="String">
      Filters to apply to the dataset. Compatible with existing filters, joined with AND.
    </ParamField>

    <ParamField body="multiSpanQuery" type="MultiSpanQueryDefinitionInput">
      Multi-span query definition for admission. Mutually exclusive with queryFilter.

      <Expandable title="MultiSpanQueryDefinitionInput fields">
        <ParamField body="subqueries" type="[MultiSpanSubqueryDefinitionInput!]!" required>
          <Expandable title="MultiSpanSubqueryDefinitionInput fields">
            <ParamField body="name" type="String!" required>
              Subquery name (A-E)
            </ParamField>

            <ParamField body="query" type="String!" required>
              Span-selection filter for this subquery
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="expression" type="String!" required>
          Boolean expression over the subquery names
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="samplingRate" type="Float">
      The sampling rate of the online task
    </ParamField>

    <ParamField body="runContinuously" type="Boolean">
      Whether the online task runs continuously
    </ParamField>

    <ParamField body="llmConfig" type="OnlineTaskLLMConfigInput">
      The LLM config for the online task

      <Expandable title="OnlineTaskLLMConfigInput fields">
        <ParamField body="integrationId" type="ID!" required>
          The selected named LLM integration id (relay global id). If provided, server will use its configuration.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          The LLM model
        </ParamField>

        <ParamField body="invocationParameters" type="JSONObject!" required>
          parameters used when running the Llm
        </ParamField>

        <ParamField body="providerParameters" type="JSONObject!" required>
          parameters used to initialize the Llm
        </ParamField>

        <ParamField body="temperature" type="Float">
          The temperature of the LLM config
        </ParamField>

        <ParamField body="provider" type="LLMIntegrationProvider">
          The LLM provider One of: `openAI`, `anthropic`, `awsBedrock`, `azureOpenAI`, `vertexAI`, `custom`, `nvidiaNim`, `gemini`, `litellm`, `fireworks`, `togetherAi`, `cursor`, `typeSafeAi`.
        </ParamField>

        <ParamField body="deploymentName" type="String">
          The deployment name of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiVersion" type="String">
          The api version of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiEndpoint" type="String">
          The api endpoint of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="region" type="String">
          The region of the LLM config
        </ParamField>

        <ParamField body="maxTokens" type="Float">
          The max tokens of the LLM config
        </ParamField>

        <ParamField body="maxCompletionTokens" type="Float">
          The max completion tokens of the LLM config
        </ParamField>

        <ParamField body="topP" type="Float">
          The top P of the LLM config
        </ParamField>

        <ParamField body="topK" type="Float">
          The top K of the LLM config
        </ParamField>

        <ParamField body="stop" type="[String!]">
          The stop associated with the config
        </ParamField>

        <ParamField body="customModelEndpointId" type="String">
          The id of the Custom Model Endpoint. Required if provider is custom.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="evaluators" type="[OnlineTaskEvaluatorInput!]">
      <Expandable title="OnlineTaskEvaluatorInput fields">
        <ParamField body="evaluatorId" type="ID!" required>
          id of evaluator to attach to online task
        </ParamField>

        <ParamField body="evaluatorVersionId" type="ID">
          Optional id of the evaluator version to pin. Omit (or null) to always run the latest version.
        </ParamField>

        <ParamField body="position" type="Int!" required>
          position of evaluator in ui
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional query used to filter over a given data granularity (ex. session, trace)
        </ParamField>

        <ParamField body="columnMappings" type="JSON">
          Mapping of evaluator variable names to dataset column names
        </ParamField>

        <ParamField body="subqueryColumnMappings" type="[SubqueryColumnMappingInput!]">
          Subquery-aware variable mappings. Populated instead of columnMappings on tasks that have a multiSpanQuery.

          <Expandable title="SubqueryColumnMappingInput fields">
            <ParamField body="variableName" type="String!" required>
              Evaluator variable name
            </ParamField>

            <ParamField body="subqueries" type="[String!]!" required>
              Source subquery names (A-E) from the task's multi\_span\_query that may populate this variable. When multiple listed subqueries match for a unit, their resolved values are concatenated. The list must contain at least one subquery; an empty list matches nothing and is rejected.
            </ParamField>

            <ParamField body="attributePath" type="String!" required>
              Span attribute path to resolve within each subquery
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="experiments" type="[ID!]" />

    <ParamField body="postProcessingSteps" type="[OnlineTaskPostProcessingStepInput!]">
      <Expandable title="OnlineTaskPostProcessingStepInput fields">
        <ParamField body="targetType" type="String!" required>
          The type of the target (e.g. dataset, annotation queue)
        </ParamField>

        <ParamField body="targetId" type="ID!" required>
          The ID of the specific target (e.g. dataset ID)
        </ParamField>

        <ParamField body="position" type="Int!" required>
          Ordering position of the post-processing step
        </ParamField>

        <ParamField body="queryFilter" type="String">
          Optional filter expression for selecting which data is added to target
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="evalTask" type="OnlineTask!">
  <Expandable title="OnlineTask fields">
    Scalar fields: `id`, `taskType`, `name`, `samplingRate`, `lastRunAt`, `queryFilter`, `createdAt`, `updatedAt`, `deletedAt`, `runContinuously`, `tracingEnabled`, `experimentIds`, `hasUnavailableEvaluatorIntegration`.

    Object fields (select subfields): `model`, `multiSpanQuery`, `createdBy`, `llmConfig`, `taskConfig`, `evaluators`, `onlineTaskEvaluators`, `allEvaluatorConfigs`, `taskRuns`, `experiments`, `dataset`, `postProcessingSteps`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation PatchEvalTaskWithEvalHub($input: PatchEvalTaskWithEvalHubMutationInput!) {
    patchEvalTaskWithEvalHub(input: $input) {
      evalTask { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "onlineTaskId": "<ID>"
    }
  }
  ```
</CodeGroup>

### migrateTaskToEvalHub

migrates a task's evaluators to eval hub

`migrateTaskToEvalHub(input: MigrateTaskToEvalHubMutationInput!): MigrateTaskToEvalHubMutationPayload`

#### Arguments

<ParamField body="input" type="MigrateTaskToEvalHubMutationInput!" required>
  <Expandable title="MigrateTaskToEvalHubMutationInput fields">
    <ParamField body="onlineTaskId" type="ID!" required>
      The online task id
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="onlineTask" type="OnlineTask!">
  <Expandable title="OnlineTask fields">
    Scalar fields: `id`, `taskType`, `name`, `samplingRate`, `lastRunAt`, `queryFilter`, `createdAt`, `updatedAt`, `deletedAt`, `runContinuously`, `tracingEnabled`, `experimentIds`, `hasUnavailableEvaluatorIntegration`.

    Object fields (select subfields): `model`, `multiSpanQuery`, `createdBy`, `llmConfig`, `taskConfig`, `evaluators`, `onlineTaskEvaluators`, `allEvaluatorConfigs`, `taskRuns`, `experiments`, `dataset`, `postProcessingSteps`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation MigrateTaskToEvalHub($input: MigrateTaskToEvalHubMutationInput!) {
    migrateTaskToEvalHub(input: $input) {
      onlineTask { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "onlineTaskId": "<ID>"
    }
  }
  ```
</CodeGroup>

### runOnlineTask

Run an online task

`runOnlineTask(input: RunOnlineTaskMutationInput!): RunOnlineTaskMutationPayload`

#### Arguments

<ParamField body="input" type="RunOnlineTaskMutationInput!" required>
  <Expandable title="RunOnlineTaskMutationInput fields">
    <ParamField body="onlineTaskId" type="ID">
      The task id for the task to be processed
    </ParamField>

    <ParamField body="dataStartTime" type="DateTime">
      Start time of the data processed by the online task
    </ParamField>

    <ParamField body="dataEndTime" type="DateTime">
      Start time of the data processed by the online task
    </ParamField>

    <ParamField body="maxSpans" type="Int">
      Maximum number of spans processed by the online task. If not specified, default to 10,000.
    </ParamField>

    <ParamField body="overrideEvaluations" type="Boolean">
      Whether to override the existing evaluation labels for the task.
    </ParamField>

    <ParamField body="isErrorRetry" type="Boolean">
      When true, the worker reproduces the source run's capped row set by omitting eval-null scan filters. Used by Rerun on Errors.
    </ParamField>

    <ParamField body="experimentIds" type="[ID!]">
      The experiment ids to run the task on
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="result" type="RunOnlineTaskResponse!">
  Result for queuing online task run on date range Union of `TaskError`, `CreateTaskRunResponse`. Use inline fragments (`... on TypeName`).
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation RunOnlineTask($input: RunOnlineTaskMutationInput!) {
    runOnlineTask(input: $input) {
      result { __typename }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {}
  }
  ```
</CodeGroup>

### cancelOnlineTaskRun

Cancel an online task run

`cancelOnlineTaskRun(input: CancelOnlineTaskRunMutationInput!): CancelOnlineTaskRunMutationPayload`

#### Arguments

<ParamField body="input" type="CancelOnlineTaskRunMutationInput!" required>
  <Expandable title="CancelOnlineTaskRunMutationInput fields">
    <ParamField body="onlineTaskRunId" type="ID!" required>
      The run id for the task to be processed
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="result" type="CancelOnlineTaskResponse!">
  Result for cancelling online task run Union of `TaskError`, `CancelTaskRunResponse`. Use inline fragments (`... on TypeName`).
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation CancelOnlineTaskRun($input: CancelOnlineTaskRunMutationInput!) {
    cancelOnlineTaskRun(input: $input) {
      result { __typename }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "onlineTaskRunId": "<ID>"
    }
  }
  ```
</CodeGroup>

### deleteOnlineTask

`deleteOnlineTask(input: DeleteOnlineTaskMutationInput!): DeleteOnlineTaskMutationPayload`

#### Arguments

<ParamField body="input" type="DeleteOnlineTaskMutationInput!" required>
  <Expandable title="DeleteOnlineTaskMutationInput fields">
    <ParamField body="onlineTaskIds" type="[ID!]!" required>
      The IDs of the online tasks to delete. Maximum 50 per request; callers must implement their own batching for larger sets.
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="success" type="Boolean!">
  True when all requested tasks were deleted (failedTasks is empty). Kept for backward compatibility.
</ResponseField>

<ResponseField name="deletedTaskIds" type="[ID!]!">
  IDs of the tasks that were successfully deleted.
</ResponseField>

<ResponseField name="failedTasks" type="[DeleteOnlineTaskFailure!]!">
  Tasks that could not be deleted, each with the reason for failure. The caller should inspect and retry or investigate individually.

  <Expandable title="DeleteOnlineTaskFailure fields">
    Scalar fields: `taskId`, `message`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation DeleteOnlineTask($input: DeleteOnlineTaskMutationInput!) {
    deleteOnlineTask(input: $input) {
      success
      deletedTaskIds
      failedTasks { message }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "onlineTaskIds": [
        "<ID>"
      ]
    }
  }
  ```
</CodeGroup>

### runExperimentTask

Run an experiment task with configurable LLM interactions

`runExperimentTask(input: RunExperimentTaskMutationInput!): RunExperimentTaskMutationPayload`

#### Arguments

<ParamField body="input" type="RunExperimentTaskMutationInput!" required>
  <Expandable title="RunExperimentTaskMutationInput fields">
    <ParamField body="datasetId" type="ID!" required>
      The dataset id for the experiment
    </ParamField>

    <ParamField body="datasetName" type="String!" required>
      The dataset name for the experiment
    </ParamField>

    <ParamField body="datasetVersionId" type="ID!" required>
      The dataset version id for the experiment
    </ParamField>

    <ParamField body="spaceId" type="ID!" required>
      The space id for the experiment
    </ParamField>

    <ParamField body="exampleOverrides" type="[ExampleOverridesInput!]!" required>
      The example overrides for the experiment

      <Expandable title="ExampleOverridesInput fields">
        <ParamField body="exampleId" type="String!" required />

        <ParamField body="overridesJSON" type="JSON!" required />
      </Expandable>
    </ParamField>

    <ParamField body="exampleIds" type="[String!]">
      The example ids to run the experiment on
    </ParamField>

    <ParamField body="examplesToRunCount" type="Int">
      The number of examples to run the experiment on
    </ParamField>

    <ParamField body="queryFilter" type="String">
      Optional dataset query filter. When set, the experiment (and associated evaluations) run only on examples matching the filter.
    </ParamField>

    <ParamField body="runConfigurations" type="[RunExperimentConfigInput!]!" required>
      The run configurations for the experiment

      <Expandable title="RunExperimentConfigInput fields">
        <ParamField body="experimentType" type="ExperimentTypeEnum!" required>
          Discriminant for what kind of work is performed per example. One of: 'llm\_generation', 'template\_evaluation', 'code\_evaluation', 'agent\_call'. One of: `llm_generation`, `template_evaluation`, `code_evaluation`, `agent_call`.
        </ParamField>

        <ParamField body="input" type="LLMInput">
          LLM input (messages, model, integration). Required for llm\_generation; provider/model fields also used for template\_evaluation. Not used for agent\_call.

          <Expandable title="LLMInput fields">
            <ParamField body="provider" type="ExternalLLMProvider!" required>
              The provider to use for the call. One of: `openAI`, `azureOpenAI`, `anthropic`, `vertexAI`, `awsBedrock`, `custom`, `nvidiaNim`, `gemini`, `litellm`, `fireworks`, `togetherAi`.
            </ParamField>

            <ParamField body="model" type="String">
              The model to use for the call.
            </ParamField>

            <ParamField body="messages" type="[LLMMessageInput!]!" required>
              The full message to send to the LLM (e.g., a prompt template with all variables filled in). Nested `LLMMessageInput` (same shape as above).
            </ParamField>

            <ParamField body="invocationParams" type="InvocationParamsInput!" required>
              The params to call the LLM with. Nested `InvocationParamsInput` (same shape as above).
            </ParamField>

            <ParamField body="providerParams" type="ProviderParamsInput!" required>
              The provider params to use for the call. Nested `ProviderParamsInput` (same shape as above).
            </ParamField>

            <ParamField body="integrationId" type="ID">
              The selected named LLM integration id (relay global id). If provided, server will use its configuration.
            </ParamField>

            <ParamField body="id" type="String">
              Unique identifier for this input
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="inputVariableFormat" type="PromptVersionInputVariableFormatEnum">
          The input variable format for determining prompt variables in the messages. Only applicable for llm\_generation. One of: `F_STRING`, `MUSTACHE`, `NONE`.
        </ParamField>

        <ParamField body="experimentName" type="String!" required>
          Name for the experiment
        </ParamField>

        <ParamField body="instanceId" type="Int!" required>
          The instance id for the experiment
        </ParamField>

        <ParamField body="promptVersionId" type="ID">
          The version ID of the prompt from Prompt Hub (if loaded from hub)
        </ParamField>

        <ParamField body="templateConfig" type="TemplateEvalConfigInput">
          Required when experimentType = 'template\_evaluation'.

          <Expandable title="TemplateEvalConfigInput fields">
            <ParamField body="template" type="String!" required>
              The evaluation prompt template.
            </ParamField>

            <ParamField body="provideExplanation" type="Boolean!" required>
              Whether to include LLM explanations in output.
            </ParamField>

            <ParamField body="classificationChoices" type="JSON">
              Map of choice label to numeric score (e.g. \{relevant: 1, irrelevant: 0}). Required when templateConfig is provided.
            </ParamField>

            <ParamField body="columnMapping" type="JSON">
              Maps named eval param keys to dataset column paths.
            </ParamField>

            <ParamField body="evaluatorVersionId" type="ID">
              The evaluator version ID from Eval Hub (if the template eval was loaded from hub)
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="codeEvalConfig" type="CodeEvalConfigInput">
          Required when experimentType = 'code\_evaluation'.

          <Expandable title="CodeEvalConfigInput fields">
            <ParamField body="codeEvalClass" type="String!" required>
              Managed evaluator class (e.g. JSONParsable, MatchesRegex) or 'Custom'.
            </ParamField>

            <ParamField body="evalParamsJson" type="String">
              JSON-encoded params for managed evaluator classes.
            </ParamField>

            <ParamField body="evaluationClass" type="String">
              Class name for managed evaluators.
            </ParamField>

            <ParamField body="customCodeEvalInfo" type="CustomCodeEvalInfoInput">
              Only set when codeEvalClass is 'Custom'. Nested `CustomCodeEvalInfoInput` (same shape as above).
            </ParamField>

            <ParamField body="columnMapping" type="JSON!" required>
              Maps named eval param keys to dataset column paths.
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="agentConfig" type="AgentConfigInput">
          Required when experimentType = 'agent\_call'.

          <Expandable title="AgentConfigInput fields">
            <ParamField body="remoteEndpointIntegrationId" type="ID!" required>
              Global ID of the remote endpoint integration to invoke.
            </ParamField>

            <ParamField body="inputTemplate" type="JSON!" required>
              Fully-merged JSON request body. \{\{column}} mustache refs are substituted at execution time using each dataset row.
            </ParamField>
          </Expandable>
        </ParamField>

        <ParamField body="requestTimeoutSeconds" type="Int">
          Per-call LLM timeout in seconds (whole number, 30-1800) for llm\_generation and template\_evaluation runs; also applies to any LLM evaluator judge calls attached to the run. Rejected for other experiment types. Null uses the system default.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="tracingMetadata" type="JSON">
      The tracing metadata for the experiment
    </ParamField>

    <ParamField body="evaluationTaskIds" type="[String!]">
      List of evaluation task IDs to run after the experiment completes
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="result" type="RunExperimentTaskResponse!">
  Result for running experiment task Union of `RunExperimentTaskSuccess`, `RunExperimentTaskError`. Use inline fragments (`... on TypeName`).
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation RunExperimentTask($input: RunExperimentTaskMutationInput!) {
    runExperimentTask(input: $input) {
      result { __typename }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "datasetId": "<ID>",
      "datasetName": "<string>",
      "datasetVersionId": "<ID>",
      "spaceId": "<ID>",
      "exampleOverrides": [
        {
          "exampleId": "<string>",
          "overridesJSON": {}
        }
      ],
      "runConfigurations": [
        {
          "experimentType": "llm_generation",
          "experimentName": "<string>",
          "instanceId": 1
        }
      ]
    }
  }
  ```
</CodeGroup>

### createPromptOptimizationTask

Create a new prompt optimization task

`createPromptOptimizationTask(input: CreatePromptOptimizationTaskMutationInput!): CreatePromptOptimizationTaskMutationPayload`

#### Arguments

<ParamField body="input" type="CreatePromptOptimizationTaskMutationInput!" required>
  <Expandable title="CreatePromptOptimizationTaskMutationInput fields">
    <ParamField body="datasetId" type="ID!" required>
      The dataset id to use for prompt optimization
    </ParamField>

    <ParamField body="experimentId" type="ID">
      Optional experiment ID to use as datasource. Only one experiment is supported for prompt optimization tasks. If not provided, the dataset will be used.
    </ParamField>

    <ParamField body="name" type="String!" required>
      The name of the prompt optimization task
    </ParamField>

    <ParamField body="runContinuously" type="Boolean!" required>
      Whether the online task runs continuously
    </ParamField>

    <ParamField body="llmConfig" type="OnlineTaskLLMConfigInput!" required>
      The LLM config for the online task

      <Expandable title="OnlineTaskLLMConfigInput fields">
        <ParamField body="integrationId" type="ID!" required>
          The selected named LLM integration id (relay global id). If provided, server will use its configuration.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          The LLM model
        </ParamField>

        <ParamField body="invocationParameters" type="JSONObject!" required>
          parameters used when running the Llm
        </ParamField>

        <ParamField body="providerParameters" type="JSONObject!" required>
          parameters used to initialize the Llm
        </ParamField>

        <ParamField body="temperature" type="Float">
          The temperature of the LLM config
        </ParamField>

        <ParamField body="provider" type="LLMIntegrationProvider">
          The LLM provider One of: `openAI`, `anthropic`, `awsBedrock`, `azureOpenAI`, `vertexAI`, `custom`, `nvidiaNim`, `gemini`, `litellm`, `fireworks`, `togetherAi`, `cursor`, `typeSafeAi`.
        </ParamField>

        <ParamField body="deploymentName" type="String">
          The deployment name of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiVersion" type="String">
          The api version of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiEndpoint" type="String">
          The api endpoint of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="region" type="String">
          The region of the LLM config
        </ParamField>

        <ParamField body="maxTokens" type="Float">
          The max tokens of the LLM config
        </ParamField>

        <ParamField body="maxCompletionTokens" type="Float">
          The max completion tokens of the LLM config
        </ParamField>

        <ParamField body="topP" type="Float">
          The top P of the LLM config
        </ParamField>

        <ParamField body="topK" type="Float">
          The top K of the LLM config
        </ParamField>

        <ParamField body="stop" type="[String!]">
          The stop associated with the config
        </ParamField>

        <ParamField body="customModelEndpointId" type="String">
          The id of the Custom Model Endpoint. Required if provider is custom.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="promptOptimizationConfig" type="PromptOptimizationConfigInput!" required>
      The prompt optimization config

      <Expandable title="PromptOptimizationConfigInput fields">
        <ParamField body="userInstructions" type="String!" required>
          The user instructions for the config
        </ParamField>

        <ParamField body="outputColumn" type="String!" required>
          The output column for the config
        </ParamField>

        <ParamField body="feedbackColumns" type="[String!]!" required>
          The feedback columns for the config
        </ParamField>

        <ParamField body="batchSize" type="Int!" required>
          The batch size for the config
        </ParamField>

        <ParamField body="promptId" type="ID!" required>
          The prompt id for the config
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="tracingEnabled" type="Boolean">
      Whether tracing is enabled for the online task
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="promptOptimizationTask" type="OnlineTask!">
  <Expandable title="OnlineTask fields">
    Scalar fields: `id`, `taskType`, `name`, `samplingRate`, `lastRunAt`, `queryFilter`, `createdAt`, `updatedAt`, `deletedAt`, `runContinuously`, `tracingEnabled`, `experimentIds`, `hasUnavailableEvaluatorIntegration`.

    Object fields (select subfields): `model`, `multiSpanQuery`, `createdBy`, `llmConfig`, `taskConfig`, `evaluators`, `onlineTaskEvaluators`, `allEvaluatorConfigs`, `taskRuns`, `experiments`, `dataset`, `postProcessingSteps`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation CreatePromptOptimizationTask($input: CreatePromptOptimizationTaskMutationInput!) {
    createPromptOptimizationTask(input: $input) {
      promptOptimizationTask { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "datasetId": "<ID>",
      "name": "<string>",
      "runContinuously": true,
      "llmConfig": {
        "integrationId": "<ID>",
        "modelName": "<string>",
        "invocationParameters": "<JSONObject>",
        "providerParameters": "<JSONObject>"
      },
      "promptOptimizationConfig": {
        "userInstructions": "<string>",
        "outputColumn": "<string>",
        "feedbackColumns": [
          "<string>"
        ],
        "batchSize": 1,
        "promptId": "<ID>"
      }
    }
  }
  ```
</CodeGroup>

### patchPromptOptimizationTask

`patchPromptOptimizationTask(input: PatchPromptOptimizationTaskMutationInput!): PatchPromptOptimizationTaskMutationPayload`

#### Arguments

<ParamField body="input" type="PatchPromptOptimizationTaskMutationInput!" required>
  <Expandable title="PatchPromptOptimizationTaskMutationInput fields">
    <ParamField body="onlineTaskId" type="ID!" required>
      The online task id for the eval task to be patched
    </ParamField>

    <ParamField body="datasetId" type="ID">
      The dataset id for the dataset to be evaluated
    </ParamField>

    <ParamField body="name" type="String">
      The name of the online task
    </ParamField>

    <ParamField body="runContinuously" type="Boolean">
      Whether the online task runs continuously
    </ParamField>

    <ParamField body="llmConfig" type="OnlineTaskLLMConfigInput">
      The LLM config for the online task

      <Expandable title="OnlineTaskLLMConfigInput fields">
        <ParamField body="integrationId" type="ID!" required>
          The selected named LLM integration id (relay global id). If provided, server will use its configuration.
        </ParamField>

        <ParamField body="modelName" type="String!" required>
          The LLM model
        </ParamField>

        <ParamField body="invocationParameters" type="JSONObject!" required>
          parameters used when running the Llm
        </ParamField>

        <ParamField body="providerParameters" type="JSONObject!" required>
          parameters used to initialize the Llm
        </ParamField>

        <ParamField body="temperature" type="Float">
          The temperature of the LLM config
        </ParamField>

        <ParamField body="provider" type="LLMIntegrationProvider">
          The LLM provider One of: `openAI`, `anthropic`, `awsBedrock`, `azureOpenAI`, `vertexAI`, `custom`, `nvidiaNim`, `gemini`, `litellm`, `fireworks`, `togetherAi`, `cursor`, `typeSafeAi`.
        </ParamField>

        <ParamField body="deploymentName" type="String">
          The deployment name of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiVersion" type="String">
          The api version of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="apiEndpoint" type="String">
          The api endpoint of the LLM config. Required for azure openai provider
        </ParamField>

        <ParamField body="region" type="String">
          The region of the LLM config
        </ParamField>

        <ParamField body="maxTokens" type="Float">
          The max tokens of the LLM config
        </ParamField>

        <ParamField body="maxCompletionTokens" type="Float">
          The max completion tokens of the LLM config
        </ParamField>

        <ParamField body="topP" type="Float">
          The top P of the LLM config
        </ParamField>

        <ParamField body="topK" type="Float">
          The top K of the LLM config
        </ParamField>

        <ParamField body="stop" type="[String!]">
          The stop associated with the config
        </ParamField>

        <ParamField body="customModelEndpointId" type="String">
          The id of the Custom Model Endpoint. Required if provider is custom.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="promptOptimizationConfig" type="PromptOptimizationConfigInput">
      The prompt optimization config

      <Expandable title="PromptOptimizationConfigInput fields">
        <ParamField body="userInstructions" type="String!" required>
          The user instructions for the config
        </ParamField>

        <ParamField body="outputColumn" type="String!" required>
          The output column for the config
        </ParamField>

        <ParamField body="feedbackColumns" type="[String!]!" required>
          The feedback columns for the config
        </ParamField>

        <ParamField body="batchSize" type="Int!" required>
          The batch size for the config
        </ParamField>

        <ParamField body="promptId" type="ID!" required>
          The prompt id for the config
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField body="tracingEnabled" type="Boolean">
      Whether tracing is enabled for the online task
    </ParamField>

    <ParamField body="clientMutationId" type="String" />
  </Expandable>
</ParamField>

#### Returns

<ResponseField name="promptOptimizationTask" type="OnlineTask!">
  <Expandable title="OnlineTask fields">
    Scalar fields: `id`, `taskType`, `name`, `samplingRate`, `lastRunAt`, `queryFilter`, `createdAt`, `updatedAt`, `deletedAt`, `runContinuously`, `tracingEnabled`, `experimentIds`, `hasUnavailableEvaluatorIntegration`.

    Object fields (select subfields): `model`, `multiSpanQuery`, `createdBy`, `llmConfig`, `taskConfig`, `evaluators`, `onlineTaskEvaluators`, `allEvaluatorConfigs`, `taskRuns`, `experiments`, `dataset`, `postProcessingSteps`.
  </Expandable>
</ResponseField>

#### Example

Only required input fields are shown. Replace `<ID>` and `<string>` placeholders with real values; the [object graph](/docs/ax/graphql-reference/queries/object-graph) page shows how to look IDs up.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation PatchPromptOptimizationTask($input: PatchPromptOptimizationTaskMutationInput!) {
    patchPromptOptimizationTask(input: $input) {
      promptOptimizationTask { id name }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "onlineTaskId": "<ID>"
    }
  }
  ```
</CodeGroup>