> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Managing LLM integrations with GraphQL

> Configure AI provider integrations, external LLM API keys, custom OpenAI-compatible endpoints and Google Cloud integrations with the Arize GraphQL API.

[AI provider integrations](/docs/ax/security-and-settings/integrations-playground/overview) connect Arize AX to an LLM provider so you can run [prompt playground](/docs/ax/improve/test-a-prompt) sessions, [LLM-as-a-judge evaluations](/docs/ax/evaluate/evaluators/llm-as-a-judge) and [managed agents](/docs/ax/agents/connect-your-harness) without pasting a key into every tool. This guide covers the mutations for creating and rotating those integrations, and the separate, narrower credential records the schema also exposes.

## Find the IDs you need

LLM integrations hang off an account, an organization, or a space. Start from `viewer` for the space ID and `account` for the account and organization IDs. See [using global node IDs](/docs/ax/graphql-reference/overview/how-to-use-graphql/using-global-node-ids) for how these opaque IDs work.

<CodeGroup>
  ```graphql Query theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  query FindAccountOrgAndSpaceIds {
    account {
      id
      name
      organizations(first: 10) {
        edges {
          node { id name }
        }
      }
    }
    viewer {
      spaces(first: 20) {
        edges {
          node { id name }
        }
      }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {}
  ```
</CodeGroup>

## Understand the three resource types

1. **`LlmIntegration`**: backs the AI Providers UI, prompt playground, evals and agent runtimes. It carries its own `apiKey`, `baseUrl` and `modelNames`, is scoped via `scopings`, and is listed on `Account.llmIntegrations` and `Space.llmIntegrations`.
2. **`ExternalLlmApiKey`** and **`CustomLlmEndpoint`**: narrower records scoped by `accountOrganizationId` directly (no further scoping, unlike `LlmIntegration`'s `scopings`), listed on `AccountOrganization.externalLlmApiKeys` and `.customLlmEndpoints`. Nothing links either back to an `LlmIntegration`; creating one does not create a usable playground integration.
3. **`GoogleCloudIntegration`**: an organization-scoped record that verifies access to a Google Cloud project. It has no query field, so the only way to see one again is the ID the mutation returned.

For most automation, create an `LlmIntegration` directly with `createLlmIntegration`. Reach for `ExternalLlmApiKey` or `CustomLlmEndpoint` only if you specifically need that narrower record.

## List configured LLM integrations

List the integrations available to the account and to a space. Space-level results include integrations scoped to that space, its organization, or the whole account.

<CodeGroup>
  ```graphql Query theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  query ListLlmIntegrations($spaceId: ID!) {
    account {
      llmIntegrations {
        id
        name
        provider
        hasApiKey
      }
    }
    node(id: $spaceId) {
      ... on Space {
        llmIntegrations {
          id
          name
          provider
          allowUseInAgents
        }
      }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "spaceId": "U3BhY2U6MTIz"
  }
  ```
</CodeGroup>

Never select `apiKey` in a listing query; use `hasApiKey` to check whether a key is set without round-tripping the secret.

## Add a provider API key and connect it as an LLM integration

An `ExternalLlmApiKey` only stores a raw credential at the organization level. To actually use the provider in the playground, evals or agents, create an `LlmIntegration` as a separate step, passing the key straight to it.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation AddExternalApiKey($input: CreateExternalLlmApiKeyInput!) {
    createExternalLlmApiKey(input: $input) {
      externalLlmApiKey { id provider createdAt }
    }
  }

  mutation CreateIntegrationFromKey($input: CreateLlmIntegrationInput!) {
    createLlmIntegration(input: $input) {
      llmIntegration { id name provider hasApiKey }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "accountOrganizationId": "QWNjb3VudE9yZ2FuaXphdGlvbjo0",
      "apiKey": "sk-ant-REPLACE_ME",
      "provider": "anthropic"
    }
  }
  ```
</CodeGroup>

The second mutation's `input` reuses the same key: `{ "accountId": "QWNjb3VudDo5", "provider": "anthropic", "name": "Team Anthropic key", "apiKey": "sk-ant-REPLACE_ME", "enableDefaultModels": true, "scopings": [{ "spaceId": "U3BhY2U6MTIz" }] }`. Omit `scopings` for an account-wide integration. For AWS or GCP, add `providerMetadata` (AWS: `roleArn`, `externalId`; GCP: `projectId`, `location`, `projectAccessLabel`).

Reference: [`createExternalLlmApiKey`](/docs/ax/graphql-reference/mutations/llm-integrations#createexternalllmapikey), [`createLlmIntegration`](/docs/ax/graphql-reference/mutations/llm-integrations#createllmintegration).

## Register a custom OpenAI-compatible endpoint

Point Arize at a self-hosted or third-party OpenAI-compatible model server with `createCustomLlmEndpoint`, or use `createLlmIntegration` with `provider: "custom"` and a `baseUrl` if you want it usable directly in the playground.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation RegisterCustomEndpoint($input: CreateCustomLlmEndpointInput!) {
    createCustomLlmEndpoint(input: $input) {
      customLlmEndpoint { id name modelName baseUrl hasApiKey }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "accountOrganizationId": "QWNjb3VudE9yZ2FuaXphdGlvbjo0",
      "name": "Internal Llama endpoint",
      "modelName": "llama-3.3-70b",
      "baseUrl": "https://my-llm-proxy.example.com/v1",
      "apiKey": "REPLACE_ME",
      "compatibleFormat": "openai"
    }
  }
  ```
</CodeGroup>

Enter the base URL including the version path (for example, `/v1`) but without an endpoint path like `/chat/completions`; Arize appends that automatically. Add `headers` (a list of `{ key, value }` pairs) for anything else your proxy requires.

Reference: [`createCustomLlmEndpoint`](/docs/ax/graphql-reference/mutations/llm-integrations#createcustomllmendpoint).

## Set up a Google Cloud (Vertex) integration

Verify access to a Google Cloud project before using Vertex AI models. Set an `arize-integration-key` label on the GCP project first, then register that same value as `projectAccessLabel`.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation SetUpGoogleCloud($input: CreateGoogleCloudIntegrationInput!) {
    createGoogleCloudIntegration(input: $input) {
      googleCloudIntegration { id projectId location }
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "accountOrganizationId": "QWNjb3VudE9yZ2FuaXphdGlvbjo0",
      "projectId": "my-gcp-project",
      "location": "us-central1",
      "projectAccessLabel": "my-arize-integration-key-value"
    }
  }
  ```
</CodeGroup>

<Warning>
  `GoogleCloudIntegration` has no list query and does not implement `Node`, so it cannot be refetched by ID either. Save the `id` this mutation returns. To use Vertex models in the playground or evals, also create an `LlmIntegration` with `provider: "googleCloud"` and the same `projectId`/`location`/`projectAccessLabel` in `providerMetadata`.
</Warning>

Reference: [`createGoogleCloudIntegration`](/docs/ax/graphql-reference/mutations/llm-integrations#creategooglecloudintegration).

## Rotate or remove a key

Update the stored key on an `LlmIntegration` without recreating it, or delete the integration outright once it is no longer in use.

<CodeGroup>
  ```graphql Mutation theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  mutation RotateIntegrationKey($input: UpdateLlmIntegrationInput!) {
    updateLlmIntegration(input: $input) {
      llmIntegration { id hasApiKey updatedAt }
    }
  }

  mutation RemoveIntegration($input: DeleteLlmIntegrationInput!) {
    deleteLlmIntegration(input: $input) {
      success
    }
  }
  ```

  ```json Variables theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
  {
    "input": {
      "integrationId": "TGxtSW50ZWdyYXRpb246NTU=",
      "apiKey": "sk-proj-NEW_KEY"
    }
  }
  ```
</CodeGroup>

Passing `null` or an empty string for `apiKey` on `updateLlmIntegration` removes the stored key instead of rotating it. `updateExternalLlmApiKey`, `deleteExternalLlmApiKey`, `deleteCustomLlmEndpoint` and `deleteGoogleCloudIntegration` follow the same shape for the narrower resource types.

Reference: [`updateLlmIntegration`](/docs/ax/graphql-reference/mutations/llm-integrations#updatellmintegration), [`deleteLlmIntegration`](/docs/ax/graphql-reference/mutations/llm-integrations#deletellmintegration).

## Gotchas and behavior notes

<AccordionGroup>
  <Accordion title="The input provider enum and the output provider enum do not match">
    `createLlmIntegration` and `updateLlmIntegration` take `provider: LlmProvider` (lowercase values like `openai`, `azureopenai`, `aws`, `googleCloud`). Reading an integration back returns `provider: LLMIntegrationProvider`, a differently-cased and differently-spelled enum (`openAI`, `azureOpenAI`, `awsBedrock`, `vertexAI`). Do not echo a value you read straight back into a mutation.
  </Accordion>

  <Accordion title="oauthConfig is required when enabling OAuth, and omitting it keeps the existing config">
    On `updateLlmIntegration`, switching `authType` to `oauth2_client_credentials` requires `oauthConfig` with `tokenUrl`, `clientId` and `clientSecret`; switching away from it deletes the stored config. Omitting `oauthConfig` while `authType` is unchanged keeps whatever is already stored.
  </Accordion>

  <Accordion title="compatibleFormat uses a narrower provider enum">
    `createCustomLlmEndpoint` and `createLlmIntegration` both type `compatibleFormat` as `ExternalLlmApiKeyProvider`, which has only 8 values and does not include `custom`, `googleCloud`, `aws`, `cursor` or `typeSafeAi`. It defaults to `openai`.
  </Accordion>
</AccordionGroup>

<CardGroup cols={3}>
  <Card title="LLM integration mutations" href="/docs/ax/graphql-reference/mutations/llm-integrations" icon="book">
    Full argument and return-type reference for all twelve mutations.
  </Card>

  <Card title="All mutations" href="/docs/ax/graphql-reference/mutations" icon="list">
    Index of every GraphQL mutation grouped by domain.
  </Card>

  <Card title="API explorer" href="/docs/ax/graphql-reference/overview/api-explorer" icon="terminal">
    Run queries and mutations interactively against your own account.
  </Card>
</CardGroup>