LLM integration mutations

Configure LLM integrations, provider keys, custom endpoints and Google Cloud. For task-oriented walkthroughs of these operations, see the LLM integration guide. Every mutation below is sent as a POST to https://app.arize.com/graphql with an x-api-key header; see Forming calls.

Mutations in this page

createLlmIntegration: Create llm integration.
updateLlmIntegration: Update llm integration.
deleteLlmIntegration: Delete llm integration.
createExternalLlmApiKey: Create external llm api key.
updateExternalLlmApiKey: Update external llm api key.
deleteExternalLlmApiKey: Delete external llm api key.
createCustomLlmEndpoint: Create custom llm endpoint.
updateCustomLlmEndpoint: Update custom llm endpoint.
deleteCustomLlmEndpoint: Delete custom llm endpoint.
createGoogleCloudIntegration: Create google cloud integration.
updateGoogleCloudIntegration: Update google cloud integration.
deleteGoogleCloudIntegration: Delete google cloud integration.

Reference

createLlmIntegration

createLlmIntegration(input: CreateLlmIntegrationInput!): CreateLlmIntegrationPayload

Arguments

CreateLlmIntegrationInput!

required

Show CreateLlmIntegrationInput fields

ID!

required

The global relay ID of the account

LlmProvider!

required

The name of the LLM provider One of: azureopenai, openai, anthropic, nvidiaNim, gemini, litellm, fireworks, togetherAi, googleCloud, aws, custom, cursor, typeSafeAi.

String!

required

The name of the integration

String

API key (will be encrypted)

String

Custom base URL for compatible providers

[String!]

Supported model names

Boolean

Enable default models for the provider

String

API compatibility format (e.g., ‘openai’)

JSON

Custom headers for requests

[LlmIntegrationScopingInput!]

Organization/Space scoping (defaults to account-wide if not provided)

Show LlmIntegrationScopingInput fields

The global relay organization ID if this integration is scoped to an organization

The global relay space ID if this integration is scoped to a specific space

JSON

Provider-specific configuration (AWS: roleArn/externalId/guardrailIdentifier/guardrailVersion, GCP: projectId/location/projectAccessLabel)

Boolean

Whether function calling is enabled for the integration

LlmIntegrationAuthType

Authentication method One of: default, proxy_with_headers, bearer_token, oauth2_client_credentials.

LlmIntegrationOauthConfigInput

OAuth 2.0 client-credentials configuration. Required when authType is oauth2_client_credentials, rejected otherwise.

Show LlmIntegrationOauthConfigInput fields

String

Token endpoint used for the client-credentials exchange. Required when enabling OAuth.

String

OAuth client ID. Required when enabling OAuth.

String

OAuth client secret (will be encrypted). Required when enabling OAuth; omit to keep the stored value.

String

Space-separated scopes. Pass null or empty string to clear, omit to keep.

String

Audience for identity providers that require one. Pass null or empty string to clear, omit to keep.

Boolean

Allow this provider to be reused for agent execution (Agent Integrations). Separate from playground/eval use.

String

Returns

LlmIntegration!

The created LLM integration

Show LlmIntegration fields

Scalar fields: id, name, provider, apiKey, hasApiKey, baseUrl, modelNames, modelsError, compatibleFormat, headers, createdAt, updatedAt, hasUnauthorizedScopings, enableDefaultModels, functionCallingEnabled, authType, allowUseInAgents, providerMetadata.Object fields (select subfields): models, creator, scopings, oauthConfig.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation CreateLlmIntegration($input: CreateLlmIntegrationInput!) {
  createLlmIntegration(input: $input) {
    llmIntegration { id name }
  }
}

{
  "input": {
    "accountId": "<ID>",
    "provider": "azureopenai",
    "name": "<string>"
  }
}

updateLlmIntegration

updateLlmIntegration(input: UpdateLlmIntegrationInput!): UpdateLlmIntegrationPayload

Arguments

UpdateLlmIntegrationInput!

required

Show UpdateLlmIntegrationInput fields

ID!

required

The global relay ID of the LLM integration to update

LlmProvider

The name of the LLM provider One of: azureopenai, openai, anthropic, nvidiaNim, gemini, litellm, fireworks, togetherAi, googleCloud, aws, custom, cursor, typeSafeAi.

String

The name of the integration

String

API key (will be encrypted). Pass null or empty string to remove. Omit to keep existing value.

String

Custom base URL for compatible providers

[String!]

Supported model names

Boolean

Enable Arize default models. Omit to keep the existing value.

Boolean

Whether function calling is enabled for the integration

String

API compatibility format (e.g., ‘openai’)

JSON

Custom headers for requests

[LlmIntegrationScopingInput!]

Organization/space scoping. If provided, replaces all existing scoping.

Show LlmIntegrationScopingInput fields

The global relay organization ID if this integration is scoped to an organization

The global relay space ID if this integration is scoped to a specific space

JSON

Provider-specific configuration (AWS: roleArn/externalId/guardrailIdentifier/guardrailVersion, GCP: projectId/location/projectAccessLabel)

LlmIntegrationAuthType

Authentication method One of: default, proxy_with_headers, bearer_token, oauth2_client_credentials.

LlmIntegrationOauthConfigInput

OAuth 2.0 client-credentials configuration. Omit to keep the stored config; switching authType away from OAuth deletes it.

Show LlmIntegrationOauthConfigInput fields

String

Token endpoint used for the client-credentials exchange. Required when enabling OAuth.

String

OAuth client ID. Required when enabling OAuth.

String

OAuth client secret (will be encrypted). Required when enabling OAuth; omit to keep the stored value.

String

Space-separated scopes. Pass null or empty string to clear, omit to keep.

String

Audience for identity providers that require one. Pass null or empty string to clear, omit to keep.

Boolean

Allow this provider to be reused for agent execution. Omit to keep the existing value.

String

Returns

LlmIntegration!

The updated LLM integration

Show LlmIntegration fields

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation UpdateLlmIntegration($input: UpdateLlmIntegrationInput!) {
  updateLlmIntegration(input: $input) {
    llmIntegration { id name }
  }
}

{
  "input": {
    "integrationId": "<ID>"
  }
}

deleteLlmIntegration

deleteLlmIntegration(input: DeleteLlmIntegrationInput!): DeleteLlmIntegrationPayload

Arguments

DeleteLlmIntegrationInput!

required

Show DeleteLlmIntegrationInput fields

ID!

required

The global relay ID of the LLM integration to delete

String

Returns

Boolean!

Whether the deletion was successful

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation DeleteLlmIntegration($input: DeleteLlmIntegrationInput!) {
  deleteLlmIntegration(input: $input) {
    success
  }
}

{
  "input": {
    "integrationId": "<ID>"
  }
}

createExternalLlmApiKey

createExternalLlmApiKey(input: CreateExternalLlmApiKeyInput!): CreateExternalLlmApiKeyPayload

Arguments

CreateExternalLlmApiKeyInput!

required

Show CreateExternalLlmApiKeyInput fields

ID!

required

The global relay id of the organization to create the external llm api key in

String!

required

The API key to use for the external llm, will be encrypted before saving

ExternalLlmApiKeyProvider!

required

The LLM provider One of: openai, azureopenai, anthropic, nvidiaNim, gemini, litellm, fireworks, togetherAi.

String

Returns

ExternalLlmApiKey!

The created llm api key

Show ExternalLlmApiKey fields

Scalar fields: id, createdAt, provider.Object fields (select subfields): creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation CreateExternalLlmApiKey($input: CreateExternalLlmApiKeyInput!) {
  createExternalLlmApiKey(input: $input) {
    externalLlmApiKey { id }
  }
}

{
  "input": {
    "accountOrganizationId": "<ID>",
    "apiKey": "<string>",
    "provider": "openai"
  }
}

updateExternalLlmApiKey

updateExternalLlmApiKey(input: UpdateExternalLlmApiKeyInput!): UpdateExternalLlmApiKeyPayload

Arguments

UpdateExternalLlmApiKeyInput!

required

Show UpdateExternalLlmApiKeyInput fields

ID!

required

The global relay id for the llm api key to update

String!

required

The new llm api key

String

Returns

ExternalLlmApiKey!

The updated llm api key

Show ExternalLlmApiKey fields

Scalar fields: id, createdAt, provider.Object fields (select subfields): creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation UpdateExternalLlmApiKey($input: UpdateExternalLlmApiKeyInput!) {
  updateExternalLlmApiKey(input: $input) {
    externalLlmApiKey { id }
  }
}

{
  "input": {
    "externalLlmApiKeyId": "<ID>",
    "apiKey": "<string>"
  }
}

deleteExternalLlmApiKey

deleteExternalLlmApiKey(input: DeleteExternalLlmApiKeyInput!): DeleteExternalLlmApiKeyPayload

Arguments

DeleteExternalLlmApiKeyInput!

required

Show DeleteExternalLlmApiKeyInput fields

ID!

required

The global relay id for the llm api key to delete

String

Returns

ExternalLlmApiKey!

The deleted llm api key

Show ExternalLlmApiKey fields

Scalar fields: id, createdAt, provider.Object fields (select subfields): creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation DeleteExternalLlmApiKey($input: DeleteExternalLlmApiKeyInput!) {
  deleteExternalLlmApiKey(input: $input) {
    externalLlmApiKey { id }
  }
}

{
  "input": {
    "externalLlmApiKeyId": "<ID>"
  }
}

createCustomLlmEndpoint

createCustomLlmEndpoint(input: CreateCustomLlmEndpointInput!): CreateCustomLlmEndpointPayload

Arguments

CreateCustomLlmEndpointInput!

required

Show CreateCustomLlmEndpointInput fields

ID!

required

The global relay id of the organization in which to create the custom llm endpoint

String!

required

The name of the custom llm endpoint

String!

required

The exact name of the provider’s model or of the model you are hosting

String!

required

The base url of your custom llm endpoint

String

An optional api key used to access your openai compatible endpoint

[CustomLlmEndpointHeadersInput]

A list of key value pairs representing HTTP headers that should be sent when making an HTTP request to the custom llm endpoint

Show CustomLlmEndpointHeadersInput fields

String!

required

The name of the header

String!

required

The value of the header

ExternalLlmApiKeyProvider!

required

The selected compatible format dictates the API that the custom endpoint gets used with. Currently only OpenAI chat completion or an Azure OpenAI request are supported. Defaults to OpenAI. One of: openai, azureopenai, anthropic, nvidiaNim, gemini, litellm, fireworks, togetherAi.

String

Returns

CustomLlmEndpoint!

The created custom llm endpoint

Show CustomLlmEndpoint fields

Scalar fields: id, accountOrganizationId, name, modelName, baseUrl, hasApiKey, createdAt, updatedAt, deletedAt, compatibleFormat.Object fields (select subfields): headers, creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation CreateCustomLlmEndpoint($input: CreateCustomLlmEndpointInput!) {
  createCustomLlmEndpoint(input: $input) {
    customLlmEndpoint { id name }
  }
}

{
  "input": {
    "accountOrganizationId": "<ID>",
    "name": "<string>",
    "modelName": "<string>",
    "baseUrl": "<string>"
  }
}

updateCustomLlmEndpoint

updateCustomLlmEndpoint(input: UpdateCustomLlmEndpointInput!): UpdateCustomLlmEndpointPayload

Arguments

UpdateCustomLlmEndpointInput!

required

Show UpdateCustomLlmEndpointInput fields

ID!

required

The global relay id of the custom llm endpoint to update

String

The name of the custom llm endpoint

String

The exact name of the provider’s model or of the model you are hosting

String

The base url of your custom llm endpoint

String

An optional api key used to access your openai compatible endpoint

[CustomLlmEndpointHeadersInput]

A list of key value pairs representing HTTP headers that should be sent when making an HTTP request to the custom llm endpoint

Show CustomLlmEndpointHeadersInput fields

String!

required

The name of the header

String!

required

The value of the header

ExternalLlmApiKeyProvider

String

Returns

CustomLlmEndpoint!

The updated custom llm endpoint

Show CustomLlmEndpoint fields

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation UpdateCustomLlmEndpoint($input: UpdateCustomLlmEndpointInput!) {
  updateCustomLlmEndpoint(input: $input) {
    customLlmEndpoint { id name }
  }
}

{
  "input": {
    "customLlmEndpointId": "<ID>"
  }
}

deleteCustomLlmEndpoint

deleteCustomLlmEndpoint(input: DeleteCustomLlmEndpointInput!): DeleteCustomLlmEndpointPayload

Arguments

DeleteCustomLlmEndpointInput!

required

Show DeleteCustomLlmEndpointInput fields

ID!

required

The global relay id of the custom llm endpoint to delete

String

Returns

CustomLlmEndpoint!

The deleted custom llm endpoint

Show CustomLlmEndpoint fields

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation DeleteCustomLlmEndpoint($input: DeleteCustomLlmEndpointInput!) {
  deleteCustomLlmEndpoint(input: $input) {
    customLlmEndpoint { id name }
  }
}

{
  "input": {
    "customLlmEndpointId": "<ID>"
  }
}

createGoogleCloudIntegration

createGoogleCloudIntegration(input: CreateGoogleCloudIntegrationInput!): CreateGoogleCloudIntegrationPayload

Arguments

CreateGoogleCloudIntegrationInput!

required

Show CreateGoogleCloudIntegrationInput fields

ID!

required

The global relay id of the organization to create the integration in

String!

required

The ID of the Google Cloud project to integrate with

String!

required

The Google Cloud location to use for the integration

String!

required

The value of the arize-integration-key label set on the project to use to verify access to the project. Set the key to arize-integration-key and the value to this value in GCP.

String

Returns

GoogleCloudIntegration!

The created integration

Show GoogleCloudIntegration fields

Scalar fields: id, projectId, location, projectAccessLabel, createdAt, updatedAt.Object fields (select subfields): creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation CreateGoogleCloudIntegration($input: CreateGoogleCloudIntegrationInput!) {
  createGoogleCloudIntegration(input: $input) {
    googleCloudIntegration { id }
  }
}

{
  "input": {
    "accountOrganizationId": "<ID>",
    "projectId": "<string>",
    "location": "<string>",
    "projectAccessLabel": "<string>"
  }
}

updateGoogleCloudIntegration

updateGoogleCloudIntegration(input: UpdateGoogleCloudIntegrationInput!): UpdateGoogleCloudIntegrationPayload

Arguments

UpdateGoogleCloudIntegrationInput!

required

Show UpdateGoogleCloudIntegrationInput fields

ID!

required

The global relay id for the llm api key to update

String

The new ID of the Google Cloud project to integrate with

String

The new Google Cloud location to use for the integration

String

The value of the arize-integration-key label set on the project to use to verify access to the project.Set the key to arize-integration-key and the value to this value in GCP.

String

Returns

GoogleCloudIntegration!

The updated llm api key

Show GoogleCloudIntegration fields

Scalar fields: id, projectId, location, projectAccessLabel, createdAt, updatedAt.Object fields (select subfields): creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation UpdateGoogleCloudIntegration($input: UpdateGoogleCloudIntegrationInput!) {
  updateGoogleCloudIntegration(input: $input) {
    googleCloudIntegration { id }
  }
}

{
  "input": {
    "googleCloudIntegrationId": "<ID>"
  }
}

deleteGoogleCloudIntegration

deleteGoogleCloudIntegration(input: DeleteGoogleCloudIntegrationInput!): DeleteGoogleCloudIntegrationPayload

Arguments

DeleteGoogleCloudIntegrationInput!

required

Show DeleteGoogleCloudIntegrationInput fields

ID!

required

The global relay id for the google cloud integration to delete

String

Returns

GoogleCloudIntegration!

The google cloud integration

Show GoogleCloudIntegration fields

Scalar fields: id, projectId, location, projectAccessLabel, createdAt, updatedAt.Object fields (select subfields): creator.

Example

Only required input fields are shown. Replace <ID> and <string> placeholders with real values; the object graph page shows how to look IDs up.

mutation DeleteGoogleCloudIntegration($input: DeleteGoogleCloudIntegrationInput!) {
  deleteGoogleCloudIntegration(input: $input) {
    googleCloudIntegration { id }
  }
}

{
  "input": {
    "googleCloudIntegrationId": "<ID>"
  }
}

​Mutations in this page

​Reference

​createLlmIntegration

​Arguments

​Returns

​Example

​updateLlmIntegration

​Arguments

​Returns

​Example

​deleteLlmIntegration

​Arguments

​Returns

​Example

​createExternalLlmApiKey

​Arguments

​Returns

​Example

​updateExternalLlmApiKey

​Arguments

​Returns

​Example

​deleteExternalLlmApiKey

​Arguments

​Returns

​Example

​createCustomLlmEndpoint

​Arguments

​Returns

​Example

​updateCustomLlmEndpoint

​Arguments

​Returns

​Example

​deleteCustomLlmEndpoint

​Arguments

​Returns

​Example

​createGoogleCloudIntegration

​Arguments

​Returns

​Example

​updateGoogleCloudIntegration

​Arguments

​Returns

​Example

​deleteGoogleCloudIntegration

​Arguments

​Returns

​Example

Mutations in this page

Reference

createLlmIntegration

Arguments

Returns

Example

updateLlmIntegration

Arguments

Returns

Example

deleteLlmIntegration

Arguments

Returns

Example

createExternalLlmApiKey

Arguments

Returns

Example

updateExternalLlmApiKey

Arguments

Returns

Example

deleteExternalLlmApiKey

Arguments

Returns

Example

createCustomLlmEndpoint

Arguments

Returns

Example

updateCustomLlmEndpoint

Arguments

Returns

Example

deleteCustomLlmEndpoint

Arguments

Returns

Example

createGoogleCloudIntegration

Arguments

Returns

Example

updateGoogleCloudIntegration

Arguments

Returns

Example

deleteGoogleCloudIntegration

Arguments

Returns

Example