Webhooks

A webhook sends an HTTP POST request to your endpoint when something changes on a prompt or evaluator in Arize AX. Use webhooks to start a deploy when a prompt version gets the production label, sync prompt versions to your own store, or tell a team when an evaluator changes.

The webhooks REST endpoints and SDK methods are in alpha. Names and fields may change. See API version stages.

How webhooks work

Arize AX splits a webhook into two parts:

Webhook: where to send events. It holds the endpoint URL, how Arize AX authenticates to it, a timeout, and optional custom headers. A webhook belongs to an organization, and any prompt or evaluator in that organization can use it.
Subscription: what to send. A subscription links one webhook to one event on one prompt or evaluator. To send three events from a prompt to the same webhook, create three subscriptions.

A webhook receives nothing until you subscribe it to at least one event.

Events

API value	`event` in the payload	Source type	Sent when
`PROMPT_VERSION_CREATED`	`prompt.version.created`	`PROMPT`	Someone saves a new version of the prompt.
`PROMPT_VERSION_LABELED`	`prompt.version.labeled`	`PROMPT`	Someone adds a label to a prompt version. Moving a label to a different version sends this event for the new version.
`PROMPT_VERSION_UNLABELED`	`prompt.version.unlabeled`	`PROMPT`	Someone removes a label from a prompt version.
`EVALUATOR_VERSION_CREATED`	`evaluator.version.created`	`EVALUATOR`	Someone saves a new version of the evaluator.

Use the API value when you create a subscription. Your endpoint receives the dotted value in the payload event field.

Limits

A webhook can have at most 20 custom headers.
The delivery timeout is between 1,000 and 60,000 ms. The default is 30,000 ms.
At most 200 webhooks can subscribe to the same event on one prompt or evaluator.
A webhook can subscribe to a given event on a given source only once.
Webhook names must be unique within an organization.

Permissions

Action	Who can do it
View webhooks	Any organization role, including Read-only Member
Create a webhook, send a test, or view delivery history	Organization Admin or Member
Update or delete a webhook	Organization Admins, and the Member who created the webhook
Add or remove a subscription	Anyone who can edit the prompt or evaluator, such as a space Admin or Member

Account Admins can do everything an Organization Admin can. See Role-based access control for role details.

Create a webhook

Go to Settings, then under Org Settings select Alert Integrations.
Click the Webhooks Integration card.
Click Add New.
Enter a Name (unique in the organization) and the URL of your endpoint.
Optionally add a Description, an Authorization Token, and custom headers with Add Custom Headers.
Click Register New Webhook.

Webhooks you create in the UI use bearer authentication. To create an HMAC-signed webhook, use the REST API, an SDK, or the CLI.

curl -X POST https://api.arize.com/v2/webhooks \
  -H "Authorization: Bearer $ARIZE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "organization_id": "YOUR_ORG_ID",
    "name": "prompt-changes",
    "url": "https://example.com/hooks/arize",
    "auth_type": "HMAC_SHA256"
  }'

The REST API takes an organization ID. Find it with ax organizations list. See Create a webhook for every field.

from arize import ArizeClient
from arize.webhooks.types import WebhookAuthType

client = ArizeClient(api_key="YOUR_API_KEY")

webhook = client.webhooks.create(
    organization="YOUR_ORG_NAME_OR_ID",
    name="prompt-changes",
    url="https://example.com/hooks/arize",
    auth_type=WebhookAuthType.HMAC_SHA256,
)

print(webhook.id)
print(webhook.signing_secret)  # shown only once

See the Python SDK webhooks reference.

import { createWebhook } from "@arizeai/ax-client";

const webhook = await createWebhook({
  organization: "YOUR_ORG_NAME_OR_ID",
  name: "prompt-changes",
  url: "https://example.com/hooks/arize",
  authType: "HMAC_SHA256",
});

console.log(webhook.id);
console.log(webhook.signingSecret); // shown only once

See the TypeScript SDK webhooks reference.

wh, err := client.Webhooks.Create(ctx, webhooks.CreateRequest{
    Organization: "YOUR_ORG_NAME_OR_ID",
    Name:         "prompt-changes",
    URL:          "https://example.com/hooks/arize",
    AuthType:     webhooks.WebhookAuthTypeHMACSHA256,
})
if err != nil {
    log.Fatal(err)
}

fmt.Println(wh.Id)
if wh.SigningSecret != nil {
    fmt.Println(*wh.SigningSecret) // shown only once
}

client comes from arize.NewClient. See the Go SDK webhooks reference.

ax webhooks create \
  --organization YOUR_ORG_NAME_OR_ID \
  --name prompt-changes \
  --url https://example.com/hooks/arize \
  --auth-type HMAC_SHA256

See the ax CLI reference.

Arize AX does not send requests to private, loopback, or internal network addresses, and it does not follow redirects. Your endpoint must be reachable from the public internet at the exact URL you register.

Authenticate deliveries

Choose an auth type when you create a webhook. You cannot change it later.

Bearer token

With BEARER (the default), Arize AX sends your token as the Authorization header on every delivery. It sends the value exactly as you enter it, so include the scheme your endpoint expects:

Authorization: Bearer my-secret-token

The token is write-only. No API response or UI page shows it after you save it. You can replace or clear it when you update the webhook.

HMAC signature

With HMAC_SHA256, Arize AX creates a signing secret and signs every delivery with it. Arize AX returns the secret only once, in the create response.

Copy the signing secret as soon as you create the webhook and store it somewhere safe. Arize AX cannot show it again, and you cannot rotate it. If you lose it, delete the webhook and create a new one.

After creation, the webhook shows only a signing_secret_hint such as whsec_…abcd so you can tell which secret it uses. Each signed delivery carries these headers:

Header	Value
`X-Arize-Webhook-Id`	The event ID. It matches `event_uuid` in the payload and stays the same across retries.
`X-Arize-Webhook-Event`	The event, for example `prompt.version.labeled`.
`X-Arize-Webhook-Timestamp`	The time Arize AX sent this request, in Unix seconds.
`X-Arize-Webhook-Signature`	`v1=` followed by the lowercase hex HMAC-SHA256 of `<timestamp>.<raw request body>`.

To verify a delivery:

Read the raw request body as bytes, before any JSON parsing.
Join the X-Arize-Webhook-Timestamp value, a ., and the raw body.
Compute HMAC-SHA256 of that string. Use the full signing secret, including the whsec_ prefix, as the key.
Hex-encode the result, add v1= in front, and compare it to X-Arize-Webhook-Signature with a constant-time compare.
Reject requests whose timestamp is too old for your needs, for example more than five minutes. Each retry carries a fresh timestamp and signature.

Python
Node.js

import hashlib
import hmac
import os
import time

from flask import Flask, abort, request

app = Flask(__name__)
SIGNING_SECRET = os.environ["ARIZE_WEBHOOK_SECRET"]
MAX_AGE_SECONDS = 300


def is_valid(secret: str, timestamp: str, body: bytes, signature: str) -> bool:
    signed = timestamp.encode() + b"." + body
    digest = hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest()
    return hmac.compare_digest("v1=" + digest, signature)


@app.post("/hooks/arize")
def arize_webhook():
    timestamp = request.headers.get("X-Arize-Webhook-Timestamp", "")
    signature = request.headers.get("X-Arize-Webhook-Signature", "")
    body = request.get_data()

    if not timestamp.isdigit() or abs(time.time() - int(timestamp)) > MAX_AGE_SECONDS:
        abort(400)
    if not is_valid(SIGNING_SECRET, timestamp, body, signature):
        abort(401)

    event = request.get_json()
    print(event["event"], event["event_uuid"])
    return "", 204

import crypto from "node:crypto";
import express from "express";

const app = express();
const SIGNING_SECRET = process.env.ARIZE_WEBHOOK_SECRET;
const MAX_AGE_SECONDS = 300;

function isValid(secret, timestamp, body, signature) {
  const digest = crypto
    .createHmac("sha256", secret)
    .update(`${timestamp}.`)
    .update(body)
    .digest("hex");
  const expected = Buffer.from(`v1=${digest}`);
  const received = Buffer.from(signature);
  return (
    expected.length === received.length &&
    crypto.timingSafeEqual(expected, received)
  );
}

app.post(
  "/hooks/arize",
  express.raw({ type: "application/json" }),
  (req, res) => {
    const timestamp = req.get("X-Arize-Webhook-Timestamp") ?? "";
    const signature = req.get("X-Arize-Webhook-Signature") ?? "";
    const age = Math.abs(Date.now() / 1000 - Number(timestamp));

    if (!/^\d+$/.test(timestamp) || age > MAX_AGE_SECONDS) {
      return res.sendStatus(400);
    }
    if (!isValid(SIGNING_SECRET, timestamp, req.body, signature)) {
      return res.sendStatus(401);
    }

    const event = JSON.parse(req.body.toString("utf8"));
    console.log(event.event, event.event_uuid);
    res.sendStatus(204);
  },
);

app.listen(3000);

Custom headers

Both auth types can send up to 20 custom headers, such as a routing key. Header values are write-only. Arize AX rejects connection headers such as Host and Content-Length. It always sets Content-Type: application/json, the bearer token when one is set, and the signature headers, so these values win over custom headers with the same names.

Open a prompt in Prompt Hub, or open an evaluator.
Click Webhooks.
Point to a webhook to open its Events menu.
Select the events to send. For a prompt, choose from Version created, Version labeled, and Version unlabeled. For an evaluator, choose Version created.

Arize AX saves each change right away. The Webhooks button shows how many webhooks the prompt or evaluator sends to. To add a destination from this menu, click Add webhook.

curl -X POST https://api.arize.com/v2/webhook-subscriptions \
  -H "Authorization: Bearer $ARIZE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_id": "YOUR_WEBHOOK_ID",
    "source_type": "PROMPT",
    "source_id": "YOUR_PROMPT_ID",
    "event": "PROMPT_VERSION_LABELED"
  }'

See Create a webhook subscription.

from arize.webhooks.types import WebhookEventType, WebhookSourceType

subscription = client.webhooks.create_subscription(
    webhook="prompt-changes",
    organization="YOUR_ORG_NAME_OR_ID",
    source_type=WebhookSourceType.PROMPT,
    source_id="YOUR_PROMPT_ID",
    event=WebhookEventType.PROMPT_VERSION_LABELED,
)

print(subscription.id)

import { createWebhookSubscription } from "@arizeai/ax-client";

const subscription = await createWebhookSubscription({
  webhook: "prompt-changes",
  organization: "YOUR_ORG_NAME_OR_ID",
  sourceType: "PROMPT",
  sourceId: "YOUR_PROMPT_ID",
  event: "PROMPT_VERSION_LABELED",
});

sub, err := client.Webhooks.CreateSubscription(ctx, webhooks.CreateSubscriptionRequest{
    Webhook:      "prompt-changes",
    Organization: "YOUR_ORG_NAME_OR_ID",
    SourceType:   webhooks.WebhookSourceTypePrompt,
    SourceID:     "YOUR_PROMPT_ID",
    Event:        webhooks.WebhookEventTypePromptVersionLabeled,
})
if err != nil {
    log.Fatal(err)
}

fmt.Println(sub.Id)

ax webhooks subscriptions create \
  --webhook prompt-changes \
  --organization YOUR_ORG_NAME_OR_ID \
  --source-type PROMPT \
  --source-id YOUR_PROMPT_ID \
  --event PROMPT_VERSION_LABELED

The event must match the source type: prompt events for PROMPT sources and EVALUATOR_VERSION_CREATED for EVALUATOR sources. The webhook must belong to the same organization as the prompt or evaluator. To find IDs, run ax prompts list or ax evaluators list.

Payloads

Every delivery is a JSON POST with Content-Type: application/json. Event deliveries share these top-level fields:

Field	Description
`event`	The event, for example `prompt.version.created`.
`event_uuid`	A unique ID for the event. Retries reuse it, so use it to drop duplicates.
`space_id`	The ID of the space that holds the prompt or evaluator.
`space_name`	The name of that space.
`account_organization_id`	The ID of the organization.
`account_organization_name`	The name of the organization.
`timestamp`	When the event happened, in RFC 3339 format (UTC).
`data`	Fields specific to the event.

changed_by in data is the ID of the user who made the change. All IDs are opaque strings.

prompt.version.created

{
  "event": "prompt.version.created",
  "event_uuid": "3f1c9a52-8d47-4b8e-9a8a-1f2e6c0d7b11",
  "space_id": "U3BhY2U6MTAx",
  "account_organization_id": "QWNjb3VudE9yZ2FuaXphdGlvbjo3",
  "space_name": "production",
  "account_organization_name": "Acme",
  "timestamp": "2026-10-07T18:22:41.512304Z",
  "data": {
    "prompt_id": "UHJvbXB0OjEyMzQ1",
    "prompt_version_id": "UHJvbXB0VmVyc2lvbjo2Nzg=",
    "prompt_name": "support-agent",
    "changed_by": "VXNlcjo0Mg=="
  }
}

prompt.version.labeled and prompt.version.unlabeled

Both events use the same shape. label is the label that was added or removed.

{
  "event": "prompt.version.labeled",
  "event_uuid": "a7e2d4c0-5b39-4f61-8c2d-9e0b7f3a6c25",
  "space_id": "U3BhY2U6MTAx",
  "account_organization_id": "QWNjb3VudE9yZ2FuaXphdGlvbjo3",
  "space_name": "production",
  "account_organization_name": "Acme",
  "timestamp": "2026-10-07T18:25:03.004871Z",
  "data": {
    "prompt_id": "UHJvbXB0OjEyMzQ1",
    "prompt_version_id": "UHJvbXB0VmVyc2lvbjo2Nzg=",
    "prompt_name": "support-agent",
    "label": "production",
    "changed_by": "VXNlcjo0Mg=="
  }
}

evaluator.version.created

{
  "event": "evaluator.version.created",
  "event_uuid": "c0b8f1e6-2a4d-4d93-b7f5-6e1a9c3d8b40",
  "space_id": "U3BhY2U6MTAx",
  "account_organization_id": "QWNjb3VudE9yZ2FuaXphdGlvbjo3",
  "space_name": "production",
  "account_organization_name": "Acme",
  "timestamp": "2026-10-07T18:30:12.771092Z",
  "data": {
    "evaluator_id": "RXZhbHVhdG9yOjQ1Ng==",
    "evaluator_version_id": "RXZhbHVhdG9yVmVyc2lvbjo4OQ==",
    "evaluator_name": "hallucination-check",
    "changed_by": "VXNlcjo0Mg=="
  }
}

test.test_event

A test delivery carries only three fields.

{
  "event": "test.test_event",
  "event_uuid": "5d9e3b7a-0c4f-4a2e-8b61-7f2c1d0e9a33",
  "timestamp": "2026-10-07T18:20:00.120938Z"
}

New fields may appear in payloads over time, so ignore fields you do not use.

Test a webhook

A test sends one test.test_event request to your endpoint and reports the HTTP status your endpoint returned.

Open the webhook from Alert Integrations > Webhooks Integration and click Test Webhook.

curl -X POST https://api.arize.com/v2/webhooks/YOUR_WEBHOOK_ID/test \
  -H "Authorization: Bearer $ARIZE_API_KEY"

result = client.webhooks.test(
    webhook="prompt-changes",
    organization="YOUR_ORG_NAME_OR_ID",
)
print(result.status_code, result.error_message)

import { testWebhook } from "@arizeai/ax-client";

const result = await testWebhook({
  webhook: "prompt-changes",
  organization: "YOUR_ORG_NAME_OR_ID",
});
console.log(result.statusCode, result.errorMessage);

result, err := client.Webhooks.Test(ctx, webhooks.TestRequest{
    Webhook:      "prompt-changes",
    Organization: "YOUR_ORG_NAME_OR_ID",
})
if err != nil {
    log.Fatal(err)
}

fmt.Println(result.StatusCode)

ax webhooks test prompt-changes --organization YOUR_ORG_NAME_OR_ID

A successful call means the test ran. Check status_code and error_message for your endpoint’s answer. status_code is 502 when Arize AX got no response, for example because the endpoint was unreachable or timed out.
Test deliveries do not work for HMAC_SHA256 webhooks. To check a signed webhook, subscribe it to an event and trigger that event, for example by saving a new prompt version.
Test deliveries do not retry and do not appear in delivery history.

Delivery history and retries

Arize AX records every attempt to deliver an event. Each record holds the event_id (the same value as event_uuid and X-Arize-Webhook-Id), the attempt_number, the payload sent, the status_code (null when no response came back), the error_message, and the time of the attempt.

Open the webhook from Alert Integrations > Webhooks Integration and select the Event Delivery tab. It lists each attempt’s HTTP status, payload, error, attempt number, and send time.

curl "https://api.arize.com/v2/webhooks/YOUR_WEBHOOK_ID/delivery-attempts?limit=50" \
  -H "Authorization: Bearer $ARIZE_API_KEY"

resp = client.webhooks.list_delivery_attempts(
    webhook="prompt-changes",
    organization="YOUR_ORG_NAME_OR_ID",
    limit=50,
)
for attempt in resp.delivery_attempts:
    print(attempt.event_id, attempt.attempt_number, attempt.status_code)

import { listWebhookDeliveryAttempts } from "@arizeai/ax-client";

const { data: attempts } = await listWebhookDeliveryAttempts({
  webhook: "prompt-changes",
  organization: "YOUR_ORG_NAME_OR_ID",
  limit: 50,
});

resp, err := client.Webhooks.ListDeliveryAttempts(ctx, webhooks.ListDeliveryAttemptsRequest{
    Webhook:      "prompt-changes",
    Organization: "YOUR_ORG_NAME_OR_ID",
    Limit:        50,
})
if err != nil {
    log.Fatal(err)
}

for _, attempt := range resp.DeliveryAttempts {
    fmt.Println(attempt.EventId, attempt.AttemptNumber)
}

ax webhooks deliveries prompt-changes --organization YOUR_ORG_NAME_OR_ID --limit 50

Retries

Arize AX counts any 2xx response as delivered. It retries an event when:

the request fails to connect or times out, or
your endpoint returns 408, 429, or any 5xx status.

It does not retry other 4xx responses or 3xx redirects. Arize AX tries each event up to six times. After a failed attempt, it waits about 1, 4, 16, 64, and then 256 minutes before the next try. Retries send the same payload and event_uuid, so the same event can reach you more than once. Use event_uuid to skip events you have already handled. To avoid timeouts, return a 2xx response quickly and do slow work after you respond.

Update a webhook

You can change a webhook’s name, description, URL, bearer token, timeout, and custom headers. You cannot change its auth type or rotate its signing secret; create a new webhook instead.

When you send headers, the new map replaces all existing headers.
Send description as null to clear it.

In the UI, open the webhook from Alert Integrations > Webhooks Integration, edit the fields on the Webhook Details tab, and click Save. For code, see the update method in the Python, TypeScript, or Go SDK reference, or run ax webhooks update.

Delete a webhook or subscription

Deleting a webhook stops all deliveries to it and removes it from every prompt, evaluator, and monitor that used it. You cannot undo this.

ax webhooks delete prompt-changes --organization YOUR_ORG_NAME_OR_ID

To stop one event without removing the webhook, delete the subscription instead. Other subscriptions and the webhook stay as they are.

ax webhooks subscriptions delete YOUR_SUBSCRIPTION_ID

In the UI, delete a webhook with the delete button in its Webhook Integration window. To remove a subscription, clear the event in the prompt’s or evaluator’s Webhooks menu.

Next steps

REST API reference

Every webhook and subscription endpoint, with all fields.

CLI reference

Manage webhooks and subscriptions with ax webhooks.

Quickstart

Instrument

Observe

Agents

Evaluate

Experimentation

Machine Learning

Settings

Security

How webhooks work

Events

Limits

Permissions

Create a webhook

Authenticate deliveries

Bearer token

HMAC signature

Custom headers

Payloads

Test a webhook

Delivery history and retries

Retries

Update a webhook

Delete a webhook or subscription

Next steps

REST API reference

CLI reference

​How webhooks work

​Events

​Limits

​Permissions

​Create a webhook

​Authenticate deliveries

​Bearer token

​HMAC signature

​Custom headers

​Subscribe a webhook to events

​Payloads

​Test a webhook

​Delivery history and retries

​Retries

​Update a webhook

​Delete a webhook or subscription

​Next steps

REST API reference

CLI reference

How webhooks work

Events

Limits

Permissions

Create a webhook

Authenticate deliveries

Bearer token

HMAC signature

Custom headers

Subscribe a webhook to events

Payloads

Test a webhook

Delivery history and retries

Retries

Update a webhook

Delete a webhook or subscription

Next steps