Skip to main content
Skills in AX Agent Improvement Loop are account-level integrations that give sandbox workers access to external systems. They are separate from Arize Skills in your local coding agent—same ideas, different runtime (managed sandbox vs your laptop). Enable skills to access external sources like Datadog and GitHub, or to expand Arize API access beyond default trace read.

​
Where to configure

LocationUse for
Settings → Agent integrations (or Manage Skills from Agent Studio / Agent Presets)Create, edit, and delete skills for the account
Agent PresetsAttach one or more skills to a preset
Agent StudioPick a preset (skills come from the preset) or attach skills for a one-off run
Repo appears on a preset only when a GitHub skill is selected.

​
Built-in skill types

​
GitHub

Lets the worker authenticate to GitHub (clone, branch, open PRs). Secrets: GH_TOKEN — personal access token or GitHub App token with the scopes your workflow needs (repo read and/or write). Repo field: After you add a GitHub skill to a preset, set the repository the worker should use for that preset.

​
Arize

Optional. Workers already have read access to traces on the bound project without this skill. When a run starts, the platform provisions a short-lived credential scoped to that project—enough for investigations, Signal, and other trace-only tasks. Add an Arize skill when you need to expand what the worker can do in Arize beyond reading spans on the bound project—for example:
  • Query or update datasets and experiments
  • Create or run evaluators
  • Apply annotations or manage other Arize resources via the API
Secrets: ARIZE_API_KEY — an API key with the permissions the workflow needs (same style as the AX CLI or API). The worker can only call APIs that key is allowed to use. Attach the skill on the preset in Agent Presets (or for a one-off run in Agent Studio). Use a dedicated key with least privilege; do not reuse a personal admin key unless the workflow requires it.

​
Datadog

Lets the worker query Datadog alongside Arize traces. Secrets:
  • DD_API_KEY
  • DD_APP_KEY
  • DD_SITE (optional; defaults to datadoghq.com)

​
Custom skill

Install a skill from a GitHub repository into the sandbox (for example, an internal runbook or a fork of an Arize skill). Required fields:
  • Name — Letters, numbers, spaces, hyphens, underscores, periods (max 253 characters).
  • Install source — GitHub repo only. Accepted: https://github.com/owner/repo, git@github.com:owner/repo, or owner/repo.
  • Description — How the harness should use the skill (max 1000 characters).
  • Installer — github (git clone).
Custom skills may use optional env vars if your skill needs them; non-custom providers require at least one secret.

​
Add a skill

1

Open Agent integrations

From Settings → Agent integrations, or Manage Skills in Agent Studio / Agent Presets.
2

Choose provider

Select GitHub, Arize, Datadog, or Custom skill.
3

Enter secrets

Paste secrets as ENV=value lines or a JSON object. The UI shows which env vars will be stored (for example GH_TOKEN, DD_API_KEY).
4

Save

The skill is available to attach on any preset in the account.
5

Attach to a preset

In Agent Presets, select the skill(s) for that preset. For GitHub, set the repo after the skill is attached.

​
Permissions

Who can do what in the app today:
ActionWho
Create / update / delete skillsAccount admin
Create / update agent presetsAccount admin
Start a session or automation (Agent Studio, Signal enable)User with write access to the space (and project permissions where applicable)
Enable Signal on a projectUser with project update permission
Delete a sandbox jobUser with access to that job in the space
Send messages / attach to a running jobUser with access to that job
Annotators cannot manage skills or presets. Starting a worker may require developer-level access or account admin when the platform provisions a short-lived service key for the sandbox. If job creation fails with a permissions error, ask an account admin to grant the needed space role or developer permission. Signal and trace-backed workers only read projects you can already access in the UI—they do not elevate privileges.

​
Security practices

  • Least privilege — GitHub tokens with read-only scope until you need PRs; Datadog keys with minimal API scopes.
  • One skill per integration — Reuse the same skill across presets instead of duplicating secrets.
  • No secrets in prompts — Put credentials only in the skill definition.
  • Review outputs — Treat PRs and investigation suggestions like any other untrusted automation output until reviewed.
  • Custom skills — Only install repos you trust; the sandbox clones and executes skill code during the run.

​
Signal and skills

The built-in Signal preset uses project traces only—no extra skills required to detect issues. To add Datadog, GitHub, or other skills, clone the Signal preset or create a new preset and point the Signal automation at it.