Skip to main content
The ax evaluators commands let you create and manage LLM-as-judge evaluators and their versions on the Arize platform.

​
ax evaluators list

List evaluators, optionally filtered by space. 
ax evaluators list [--space-id <id>] [--limit <n>] [--cursor <cursor>]
OptionDescription
--space-idFilter evaluators by space ID
--limitMaximum number of results to return (default: 15)
--cursorPagination cursor for the next page
Examples: 
ax evaluators list --space-id sp_abc123
ax evaluators list --space-id sp_abc123 --output evaluators.json

​
ax evaluators get

Get an evaluator by ID, with its resolved version. 
ax evaluators get <evaluator-id> [--version-id <id>]
OptionDescription
--version-idSpecific version ID to retrieve (default: latest version)
Examples: 
ax evaluators get ev_abc123
ax evaluators get ev_abc123 --version-id evv_xyz789

​
ax evaluators create

Create a new evaluator with an initial version. Required options will be prompted interactively if not passed as flags. 
ax evaluators create \
  --name <name> \
  --space-id <id> \
  --commit-message <message> \
  --template-name <name> \
  --template <template-string> \
  --ai-integration-id <id> \
  --model-name <model>
OptionDescription
--nameEvaluator name (must be unique within the space)
--space-idSpace ID to create the evaluator in
--commit-messageCommit message for the initial version
--template-nameEval column name (alphanumeric, spaces, hyphens, underscores)
--templatePrompt template string with {{variable}} placeholders
--ai-integration-idAI integration global ID
--model-nameModel name (e.g. gpt-4o)
--descriptionOptional evaluator description
--include-explanationsInclude reasoning explanation alongside the score (flag)
--use-function-callingPrefer structured function-call output when supported (flag)
--invocation-paramsJSON object of model invocation parameters (e.g. '{"temperature": 0}')
--provider-paramsJSON object of provider-specific parameters
Example: 
ax evaluators create \
  --name "Relevance" \
  --space-id sp_abc123 \
  --commit-message "Initial version" \
  --template-name "Relevance" \
  --template "Is the response relevant to the query?\nQuery: {{input.value}}\nResponse: {{output.value}}" \
  --ai-integration-id ai_xyz789 \
  --model-name gpt-4o \
  --include-explanations \
  --invocation-params '{"temperature": 0}'

​
ax evaluators update

Update an evaluator’s name or description. At least one of --name or --description is required. 
ax evaluators update <evaluator-id> [--name <name>] [--description <desc>]
OptionDescription
--nameNew evaluator name
--descriptionNew evaluator description
Example: 
ax evaluators update ev_abc123 --name "Relevance v2" --description "Updated scoring rubric"

​
ax evaluators delete

Delete an evaluator and all its versions. This operation is irreversible. 
ax evaluators delete <evaluator-id> [--force]
OptionDescription
--forceSkip the confirmation prompt
Examples: 
ax evaluators delete ev_abc123
ax evaluators delete ev_abc123 --force

​
ax evaluators list-versions

List all versions of an evaluator. 
ax evaluators list-versions <evaluator-id> [--limit <n>] [--cursor <cursor>]
OptionDescription
--limitMaximum number of versions to return (default: 15)
--cursorPagination cursor for the next page
Example: 
ax evaluators list-versions ev_abc123

​
ax evaluators get-version

Get a specific evaluator version by ID. 
ax evaluators get-version <version-id>
Example: 
ax evaluators get-version evv_xyz789

​
ax evaluators create-version

Create a new version of an existing evaluator. Versions are immutable once created; the new version becomes the latest immediately. Required options will be prompted interactively if not passed as flags. 
ax evaluators create-version <evaluator-id> \
  --commit-message <message> \
  --template-name <name> \
  --template <template-string> \
  --ai-integration-id <id> \
  --model-name <model>
OptionDescription
--commit-messageCommit message describing the changes in this version
--template-nameEval column name
--templateUpdated prompt template string with {{variable}} placeholders
--ai-integration-idAI integration global ID
--model-nameModel name (e.g. gpt-4o)
--include-explanationsInclude reasoning explanation alongside the score (flag)
--use-function-callingPrefer structured function-call output when supported (flag)
--invocation-paramsJSON object of model invocation parameters
--provider-paramsJSON object of provider-specific parameters
Example: 
ax evaluators create-version ev_abc123 \
  --commit-message "Improved prompt for edge cases" \
  --template-name "Relevance" \
  --template "Rate the relevance of the response on a scale of 0 to 1.\nQuery: {{input.value}}\nResponse: {{output.value}}" \
  --ai-integration-id ai_xyz789 \
  --model-name gpt-4o