Create task

Creates a new task. Supported task types:

For run_experiment tasks the run configuration is stored on the task. Each trigger (POST /v2/tasks/{task_id}/trigger) supplies per-run fields (experiment_name, optional example subset, etc.) and starts an async run. Poll GET /v2/task-runs/{run_id} until status reaches a terminal state.

Payload Requirements (template_evaluation / code_evaluation)

• At least one evaluator is required.
• Duplicate evaluator IDs are not allowed.
• When dataset_id is provided, experiment_ids must contain at least one entry.
• sampling_rate and is_continuous are only supported on project-based tasks.
• System-managed fields (id, created_at, updated_at) are rejected on input.

Payload Requirements (run_experiment)

• dataset_id is required; project_id must be omitted.
• run_configuration is required; evaluators, experiment_ids, sampling_rate, is_continuous, and query_filter must be omitted.

Valid example (template_evaluation, project-based)

{
  "name": "Production Hallucination Check",
  "type": "template_evaluation",
  "project_id": "TW9kZWw6MTIzOmFCY0Q=",
  "sampling_rate": 1.0,
  "is_continuous": true,
  "evaluators": [
    {
      "evaluator_id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
      "column_mappings": {"input": "attributes.input.value", "output": "attributes.output.value"}
    }
  ]
}

Invalid example (run_experiment missing run_configuration)

{
  "name": "My Experiment",
  "type": "run_experiment",
  "dataset_id": "RGF0YXNldDo1NjpxUndY"
}