Skip to main content
Available in arize-phoenix-client 3.4.0+, with trajectory traces in 3.5.0+ Phoenix now ships a job plugin for Harbor. Select it on a Harbor run and the job lands in Phoenix as a dataset, one experiment per resolved agent configuration, a run per trial, and every verifier reward as a named evaluation. No tracing code goes inside the sandbox.
Phoenix dataset page for a Harbor dataset showing eight experiments, with annotation score comparison, run latency, and cost charts above a table listing each experiment's agent configuration, infra_ok score, and a link to its traces

Each agent and model configuration in a Harbor job becomes its own experiment on the job's dataset, so reward, latency, and cost line up across runs.

​
Rewards Become Scores

reward distributions show up in experiment comparison. An infra_ok evaluation helps distinguish low task scores from execution errors. health, which tells a failed agent apart from a broken sandbox.
Phoenix experiment comparison for a single Harbor trial, showing the task input and reference output beside the agent's response, with reward, infra_ok, agent_turn_count, and tool_call_count evaluations and their averages

One trial in experiment comparison: the task instruction, the optional reference output, and the agent's final response, with the verifier's rewards recorded as evaluations.

​
Trajectories Become Traces

Phoenix experiment run trace for a Harbor trial, with the harbor.trial root span above the codex agent span and its iteration, openai/gpt-5.6 LLM, and exec_command tool spans, beside the selected tool span's input and output

A Harbor trial's trajectory as a Phoenix trace, opened from its experiment run: the harbor.trial root, the agent beneath it, and an LLM and a tool span for each iteration.

  • trace_mode="atif" is the default. The plugin reads the saved trajectory after the final trial attempt and converts it into one trace and one session, with AGENT, LLM, and TOOL spans under a harbor.trial root. A multi-step trial adds a span for each attempted step, and context-management operations appear as compaction spans.
  • Tracing never changes a score. If the agent saves no valid trajectory, the plugin logs a warning and records the run and its evaluations without a trace. Pass trace_mode=null on the Harbor command line to turn tracing off entirely.
  • The sandbox stays offline. The plugin converts trajectories after the trial, so the agent’s environment needs no Phoenix endpoint and no Phoenix credentials.

​
Run a Job with the Plugin

Install Harbor and the plugin into the same Python environment:
The plugin registers under the name arize-phoenix, so a Harbor run picks it up directly:
Configure it in Python when you want the dataset name or the naming template under version control:
  • Experiment names follow a template. experiment_name_template accepts fields like {job.name}, {agent.name}, {agent.model}, and {agent.short_digest}, and
  • Resuming the same Harbor job reuses its experiments without duplicating recorded runs.
  • Resuming is idempotent, so a re-run adds to the same experiment instead of duplicating it. The plugin requires Harbor 0.21.0+, Python 3.12+, and Phoenix server 15.0+. Endpoint and API key fall back to The plugin requires Harbor 0.21.0+ on Python 3.12+. Endpoint and API key fall back to PHOENIX_COLLECTOR_ENDPOINT and PHOENIX_API_KEY.

Harbor

Install the plugin, map its objects, and read its scores correctly

Importing ATIF Trajectories

The trajectory format the plugin converts