Skip to main content
Trace Devin CLI interactions, model calls, token usage, and tool calls in Phoenix for full observability.
Devin is Cognition’s autonomous software engineer. The coding-harness-tracing toolkit instruments the Devin CLI and streams OpenInference spans to Phoenix. Each agent interaction emits its own trace: a root agent span, per-generation LLM spans with real token counts, and tool spans for tool calls. Devin’s hook payloads are thin, carrying no session ID and no token or model data, so the rich content is read from Devin’s live SQLite session database instead. The harness registers a Stop hook that fires at the end of each agent response: it resolves the session, reads the generations that have appeared since the last emission, and emits one self-contained trace for that interaction. Traces therefore appear as each interaction completes, not only after the session exits. A SessionEnd hook is also registered as a final flush for an interrupted last turn.

​
Launch Phoenix

The fastest way to get started with Phoenix is by signing up for a free Phoenix Cloud account. If you prefer, you can also run Phoenix in a notebook, self-host it, or use it directly from your terminal. Go to the settings page in your Phoenix instance to find your endpoint and API key. A self-hosted Phoenix defaults to http://localhost:6006; the API key is only required when auth is enabled.

​
Install

The curl installer is the simplest — it runs a short wizard that saves your Phoenix credentials for you. Use a local clone if you’d rather run the installer from a checkout of the source. macOS / Linux:
Windows (PowerShell):

​
Local clone

The installer prompts for your backend — select Phoenix, then enter your endpoint and optional API key — and your project name. It writes credentials to ~/.arize/harness/config.json and registers Stop and SessionEnd command hooks under the top-level hooks key in Devin’s user config: ~/.config/devin/config.json on macOS and Linux, %APPDATA%\devin\config.json on Windows.
Devin allows comments in its config files. The installer reads them, but writes the file back as plain JSON, so comments in a config it has to modify are not preserved. A config it cannot parse is left untouched and the install aborts with an error rather than overwriting your settings.

​
Configuration

Credentials live in ~/.arize/harness/config.json. Environment variables override values in config.json and can be set in your shell profile so they apply to every Devin session.
On the Phoenix backend, set the project name with PHOENIX_PROJECT (or PHOENIX_PROJECT_NAME); ARIZE_PROJECT_NAME is ignored here. ARIZE_TRACE_ENABLED is a backend-agnostic harness setting and keeps the ARIZE_ prefix regardless of destination.

​
Redaction controls

Each ARIZE_LOG_* flag accepts "true" or "false" and defaults to "true". Set to "false" to opt out per category:

​
Default settings

On Windows only the config file moves, to %APPDATA%. The sessions database stays home-relative at %USERPROFILE%\.local\share\devin\cli\sessions.db, the same layout as macOS and Linux. If traces do not appear, check ~/.arize/harness/logs/devin.log for the database path the hook tried.

​
Observe

Once tracing is enabled, Devin activity is streamed to Phoenix. Each agent interaction is captured as a trace. You’ll see:
  • Interaction traces — one root agent span per agent response, carrying the user prompt as input, the final assistant text as output, the model name, and interaction token totals
  • LLM spans — one per real model generation, with that generation’s prompt, completion, and cache token counts, model name, and reasoning content
  • Tool spans — one per tool call, parented to the LLM span that issued it, with the serialized tool arguments as input
  • Session grouping — interactions from the same session grouped by session.id
Real generations are deduped by metadata.request_id. Devin rebuilds the message chain as the conversation grows, so the same generation reappears under new node IDs, and a per-session, per-request watermark ensures each generation is emitted exactly once. The database is opened read-only in WAL-respecting mode, so the newest turn’s rows are visible without disturbing Devin’s writers.

​
Span shape

The root agent span carries session.id, input.value, output.value, llm.model_name, the llm.token_count.* totals (including prompt_details.cache_read and prompt_details.cache_write), project.name, an optional user.id, and devin.backend for the agent backend.

​
What does not carry over

  • LLM spans do not carry input.value. The per-generation prompt messages sent to the model are not reconstructed from the database, so the interaction’s user prompt lives on the root agent span instead. A generation that issued only tool calls, with no assistant text or reasoning, has an empty output.value; its visible output appears on the later generation that answers the user. No output is lost, it is attributed to the generation that produced it.
  • Tool output is best-effort. Tool results are not reliably present in the live database at the moment Stop fires, so a tool span may carry only its input arguments.

​
Reference

For the full list of environment variables, default file paths, and troubleshooting steps, see the Devin tracing README. Errors always land in ~/.arize/harness/logs/devin.log. Set export ARIZE_VERBOSE=true before launching Devin to also log routine hook activity.

​
Uninstall

Uninstall removes only the Stop and SessionEnd hook entries the installer added, leaving any hooks you added yourself untouched.

​
Resources

Arize Coding Harness Tracing

OpenInference

Devin