Skip to main content

Documentation Index

Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

Trace Codex CLI agent turns, tool calls, and API requests in Arize AX for full observability.
Codex is OpenAI’s open-source CLI coding agent. The Arize Coding Harness Tracing instruments Codex notify hook events and exports OpenInference spans to Arize AX or Phoenix. Each agent turn is captured as an LLM span with tool calls, token usage, and API request details.

​
Launch Arize AX

To get started, sign up for a free Arize AX account and get your Space ID and API Key:
  1. Log in at app.arize.com
  2. Click Settings and copy the Space ID
  3. Open the API Keys tab and create or copy an API key

​
Install

​
Curl installer

macOS / Linux: 
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- codex
Windows (PowerShell): 
iwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat
& $env:TEMP\install.bat codex

​
Local clone

git clone https://github.com/Arize-ai/coding-harness-tracing.git
cd coding-harness-tracing
./install.sh codex         # macOS / Linux
install.bat codex          # Windows
Open a new shell after install so the update takes effect.

​
Configuration

Credentials live in ~/.arize/harness/config.yaml. To override per category, set environment variables in ~/.codex/arize-env.sh — the notify hook sources this file automatically. Env values take precedence over config.yaml.

​
Phoenix (self-hosted)

export PHOENIX_ENDPOINT="http://localhost:6006"
export ARIZE_PROJECT_NAME="codex"
export ARIZE_TRACE_ENABLED="true"

​
Arize AX (cloud)

export ARIZE_API_KEY="<your-api-key>"
export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_PROJECT_NAME="codex"
export ARIZE_TRACE_ENABLED="true"

​
Redaction controls

Each ARIZE_LOG_* flag accepts "true" or "false" and defaults to "true". Set to "false" to opt out per category: 
export ARIZE_LOG_PROMPTS="false"
export ARIZE_LOG_TOOL_DETAILS="false"
export ARIZE_LOG_TOOL_CONTENT="false"
FlagRedacts
ARIZE_LOG_PROMPTSUser prompt and assistant response text
ARIZE_LOG_TOOL_DETAILSTool names and arguments
ARIZE_LOG_TOOL_CONTENTTool call output content

​
Observe

Once tracing is enabled, Codex activity is streamed to Arize AX. You’ll see:
  • Turn traces — each agent turn (user prompt to assistant response) as a parent LLM span
  • Tool call spans — one per tool decision and result pair
  • API request spans — one per API or WebSocket request
  • Session grouping — all turns from the same session grouped by session.id
  • Token usage — prompt, completion, and total token counts on every turn span

​
Hooks Captured

Codex emits a single notify event with subtypes for each lifecycle stage. The Arize AX hook fans them out into the spans listed above.
Notify subtypeSpan KindDescription
agent-turn-startLLMParent turn span starts; records the user prompt
agent-tool-callTOOLTool invocation, records name + input + result
agent-api-requestLLMOne span per API / WebSocket request to the model
agent-turn-completeLLMTurn span ends; records output and token counts
agent-session-completeCHAINSession teardown, state file cleanup

​
Reference

For the full list of environment variables, default file paths, and troubleshooting steps, see the Codex tracing README.

​
Uninstall

curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall codex

​
Resources

Arize Coding Harness Tracing

OpenInference

Codex CLI