TypeSafe AI

TypeSafe AI answers typed questions about a piece of state through its System One API. A request sends a state plus a map of named questions built from three primitives — Noul (yes/no), Choice (labeled alternatives), and Score (an ordered rubric) — and returns one typed answer per question. Arize AX captures every call through the openinference-instrumentation-typesafe package, which wraps both TypeSafeClient.system_one and AsyncTypeSafeClient.system_one as OpenInference LLM spans.

This is the Python guide. For the TypeScript / JavaScript instrumentor, see TypeSafe AI (JS).

A System One call is not a chat exchange, so the state and the answers are recorded as input.value and output.value rather than as llm.input_messages / llm.output_messages.

Prerequisites

Python 3.10+
typesafe-sdk 0.6.0 or newer — the range the instrumentor patches
An Arize AX account (sign up)
A TYPESAFE_API_KEY from TypeSafe AI

Launch Arize AX

Sign in to your Arize AX account.
From Space Settings, copy your Space ID and API Key. You will set them as ARIZE_SPACE_ID and ARIZE_API_KEY below.

Install

pip install arize-otel openinference-instrumentation-typesafe typesafe-sdk

Configure credentials

export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_API_KEY="<your-api-key>"
export ARIZE_PROJECT_NAME="typesafe-tracing-example"
export TYPESAFE_API_KEY="<your-typesafe-api-key>"

Setup tracing

# instrumentation.py
import os

from arize.otel import register
from openinference.instrumentation.typesafe import TypeSafeAIInstrumentor

tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
    project_name=os.environ["ARIZE_PROJECT_NAME"],
)

TypeSafeAIInstrumentor().instrument(tracer_provider=tracer_provider)
print("Arize AX tracing initialized for TypeSafe AI.")

Run TypeSafe AI

This example asks all three question types about one support ticket in a single call.

# example.py

# Importing instrumentation first ensures the instrumentor is installed
# before the first `system_one` call.
from instrumentation import tracer_provider

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

# The client reads TYPESAFE_API_KEY from the environment and defaults to
# the jev-latest model.
client = TypeSafeClient()

response = client.system_one(
    state={"document": "I was charged twice. Please fix this ASAP."},
    questions={
        "billing": Noul(instructions="Is this ticket about billing?"),
        "tone": Choice(
            instructions="What is the customer's tone?",
            criteria={"calm": None, "frustrated": None, "angry": None},
        ),
        "urgency": Score(
            instructions="How urgent is this ticket?",
            criteria=["can wait", "this week", "today"],
        ),
    },
)

print(response.nouls["billing"].noul)
print(response.choices["tone"].choice)
print(response.scores["urgency"].score)

Questions can be passed as the SDK objects shown above or as raw dictionaries — the instrumentor records either form.

Expected output

Arize AX tracing initialized for TypeSafe AI.
0.98
frustrated
1.99

Verify in Arize AX

Open your Arize AX space and select project typesafe-tracing-example.
You should see a new trace within ~30 seconds containing a single TypeSafeClient LLM span carrying:
- input.value — the request body (state, model, questions) as JSON
- output.value — the response body (model, answers, usage) as JSON
- llm.invocation_parameters — the call configuration: the model and any extra_body fields
- llm.request.model_name (jev-latest — the instrumentor falls back to the client default when the call omits model) and llm.response.model_name (the resolved version, for example jev-1.13.0)
- llm.token_count.prompt, llm.token_count.completion, and llm.token_count.total, when the API reports usage
If no traces appear, see Troubleshooting.

Check from the skill, CLI, or SDK

Confirm spans are actually reaching your Arize AX project. Use whichever fits your workflow — the skill and CLI work for any framework; the SDK check is shown for each language.

Arize skill (agent)
AX CLI
SDK

Install the Arize Skills plugin and let your coding agent check for you:

npx skills add Arize-ai/arize-skills

Then prompt your agent:

Use the arize-trace skill to export and analyze recent traces from my project. Confirm spans are arriving, and summarize any errors or latency issues.

Export recent spans for your project — any rows mean traces are landing:

ax spans export "$ARIZE_PROJECT_NAME" --space "$ARIZE_SPACE_ID" \
  --limit 5 --stdout | jq 'length'

A non-zero count confirms spans reached Arize AX. Run ax auth login first if you have not authenticated. See the ax spans reference.

Query the project’s spans and check that at least one came back.

import os
from arize import ArizeClient

client = ArizeClient(api_key=os.environ["ARIZE_API_KEY"])
resp = client.spans.list(
    project=os.environ["ARIZE_PROJECT_NAME"],
    space=os.environ["ARIZE_SPACE_ID"],
    limit=5,
)
count = len(resp.spans)
print(
    f"{count} span(s) found" if count else "No spans yet — recheck setup"
)

// Reads ARIZE_API_KEY from the environment.
import { listSpans } from "@arizeai/ax-client";

const { data: spans } = await listSpans({
  project: process.env.ARIZE_PROJECT_NAME!,
  space: process.env.ARIZE_SPACE_ID!,
  limit: 5,
});
const count = spans.length;
console.log(
  count ? `${count} span(s) found` : "No spans yet — recheck setup",
);

client, err := arize.NewClient(
    arize.Config{APIKey: os.Getenv("ARIZE_API_KEY")},
)
if err != nil {
    log.Fatal(err)
}
resp, err := client.Spans.List(ctx, spans.ListRequest{
    Project: os.Getenv("ARIZE_PROJECT_NAME"),
    Space:   os.Getenv("ARIZE_SPACE_ID"),
    Limit:   5,
})
if err != nil {
    log.Fatal(err)
}
fmt.Printf("%d span(s) found\n", len(resp.Spans))

SDK span references: Python · TypeScript · Go.

Trace the async client

AsyncTypeSafeClient is instrumented by the same TypeSafeAIInstrumentor() call — no extra setup. Its spans are named AsyncTypeSafeClient and carry identical attributes.

# example_async.py
from instrumentation import tracer_provider

import asyncio

from typesafe_sdk import AsyncTypeSafeClient, Noul


async def main() -> None:
    async with AsyncTypeSafeClient() as client:
        response = await client.system_one(
            state={"document": "I was charged twice."},
            questions={
                "billing": Noul(instructions="Is this about billing?"),
            },
        )
    print(response.nouls["billing"].noul)


asyncio.run(main())

Mask sensitive payloads

The state is the document you are asking about, so it often carries customer data. Because the state and the questions are recorded only in input.value, hide_inputs keeps the whole request off the span, and hide_outputs does the same for the answers:

import os

from arize.otel import register
from openinference.instrumentation import TraceConfig
from openinference.instrumentation.typesafe import TypeSafeAIInstrumentor

tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
    project_name=os.environ["ARIZE_PROJECT_NAME"],
)

TypeSafeAIInstrumentor().instrument(
    tracer_provider=tracer_provider,
    config=TraceConfig(hide_inputs=True, hide_outputs=True),
)

Model names and token counts survive masking, so cost and latency dashboards keep working. llm.invocation_parameters holds no request content — only the model and any extra_body fields — so mask it with hide_llm_invocation_parameters only if those are sensitive. You can also suppress tracing for a block with suppress_tracing(), and attach session, user, metadata, and tag information with the using_session, using_user, and using_attributes context managers.

Troubleshooting

No traces in Arize AX. Confirm ARIZE_SPACE_ID and ARIZE_API_KEY are set in the same shell that runs example.py.
TypeSafe spans missing but other spans present. TypeSafeAIInstrumentor().instrument(...) must run before the first system_one call. Import order is not the constraint here: the instrumentor patches the method on the TypeSafeClient class, so importing typesafe_sdk — or even constructing a client — before instrumenting is fine. Importing instrumentation first is simply the easiest way to guarantee the call order.
No span for client.models.list(). Expected — the instrumentor only wraps system_one.
TypeSafeAuthenticationError. Verify TYPESAFE_API_KEY is set and valid; the client reads it from the environment unless you pass api_key explicitly.
Instrumentor installs but never patches. It declares typesafe-sdk >= 0.6.0 as an instrumented dependency, and OpenTelemetry’s base instrumentor logs a dependency conflict and skips patching rather than raising when the installed SDK falls outside that range. Check the version with pip show typesafe-sdk.
Answers present but a question is missing from the response. The SDK drops answer types it doesn’t model yet and logs a warning; the raw payload is still on response.raw_http_response.

OpenTelemetry

LLM Providers

Agent Frameworks

Coding Agents

Platforms

Orchestration

Evaluation

Prerequisites

Launch Arize AX

Install

Configure credentials

Setup tracing

Run TypeSafe AI

Expected output

Verify in Arize AX

Check from the skill, CLI, or SDK

Trace the async client

Mask sensitive payloads

Troubleshooting

Resources

TypeSafe AI Documentation

OpenInference TypeSafe Instrumentor (Python)

Runnable TypeSafe Tracing Examples

TypeSafe AI SDK on PyPI

TypeSafe AI (JS/TS) tracing

​Prerequisites

​Launch Arize AX

​Install

​Configure credentials

​Setup tracing

​Run TypeSafe AI

​Expected output

​Verify in Arize AX

​Check from the skill, CLI, or SDK

​Trace the async client

​Mask sensitive payloads

​Troubleshooting

​Resources

TypeSafe AI Documentation

OpenInference TypeSafe Instrumentor (Python)

Runnable TypeSafe Tracing Examples

TypeSafe AI SDK on PyPI

TypeSafe AI (JS/TS) tracing

Prerequisites

Launch Arize AX

Install

Configure credentials

Setup tracing

Run TypeSafe AI

Expected output

Verify in Arize AX

Check from the skill, CLI, or SDK

Trace the async client

Mask sensitive payloads

Troubleshooting

Resources