Together AI

Instrument TogetherAI and send OpenTelemetry traces to Arize

Together.AI provides low cost inference for AI models in production. Arize supports instrumenting Together AI API calls using our OpenAI instrumentation. You can create a free TogetherAI account and generate a Together AI API Key to get started.

In this example we will instrument an LLM application built using Together AI and the OpenAI library.

Python

Install dependencies

pip install openai openinference-instrumentation-openai arize-otel

Register for OpenTelemetry and activate the OpenAIInstrumentor

# Import open-telemetry dependencies
from arize.otel import register

# Setup OTel via our convenience function
tracer_provider = register(
    space_id = "your-space-id", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    project_name = "your-project-name", # name this to whatever you would like
)

# Import the automatic instrumentor from OpenInference
from openinference.instrumentation.openai import OpenAIInstrumentor

# Finish automatic instrumentation
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

Now start asking questions to your LLM app and watch the traces being collected by Arize.

import os
import openai

client = openai.OpenAI(
  api_key=os.environ.get("TOGETHER_API_KEY"),
  base_url="https://api.together.xyz/v1",
)

response = client.chat.completions.create(
  model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
  messages=[
    {"role": "system", "content": "You are a travel agent. Be descriptive and helpful."},
    {"role": "user", "content": "Tell me the top 3 things to do in San Francisco"},
  ]
)

print(response.choices[0].message.content)