Logging Latent Metadata

Sometimes you might want to log additional metadata after a span has already been ingested. This is especially useful when your system enriches interactions with more information that wasn’t available at generation time.

Logging Metadata via the Python SDK

You can attach metadata, corrections, or other enriched information to specific spans using the log_metadata() function in the Arize Python SDK

Step 1: Set Up the SDK

import os
import pandas as pd
from arize.pandas.logger import Client

# Replace these with your actual values
API_KEY = os.environ.get("ARIZE_API_KEY")       # Found in the Arize UI
SPACE_ID = os.environ.get("ARIZE_SPACE_ID")     # Found in the Arize UI
PROJECT_NAME = "YOUR_PROJECT_NAME"

arize_client = Client(
    space_id=SPACE_ID,
    api_key=API_KEY
    )

Step 2: Create Your Metadata Patch

To update metadata on spans, your DataFrame must include:

• context.span_id – identifies the span to update

• Either:

  • A patch_document column containing JSON objects

  • One or more attributes.metadata.* columns

  • Or both (in which case, the patch_document is applied after the field-level updates and will override any conflicting keys)

Example Structures

# Method 1: Using patch_document
metadata_df = pd.DataFrame({
    "context.span_id": ["span1"],
    "patch_document": [{"status": "reviewed"}],
})

# Method 2: Using metadata-prefixed columns
metadata_df = pd.DataFrame({
    "context.span_id": ["span1"],
    "attributes.metadata.status": ["reviewed"],
    "attributes.metadata.tag": ["important"]
})

# Method 3: Combining both
metadata_df = pd.DataFrame({
    "context.span_id": ["span1"],
    "attributes.metadata.tag": ["important"],
    "patch_document": [{"status": "reviewed"}],  # This takes precedence
})

Type Handling

Input Type	Behavior
string, int, float	Fully supported
bool	Automatically converted to string ("true" / "false")
Objects / Arrays	Serialized to JSON strings
None / null	Stored as JSON null (does not remove the field)

⚠️ null values do not remove fields—they explicitly set them to null.

Step 3: Log Metadata

try:
    response = arize_client.log_metadata(
        dataframe=metadata_df,
        project_name=PROJECT_NAME,
        validate=True,
        verbose=True
    )

if response:
    print("✅ Metadata logged!")
    print(f"Spans updated: {response.spans_updated}")
else:
    print("⚠️ No response returned. Check logs or UI.")
except Exception as e:
    print(f"❌ Error logging metadata: {e}")

Response Format

Success

{
  "spans_processed": 1,
  "spans_updated": 1,
  "spans_failed": 0,
  "errors": []
}

Error

{
  "spans_processed": 1,
  "spans_updated": 0,
  "spans_failed": 1,
  "errors": [
    {
      "span_id": "61dbffb32f20ddb6",
      "error_message": "[type_conflict] Type conflict: ... Int vs. Utf8"
    }
  ]
}

Common Error Types

Error Type	Description
parse_failure	Invalid JSON in patch document
patch_failure	Patch could not be applied
type_conflict	Existing field has a different data type
segment_not_found	Span ID not found
connection_failure	Network or API connectivity issue
druid_rejection	Backend rejected update (e.g., schema mismatch)