LLMRunMetadata
Ingest metadata about your LLM inferences
Arize class to map up to 4 columns: total_token_count_column_name
, prompt_token_count_column_name
, response_token_count_column_name
, andresponse_latency_ms_column_name
class LLMRunMetadata:
total_token_count: Optional[int] = None
prompt_token_count: Optional[int] = None
response_token_count: Optional[int] = None
response_latency_ms: Optional[Union[int,float]] = None
Parameters
Data Type
Description
total_token_count
int
The total number of tokens used in the inference, both in the prompt sent to the LLM and in its response
promt_token_count
int
The number of tokens used in the prompt sent to the LLM
response_token_count
int
The number of tokens used in the response returned by the LLM
response_latency_ms
int or float
The latency (in ms) experienced during the LLM run
Code Example
from arize.utils.types import LLMRunMetadata
# Declare LLM run metadata
llm_run_metadata = LLMRunMetadata(
total_token_count = 4325,
prompt_token_count = 2325,
response_token_count = 2000,
response_latency_ms = 20000,
)
Last updated
Was this helpful?