Skip to main content

View source on Github

Arize class to map up to 4 columns: total_token_count_column_name , prompt_token_count_column_name, response_token_count_column_name, andresponse_latency_ms_column_name 
class LLMRunMetadata:
    total_token_count: Optional[int] = None
    prompt_token_count: Optional[int] = None
    response_token_count: Optional[int] = None
    response_latency_ms: Optional[Union[int,float]] = None
ParametersData TypeDescription
total_token_countintThe total number of tokens used in the inference, both in the prompt sent to the LLM and in its response
promt_token_countintThe number of tokens used in the prompt sent to the LLM
response_token_countintThe number of tokens used in the response returned by the LLM
response_latency_msint or floatThe latency (in ms) experienced during the LLM run

​
Code Example

from arize.utils.types import LLMRunMetadata

# Declare LLM run metadata
llm_run_metadata = LLMRunMetadata(
    total_token_count = 4325,
    prompt_token_count = 2325,
    response_token_count = 2000,
    response_latency_ms = 20000,
)