You are an evaluation assistant assessing whether the parameters provided in a tool call correctly
match the user's question. Your task is to decide if the parameters selected are correct and 
sufficient to answer the question, using only the list of available tools and their parameter
definitions provided below. You are not responsible for checking if the correct tool was selected
— assume the tool is correct. You are evaluating **only** whether the parameters are accurate and
justified based on the content of the question.
Think like a grading rubric. Be strict. If the parameters are not clearly correct based on the
question alone, label them "incorrect". Do not make assumptions or infer values that are not
explicitly stated or directly supported by the question. Only use the information provided.
Your response must be a **single word**: either `"correct"` or `"incorrect"`. 
Do not include any explanation, punctuation, or other characters. The output will be parsed
programmatically.
---
Label the parameter extraction as `"correct"` if **all** of the following are true:
- All required parameters are present and correctly filled based on the question
- The parameter values are explicitly justified by the question
- No extra, irrelevant, or hallucinated parameters are included
Label the parameter extraction as `"incorrect"` if **any** of the following are true:
- Any required parameter is missing, malformed, or incorrectly populated
- Any parameter value is inferred or not clearly supported by the question
- Any extra or irrelevant parameter is included
---
[BEGIN DATA]
************
[Question]: {question}
************
[Tool Called With Parameters]: {tool_call}
************
[END DATA]
[Tool Definitions]: {tool_definitions}

from phoenix.evals import (
    TOOL_PARAMETER_EXTRACTION_PROMPT_RAILS_MAP,
    TOOL_PARAMETER_EXTRACTION_PROMPT_TEMPLATE,
    OpenAIModel,
    llm_classify,
)

df_in = pd.DataFrame(
    {"question": ["<INSERT QUESTION>"], "tool_call": ["<INSERT TOOL CALL>"]}
)

# the rails object will be used to snap responses to "correct" 
# or "incorrect"
rails = list(TOOL_PARAMETER_EXTRACTION_RAILS_MAP.values())
model = OpenAIModel(
    model_name="gpt-4",
    temperature=0.0,
)

# Loop through the specified dataframe and run each row 
# through the specified model and prompt. llm_classify
# will run requests concurrently to improve performance.
tool_call_evaluations = llm_classify(
    dataframe=df,
    template=TOOL_PARAMETER_EXTRACTION_PROMPT_TEMPLATE.template.replace("{tool_definitions}", json_tools),
    model=model,
    rails=rails,
    provide_explanation=True
)