Documentation Index
Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt
Use this file to discover all available pages before exploring further.
Binary Classification Cases
| Binary Classification Cases | Expected Fields | Performance Metrics |
|---|
| Case 1: Supports Only Classification Metrics | prediction label, actual label | Accuracy, Recall, Precision, FPR, FNR, F1, Sensitivity, Specificity |
| Case 2: Supports Classification, AUC, Log Loss Metrics | prediction score, prediction label, actual label | AUC, PR-AUC, Log Loss, Accuracy, Recall, Precision, FPR, FNR, F1, Sensitivity, Specificity |
| Case 3: Supports AUC & Log Loss Metrics | prediction score, actual label | AUC, PR-AUC, Log Loss |
Case #1 - Supports Only Classification Metrics
Python Pandas Batch
Python Single Record
Data Connector
Example Row| state | pos_approved | zip_code | age | prediction_label | actual_label | prediction_ts |
|---|
ca | True | 12345 | 25 | not_fraud | fraud | 1618590882 |
Code Example
# feature & tag columns can be optionally defined with typing:
tag_columns = TypedColumns(
inferred=["name"],
to_int=["zip_code", "age"]
)
schema = Schema(
prediction_id_column_name="prediction_id",
timestamp_column_name="prediction_ts",
prediction_label_column_name="prediction_label",
actual_label_column_name="actual_label",
feature_column_names=["state", "pos_approved"],
tag_column_names=tag_columns,
)
response = arize_client.log(
model_id='sample-model-1',
model_version='v1',
model_type=ModelTypes.BINARY_CLASSIFICATION,
metrics_validation=[Metrics.CLASSIFICATION],
environment=Environments.PRODUCTION,
dataframe=example_dataframe,
schema=schema
)
Code Example
# features & tags can be optionally defined with typing
features = {
'state': 'ca',
'pos_approved': TypedValue(value=True, type=ArizeTypes.INT)
}
tags = {
'zip_code': '12345',
'age': '25'
}
response = arize_client.log(
model_id='sample-model-1',
model_version='v1',
model_type=ModelTypes.BINARY_CLASSIFICATION,
environment=Environments.PRODUCTION,
features=features,
tags=tags,
prediction_label="not fraud",
actual_label="fraud"
)
Download an example Parquet file:Open parquet reader[here] (https://parquetreader.com/result).Learn how to upload files via various Data Connectors:Google Cloud Storage (GCS)
Case #2 - Supports Classification & AUC/Log Loss Metrics
Python Pandas
Python Single Record
Data Connector
Example Row| state | pos_approved | zip_code | age | prediction_label | actual_label | prediction_score | prediction_tsa |
|---|
ca | True | 12345 | 25 | not_fraud | fraud | 0.3 | 1618590882 |
# feature & tag columns can be optionally defined with typing:
tag_columns = TypedColumns(
inferred=["name"],
to_int=["zip_code", "age"]
)
schema = Schema(
prediction_id_column_name="prediction_id",
timestamp_column_name="prediction_ts",
prediction_label_column_name="prediction_label",
prediction_score_column_name="prediction_score",
actual_label_column_name="actual_label",
feature_column_names=["state", "pos_approved"],
tag_column_names=tag_columns,
)
response = arize_client.log(
model_id='sample-model-1',
model_version='v1',
model_type=ModelTypes.BINARY_CLASSIFICATION,
metrics_validation=[Metrics.CLASSIFICATION, Metrics.AUC_LOG_LOSS],
environment=Environments.PRODUCTION,
dataframe=test_dataframe,
schema=schema,
)
Code Examplefeatures = {
'state': 'ca',
'pos_approved': True,
'item_count': 10
}
# features & tags can be optionally defined with typing
tags = {
'zip_code': TypedValue(value='12345', type=ArizeTypes.INT),
'age': '25'
}
response = arize_client.log(
model_id='sample-model-1',
model_version='v1',
model_type=ModelTypes.BINARY_CLASSIFICATION,
environment=Environments.PRODUCTION,
features=features,
tags=tags,
prediction_label=("not fraud", 0.3),
actual_label="fraud"
)
Download an example Parquet file:Open parquet reader here.Learn how to upload files via various Data Connectors:Google Cloud Storage (GCS)
Case #3: Supports AUC & Log Loss Metrics
Python Pandas
Python Single Record
Example Row
| state | pos_approved | zip_code | age | actual_label | prediction_score | prediction_ts |
|---|
ca | True | 12345 | 25 | fraud | 0.3 | 1618590882 |
Code Example
# feature & tag columns can be optionally defined with typing:
tag_columns = TypedColumns(
inferred=["name"],
to_int=["zip_code", "age"]
)
schema = Schema(
prediction_id_column_name="prediction_id",
timestamp_column_name="prediction_ts",
prediction_score_column_name="prediction_score",
actual_label_column_name="actual_label",
feature_column_names=["state", "pos_approved"],
tag_column_names=tag_columns,
)
response = arize_client.log(
model_id='sample-model-1',
model_version='v1',
model_type=ModelTypes.BINARY_CLASSIFICATION,
metrics_validation=[Metrics.CLASSIFICATION, Metrics.REGRESSION, Metrics.AUC_LOG_LOSS],
environment=Environments.PRODUCTION,
dataframe=test_dataframe,
schema=schema,
)
To declare prediction_score ONLY, pass a tuple of an empty string and your prediction_score through the prediction_label argument.
Code Example
# features & tags can be optionally defined with typing
features = {
'state': 'ca',
'pos_approved': TypedValue(value=True, type=ArizeTypes.INT)
}
tags = {
'zip_code': '12345',
'age': '25'
}
response = arize_client.log(
model_id='sample-model-1',
model_version='v1',
model_type=ModelTypes.BINARY_CLASSIFICATION,
environment=Environments.PRODUCTION,
features=features,
tags=tags,
prediction_label=("", 0.3),
actual_label="fraud",
)
Default Actuals
For some use cases, it may be important to treat a prediction for which no corresponding actual label has been logged yet as having a default negative class actual label.
For example, consider tracking advertisement conversion rates for an ad clickthrough rate model, where the positive class is click and the negative class is no_click. For ad conversion purposes, a prediction without a corresponding* *actual label for an ad placement is equivalent to logging an explicit no_click actual label for the prediction. In both cases, the result is the same: a user has not converted by clicking on the ad. For AUC-ROC, PR-AUC, and Log Loss performance metrics, Arize supports treating predictions without an explicit actual label as having the negative class actual label by default. In the above example, a click prediction without an actual would be treated as a false positive, because the missing actual for the prediction would, by default, be assigned to the no_click negative class.
This feature can be enabled for monitors and dashboards via the model performance config section of your model’s config page.
Quick Definitions
Prediction Label: The classification label of this event (Cardinality = 2)
Actual Label: The ground truth label (Cardinality = 2)
Prediction Score: The likelihood of the event (Probability between 0 to 1)
Actual Score: The ground truth score (0 or 1)
