Multi-Class Classification
How to log your model schema for multiclass classification models
Multi-Class Classification Overview
A classification model with more than two classes.
Supported Metrics
Micro-Averaged Precision, Micro-Averaged Recall, Macro-Averaged Precision, Macro-Averaged Recall, Precision for a Class, Recall for a Class
How To Log Multi-Class Data
Log multi-class classification models based on your use case
Single-Label
A prediction that has 1 label i.e. A passenger can only be in EITHER economy, business, OR first-class
prediction scores (dictionary)
actual scores (dictionary, optional)
Multi-Label
A prediction that has multiple labels i.e. A song can be multiple genres such as 'pop-rock'
prediction scores (dictionary)
threshold scores (dictionary)
actual scores (dictionary, optional)
Single-Label Use Case
Example Row
[{"class_name": "economy_class", "score": 0.81},{"class_name": "business_class", "score": 0.42},{"class_name": "first_class", "score": 0.35}]
[{"class_name": "economy_class", "score": 1}]
Note: class economy_class has the highest prediction score and will be the prediction label
Code Example
schema = Schema(
prediction_id_column_name="prediction_id",
prediction_score_column_name="prediction_scores",
actual_score_column_name="actual_scores",
)
response = arize_client.log(
model_id='multiclass-classification-single-label-example',
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
dataframe=example_dataframe,
schema=schema,
environment=Environments.PRODUCTION,
)# Predicting likelihood of Economy, Business, or First Class
"""
example_record = {
"prediction_scores":{
"economy_class":0.81,
"business_class":0.42,
"first_class":0.35
},
"actual_scores": {
"first_class": 1
}
}
"""
prediction_label = MultiClassPredictionLabel(
prediction_scores=record["prediction_scores"],
)
actual_label = MultiClassActualLabel(
actual_scores=record["actual_scores"],
)
response = arize_client.log(
model_id="multiclass-classification-single-label-example",
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
prediction_id=record["prediction_id"],
prediction_label=prediction_label,
actual_label=actual_label,
environment=Environments.PRODUCTION
)Note: class economy_class has the highest prediction score and will be the prediction label
Multi-Class Prediction
Log multi-class prediction values using the MultiClassPredictionLabel object, which can be assigned to the prediction_label parameter.
Prediction scores are required to use MULTI_CLASS models in Arize.
class MultiClassPredictionLabel(
prediction_scores: Dict[str, Union[float, int]]
)Multi-Class Actual Values
Log multi-class actual values by using the MultiClassActualLabel object, which can be assigned to the actual_label parameter.
Actual values are optional, and can be sent later via delayed actuals.
class MultiClassActualLabel(
actual_scores: Dict[str, Union[float, int]]
)For more information on Python Single Record Logging API Reference, visit here:
Single Record LoggingMulti-Label Use Case
Example Row
[{"class_name": "jazz", "score": 0.81},{"class_name": "rock", "score": 0.42},{"class_name": "pop", "score": 0.35}]
[{"class_name": "jazz", "score": 0.5},{"class_name": "rock", "score": 0.4},{"class_name": "pop", "score": 0.6}]
[{"class_name": "rock", "score": 1}]
Note: classes jazz and rock have prediction scores > threshold scores and will be part of the prediction label.
Code Example
schema = Schema(
prediction_id_column_name="prediction_id",
prediction_score_column_name="prediction_scores",
multi_class_threshold_scores_column_name="threshold_scores"
actual_score_column_name="actual_scores",
)
response = arize_client.log(
model_id='multiclass-classification-multi-label-example',
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
dataframe=example_dataframe,
schema=schema,
environment=Environments.PRODUCTION,
)#predicting song genre within pop, rock, and jazz categories
"""
example_record = {
"prediction_id": "57437c5d-0cd2-48bf-aa91-379302fd0fe4",
"prediction_scores": {
"pop": 0.5044088627000001,
"rock": 0.3688503145,
"jazz": 0.1267408228
},
"threshold_scores": {
"pop": 0.50,
"rock": 0.40,
"jazz": 0.10
},
"actual_scores": {
"pop": 1,
"jazz": 1
}
"""
prediction_label = MultiClassPredictionLabel(
prediction_scores=record["prediction_scores"],
threshold_scores=record["threshold_scores"], #required for multi-label
)
actual_label = MultiClassActualLabel(
actual_scores=record["actual_class"],
)
response = arize_client.log(
model_id="multiclass-classification-multi-label-example",
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
prediction_id=record["prediction_id"],
prediction_label=prediction_label,
actual_label=actual_label,
environment=Environments.PRODUCTION
)Multi-Class Prediction & Threshold Values
Log multi-class prediction and threshold values using the MultiClassPredictionLabel object, which can be assigned to the prediction_label parameter.
Prediction scores are required to use MULTI_CLASS models in Arize.
class MultiClassPredictionLabel(
prediction_scores: Dict[str, Union[float, int]]
threshold_scores: Dict[str, Union[float, int]] = None #for multi-label use cases
)Multi-Class Actual Values
Log multi-class actual values by using the MultiClassActualLabel object, which can be assigned to the actual_label parameter.
Actual values are optional, and can be sent later via delayed actuals.
class MultiClassActualLabel(
actual_scores: Dict[str, Union[float, int]]
)For more information on Python Single Record Logging API Reference, visit here:
Single Record LoggingInferring Labels From Uploaded Scores
To calculate metrics and visualize & troubleshoot data for multi-class models, Arize automatically infers prediction & actual labels from the scores that you upload.
Learn how each case is determined below.
Single-Label
For each prediction, the class with the highest prediction score is the prediction label
The class with an actual score of 1 is the actual label
Multi-Label
For each class, there must exist a prediction score and threshold score. If the prediction score > threshold score, the class is a part of the prediction label
Each class with an actual score of 1 is part of the actual label
Last updated
Was this helpful?