Image Segmentation

Arize_Tutorial_CV_Instance_Segmentation.ipynbGoogle Docs

Instance

What is Instance Segmentation?

Instance segmentation assigns a class label to each polygon but also distinguishes between different objects of the same class, providing unique instances for each object.

Logging Instance Segmentation Data

To log instance segmentation data into Arize, use the following code:

from arize.utils.types import (
    Schema,
    EmbeddingColumnNames,
    InstanceSegmentationPredictionColumnNames,
    InstanceSegmentationActualColumnNames
)

embedding_feature_column_names = {
    "image_embedding": EmbeddingColumnNames(
        vector_column_name="image_vector", link_to_data_column_name="url"
    )
}
instance_segmentation_prediction_column_names = InstanceSegmentationPredictionColumnNames(
    polygon_coordinates_column_name="prediction_polygons",
    categories_column_name="prediction_categories",
    scores_column_name="prediction_scores",
    bounding_boxes_coordinates_column_name="prediction_bboxes"
)

instance_segmentation_actual_column_names = InstanceSegmentationActualColumnNames(
    polygon_coordinates_column_name="actual_polygons",
    categories_column_name="actual_categories",
    bounding_boxes_coordinates_column_name="actual_bboxes"
)

Semantic

What is Semantic Segmentation?

Semantic segmentation assigns a class label to each pixel in an image. It groups all polygons of the same class into a single group, without distinguishing individual object instances.

Logging Semantic Segmentation Data

To log semantic segmentation data into Arize, use the following code:

from arize.utils.types import (
    Schema,
    EmbeddingColumnNames,
    SemanticSegmentationColumnNames,
)
embedding_feature_column_names = {
    "image_embedding": EmbeddingColumnNames(
        vector_column_name="image_vector", link_to_data_column_name="url"
    )
}
semantic_segmentation_prediction_column_names = SemanticSegmentationColumnNames(
    polygon_coordinates_column_name="prediction_polygons",
    categories_column_name="prediction_categories",
)

#OPTIONAL: Log actual data 
semantic_segmentation_actual_column_names = SemanticSegmentationColumnNames(
    polygon_coordinates_column_name="actual_polygons",
    categories_column_name="actual_categories",
)

Sending Data to Arize

For both semantic and instance segmentation, use the OBJECT_DETECTION model type when sending data to Arize:

response = arize_client.log(
    dataframe=test_demo,
    schema=schema,
    model_id=model_id,
    model_version=model_version,
    model_type=ModelTypes.OBJECT_DETECTION,
    environment=Environments.PRODUCTION,
)

Note: You need to define your schema appropriately based on whether you're using semantic or instance segmentation. The schema should include your embedding feature column names and the respective segmentation column names to view polygons in the platform.