Object Detection

Object Detection Model Overview

Object detection models identify and locate objects within images or videos by assigning them specific bounding boxes. Applicable Metrics: Accuracy, Euclidian Distance (embeddings) Click here for all valid model types and metric combinations.

Object Detection Code Example

Python Batch

Example Row

image_vector	image_link	prediction_bboxes	actual_bboxes	prediction_categories	actual_categories	actual_super_categories	prediction_scores
`[0.24713118374347687, 0.7061651349067688, 1.12...`	```javascript
”https://link-to-my-image.png”
```	`[[50.43, 109.49, 538.21...`	`[[55.39, 107.72, 539.25, 362.9], [554.41, 194....`	`[bus]`	`[bus, person, person]`	`[vehicle, person, person]`	`[0.9997552]`	```
1618590882

```python
embedding_feature_column_names={
    "image_embedding": EmbeddingColumnNames(
        vector_column_name="image_vector",
        link_to_data_column_name="url"
    )
}
object_detection_prediction_column_names=ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name="prediction_bboxes",
    categories_column_name="prediction_categories",
    scores_column_name="prediction_scores"
)
object_detection_actual_column_names=ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name="actual_bboxes",
    categories_column_name="actual_categories",
)

# feature & tag columns can be optionally defined with typing:
tag_columns = TypedColumns(
    inferred=["name"],
    to_int=["zip_code", "age"]
)

# Defina the Schema, including embedding information
schema = Schema(
    prediction_id_column_name="prediction_id",
    timestamp_column_name="prediction_ts",
    tag_column_names=tag_columns,
    embedding_feature_column_names=embedding_feature_column_names,
    object_detection_prediction_column_names=object_detection_prediction_column_names,
    object_detection_actual_column_names=object_detection_actual_column_names,
)

response = arize_client.log(
    model_id= "CV-object-detection",
    model_version= "v1",
    model_type=ModelTypes.OBJECT_DETECTION,
    environment=Environments.PRODUCTION,
    dataframe=df,
    schema=schema
)

Google Colaboratory

For more details on Python Batch API Reference, visit here:

Python Batch API Reference

Object Detection Prediction & Actual Values

Arize supports logging object detection prediction and actual values using the ObjectDetectionColumnNames object, which can be assigned to the prediction/actual schema parameters, object_detection_prediction_column_names and object_detection_actual_column_names.Object prediction or actual declaration is required to use the object detection model type in Arize.

class ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name: str
    categories_column_name: str
    scores_column_name: Optional[str] = None # actual ground truth labels wont have scores
)

Embedding Features

In addition to object detection prediction and actual values, Arize supports logging the embedding features associated with the images in an object detection model using the EmbeddingColumnNames object.

The vector_column_name should be the name of the column where the embedding vectors are stored. The embedding vector is the dense vector representation of the unstructured input. ⚠️ Note: embedding features are not sparse vectors.
The link_to_data_column_name should be the name of the column where the URL links to the source images are stored.

{ 
    "embedding_display_name": EmbeddingColumnNames(
        vector_column_name="image_vector", 
        link_to_data_column_name="image_link" 
    ) 
}

See here for more information on embeddings and options for generating them.

Quickstart

Instrument

Observe

Evaluate

Improve

Agents

Machine Learning

Settings

Security