ML Observability Fundamentals

As more machine learning models are deployed into production, it’s imperative to have the right skillset to monitor, troubleshoot, and explain model performance. Our self-paced, ML Observability Fundamentals Course is designed help data scientists and ML practitioners gain confidence taking their models from research to production. Through our pre-recorded instructor videos, checkpoint questions and unit labs, you will gain a hands-on understanding of how to identify where a model is underperforming, troubleshoot model and data issues, and how to proactively mitigate future degradations.

Enroll now

Course Curriculum

Introduction Welcome to ML Observability

Overview of the ML Observability course, what to expect and tools you will need to complete the course. Find out how this course will help you gain confidence in taking models from research to production.

Video icon

Course video (7 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (7 minutes)

Enroll now →
Unit 1 Performance tracing

ML performance tracing is the methodology for pinpointing the source of a model performance problem and mapping back to the underlying data issue causing that problem. Learn how to performance trace a model to find the root cause of the performance degradation by comparing different model versions at the slice level.

Video icon

Course video (8 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (6 minutes)

Enroll now →
Unit 2 Drift Detection

Learn how to identify the various types of drift in productionalized ML models. From data drift to concept drift, these different forms of data movement often go undetected as they occur gradually over time, so it’s crucial to drift trace a model to find and resolve the root cause of the model’s recent drift.

Video icon

Course video (10 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (6 minutes)

Enroll now →
Unit 3 Data Quality Management

Learn how to get your own ML models into the Arize platform in order to begin monitoring for performance, drift, and common data quality issues in production.

Video icon

Course video (7 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (6 minutes)

Enroll now →
Unit 4 Feature Importance & Explainability

Learn about how teams in industry are leveraging Explainability methods to compute feature importance values and how these values can be leveraged in production RCA.

Video icon

Course video (8 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (5 minutes)

Enroll now →
Unit 5 Fairness & Bias Tracing

Learn about the types of bias that can seep into your ML models in production, as well as how to uncover the features and cohorts likely contributing to algorithmic bias.

Video icon

Course video (12 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (5 minutes)

Enroll now →
Unit 6 Unstructured Data & Embeddings

Applying ML Observability to unstructured use cases is especially important due to the lack of visibility into how embeddings perform regularly. Learn how to analyze embedding drift in unstructured data and troubleshoot with an interactive 2D or 3D UMAP.

Video icon

Course video (10 minutes)

Quiz

Quiz (10 minutes)

Lab

Lab (6 minutes)

Enroll now →

Skills you will learn

ML Monitoring in Production
A/B Testing for Models and Model Versions
Model Drift and Performance Diagnosis
Model Fairness & Bias Evaluations
Feature Importance & Model Explainability
Unstructured Data & Embedding Tracking

Technical requirements

Python version
OS

Earn your certificate in only a few hours

Enroll now
Arize University certificate

Completion requirements

Upon completing all units of the ML Observability Curriculum, you will receive a Certificate of Completion to highlight your new skillset. Completion of all the unit labs, as well as the passing of each unit quiz are required for completion.

Join the Arize Machine Learning Observability Course

Enroll for free