Documentation Index
Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt
Use this file to discover all available pages before exploring further.
What’s New
August 29, 2022 Embedding Drift Monitors Use embeddings drift monitors to receive automatic alerts when your unstructured data drifts. Embedding drift is measured by calculating the Euclidean Distance between embedding vectors.
Drift Monitor Metric: JS Distance
JS distance is a symmetric derivation of KL divergence, and it is used to measure drift. Use JS distance to compare distributions with low variance to measure the similarity between two distributions. Learn how to use and calculate JS Distance here.
File Importer API
Programmatically create, query, and delete jobs & files using our public facing GraphQL API. Our GraphQL API provides a direct path between cloud storage and the Arize platform to easily set up jobs and automate job creation. Walk through a step-by-step tutorial here.Interested in using the file importer API? Reach out to support@arize.com for access.
File Importer - Dry Run
August 1, 2022 Streamline data ingestion feedback on an import job by quickly surfacing obvious errors without necessitating a read-through of entire files. Note: This does not guarantee a successful import job. Learn more about data ingestion here.
File Importer - Audit Trail
Gain a transparent view of your data and easily resolve errors with the audit trail. View a ledger of files that show:- # of records successfully imported
- Failed files with error messages
- Pending imports for each job
Enhancements
August 29, 2022Python SDK 5.0.2
-
Removed
bulk_log(). Learn how to log data to Arize_ here_. -
Deprecated
CATEGORICAL_ & _BINARY* *models. -
Model typeparameter field is now **required **(previously optional). -
Model versionis now optional (previously required). If you do not set a model version, it will appear as “no_version” in the UI. -
Schema* *type is now imported fromarize.utils.types. -
Added parameter
environmentto the real-time logger. This allows to log in real-time training/validation records.****- The
environmentparameter is required.
- The
Model Schema API
August 15, 2022 Query a model’s schema through our public-facing GraphQL API. This allows users to programmatically paginate through a model’s features, tags, predictions, and actuals, unlocking the ability to dynamically create monitors that track these values over time. An example of this functionality can be found in the Create Data Quality Monitor Colab. Learn more about the Model Schema API here, and more about our GraphQL API here.Quick Start Guide
The quick start guide gives you a brief intro to core Arize workflows. Re-launch the quickstart guide by clicking the rocketship icon on the bottom left of any page to check out the new short guides. Walk through key workflows for bias tracing, embeddings troubleshooting, performance tracing, and more.
Performance Monitor Metric: MASE
Use MASE as your evaluation metric for forecasting models to understand your model performance better. MASE is recommended to determine the comparative accuracy of forecasts.
Drift Monitor Metric: KL Divergence
Choose between PSI and KL Divergence when measuring drift. Use KL divergence if you have a distribution with a high variance. Learn more about KL divergence here.
Data Quality Monitor Metrics: Percentiles
Evaluate P50, P95, and P99 for data quality monitors to gain a more representative understanding of both your median and outlier data performance. Learn more about data quality monitors here.
Pagerduty Simple Install
August 1, 2022 Connect to PagerDuty using the simple install flow to reduce manually moving back and forth between platforms. This flow automatically directs you to your PagerDuty services to simplify the integration process. Learn more about our PagerDuty integration here.
Alerting Integration Setup - Priority/Severity Setting
Categorize the severity of your alert within a model or monitor to enhance metadata and build flows specific to your use case.
Color by Confusion Matrix
Color by Confusion Matrix is now available for the UMAP point cloud to bring performance troubleshooting to Embeddings. You can now select a Positive Class and get a high-level overview of performance and drift in the UMAP visualization.
In the News
August 29, 2022Four Crisis-Tested Lessons For Leading Effective ML Teams
Recently, Arize co-hosted an event with Vectice featuring the “Voices of ML Leaders.” Miss it? Here are four takeaways from leaders at Microsoft, Kohl’s, and Yelp on how to lead effective ML teams:- Tie Model Metrics To Business KPIs Upfront
- Invest All the Way Through the ML Lifecycle
- Consider Threading the Needle With Central ML
- Assess New Talent By Simulating Real-World Problems
Ray + Arize: Productionize ML for Scale and Usability
Arize and Ray are partnering to help teams better productionize ML for scale and usability! Learn more and about Ray’s distributed ML framework and Arize’s ML observability platform and follow along with a code example that shows the scaffolding of both technologies working in tandem in this blog by** **Dat Ngo, Arize Solutions Architect.
The Next Generation of Machine Learning Monitoring
The increasing reliance on AI systems means model monitoring needs to go the extra mile and encompass** scale** as a priority. For Arize, this means ML monitoring must accommodate hundreds or even thousands of models with thousands of features – all with unique requirements and little human intervention. Aman Khan, Group Product Manager at Arize covers how Arize’s next generation of model monitoring is helping teams catch issues in production sooner with less oversight through automation, programmatic monitoring access, and native alerting integrations with Pagerduty and OpsGenie.
Arize + Hugging Face = Better Performance, Lower Costs for Unstructured Models
August 15, 2022 Arize AI and Hugging Face are partnering to help organizations train unstructured models and monitor and troubleshoot those models in production, lowering costs and maximizing performance. Learn more about challenges with NLP models, follow along with a code example on obtaining embeddings from a transformer model, and see how Arize and Hugging Face can help improve your unstructured data workflows in this informative post.
- Hundreds of extra hours freed up per year across the team
- A payback period of under a year; >100% ROI
- Improved model performance from proactively surfacing feature drift and performance impact score at a cohort-level
- Robust drift monitoring for structured data, with the plans to implement embedding drift monitoring for NLP models
- Immediate visibility when issues arise based on predefined and automated thresholds, maximizing internal visibility and speeding up mean time-to-resolution
Why Machine Learning in Ad Tech Is Ready for Liftoff
August 1, 2022 Yunshi Zhao, an ML Engineer at Liftoff Mobile, details her career arc and shares best practices for deploying ML models in ad tech in this interview with** **Arize ML Engineer, Amber Roberts.
