Co-Authored by Richard Young, Director, Partner Solutions Architecture & Noah Smolen, Head of Partnerships.
We’re excited to announce that Arize AX now supports NVIDIA NIM as a native AI model provider. Enterprises running NIM-deployed models can now connect them directly to the Arize platform and access them from the playground, run experiments, online evaluations, and enable production monitoring — now as a dedicated first-class integration within Arize.
What is NVIDIA NIM?
NVIDIA NIM microservices, part of NVIDIA AI Enterprise, are easy-to-use microservices designed for secure, reliable deployment of high-performance AI model inferencing across clouds, data centers, and workstations. Each NIM deployment includes a model paired with a production-optimized inference engine and exposes an API for application integration.
You can deploy NIM anywhere NVIDIA GPUs are available — whether in the cloud, an on-prem data center, or a local workstation.
NVIDIA NIM supports an extensive catalog of model families including NVIDIA Nemotron, Meta Llama, Mistral, Mixtral, and others. It’s designed for teams that need high-performance, self-hosted inference with full control over where their data lives. For enterprises with data residency requirements or compliance constraints, self-hosted NIM is often the only viable path to production. This complements Arize’s enterprise-ready self-hosted deployment option for Arize AX.
Our native integration now also enables seamless access to models through build.nvidia.com, NVIDIA’s developer platform for exploring, testing, and deploying NIM microservices and NVIDIA AI Blueprints. As a central hub for experimentation, it gives developers a fast path from concept to deployment using production-ready APIs and workflows.
Why Native NIM Support Matters
Deploying a model through NIM is fast, but understanding how that model behaves in production takes much longer. This integration bridges that gap, accelerating the time to discover which prompts fail, which responses drift, and where edge cases slip through.
For enterprise teams building agentic systems, the challenge is even greater. Model outputs trigger downstream actions, creating complex systems with feedback loops where silent failures can carry real consequences. Pre-deployment benchmarks aren’t enough — teams need continuous evaluation against production data, not just a snapshot from last month’s test suite.
With NVIDIA NIM natively integrated in Arize AX, teams get both layers in one place: NVIDIA’s inference performance and model access, plus Arize’s evaluation and improvement workflows. No custom endpoint configuration. No wrapper code. Simply connect your NIM endpoint under Settings → AI Providers, and your models are immediately available across playground, experiments, and evaluations.
Arize AX’s Role in the NVIDIA Ecosystem
With this integration, the collaboration between Arize and NVIDIA grows even stronger.
The Arize integration with NVIDIA NeMo enables teams to build and manage continuously improving agentic systems by making it seamless to implement data flywheels. Arize surfaces production failures through online evaluations, routes problem cases for human annotation, and triggers fine-tuning jobs through NVIDIA NeMo Customizer. NVIDIA NIM serves as the inference layer that closes the loop, deploying the improved model back into production.
With NIM now a native provider in Arize, the full cycle runs through a connected workflow:
- Deploy a model via NIM
- Observe and evaluate production traffic in Arize
- Curate failure examples into labeled datasets
- Fine-tune an improved model with NeMo Customizer
- Validate with Arize experiments, redeploy via NIM, and repeat
Built on Arize’s OpenTelemetry-based observability architecture, Arize integrates seamlessly with any orchestration framework or agent stack. When NIM serves as the inference layer, Arize provides full visibility and continuous evaluation across all workflows.
The Path Forward
As more enterprise teams move AI inference on-premises — driven by compliance requirements, latency needs, or data residency rules — the NVIDIA NIM-powered infrastructure layer and the Arize evaluation layer need to work together natively. This integration is a step toward making that the default, not the exception.
For organizations building on NVIDIA AI Infrastructure, Arize provides the observability foundation needed to deploy models confidently, evaluate them continuously, and improve them systematically.