Individual conditional explanation (ICE) plots visualize one line per instance to show how the instance’s prediction changes when a feature changes. Although ICE curves can uncover heterogeneous relationships, this approach requires drawing features on multiple overlapping surfaces, limiting its utility in practice as it is hard for humans to read when many lines overlap.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- ICE plots show per-row prediction curves as one feature sweeps across a grid while others stay fixed.
- Heterogeneous slopes reveal subgroups the model treats differently; flat average PDPs can hide this.
- Overplotting many lines reduces readability; use centered ICE, clustering, or subset sampling.
- ICE explains model behavior locally, not causal effects in the business world.
- Pair ICE with global importance and slice metrics before changing features or retraining.
ICE versus partial dependence
Partial dependence plots (PDP) average model predictions over a grid of feature values, marginalizing other inputs via the dataset. They show the global average effect.
ICE keeps other features at each row’s observed values and plots that row’s predicted response as the focal feature moves. Averaging ICE lines approximates PDP.
When ICE lines fan out with different slopes, the model’s relationship with the feature is interaction-heavy or cohort-specific. A single PDP slope would mislead.
How to build ICE plots
Pick a focal feature and a grid of values (within training support).
For each row, replace only that feature with grid values, hold others constant, score the model, plot prediction versus grid value.
One line equals one instance. Color by cluster, cohort, or outcome to reduce visual noise.
Centered ICE (CICE) subtracts each line’s value at a reference point so lines fan around zero, easing comparison.
Practical uses
Credit and fraud teams inspect whether score sensitivity to income differs for thin-file applicants versus established customers.
Ranking models check if score monotonicity with a price feature holds for all products or only some.
Before enforcing monotonic constraints, ICE reveals where the unconstrained model violates business rules on subsets.
Limitations
High-dimensional models with strong correlations make counterfactual sweeps off-manifold.
Too many overlapping lines defeat human parsing; aggregate or sample.
Expensive for large grids and huge models; run offline on sampled batches.
Explanations describe the fitted model, which may encode spurious patterns.
Deployment and monitoring guides in AI model lifecycle management situate interpretability reviews in release checklists.
Confidence-focused classification workflows in shipping image classification models with confidence complement ICE with slice metrics before launch.
When interpretability supports human review in complex systems, store plots or scores in eval pipelines described in LLM and agent evaluation platforms.
Sampling for readability
Draw stratified samples across outcome deciles before plotting ICE so lines represent successes and failures, not only average customers.
Cluster ICE lines with k-means on curve shapes to show representative modes instead of spaghetti plots.
Monotonicity constraints
Some credit products require score monotonicity in income or utilization. ICE reveals violations per segment before applying constrained booster training.
Practitioner checklist
Before changing production settings, confirm labels, thresholds, and monitor windows match the definitions used in your last offline eval. Snapshot dashboards when incidents start so postmortems compare apples to apples.
Run slice-level reviews on high-traffic cohorts weekly even when global metrics look flat. Many failures appear first in one locale, product line, or prompt route.
Document model version, featurizer hash, and data window in every incident ticket. Future you needs that context to interpret drift charts.
When metrics disagree (offline vs online, precision vs recall), write down the business cost of each error type before picking a fix. Metrics are proxies; costs are the decision.
Share eval harness links in release notes so support and PM teams know which golden tasks must pass before they announce improvements.
FAQ
How many ICE lines should I plot?
Start with tens to low hundreds, or cluster representatives. Full datasets rarely plot well.
Can ICE handle categorical features?
Yes by sweeping across category levels with other features fixed; watch unsupported combinations.
Is ICE the same as SHAP dependence plots?
Both show feature effects; SHAP dependence uses attribution values while ICE uses raw predictions over sweeps.
Should ICE drive feature removal?
Use it to generate hypotheses; validate with ablation and slice performance before dropping features.
Do ICE plots work on LLM scorers?
Only when inputs are structured features feeding a differentiable or tree scorer, not free-form text alone.
