JS distance is a symmetric derivation of KL divergence, and it is used to measure drift. In addition to being an actual metric (as opposed to KL), it is bounded by the square root of ln(2). For two distributions P and Q, you compute a mixture distribution, measure KL divergence from each side to that mixture, average the two, and take the square root. Use JS distance to compare distributions with low variance.
If you histogram production features, JS distance returns a finite score even when one bin goes to zero, treats P versus Q the same as Q versus P, and gives you a fixed scale for dashboards.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- JS distance is the square root of Jensen-Shannon divergence, a symmetric average of KL divergences to the midpoint distribution.
- Values range from 0 (identical binned distributions) to sqrt(ln(2)), roughly 0.833, so scores are comparable across features.
- Unlike raw KL divergence, JS distance is a true metric: non-negative, symmetric, and zero only when the distributions match.
- Binning choices matter. JS distance compares histograms, not raw samples. Align bins to your reference window or use fixed edges from training data.
- JS distance shines on low-variance continuous features and embedding histograms where KL would explode or PSI over-smooths.
From KL divergence to JS distance
Kullback-Leibler divergence measures how many extra bits you need to encode samples from Q using a code optimized for P. It is useful but asymmetric: KL(P || Q) generally differs from KL(Q || P). It is also unbounded. When Q assigns zero mass where P does not, KL diverges.
Jensen-Shannon divergence fixes the symmetry problem by introducing the mixture M = (P + Q) / 2 and averaging KL(P || M) and KL(Q || M). The result is bounded between 0 and ln(2). JS distance is the square root of that value, which restores the triangle inequality and makes distance comparisons mathematically clean.
Libraries take two probability vectors and return the distance. The engineering work is upstream: choosing bins, handling zeros, and aligning reference and current windows.
Why bounded and symmetric matter in production
Drift dashboards fail when the metric scale changes with every alert. JS distance caps the range at sqrt(ln(2)). Zero means no drift under your binning. Values near the maximum mean the histograms barely overlap.
Symmetry matters when you swap baseline and current windows during investigations. KL is not symmetric. JS distance returns the same number whether you label production as P or Q.
When to use JS distance vs other drift metrics
JS distance compares binned probability mass. That makes it a sibling of PSI and a complement to Kolmogorov-Smirnov, which operates on sorted continuous values without explicit bins.
Reach for JS distance when:
- Variance is low. Features like calibrated scores clustered between 0.4 and 0.6 produce narrow histograms. JS distance detects small redistributions that mean shifts miss.
- You already bin embeddings. Cluster assignment proportions or latent-space histograms are natural inputs. JS distance summarizes how those proportions moved.
- You need a proper metric on probabilities. If downstream logic treats distance as a distance (clustering alert types, combining scores), JS qualifies. Raw KL does not.
Prefer Kolmogorov-Smirnov when you want to avoid bin width decisions and the feature is truly continuous with wide support. Prefer PSI when stakeholders expect decile buckets and you report proportional shifts in familiar language. JS distance sits between them: bin-based like PSI, but with a symmetric, bounded scale closer to what mathematicians expect from a distance.
Both JS distance and model, concept, and data drift workflows assume you defined a reference window and a current window. Document both. JS distance between this week and last week differs from this week versus training data, and the corrective action differs too.
Binning and numerical details
JS distance is only as good as your histogram. Common patterns:
- Fixed edges from training. Freeze bin boundaries when you deploy. Current traffic slides through the same grid the model saw during validation.
- Quantile bins from reference. Choose edges so each training bin holds equal mass, then apply those edges to production. Reduces sensitivity to long tails.
- Smoothing zeros. Add a tiny epsilon to every bin before normalization so KL terms stay finite. Most libraries do this implicitly; verify before trusting edge cases.
Watch bin count. Too few bins wash out real shifts; too many bins leave sparse counts that jump randomly week to week. For embedding clusters, use the cluster count itself as the bin dimension.
Because JS distance is bounded, a move from 0.05 to 0.15 is meaningful. Set thresholds per feature using historical quiet periods the same way you would for PSI or KS effect sizes.
JS distance in the monitoring stack
Instrument numeric and embedding-derived features on a schedule aligned with your AI model lifecycle management reviews. Store JS distance alongside the bin definitions, window sizes, and row counts. When drift fires during a release, you want to replay the exact histogram, not re-derive bins from memory.
Pair JS distance with histogram diffs so alerts show which bins or clusters moved, not only that something changed.
FAQ
What is the difference between JS distance and Jensen-Shannon divergence?
Jensen-Shannon divergence is the averaged KL divergence to the mixture distribution. JS distance is its square root. The divergence is bounded by ln(2); the distance is bounded by sqrt(ln(2)). Many monitoring tools report one or the other; check which version your library returns before setting thresholds.
Why use JS distance instead of KL divergence?
KL is asymmetric and unbounded. JS distance is symmetric, bounded, and defined even when bins have zero counts (after smoothing). Those properties make it easier to operationalize in drift alerts.
Does JS distance require binned data?
Yes. You compare two probability vectors, usually from histograms. Kolmogorov-Smirnov, by contrast, can work directly on sorted continuous samples without choosing bin edges.
When is JS distance a poor choice?
When binning destroys the signal (high-cardinality IDs), when sample size is too small to fill bins reliably, or when you need ordinal information preserved across the full real line without arbitrary edges. In those cases, use KS on continuous features or frequency tests on categoricals.
How do I set alert thresholds on JS distance?
Compute JS distance across rolling windows from a period with no known incidents. Use the high quantiles of that history as a starting threshold, then tune false-positive rate per feature importance. The bounded scale makes cross-feature ranking easier than with raw KL.
