
What is Kullback-Leibler (KL) Divergence?
Kullback-Leibler Divergence (KL Divergence)
The Kullback-Leibler Divergence metric is calculated as the difference between one probability distribution from a reference probability distribution. KL divergence is sometimes referred to as ‘relative entropy’ and best used when one distribution is much smaller in sample and has a large variance.
