Kullback-Leibler (KL) divergence

Measures the dissimilarity between two probability distributions ($P$ and $Q$), representing the relative entropy or information loss when $Q$ is used to approximate $P$.

It is non-negative, equals zero only when distributions are identical, and is asymmetric.

A simple interpretation of the KL divergence of $P$ from $Q$ is the expected excess surprisal from using the approximation $Q$ instead of $P$ when the actual is $P$.