Entropy
Definitions
\[ H(X)=\log\frac{1}{p_X(X)}.\]
The base of the logarithm defines the unit of entropy. If the logarithm is to the base 2, the unit of entropy is the bit. If the if the logarithm is to the base \(e\), the unit of entropy is the nat.