Mutual information and diagnostic testing primer
Assume that a test (T) is used to examine whether a disease (D) is
present in a group of N patients. For a diagnostic test, the
values of specificity, sensitivity as well as the counts of true
positive (TP), true negative (TN), false positive (FP), and false
negative (FN) results depend on whether the test turns out to be
positive \(\left(T+\right)\), with probability t , or negative\(\left(T-\right)\), and whether the disease is present\(\left(D+\right)\), with probability p , or absent\(\left(D-\right)\). To assist the reader, Table 1 summarizes the
calculations of specificity, sensitivity, TP, TN, FP, and FN. Unabridged
derivations are presented in the appendix.
The uncertainty of the state of disease prior to performing the
diagnostic test is best expressed as entropy 4,15,18:
\(H\left(D\right)=-\left(p\operatorname{}p+\left(1-p\right)\operatorname{}\left(1-p\right)\right)\),
where \(p\) is the probability of disease. The uncertainty due to the
test is:
\begin{equation}
H\left(T\right)=-\left(\text{\ t}\operatorname{}t+\left(1-t\right)\operatorname{}\left(1-t\right)\right),\nonumber \\
\end{equation}where \(t\) is the probability of disease estimated by the diagnostic
test T.
The MI is computed as:
\begin{equation}
I\left(D,T\right)=H\left(D\right)+H\left(T\right)-H\left(D,T\right).\nonumber \\
\end{equation}where \(H\left(D,T\right)\) is the joint entropy of disease and
diagnostic test. MI can also be expressed in terms of the conditional
entropy as well as the conditional probabilities of every test/disease
outcome combination:
\begin{equation}
H\left(D\middle|T\right)=H\left(D,T\right)-H\left(T\right)\nonumber \\
\end{equation}Hence, the mutual information is also defined as:
\begin{equation}
I\left(D,T\right)=H\left(D\right)-H\left(D\middle|T\right)\nonumber \\
\end{equation}From the latter expression it is evident that MI explicitly describes
the amount of diagnostic uncertainty that can be reduced by the
diagnostic test. Clinically, it is particularly useful to express MI in
relative terms, as it can indicate explicitly the percentage of
diagnostic uncertainty a diagnostic test can reduce. Relative MI (RMI)
is defined as:
\begin{equation}
I_{R}\left(D,T\right)=\frac{I\left(D,T\right)}{H\left(D\right)}=1-\frac{H(D|T)}{H(D)}\nonumber \\
\end{equation}The quantity \(\frac{H(D|T)}{H(D)}\), is the relative entropy associated
with the test result (i.e. the percentage of uncertainty reduced by the
test result).