Mutual information and diagnostic testing primer
Assume that a test (T) is used to examine whether a disease (D) is present in a group of N patients. For a diagnostic test, the values of specificity, sensitivity as well as the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) results depend on whether the test turns out to be positive \(\left(T+\right)\), with probability t , or negative\(\left(T-\right)\), and whether the disease is present\(\left(D+\right)\), with probability p , or absent\(\left(D-\right)\). To assist the reader, Table 1 summarizes the calculations of specificity, sensitivity, TP, TN, FP, and FN. Unabridged derivations are presented in the appendix.
The uncertainty of the state of disease prior to performing the diagnostic test is best expressed as entropy 4,15,18:
\(H\left(D\right)=-\left(p\operatorname{}p+\left(1-p\right)\operatorname{}\left(1-p\right)\right)\),
where \(p\) is the probability of disease. The uncertainty due to the test is:
\begin{equation} H\left(T\right)=-\left(\text{\ t}\operatorname{}t+\left(1-t\right)\operatorname{}\left(1-t\right)\right),\nonumber \\ \end{equation}
where \(t\) is the probability of disease estimated by the diagnostic test T.
The MI is computed as:
\begin{equation} I\left(D,T\right)=H\left(D\right)+H\left(T\right)-H\left(D,T\right).\nonumber \\ \end{equation}
where \(H\left(D,T\right)\) is the joint entropy of disease and diagnostic test. MI can also be expressed in terms of the conditional entropy as well as the conditional probabilities of every test/disease outcome combination:
\begin{equation} H\left(D\middle|T\right)=H\left(D,T\right)-H\left(T\right)\nonumber \\ \end{equation}
Hence, the mutual information is also defined as:
\begin{equation} I\left(D,T\right)=H\left(D\right)-H\left(D\middle|T\right)\nonumber \\ \end{equation}
From the latter expression it is evident that MI explicitly describes the amount of diagnostic uncertainty that can be reduced by the diagnostic test. Clinically, it is particularly useful to express MI in relative terms, as it can indicate explicitly the percentage of diagnostic uncertainty a diagnostic test can reduce. Relative MI (RMI) is defined as:
\begin{equation} I_{R}\left(D,T\right)=\frac{I\left(D,T\right)}{H\left(D\right)}=1-\frac{H(D|T)}{H(D)}\nonumber \\ \end{equation}
The quantity \(\frac{H(D|T)}{H(D)}\), is the relative entropy associated with the test result (i.e. the percentage of uncertainty reduced by the test result).