Table 5. Comparison of overall accuracy and Cohen’s kappa
coefficient of operators’ ICT kits based on rRT-PCR Ct value
categories.
Table 5: Comparative analysis of overall accuracy and
Cohen’s kappa coefficient among the operators’ ICT kits based on their
Ct value categories. N represents the total number of samples analyzed,
while n represents only samples that tested positive. Thirty-seven (37)
samples tested negative.
Discussion
This study evaluated the diagnostic performance characteristics of these
brands of COVID-19 ICT test kits on the Ghanaian market, namely, DG
Rapid (DGT), SD Rapid (SD), and SS Rapid (SS). Rapid antigen tests offer
several advantages, including affordability, faster turnaround time, and
the ability to diagnose patients at their point-of-care. These
advantages are essential and critical, especially in resource-limited
settings, where rRT-PCR testing may not be readily available. This study
demonstrated that DG Rapid and SD Rapid antigen test kits performed
relatively better in detecting SARS-CoV-2 than SS Rapid.
The findings for DG Rapid and SD Rapid are consistent with previous
studies that have reported the effectiveness of rapid antigen tests in
detecting COVID-19 (13). These studies showed a relatively lower
sensitivity with SARS-CoV-2 antigen rapid diagnostic test kits (Ag-RDTs)
compared to the clinical reference standard, which is the real-time
reverse transcriptase- polymerase chain reaction (rRT-PCR). A recent
systematic review and meta-analysis that evaluated the accuracy of
commercially available SARS-CoV-2 Ag-RDTs revealed a pooled sensitivity
of 71.2% (13). A sensitivity of 70% and 59% (95% CI) respectively
was observed in 262 study participants in Uganda and Cameroon,
respectively, using the SD Rapid RDT (14,15). In Cameroon, the RDTs’
sensitivity of 59% (95% CI) increased to 69% (95% CI) when only
symptomatic participants were considered. Another study conducted at a
teaching hospital in northern Ghana evaluated the sensitivity of the SD
Rapid RDT with 193 participants as 64% (95% CI) (16). Our study,
however, found a sensitivity of 74% (95% CI) for SD Rapid RDT, which
is comparable to what was observed in Uganda and Cameroon. Moreover,
among the three brands, DG Rapid demonstrated a higher sensitivity of
79% (95% CI), followed by SD Rapid with 74% (95% CI). However, the
SS Rapid RDT demonstrated a lower sensitivity of 50% (95% CI) compared
to the other two brands. This lower sensitivity for SS Rapid may limit
its usefulness as a standalone diagnostic tool, as it may lead to false
negative results. False negative results may result in a delay in
diagnosis, thereby increasing the risk of virus transmission. It is
noteworthy that these values of sensitivity observed in our study are
much below the performance reported by the manufacturers (DG Rapid,
2022; SD Rapid, 2020; SS Rapid, 2021). Possible explanations for the
lower sensitivity observed could be due to factors such as variations in
the batch of ICT kits used, variations in the concentration of extracted
antigens, differences in processing techniques, and variations in the
storage conditions of the kits, especially in the market (18).
In this study, the specificity was 100% (95% CI), which is comparable
to the 100% (95% CI) stated by the manufacturers but higher than the
92% (95% CI) documented in the studies conducted in both Uganda and
Cameroon. The possible explanations given for the lower specificity in
these other studies were cross-reacting antibodies from previous
infections or variations in environmental testing temperatures (24°C -
37°C) in the general wards and the COVID-19 isolation center where the
tests were carried out (16). The high specificity exhibited by all three
test kits in our study is an important attribute as it ensures that
individuals without the virus are correctly identified, reducing the
risk of false-positive results. False positives can lead to unwarranted
quarantine, isolation, and treatment, with significant social and
economic consequences.
Regarding overall accuracy, this study reported values of 89% (for DG
Rapid), 87% (SD Rapid), and 75% (SS Rapid) at 95% CI for the three
SARS-CoV-2 ICT kits, which are lower than the manufacturers’ claims of
99%, 98%, and 99%, respectively. However, it is essential to
recognize that overall accuracy can vary with disease prevalence, making
it less reliable as a single summary measure of a test’s validity. The
prevalence-dependent nature of overall accuracy introduces challenges,
leading to warnings against its use. Estimates of overall accuracy can
be misleading when obtained from populations with significantly
different disease prevalence from the target population where the test
is intended for application (19).
To assess the agreement between the ICT kits and rRT-PCR, Cohen’s kappa
coefficient was employed. Specifically, it was used to determine the
level of agreement between the performance of the ICT kits and rRT-PCR.
Our findings indicated that only DG Rapid demonstrated excellent
agreement, while SD Rapid and SS Rapid exhibited good agreement compared
to rRT-PCR. This observation could be because DG Rapid demonstrated a
higher sensitivity of 79% and a higher NPV of 82% compared to the
other brands. Other performance indicators, specifically, specificity
and PPV, were the same (100%) for all three brands and thus did not
have an impact on the kappa value calculation. In a study conducted in
India, Cohen’s kappa calculated for SD Rapid and rRT-PCR showed a good
agreement, with a Cohen’s kappa of 64.4%(20). Another study conducted
in Ethiopia found that a SARS-CoV-2 antigen rapid test kit and rRT-PCR
had a kappa value of agreement of 73.5% which indicates good agreement
between the two tests (21). While we could not find specific research
articles on DG and SS Rapid kits, the evidence from the study conducted
in both India and Ethiopia suggests a consistent trend of good agreement
between SARS-CoV-2 antigen rapid test kits such as SD Rapid and rRT-PCR.
By convention, a lower Ct value signifies a higher viral load, while a
higher Ct value suggests a lower viral load (13). All three brands
demonstrated better detection limits for higher viral load (Ct values
≤29.99), which is often the case in the pre-symptomatic phase (1–3 days
before the onset of symptoms) and the early symptomatic phase (during
the first 5 – 7 days of illness) of SARS-CoV-2 infection (22).
Conversely, they displayed less favorable detection limits for Ct values
>35.00.. Consequently, the DG Rapid ICT kits exhibit a
lower detection threshold when Ct values exceed 35.00, despite their
excellent agreement with rRT-PCR. It is essential to acknowledge that
our study utilized frozen archived nasopharyngeal samples, which may not
represent the current situation accurately. The performance of these
antigen test kits can be influenced by factors such as viral load and
the type of specimen used for testing(13). Therefore, it is crucial to
evaluate the performance of these rapid antigen test kits on fresh
samples and in real-life settings.