Assessing agreement between malaria slide density readings
1 Tropical Epidemiology Group, Department of Epidemiology and Public Health, London School of Hygiene and Tropical Medicine, London, UK
2 Department of Microbiology, Immunology and Tropical Medicine, The George Washington University, Washington, USA
3 Department of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, UK
4 Infectious Diseases Unit, Department of Medicine, Karolinska University Hospital, Karolinska Institutet, Stockholm, Sweden
5 Department of Parasitology, Muhimbili University of Health and Allied Sciences, Dar es Salaam, Tanzania
6 Division of International Health, Karolinska Institutet, Stockholm, Sweden
7 The Nordic school of Public Health, Gothenburg, Sweden
8 Health Protection Agency Malaria Reference Laboratory, London School of Hygiene and Tropical Medicine, Keppel Street, London, UK
Malaria Journal 2010, 9:4 doi:10.1186/1475-2875-9-4Published: 4 January 2010
Several criteria have been used to assess agreement between replicate slide readings of malaria parasite density. Such criteria may be based on percent difference, or absolute difference, or a combination. Neither the rationale for choosing between these types of criteria, nor that for choosing the magnitude of difference which defines acceptable agreement, are clear. The current paper seeks a procedure which avoids the disadvantages of these current options and whose parameter values are more clearly justified.
Methods and Results
Variation of parasite density within a slide is expected, even when it has been prepared from a homogeneous sample. This places lower limits on sensitivity and observer agreement, quantified by the Poisson distribution. This means that, if a criterion of fixed percent difference criterion is used for satisfactory agreement, the number of discrepant readings is over-estimated at low parasite densities. With a criterion of fixed absolute difference, the same happens at high parasite densities. For an ideal slide, following the Poisson distribution, a criterion based on a constant difference in square root counts would apply for all densities. This can be back-transformed to a difference in absolute counts, which, as expected, gives a wider range of acceptable agreement at higher average densities. In an example dataset from Tanzania, observed differences in square root counts correspond to a 95% limits of agreement of -2,800 and +2,500 parasites/μl at average density of 2,000 parasites/μl, and -6,200 and +5,700 parasites/μl at 10,000 parasites/μl. However, there were more outliers beyond those ranges at higher densities, meaning that actual coverage of these ranges was not a constant 95%, but decreased with density. In a second study, a trial of microscopist training, the corresponding ranges of agreement are wider and asymmetrical: -8,600 to +5,200/μl, and -19,200 to +11,700/μl, respectively. By comparison, the optimal limits of agreement, corresponding to Poisson variation, are ± 780 and ± 1,800 parasites/μl, respectively. The focus of this approach on the volume of blood read leads to other conclusions. For example, no matter how large a volume of blood is read, some densities are too low to be reliably detected, which in turn means that disagreements on slide positivity may simply result from within-slide variation, rather than reading errors.
The proposed method defines limits of acceptable agreement in a way which allows for the natural increase in variability with parasite density. This includes defining the levels of between-reader variability, which are consistent with random variation: disagreements within these limits should not trigger additional readings. This approach merits investigation in other settings, in order to determine both the extent of its applicability, and appropriate numerical values for limits of agreement.