Evaluation of the Palutop+4 malaria rapid diagnostic test in a non-endemic setting

Abstract

Background

Palutop+4 (All. Diag, Strasbourg, France), a four-band malaria rapid diagnostic test (malaria RDT) targeting the histidine-rich protein 2 (HRP-2), Plasmodium vivax- specific parasite lactate dehydrogenase (Pv-pLDH) and pan Plasmodium-specific pLDH (pan-pLDH) was evaluated in a non-endemic setting on stored whole blood samples from international travellers suspected of malaria.

Methods

Microscopy corrected by PCR was the reference method. Samples include those infected by Plasmodium falciparum (n = 323), Plasmodium vivax (n = 97), Plasmodium ovale (n = 73) and Plasmodium malariae (n = 25) and 95 malaria negative samples.

Results

The sensitivities for the diagnosis of P. falciparum, P. vivax, P. malariae and P. ovale were 85.1%, 66.0%, 32.0% and 5.5%. Sensitivities increased at higher parasite densities and reached 90.0% for P. falciparum >100/μl and 83.8% for P. vivax > 500/μl. Fourteen P. falciparum samples reacted with the Pv-pLDH line, one P. vivax sample with the HRP-2 line, and respectively two and four P. ovale and P. malariae samples reacted with the HRP-2 line. Two negative samples gave a signal with the HRP-2 line. Faint and weak line intensities were observed for 129/289 (44.6%) HRP-2 lines in P. falciparum samples, for 50/64 (78.1%) Pv-pLDH lines in P. vivax samples and for 9/13 (69.2%) pan-pLDH lines in P. ovale and P. malariae samples combined. Inter-observer reliabilities for positive and negative readings were excellent for the HRP-2 and Pv-pLDH lines (overall agreement > 92.0% and kappa-values for each pair of readers ≥ 0.88), and good for the pan-pLDH line (85.5% overall agreement and kappa-values ≥ 0.74).

Conclusions

Palutop+4 performed moderately for the detection of P. falciparum and P. vivax, but sensitivities were lower than those of three-band malaria RDTs.

Background

Malaria is a widespread and life-threatening disease. Each year, 10,000 malaria cases are reported among returned international travellers, and the real number of cases is estimated at 30,000 [1]. Prompt diagnosis is essential for the treatment and outcome, and malaria rapid diagnostic tests (malaria RDTs) may be of help to non-experienced laboratory staff in non-endemic settings [2, 3].

Malaria RDTs are immunochromatographic tests targeting specific antigens of one or more Plasmodium species. Malaria RDTs are available as strips or cassettes, and display visible cherry-red to purple coloured control and test lines. The initially developed two-band malaria RDTs had, besides a control line, a Plasmodium falciparum-specific line targeting histidine-rich protein-2 (HRP-2) or P. falciparum-specific parasite lactate dehydrogenase (Pf-pLDH). Later developed three-band malaria RDTs additionally detected an antigen common to the four main Plasmodium species, such as aldolase or the pan Plasmodium-specific pLDH (pan-pLDH). In addition, a few four band malaria RDTs are on the market, such as Palutop+4 (All. Diag, Strasbourg, France). This malaria RDT detects three antigens: P. falciparum-specific HRP-2, Plasmodium vivax-specific pLDH (Pv-pLDH) and pan-pLDH. Four-band malaria RDTs may distinguish between infections with P. falciparum, P. vivax or another Plasmodium species and, therefore, the intended field of application includes non-endemic settings such as travel clinics. In this study, Palutop+4 was evaluated on stored blood samples of returned international travellers.

Methods

Study design

In this retrospective study, Palutop+4 was evaluated with a collection of stored samples obtained from international travellers. Tests were carried out in the reference laboratory of the Institute of Tropical Medicine (ITM) in Antwerp, Belgium. The study design was in compliance with the STARD guidelines for presentation of diagnostic studies [4].

Patients and samples

Samples were selected from a collection of EDTA-blood samples stored at -70°C and obtained from patients presenting at the outpatient clinic of ITM. The patients were international travellers and, to a lesser extent, immigrants returning from visits to their native countries. In addition, samples sent by Belgian laboratories to ITM in the scope of the national reference function were included. The 530 samples collected at ITM, were aliquoted and frozen at -70°C the day of collection. Between collection and storage at -70°C, the samples remained for a maximum of 10 hours at ambient temperatures below 25°C. The 83 samples submitted by Belgian laboratories to ITM for second opinion and confirmation were sent by mail and had been exposed to ambient temperature for the period of shipment, which was generally less than 24 hours and ranged to a maximum of 48 hours. The delays of shipment and processing before storage at -70°C had been validated before and were compliant with routine laboratory procedures. In addition, samples from symptomatic patients without malaria parasites (as tested with microscopy and polymerase chain reaction (PCR)) were included.

Reference method

Microscopy, corrected by PCR, was used as the reference method. Standard microscopy including determination of parasite density was performed [5]. Malaria diagnosis at Central Laboratory of Clinical Biology is accredited in accordance with the requirements of the ISO 15189:2007 norm. The laboratory technicians have received a detailed training and their performance and agreement are monitored by participation to internal and external quality control assessments. As a standard procedure, all slides with Plasmodium spp. are confirmed by a second blinded microscopist, as well as those slides that show results different from those obtained by RDTs and PCR. PCR analysis was performed on all discordant samples, with a species-specific real-time PCR as described previously [5].

Test platform

Palutop+4 is a lateral flow immunochromatographic malaria RDT in a cassette format. Four lines are present, a control line which indicates whether the test is valid, a HRP-2 line, a Pv-pLDH line and a pan-pLDH line. According to the manufacturer's instructions, Palutop+4 can detect P. falciparum, P. vivax, Plasmodium ovale and Plasmodium malariae. Further it can distinguish between P. falciparum (unique HRP-2 line visible), P. vivax (unique Pv-pLDH line visible), the other two Plasmodium species (unique pan-pLDH line visible) or mixed Plasmodium infections (all other combinations of test lines). For the evaluation, test kits of two different lot numbers were used, 91055 and 91057. Test kits were stored at a dry place between 18°C and 30°C.

Test procedure

Tests were performed according to the instructions of the manufacturer except that samples (5 μl) where loaded with a transfer pipette (Finnpipette, Helsinki, Finland) instead of the plastic loop supplied. In case the control line did not appear, the result was interpreted as invalid and the test was repeated. Test lines were scored for intensity as negative (no line visible), faint (barely visible line), weak (paler than the control line), medium (equal to the control line) or strong (stronger than the control line) [5]. Readings were performed 15 minutes after application of the sample and diluent, by three blinded observers. The results of the readings were based on consensus agreement, which means that a positive result was defined as a result read positive by at least two out of three different observers. Inter-reader reliabilities were assessed for positive and negative readings as well as for the intensity readings. To assess reproducibility, a panel of 16 samples (including four P. falciparum, four P. vivax, four P. ovale and four P. malariae samples) was tested on five successive occasions.

Statistical analysis

Samples infected with P. falciparum, P. vivax and the two other Plasmodium species were considered separately. For P. falciparum, samples with pure gametocytaemia were included among the positive samples. Tables 1, 2 and 3 list the definitions used for calculation of test characteristics for P. falciparum, P. vivax and the two other Plasmodium species respectively. Sensitivity and specificity were calculated with 95% confidence intervals (CI) and differences were tested for significance using the Yates chi-square test or, when this was not possible, the Fisher exact probability test. A p-value of < 0.05 was considered significant. Reliabilities for positive and negative readings and line intensities were calculated as percentage agreements for all three readers and kappa values for each pair of readers. Associations between line intensity readings and parasite densities were assessed for strength of association with Cramer's V for categorical variables.

Table 1 Interpretation of test results for Palutop+4 for the detection of P. falciparum*

Table 2 Interpretation of test results for Palutop+4 for the detection of P. vivax

Table 3 Interpretation of test results of Palutop+4 for the detection of P. ovale and P. malariae

Duration of storage

To examine the effect of the duration of storage on the test performance, the sensitivity of samples obtained between 1995 and 2000 was compared with those obtained between 2001 and 2008.

Ease of use

Three experienced laboratory technicians scored the ease of use of Palutop+4 test and the clarity of manufacturer's instructions with a standardized list [5].

Ethical review

The study was reviewed and approved by the Institutional Review Board of ITM and by the Ethical Committee of Antwerp University, Belgium.

Results

Sample collection

A total of 613 samples was selected,. They had been collected from December 1995 to August 2008. According to microscopy and after correction by PCR analysis, 323 of these samples were positive for P. falciparum, 97 for P. vivax, 73 for P. ovale and 25 for P. malariae. In addition, 95 microscopic and PCR negative samples were included. The majority (270/323) of P. falciparum infections was acquired in Africa. They were obtained from 613 patients, with a male-to-female ratio of 2.07:1, and median age of 36.5 years (range 1 - 84 years). Only a minority (nine patients, 1.5%) were children less than five years old.

Invalid test results

One of the 613 samples gave an invalid result at initial testing. Upon repetition, the test performed well.

Sensitivity, specificity and species mismatch

RDTs were performed between November and December 2008. Table 4 lists the results of the three test lines and Table 5 lists the sensitivities for the different species and parasite densities. For the detection of P. falciparum, the overall sensitivity was 85.1% and was related to parasite densities, with values at parasite densities > 100/μl which were significantly higher as compared to those ≤ 100/μl (90.0% versus 67.9% respectively, p < 0.001). For the detection of P. vivax, the sensitivity was 66.0%, with higher values above and lower values below the parasite density breakpoint of 500/μl (83.8% versus 24.1%, p < 0.001). The sensitivities for the detection of P. malariae and P. ovale were 32.0% and 5.5% respectively. The differences between parasite densities below and above 500/μl did not reach statistical significance. The specificities for P. falciparum, P. vivax and P. ovale/P. malariae were 96.9% (94.0%-98.5%), 100% (99.1%-100%) and 100% (99.1%-100%) respectively. The two malaria negative patients with a false positive test result (Table 4) did not have a recent history of malaria. Species mismatch was observed in 21/518 Plasmodium-positive samples (4.1%, Table 4): fourteen P. falciparum samples reacted with all three test lines, one P. vivax sample reacted exclusively with the HRP-2 line, and respectively two and four P. ovale and P. malariae samples reacted with the HRP-2 line. In the case of P. falciparum, cross-reactions with the Pv-pLDH line occurred more frequently at parasite densities of ≥ 100,000/μl compared to lower parasite densities (9/61 versus 5/262, p < 0.001). There was no relation between the duration of storage of the samples and the sensitivity, specificity and line intensities of the test.

Table 4 Test results of all samples (n = 613)

Table 5 Sensitivity of Palutop+4 for the detection of all Plasmodium species related to parasite densities*

Line intensities

For P. falciparum samples, 129/289 (44.6%) HRP-2 readings gave faint or weak line intensities. For P. vivax samples, faint and weak line intensities accounted for 78.1% (50/64) positive Pv-pLDH line readings, as well as for all (n = 50) positive pan-pLDH line readings. For the P. ovale and P. malariae samples combined, two and seven out of 13 positive pan-pLDH readings were faint and weak respectively, the remaining four showed medium line intensities. Line intensities for the HRP-2 and pan-pLDH lines were related to parasite densities (HRP-2: V = 0.568, p < 0.001; Pv-pLDH: V = 0.293, p = 0.14; pan-pLDH: V = 0.397, p < 0.001).

Of interest is that in the case of positive P. falciparum samples, the unique presence of a HRP-2 line pointed in most cases (90.5%, 115/127 samples) to a parasite density below 1000/μl. The co-presence of both HRP-2 and pan-pLDH lines predicted parasite densities above 1000/μl in 115/148 (77.7%) of samples.

Inter-reader reliability and reproducibility

For the HRP-2 line and the Pv-pLDH line, the inter-reader reliability for positive and negative test results was excellent: for the HRP-2 line, there was 92.0% agreement between the three readers and kappa values between the three pairs of readers were 0.89, 0.88 and 0.88. For the Pv-pLDH line, overall agreement was 96.4%, and kappa values were 0.91, 0.88 and 0.90. The inter-reader reliability for the pan-pLDH line was lower, with 85.5% overall agreement and kappa values of 0.74, 0.76 and 0.88 for the three pairs of readers.

For the line intensity results, the overall agreement for the HRP-2 line was 69.0% with kappa values of 0.67, 0.72, and 0.67 for the three pairs of readers. For the Pv-pLDH line intensity, the overall agreement between the three readers was 91.4% with kappa values of 0.81, 0.72 and 0.73 for the three pairs of readers. For the pan-pLDH line intensity, the overall agreement between the three readers was 71.1% with kappa values of 0.63, 0.66 and 0.70 for the three pairs of readers.

Test results (positive or negative) and line intensity readings were reproducible. Consistent test results upon five test occasions were obtained for 12 out of 16 samples. From the four remaining samples, three and one had consistent results upon four and three occasions respectively. These four samples included three P. malariae samples that showed positive and negative pan-pLDH lines upon successive testing and a single P. falciparum sample that showed an additional Pv-pLDH line on a single test occasion. All inconsistent results for line intensity readings had discordances only within one category of difference in line intensity.

Duration of storage

There was no significant difference between test sensitivities of samples that had been stored for long versus shorter periods, for any of the four Plasmodium species (Table 6).

Table 6 Comparison of test sensitivities between samples that had been stored for long and short periods

Ease of use

Palutop+4 was scored as easy to use and practical, and the instructions were scored as clear and simple to perform by all three technicians. However, in roughly one third of samples, the clearance of the test strip was not optimal and the strip remained red-coloured after completion of the test procedure.

Discussion

This retrospective study showed that Palutop+4 performed moderately well for the detection of P. falciparum and P. vivax and poorly for the detection of P. ovale and P. malariae. Sensitivities for P. falciparum and P. vivax at the parasite density breakpoints of 100/μl and 500/μl were 90.0% and 83.8%, respectively. Species mismatches occurred in 4.1% of Plasmodium-positive samples. Test results were reliable and reproducible and the test was scored as practical.

For this study, several limitations should be considered. For instance, its retrospective design impeded the possibility to test patients for known causes of false positive test results such as the presence of the rheumatoid factor [6]. Further, a transfer pipette was used instead of the test's application loop, thereby biasing any test errors due to volume error [7]. Likewise, tests were performed in a reference setting and results should not be extrapolated to field settings [8]. Furthermore, the storage-time of the samples could affect antigen stability [7], although in the present study the samples were not exposed to repeated thawing and freezing procedures. There were no significant differences in sensitivities between samples that had been stored for long versus short periods for any of the four Plasmodium species. Other evaluations, including the World Health Organization (WHO) Malaria RDT Evaluation Programme, also used stored samples [9]. In addition, for HRP-2, a prospective study revealed similar results between fresh and stored samples for HRP-2 detection: 125 samples were tested upon collection and after a storage duration of one to three years they were tested again. In only six of the 125 samples the results were discordant [10]. Finally, it should be noted that the present study used stringent criteria for defining test characteristics: as an example, the 14 P. falciparum samples that showed both the HRP-2 line and the Pv-pLDH line were scored as "species mismatches", although the error of diagnosing a mixed P. falciparum - P. vivax infection instead of a pure P. falciparum infection can be considered as a minor error. If these 14 results would have been scored as correct, sensitivity would increase to 89.5%. Although most papers on malaria RDTs do not describe in detail the criteria for defining test characteristics, it can be assumed that a previous study on Palutop+4 [11] scored this minor species-mismatch as correct, thereby explaining for the higher sensitivity.

The WHO list of malaria RDT manufacturers with adequate evidence of good manufacturing practice published online [12] actually includes seven four-band malaria RDTs. Palutop+4 is the only test for which published evaluations are available. In an endemic field setting, Rakotonirina et al found sensitivities for Palutop+4 of 95.4% and 94.2% for P. falciparum and P. vivax, respectively [11]. Unlike the present study, they did not include P. ovale or P. malariae positive patients. The present study showed sensitivities for P. falciparum and P. vivax which are lower as compared to their findings: apart from the possibility of a different definition of test characteristics discussed above, these lower results may also be due to a higher number of samples with low parasite densities and pure gametocytaemia. Further the present study included a higher amount of samples (323 versus 87 P. falciparum samples and 97 versus 17 P. vivax samples), thereby generating smaller confidence intervals. As in the present study, Rakotonirina et al also demonstrated species mismatch for one out of 87 P. falciparum samples (diagnosed as P. ovale or P. malariae) and two out of 17 P. vivax samples (one diagnosed as P. falciparum and one as P. ovale or P. malariae) [11].

Compared to other malaria RDTs in a non-endemic setting, Palutop+4 has a lower sensitivity for the detection of P. falciparum: reported sensitivities for two- and three-band malaria RDTs ranged from 87.5%-99.0% [13–18], with one exception of 76.2% [19]. In addition, the present study found the sensitivity of Palutop+4 at parasite densities above 100/μl to be lower than the 95% value recommended by the WHO [20]. The sensitivity for the detection of P. vivax (66.0%) is in line with other malaria RDTs. For the pan-pLDH, sensitivities of 35.3%[20] and 62.0%-95.0% [13, 15, 17] and for the aldolase, sensitivities of 46.0%-93.0% [15] have been reported. Studies in endemic settings that evaluated a dedicated Pv-pLDH two-band malaria RDT reported overall sensitivities of 96.4% and 93.4% [21, 22], and in a comparable study design in the Antwerp laboratory, this malaria RDT showed an overall sensitivity of 88.0% for the diagnosis of P. vivax [23]. For P. ovale and P. malariae, it is more difficult to make comparisons, as sensitivities vary a lot among different studies, with values for pan-pLDH and aldolase ranging from 36.0% to 95.0% and from 7.0% to 80.0% respectively [13, 15]. Part of the lower sensitivity of Palutop+4 for these species may be explained by the four-band design: optimal conditions, such as the pH and ionic strength of the diluent and coating of the nitrocellulose strips, must be met for three different antigen-antibody combinations, with an inevitable trade-off in test characteristics for one or more of the target antigens. Regarding species mismatches, cross-reactions of P. falciparum with high parasite density and the Pv-pLDH line have been described earlier [23]. Also, the number of P. malariae cross-reactions with the HRP-2 line was similar to other findings [5]. Cross-reactions for P. vivax or P. ovale are rarely observed [24].

Although Palutop+4 proved to be reliable and reproducible, most of the test line intensities, in particular those for Pv-pLDH and pan-pLDH, scored faint or weak. In addition, there was incomplete clearance of the test strips in about one third of specimens. These two facts may interfere with correct reading: misinterpretation of faint line intensities as negative test results is a frequent pitfall for inexperienced staff both in endemic and non-endemic settings [17, 25]. Although the unique presence of the HRP-2 line and its co-presence with the pan-pLDH line were indicative for the parasite density, this prediction was not as strict and conclusive as observed in other malaria RDTs [5, 26].

Conclusions

In view of its theoretical ability to diagnose and differentiate among at least three of the four common Plasmodium species, the intended use of Palutop+4 is in non-endemic settings. However, the lower sensitivities of Palutop+4 for the P. ovale and P. malariae species must be taken into account and compared to three-band malaria RDTs. The four-band design tends to have a lower sensitivity for the detection of P. falciparum and P. vivax, even at higher parasite densities. Further improvement of test conditions may favour the use of four-band malaria RDTs in the future.