Serum Samples

We used 14 serum samples (coded A–N) from persons who had received nonadjuvanted or adjuvanted split-product vaccine derived from reassortant clade 1 virus (A/Vietnam/1194/2004 or A/Vietnam/1203/2004). Samples included 3 prevaccination and postvaccination paired samples and 4 postvaccination samples. Postvaccination samples were generally obtained within 42 days of vaccination. Samples A and L were identical duplicates from a person primed with clade 0 vaccine and boosted with adjuvanted clade 1 vaccine. Sample K was from a person known to show nonspecific assay reactivity for antibodies to H5. Sample M was pooled negative human serum. Sheep antiserum to influenza subtype H5N1 (samples O and P) was supplied by the US National Institute of Allergy and Infectious Diseases and the UK National Institute for Biologic Standards and Controls, respectively. Serum P was produced by intramuscular administration of 20 µg hemagglutinin (HA) from A/Vietnam/1194/NIBRG-14 with Freund complete adjuvant, followed by 3 more injections with 10 µg HA with Freund incomplete adjuvant. Serum was collected 5 weeks after the first vaccination. Serum O was produced in a similar manner, with bromelin-cleaved purified HA. A single animal was vaccinated with A/turkey/Wisconsin/68 H5 bromelin-cleaved HA and received booster vaccinations with purified HA from A/Vietnam/1203/2004.

Candidate Standard 07/150

Standard 07/150 contained freeze-dried plasma from 9 persons who had received inactivated whole-virus A/Vietnam/1194/NIBRG-14 vaccine. Four donations (total volume 3 L) were obtained from Omninvest, Hungary (vaccine contained aluminum phosphate), and 5 donations (total volume 2 L) were obtained from Sinovac, People’s Republic of China (vaccine contained aluminum hydroxide). Persons gave informed consent after studies had received approval by appropriate ethics committees. Donations were negative for antibodies to HIV-1, HIV-2, hepatitis B surface antigen, and hepatitis C RNA. Plasma was pooled and freeze dried at the National Institute for Biologic Standards and Controls, according to standard procedures (15) to produce 1-mg ampoules and stored at –20°C. Stability studies found no significant change in titers after 8 months at –20°C, +4°C, or +20°C when compared with samples stored at –70°C.

Virus Reagents

Reassortant subtype H5N1 influenza viruses were prepared by reverse genetics from wild-type viruses, amplified in 10-day-old embryonated hens’ eggs, and stored at –80°C. Each virus passed internationally approved safety testing (16), which permitted use at Biosafety Level 2–enhanced facilities. The National Institute for Biologic Standards and Controls supplied NIBRG-14 (A/Vietnam/1194/2004, clade 1) and NIBRG-23 (A/turkey/Turkey/1/2005, clade 2.2) viruses, and the Centers for Disease Control and Prevention supplied IBCDC-RG5 (A/Anhui/1/2005, clade 2.3.4).

Study Design

Fifteen laboratories from 9 countries agreed to participate and were assigned a code from 1 to 15. One additional laboratory returned titers from 1 neutralization and 1 pseudotype assay and was excluded from analysis.

The participating laboratories were sent reagents on solid CO₂, asked to store serum at –20°C and viruses at –70°C, and instructed to reconstitute 07/150 with 1 mL distilled water and to test it and the serum for antibodies to each antigen, using hHI and neutralization, on at least 3 separate occasions. Suggested protocols were supplied, but participating laboratories could use in-house assays.

Statistical Analyses

Neutralization and hHI data consisted of replicate absolute titers, expressed as the reciprocal of serum dilution, and represented the last dilution giving a positive response from a doubling-dilution series. If the initial dilution did not give a positive response, the titer was recorded as less than the reciprocal initial dilution, e.g., <10 if the starting dilution was 1:10. Serum was interpreted as negative if no titer was detected and positive if any titer was detected. For calculation, negative titers were assigned the value of half the minimum detectable titer, and titers greater than the final dilution were assigned a value 2× the largest titer. These values represent the hypothetical adjacent dilution steps in the doubling-dilution series. This convention enables comparison of overall mean titers among groups on a consistent basis.

We calculated the geometric mean titer (GMT) for each serum, virus, and assay combination. Overall titers were calculated as the GMT of the individual laboratory means. Interlaboratory variation was expressed as percentage geometric coefficient of variation (%GCV) between the individual laboratory GMTs. The distribution of hHI or neutralization titers does not represent a continuous variable, and the results from using different viruses within laboratories are not independent. Thus, use of parametric modeling techniques, such as analysis of variance, to characterize intra- and interlaboratory variability was precluded.

To assess intralaboratory variation, we calculated the percentage of endpoints of replicate tests for identical serum samples A and L that differed >2-fold or >4-fold for each antigen and assay in each laboratory. We also compared the percentage of replicate tests returned for all serum samples and postvaccination samples that differed by >2-fold or >4-fold for each antigen and assay.

To assess interlaboratory variation, we compared differences between hHI and neutralization GMTs for 07/150 by different laboratories by using a paired nonparametric Wilcoxon signed-rank test for each antigen separately. For each laboratory, the difference in GMT between hHI and neutralization for NIBRG-14 was calculated, and these differences were compared with zero by using the Wilcoxon signed-rank test. Similarily, the results for hHI with each antigen were compared, taking the laboratory differences between the hHI GMT for NIBRG-14 and NIBRG-23 and the differences between NIBRG-14 and IBCDC-RG5 and comparing these differences with zero. The same was done for neutralization assays. We also compared differences among overall (for all laboratories) mean GMT for all serum samples by using a paired nonparametric Wilcoxon signed-rank test for each antigen separately; e.g., for NIBRG-14, the difference between the overall mean GMT for hHI and neutralization was calculated for each sample, and these differences were compared with zero. The nonparametric tests use the ranks of observed titers to calculate the significance of differences among groups and are unaffected by the value chosen to represent titers below the initial dilution or greater than the highest dilution used in the individual assays.

To assess a standard’s ability to improve interlaboratory agreement, we expressed titers relative to 07/150 by taking the ratio of the GMT for a sample to the GMT for 07/150 and multiplying it by an assigned value for 07/150. The assigned value was the overall GMT by hHI and neutralization. The effect on interlaboratory agreement and %GCV is independent of the value chosen.

To evaluate improvement in interlaboratory agreement from expressing titers relative to 07/150 (or sheep antiserum), we calculated %GCV between laboratory GMTs, both absolute and relative, for each sample. We then calculated the difference between the %GCV of the laboratory GMT of absolute titers and the %GCV of the laboratory GMT of relative titers. Using the Wilcoxon signed-rank test for each antigen separately, we compared these differences with zero.

Assays

All participating laboratories returned at least 3 replicates by both assays, except laboratory 11, which did not perform hHI. Negative serum M was excluded because all titers were negative, except in laboratory 3, which reported 1 positive ( titer 45) and 2 negative neutralization titers.

Intralaboratory Reproducibility: Identical Samples A and L

The numbers of intralaboratory comparisons of samples A and L differing by >2-fold were 1 (2.4%), 2 (4.8%), and 1 (2.4%) of 42 for NIBRG-14, NIBRG-23, and IBCDC-RG5, respectively, by hHI compared with 3 (7.1%), 3 (7.1%), and 1 (2.4%) of 42 by neutralization. Overall, 4 of 126 (3.1%) comparisons of identical samples by hHI differed by >2-fold compared with 7 of 127 (5.5%) by neutralization. No samples differed by >4-fold by either assay.

Intralaboratory Reproducibility: Replicate Assays

Figure 1

Figure 1. Intralaboratory reproducibility showing proportion (%) of replicate assays differing by >2-fold (A, B) and >4-fold (C, D) by horse hemagglutination-inhibition and neutralization assays for each participating laboratory for all serum samples...

The proportion of serum samples for which replicate assays differed by >2-fold and >4-fold was assessed in each laboratory for all serum and postvaccination serum samples (Figure 1). By hHI, 13 of 14 (93%) laboratories reported >90% replicate titers within a 2-fold range. By neutralization, intralaboratory reproducibility was more variable; 9 of 15 (60%) laboratories reported >90% replicate titers within a 2-fold range. For postvaccination serum, greater variability was found with neutralization; laboratories 7, 14, and 15 reported >25% replicates differing by >2-fold. Three laboratories reported 3.7%–7.4% of replicate samples differing by >4-fold by either assay.

Interlaboratory Reproducibility of Absolute Titers: 07/150

All laboratories reported positive titers to 07/150, but variation was marked (Table 1). By hHI, the GMTs, ranges, and %GCVs in all laboratories to NIBRG-14, NIBRG-23, and IBCDC-RG5 were 140 (25–406; 16-fold difference, 112%), 102 (25–320; 13-fold difference, 109%), and 91 (25–256; 10-fold difference, 101%), respectively. By neutralization, the GMTs, ranges, and %GCVs of titers to NIBRG-14, NIBRG-23, and IBCDC-RG5 were 518 (127–2032; 16-fold difference, 120%), 291 (52–1810; 35-fold difference, 140%), and 299 (80–806; 10–fold difference, 109%), respectively. Titers were higher with neutralization than with hHI (p = 0.003, NIBRG-14; p = 0.005, NIBRG-23; p = 0.005, IBCDC-RG5; Wilcoxon signed-rank test for all comparisons). The GMT for NIBRG-14 was higher than that for clade 2 viruses (hHI: p = 0.039, NIBRG-23; p = 0.028, IBCDC-RG5; neutralization: p = 0.004, NIBRG-23; p = 0.008, IBCDC-RG5; Wilcoxon signed-rank test).

Interlaboratory Reproducibility of Absolute Titers

GMT and %GCV for each serum sample, virus, and assay were calculated (Table 2). For postvaccination serum, both assays showed higher titers to NIBRG-14 than to clade 2 viruses. For all serum, neutralization gave higher titers than hHI (p = 0.001, NIBRG-14; p = 0.008, NIBRG-23; p = 0.001, IBCDC-RG5; Wilcoxon signed-rank test) and fewer negative values but displayed more range variation, particularly in prevaccination serum. Absolute titers for sheep serum samples O and P were highly variable; %GCV was 147%–582% for hHI and 117%–283% for neutralization.

When summarized over all serum samples, the best interlaboratory agreement was for IBCDC-RG5 by hHI and neutralization; %GCVs were 108% and 112%, respectively (Table 3). The worst interlaboratory agreement was for neutralization with NIBRG-23 and NIBRG-14; %GCVs were 185% and 183%, respectively.

Reproducibility of Relative Titers: 07/150 or Sheep Serum as Standard

Figure 2

Figure 2. Range of hemagglutination-inhibition (HI) and neutralization titers to clade 1 homologous NIBRG-14 virus in postvaccination serum sample F: the number of laboratories reporting specific titer dilution of absolute titers and titers...

Figure 3

Figure 3. Range of hemagglutination-inhibition (HI) and neutralization titers to clade 2.2 and 2.3 heterologous NIBRG-23 and IBCDC-RG5 viruses in postvaccination serum sample F: the number of laboratories reporting specific titer dilution of...

To evaluate the ability of 07/150 to improve interlaboratory agreement, GMTs were expressed relative to 07/150 for each sample (Table 2) and then summarized for all samples (Table 3). For all serum, interlaboratory reproducibility improved significantly for NIBRG-14; the median %GCV for hHI decreased from 125% to 61% (p = 0.001) and for neutralization from 183% to 81% (p = 0.002, Wilcoxon signed-rank test). However, for clade 2 viruses, interlaboratory variation did not change significantly. Figures 2 and 3 display the range of absolute and relative 07/150 titers for each antigen in serum F (shown as an example of postvaccination serum with midpoint GMT and wide range of values).

For sheep antiserum, the interlaboratory variability was increased because some laboratories reported negative hHI titers, resulting in high %GCV when test serum samples were expressed relative to them (Table 3). However, when these laboratories were excluded from analysis, the interlaboratory variation for NIBRG-14 by hHI became comparable to that found for 07/150. Laboratory 5 reported negative hHI titers for serum P; when that laboratory was excluded from analysis, the range of %GCV by hHI improved from 689%–796% to 51%–71%. Laboratories 5, 6, and 12 reported negative hHI titers for serum O; when they were excluded, the range of %GCV improved from 306%–442% to 39%–113%. When neutralization titers were expressed relative to serum O, interlaboratory variation to NIBRG-14 was reduced, in contrast with serum P, for which interlaboratory variation by hHI or neutralization did not improve for any antigen, even when laboratory 5, which failed to detect antibody in this sample, was excluded.

Relationship between hHI and Neutralization

Because a serum HI titer ≈40 is considered seroprotective (2), the establishment of a consistent equivalence factor between an hHI titer of 40 and neutralization would be useful. However, the relationship of hHI and neutralization is dependent on the virus-serum-laboratory combination and cannot be generalized. Equivalence factors display large differences of 0.1–40.3 based on absolute titers and 0.3–6.3 based on titers relative to 07/150 for NIBRG-14 (Appendix Table 1).

Serum K: False-positive Serum

Serum K was prevaccination serum from a person with detectable antibodies against influenza subtype H5 (by neutralization and hHI in the trial laboratory) but no known exposure to influenza subtype H5N1. Using NIBRG-14, NIBRG-23, and IBCDC-RG5 as test antigens, 13/14 (93%), 12/14 (86%), and 3/14 (21%) laboratories reported positive titers by both hHI and neutralization, respectively.

Assay Operating Protocols

Thirteen laboratories supplied hHI protocols. Although similar (Appendix Table 2), they differed in some respects: pretest serum hemabsorption, erythrocyte suspension concentration (<1% vol/vol or >1% vol/vol), and time and temperature of erythrocyte-settling period (60 or >120 min, 4°C or room temperature). Although no relationship between protocol and intralaboratory reproducibility was found, laboratories that used lower erythrocyte concentrations or read plates at 4°C tended to report higher titers. Laboratories that performed pretest hemabsorption tended to report lower titers.

Thirteen laboratories supplied neutralization protocols (Appendix Table 3) that were grouped into 3 broad methods: use of cell suspension for virus infection with short incubation time to endpoint (<26 hours), use of cell suspension with long incubation (>3 days), and use of cell monolayer for infection with long incubation (>3 days). Although no parameters were clearly associated with reproducibility, laboratories that used monolayers tended to report lower titers than those that used cell suspensions, and those that used longer incubation times had more interlaboratory variation by more frequently reporting titers at either end of the range (i.e., highest or lowest) than laboratories that used shorter times. Expression of initial serum dilution varied among laboratories as dilution of test serum was calculated either before or after the addition of virus.