Volume 19, Number 4—April 2013
Effect of 10-Valent Pneumococcal Vaccine on Pneumonia among Children, Brazil
Pneumonia is most problematic for children in developing countries. In 2010, Brazil introduced a 10-valent pneumococcal conjugate vaccine (PCV10) to its National Immunization Program. To assess the vaccine’s effectiveness for preventing pneumonia, we analyzed rates of hospitalization among children 2–24 months of age who had pneumonia from all causes from January 2005 through August 2011. We used data from the National Hospitalization Information System to conduct an interrupted time-series analysis for 5 cities in Brazil that had good data quality and high PCV10 vaccination coverage. Of the 197,975 hospitalizations analyzed, 30% were for pneumonia. Significant declines in hospitalizations for pneumonia were noted in Belo Horizonte (28.7%), Curitiba (23.3%), and Recife (27.4%) but not in São Paulo and Porto Alegre. However, in the latter 2 cities, vaccination coverage was less than that in the former 3. Overall, 1 year after introduction of PCV10, hospitalizations of children for pneumonia were reduced.
Streptococcus pneumoniae infections are the leading cause of bacterial pneumonia, meningitis, and sepsis among children (1,2); in developing countries, these infections account for almost a half million deaths among children <5 years of age (3). In Brazil, the largest country in South America, the role of S. pneumoniae in pneumonia in children is considerable (4,5).
Brazil is composed of 5 administrative regions with different climatic and socioeconomic characteristics. In 2010, the estimated population of infants (children <12 months of age) was ≈2,800,000, and the infant mortality rate was 17 deaths per 1,000 live births (6,7). In Brazil, the main reason for hospitalization of infants is pneumonia (6).
Vaccination with pneumococcal conjugate vaccine (PCV) is a public health intervention to prevent pneumococcal disease. PCV has been in use since 2000, when a 7-valent vaccine (PCV7) was licensed in the United States for routine use in children. In 2010, PCV7 was replaced by a 13-valent vaccine. Recently, a 10-valent pneumococcal conjugate vaccine (PCV10) was licensed in Brazil; this vaccine includes the same serotypes that are in PCV7 (4, 6B, 9V, 14, 18C, 19F, 23F), plus 3 more (1, 5, and 7F) (8).
In 2010, Brazil introduced PCV10 into its routine National Immunization Program. Previously, no PCV had been incorporated into the routine immunizations. The vaccination was introduced in all cities from March through September 2010; 3 doses (at 2, 4, and 6 months of age) plus 1 booster (at 12–15 months of age) were recommended. Two routine catch-up schedules were also in place: 1) two doses for children 7–11 months of age plus a booster at 12–15 months of age, and 2) one dose for children 12–24 months of age. PCV10 is not given to children >24 months of age (9).
In Brazil, vaccination of children with PCV10 is free through the National Unified Health System (10). By October 2011, the mean vaccination coverage rapidly reached 80% for a full primary series for children <12 months of age in >5,000 municipalities (Brazilian Ministry of Health, unpub. data).
Studies that assessed the effect of PCV7 found a statistically significant reduction in the overall incidence of invasive pneumococcal disease and hospitalizations for pneumonia among children <2 years of age shortly after the first year of vaccination (11–14). Our aim was to assess the effectiveness of PCV10 for reducing hospitalizations for all-cause pneumonia. We analyzed trends in rates of hospitalization for pneumonia among children soon after the introduction of PCV10 in Brazil. Ethical approval was granted by the Ethics Committee, Federal University of Goiás, Goiania, Brazil.
We conducted an interrupted time-series analysis by using individual-level secondary data from the Hospitalization Information System of the National Unified Health System from January 2005 through August 2011. The Hospitalization Information System records ≈75% of all hospitalizations in Brazil and 60%–80% of the hospitalizations for the cities in the analyses (15). During the study period, there were no major changes in the amount of hospital care provided by the National Unified Health System.
Variables in the Hospitalization Information System are demographics, date of admission/discharge, residential address, hospital code, and International Classification of Diseases 10th Revision (ICD-10) codes for primary and secondary diagnoses. Because the Hospitalization Information System database is mainly used for reimbursement purposes, the likelihood that hospitalizations would be underreported and that data would be missing are small (16).
The structure of the Hospitalization Information System made it possible for 1 episode of hospitalization for a given patient to be recorded multiple times. Additional records might be generated when patients remain hospitalized longer than anticipated. To avoid including duplicate records, we used a deterministic record linkage algorithm to find records for the same patient (17). We then considered that consecutive records of the same patient with a 14-day interval between discharge and reentry belonged to the same episode of disease (18).
We studied 5 state capital cities in Brazil: Belo Horizonte, Curitiba, Recife, São Paulo, and Porto Alegre (Figure 1). The cities were initially selected from a list of 10 cities participating in an ongoing case–control study evaluating the effect of PCV10 vaccination on pneumococcal disease. The selection of cities for the case–control study was based on data quality and willingness of the local surveillance teams to participate in the study. Of the initial 10 cities, 5 were excluded a priori from the time-series analysis; 3 cities had not reached vaccination coverage of at least 75% for the first dose of vaccine (primary series) 3 months after vaccine introduction, and 2 cities were excluded because of poor data quality in the initial descriptive analyses. The 5 chosen cities account for 50% of the population of the state capitals of the country and are located in 3 of the 5 administrative regions of the country.
The annual numbers of live births were obtained from the Live Birth Information System and used to calculate the annual population of children 2–24 months of age. The monthly population was calculated by interpolating an exponential growth model to the annual data.
We identified Hospitalization Information System records of children 2–24 months of age who were hospitalized from January 2005 through July 2011 with specific ICD-10 codes: pneumonia (J12-J18), bronchiolitis (J21), respiratory causes (J00–J99), nonrespiratory causes, and all causes (19). We considered nosocomial pneumonia more likely to be reported as a secondary discharge diagnosis; therefore, only the primary diagnosis of the first record of each episode of disease was used in all data analyses.
In a descriptive analysis, which included only the prevaccination period, we obtained average annual numbers and rates of pneumonia hospitalizations for each city and the proportion of pneumonia out of all respiratory causes and out of all causes of hospitalizations. Specific pneumococcal pneumonia–coded cases in the Hospitalization Information System represented only 0.06% of the pneumonia cases reported in the system because confirming bacteriologic pneumonia in children is difficult; thus, we considered use of pneumococcal pneumonia–coded cases not appropriate in a time-series analysis.
In Brazil, the National Immunization Program, established in 1973, led to high rates of coverage (20). PCV10 vaccination was introduced in March 2010 in all selected cities except Porto Alegre, where it started in June 2010. PCV10 coverage data for each city were obtained from the National Immunization Program vaccine coverage database of the National Unified Health System, in which number of vaccine doses and administrative vaccination coverage are made available for all municipalities in the country. Numerator data are obtained from the number of doses administered in the vaccination rooms, by vaccine type, patient age, and municipality. PCV10 coverage for a full PCV10 primary series (3 doses) was estimated as the number of third doses of PCV10 administered (numerator) to children <12 months of age divided by the number of births in a population over time in each municipality (denominator) multiplied by 100 (see Technical Appendix, for sources of data on vaccination coverage).
We calculated the moving average of vaccine coverage for every 3-month period. The value attributed to a given month was the average vaccine coverage in that month and the coverage for the months before and after the given month. This calculation was done to smooth out short-term vaccination coverage fluctuations (Figure 2).
In the interrupted time-series analysis, 3 immunization periods were defined: prevaccination, transition, and postvaccination. The prevaccination period was January 2005–February 2010 and had 62 time points (monthly data) in the final model (except for Porto Alegre, which had 65 time points because vaccination introduction was delayed for 3 months). The transition period was the time of vaccine introduction through 4 months after. The postvaccination period was the time after the transition period; it comprised 14 time points in the analysis (3 fewer for Porto Alegre). The transition period was excluded from analysis, although it is shown in the figures. The time-series analysis was based on a generalized linear model for rates of hospitalizations for pneumonia and for nonrespiratory causes by using the negative binomial distribution with a logarithmic link function and an offset equal to the log of the population divided by 100,000 (21). Residual analyses showed no substantial deviations from model assumptions. The main outcome was rates of hospitalization for all-cause pneumonia. The explanatory variables in the model were calendar month (to control for seasonality), linear trend over time (to control for preexisting trends), and a variable equal to 1 after vaccination and 0 otherwise. After estimation of the models, 2 outputs were presented: 1) the percentage change in hospitalization rates, which compare the prevaccination and postvaccination periods and their corresponding p values and 95% CIs, and 2) a graph showing the predicted hospitalization rates for pneumonia and their 95% CIs for the postvaccination period based on models fitted with data for the prevaccination period. With the latter output, it is possible to visually evaluate the observed and the predicted monthly hospitalization rates for the postvaccination period; that is, the rates that would have resulted had the changes during the transition period not taken place.
Sensitivity analyses for each city compared models with and without removal of the months of July and August 2009. This comparison was an attempt to reduce potential bias resulting from the influenza epidemic. To compare the estimated percentage change for hospitalizations for pneumonia and nonrespiratory causes, we performed separate time-series analyses. In theory, the vaccination-induced percentage change would be higher for hospitalizations for pneumonia than for nonrespiratory causes, although a minority of possible pneumococcal disease codes was included in nonrespiratory causes (such as meningitis). The percentage change for hospitalizations for nonrespiratory causes was expected to reflect influences other than the PCV10 vaccination effect that might have concomitantly affected the data series. We expected the effect of reporting/processing delays to still be present during the postvaccination period, therefore inducing an artificially lower number of hospitalizations in more recent months (although we had discarded the most recent ones from analysis). To compensate for this effect, we calculated the differences between the percentage changes in rates of hospitalizations for pneumonia and nonrespiratory causes for each city. The equality of the percentage changes for hospitalization for pneumonia and nonrespiratory causes was tested by using the Wald test (22).
The observed trends for bronchiolitis, respiratory, and all-cause hospitalizations are shown for comparison. The linkage/classification procedures were conducted by using STATA version 12.0 (www.stata.com/), and the statistical analysis was done by using R (www.r-project.org/).
In the 5 cities, 197,975 hospitalizations of children 2 months to 2 years of age were identified during the study period; 109,155 (55.1%) were for respiratory causes, including 59,636 (30.1%) for pneumonia. During the prevaccination period, the rates of pneumonia hospitalizations varied substantially by city (Table 1).
Figure 2 shows the moving average for PCV10 coverage (percentage of children <12 months of age who received all 3 doses of PCV10). Vaccination coverage varied by city. Belo Horizonte and Curitiba rapidly reached 100% coverage and maintained stable rates of ≈100% from September 2010 on. Recife showed a tendency toward sustained and continuously rising coverage over the study period, eventually reaching ≈100%. Porto Alegre reached 100% coverage on January 2011, followed by a gradual decrease to 85% in July 2011. São Paulo coverage increased to 90% in November 2010, after which it continuously declined, reaching 75% in July 2011.
Trends in patterns of hospitalization rates for pneumonia, respiratory causes, and all causes are shown for each city (Figure 3). Seasonal variations are evident for all cities. The contribution of pneumonia to the total number of hospitalized patients varied widely by city but not by years. Rates of hospitalization for all causes, particularly from mid-2007 on, decreased notably for Belo Horizonte and Recife; however, the observed reductions in rates of hospitalization for pneumonia for these cities and for Curitiba seem to be restricted to the postvaccination period (from mid-2010 on).
Rates of hospitalization for bronchiolitis were lower than those for pneumonia in all cities except Porto Alegre (Figure 4). The seasonal variations of bronchiolitis and pneumonia were mostly parallel and were found for all cities. Hospitalization rates progressively increased in the more recent years for Porto Alegre, São Paulo, and possibly Curitiba.
Table 2 and Figure 5 show results derived from the same time-series models. During the postvaccination period, rates of hospitalization for pneumonia decreased significantly (p<0.001) in Belo Horizonte (−40.3%), Curitiba (−37.6%), and Recife (−49.3%). Rate reductions were borderline significant for São Paulo (−13.4%; p = 0.074) and Porto Alegre (−23.5%; p = 0.052) (Table 2). Rates of hospitalization for nonrespiratory causes also decreased in all cities, albeit at a lower rate. The following differences between the percentage changes in hospitalization rates for pneumonia and nonrespiratory causes represent our best estimate of the vaccination effect: Belo Horizonte (−28.7%), Curitiba (−23.3%), Recife (−27.4%), São Paulo (−1.8%), and Porto Alegre (−2.3%). During the postvaccination period, reductions in rates of hospitalization for pneumonia did not differ significantly from rates of hospitalization for nonrespiratory causes in São Paulo (p = 0.827) and Porto Alegre (p = 0.845).
Figure 5 compares the observed monthly rates of hospitalization for pneumonia with the forecasted values that were modeled with use of data exclusively from the prevaccination period. For Belo Horizonte, Curitiba, and Recife, the observed numbers are close to or below the lower limit of the 95% CI, particularly for the most recent months.
This study indicates that the introduction of PCV10 through the routine immunization program in Brazil has effectively lowered rates of hospitalization for pneumonia among children. Rates of hospitalization for all causes declined in 3 of the 5 cities studied (Belo Horizonte, Curitiba, and Recife). In the other 2 cities (São Paulo and Porto Alegre), these rates did not decline significantly, possibly because vaccination coverage for these 2 cities in 2011 was lower (≈80%) than it was in the other 3 cities (>90%). Another possible reason is that Porto Alegre started its vaccination program 3 months after the other cities, so its postvaccination period was shorter, leaving less time for the vaccination to become effective.
Comparison of our results with those of other studies is not straightforward because, to our knowledge, no comparable studies have been published (e.g., effects of PCV10 on rates of hospitalization for all causes). PCV10 has recently been introduced in some countries in North America and Europe. Preliminary evaluations indicate a reduction of invasive pneumococcal disease. In the province of Quebec, Canada, PCV10 was introduced to the routine immunization schedule 5 years after PCV7 was introduced. Data obtained by a sentinel laboratory surveillance network showed lower incidence of invasive pneumococcal disease among children vaccinated with PCV10 than with PCV7 (35.3 vs. 64.1 cases/100,000 person-years) (23). In Finland, results of a recent field trial found a marked decrease in the incidence of invasive pneumococcal disease among children who were vaccinated according to a 3+1 or a 2+1 immunization schedule; vaccine effectiveness reached 100% (95% CI 83%–100%) and 92% (95% CI 58%–100%), respectively, after 2 years (24). The Clinical Otitis Media and Pneumonia Study conducted at urban sites in Argentina, Colombia, and Panama showed that the efficacy of PCV10 for reducing community-acquired pneumonia and alveolar consolidation among children was 7.3% and 23.4%, respectively (25).
For PCV7, studies have already documented its effect on rates of hospitalization for pneumonia among children (11,12,26–28). In the United States, the rates of hospitalization for pneumonia were reduced 39%–52%. However, aside from the use of different vaccines, our study is not directly comparable. Vaccination coverage was generally lower in the United States, increasing from 68% to 83% during the postvaccination period, which was much (4 years) longer. This longer time might have allowed time for herd immunity to protect the nonvaccinated population (26,28). Also, the illnesses compared in each study were not the same; the United States study evaluated dehydration and diarrhea, whereas our study evaluated all nonrespiratory conditions since the rotavirus vaccine was introduced in 2006 to Brazil.
The introduction of rotavirus vaccination might actually be one of the best explanations for the decreasing trends of all-cause hospitalizations in the target age group during the study period. Another explanation is the rapid increase in coverage of the Family Health Programme. This program reached 85% of Brazilian municipalities in 2010 and greatly reduced deaths and hospitalizations of infants for primary-care sensitive diseases like diarrhea and for lower respiratory tract diseases (29–31).
Across all 5 cities, we found differences in rates of hospitalization for pneumonia before introduction of PCV10. Marked regional differences had already been documented (32). Possible reasons, other than differences in health care provision, are variations in epidemiology, demographics, socioeconomic status, and climate. Health care provisions might play a progressively lesser role in explaining the differences in rates of hospitalization for pneumonia because the results of National Household Sample Surveys show a trend toward equity in access to and use of health care facilities (31,33).
Several potential limitations of our study should be highlighted. Our data represent only the population served by National Unified Health System in Brazil. The findings observed for the 5 capital cities cannot be considered representative of the entire country. We attempted to include mostly community-acquired cases of pneumonia by restricting our analysis to the primary diagnosis for hospitalization. By doing so, we hypothetically increased the proportion of hospitalizations for pneumococcal pneumonia out of all hospitalizations for pneumonia. Because only a few cases have pneumonia listed as a secondary diagnosis, we missed only a few community-acquired cases of pneumonia by not including it.
Information about the extent of coding errors in Brazil is scarce. None of the information is specific to pneumonia. However, the clinical diagnosis of pneumonia has been shown to have high sensitivity and low specificity for ascertaining pneumococcal pneumonia; thus, any bias resulting from misclassification of ICD-10 would be toward reduction of the observed effect of vaccination (34).
Our results could also have been influenced by changes in disease diagnosis and management over time. We observed an increase in hospitalizations for bronchiolitis in the cities of Curitiba, Porto Alegre, and São Paulo. Although this increase might represent a real increase in disease incidence and/or severity, a more likely reason could simply be improvements in the diagnosis of bronchiolitis. Thus, hospitalizations for bronchiolitis, which would otherwise be coded as nonspecific pneumonia or other lower respiratory infections, could be increasingly coded correctly in these locations. Potentially, this reduction in misclassification over time would tend to increase the observed effect of vaccination in these cities, but no vaccination effect was observed in Porto Alegre and São Paulo. We have not identified other major changes in diagnosis and reporting habits, including those motivated by knowledge of PCV introduction or the fact that its effect was being assessed.
The study was conducted at the time of the 2009 influenza pandemic, and an influenza A (H1N1) vaccination campaign was conducted in Brazil as a time-limited intervention. This campaign took place from March through June 2010 and achieved high vaccination coverage among children <2 years of age (Brazilian Ministry of Health, unpub. data). This age group was only slightly affected by the pandemic, as evidenced by the lack of a temporary increase in the rates of hospitalization for pneumonia and rates of hospitalization for influenza (data not shown because numbers were so small). Therefore, we consider it unlikely that the pandemic or its vaccination campaign have biased our results.
Any study that uses a time-series method to determine the early effects of a vaccine can be challenged by fluctuations in vaccination coverage and by the natural lag period between vaccination and protection. Moreover, the limitations of using vaccine coverage estimates derived from secondary data collected for administrative reasons are obvious because of the fact that coverage goes beyond 100% for the initial months after start of a vaccination program (Figure 2). The entry of a new vaccine into the immunization program in Brazil usually attracts infants <6 months of age and infants from areas surrounding the municipality. For both situations, the number of doses administered per month are higher than the number of live births per month.
A major challenge to our analysis was dealing with unavoidable delays for reporting the estimated early effect of PCV10 vaccination. Although we found that the number of hospitalizations for all causes decreased during the most recent months of our series, to run the time-series models we still needed as many data points as possible after the vaccine was introduced. The chosen strategy was to subtract the declines for the nonrespiratory hospitalization rates from the pneumonia rates. By doing so, we accounted for as much of the effect of reporting delays as possible.
In conclusion, our data demonstrate that 1 year after its introduction to Brazil, PCV10 reduced hospitalizations for pneumonia among children in 3 of the 5 cities studied. To ascertain the sustainability of this reduction, prospective analyses covering a longer time after introduction of the vaccination program are needed.
Dr Afonso is a PhD student in epidemiology and public health at the Federal University of Goiás, Goiânia, and an associate professor in the Department of Pediatrics. Her scientific interests include the epidemiology of respiratory tract infections.
We thank Sérgio Norio Nakamura, and Marcelo Netto do Carmo for their thoughtful input. We also are grateful for the reviewers’ valuable comments and advice that improved the article.
This study was supported by the Ministry of Health of Brazil, Department of Heath Analysis, and National Immunization Program. A.L.A. is a research fellow of the National Council for Scientific and Technological Development, Brazil (research grant no. 306096/2010-2) and of the National Institute of Science and Technology for Health Technology Assessment. A.L.A. received a research grant from GlaxoSmithKline and financial support to attend scientific meetings from Pfizer and GlaxoSmithKline and has served as an advisor for Pfizer.
- Black RE, Cousens S, Johnson HL, Lawn JE, Rudan I, Bassani DG, Global, regional, and national causes of child mortality in 2008: a systematic analysis. Lancet. 2010;375:1969–87.
- O’Brien KL, Wolfson LJ, Watt JP, Henkle E, Deloria-Knoll M, McCall N, Burden of disease caused by Streptococcus pneumoniae in children younger than 5 years: global estimates. Lancet. 2009;374:893–902.
- World Health Organization. Estimated Hib and pneumococcal deaths for children under 5 years of age, 2008 [cited 2012 May 10]. http://www.who.int/immunization_monitoring/burden/Pneumo_hib_estimates/en/index.html
- Andrade AL, Oliveira R, Vieira MA, Minamisava R, Pessoa VJ, Brandileone MC, Population-based surveillance for invasive pneumococcal disease and pneumonia in infants and young children in Goiania, Brazil. Vaccine. 2012;30:1901–9.
- Novaes HM, Sartori AM, Soarez PC. Hospitalization rates for pneumococcal disease in Brazil, 2004–2006. Rev Saude Publica. 2011;45:539–47.
- Brazilian Ministry of Health. Informações de Saúde, 2010. DATASUS [cited 2012 Jul 01]. http://www2.datasus.gov.br/DATASUS/index.php?area=02
- United Nations Children’s Fund. The state of the world’s children 2012: children in an urban world, 2012 [cited 2012 Jun 30]. http://www.unicef.org/sowc2012/pdfs/SOWC%202012-Main%20Report_EN_13Mar2012.pdf
- World Health Organization. Pneumococcal vaccines WHO position paper–2012–recommendations. Vaccine. 2012;30:4717–8.
- Brazilian Ministry of Health. Proposta para introdução da vacina pneumocócica 10-valente (conjugada) no calendário básico de vacinação da criança: incorporação março—2010 [cited 2011 Nov 10]. http://portal.saude.gov.br/portal/arquivos/pdf/intro_pneumococica10_val_04_02_10_ver_final.pdf
- Victora CG, Barreto ML, do Carmo Leal M, Monteiro CA, Schmidt MI, Paim J, Health conditions and health-policy innovations in Brazil: the way forward. Lancet. 2011;377:2042–53.
- Centers for Disease Control and Prevention. Pneumonia hospitalizations among young children before and after introduction of pneumococcal conjugate vaccine—United States, 1997–2006. MMWR Morb Mortal Wkly Rep. 2009;58:1–4 .
- De Wals P, Robin E, Fortin E, Thibeault R, Ouakki M, Douville-Fradet M. Pneumonia after implementation of the pneumococcal conjugate vaccine program in the province of Quebec, Canada. Pediatr Infect Dis J. 2008;27:963–8.
- Rückinger S, van der Linden M, Reinert RR, von Kries R, Burckhardt F, Siedler A. Reduction in the incidence of invasive pneumococcal disease after general vaccination with 7-valent pneumococcal conjugate vaccine in Germany. Vaccine. 2009;27:4136–41.
- Vestrheim DF, Lovoll O, Aaberge IS, Caugant DA, Hoiby EA, Bakke H, Effectiveness of a 2+1 dose schedule pneumococcal conjugate vaccination programme on invasive pneumococcal disease among children in Norway. Vaccine. 2008;26:3277–81.
- Brazilian Ministry of Health. Pesquisa nacional por amostra de domicílios–suplemento saúde—2008 [cited 2012 Oct 17]. http://tabnet.datasus.gov.br/cgi/tabnet.exe?pnad2008/pnad.def
- Bittencourt SA, Camacho LA, Leal Mdo C. Hospital information systems and their application in public health [in Portuguese]. Cad Saude Publica. 2006;22:19–30.
- Pacheco AG, Saraceni V, Tuboi SH, Moulton LH, Chaisson RE, Cavalcante SC, Validation of a hierarchical deterministic record-linkage algorithm using data from 2 different cohorts of human immunodeficiency virus–infected persons and mortality databases in Brazil. Am J Epidemiol. 2008;168:1326–32.
- Lanata CF, Rudan I, Boschi-Pinto C, Tomaskovic L, Cherian T, Weber M, Methodological and quality issues in epidemiological studies of acute lower respiratory infections in children in developing countries. Int J Epidemiol. 2004;33:1362–72.
- World Health Organization. ICD-10 version: 2010. International Statistical Classification of Diseases and Related Health Problems, 10th Revision 2010 [cited 2011 Dec 12]. http://apps.who.int/classifications/icd10/browse/2010/en
- Barreto ML, Teixeira MG, Bastos FI, Ximenes RA, Barata RB, Rodrigues LC. Successes and failures in the control of infectious diseases in Brazil: social and environmental context, policies, interventions, and research needs. Lancet. 2011;377:1877–89.
- McCullagh P, Nelder JA. Generalized linear models. 2nd ed. Boca Raton (FL): Chapman & Hall/CRC; 1998.
- Sen PK, Singer JM, Lima ACP. From finite sample to asymptotic methods in statistics. Cambridge: Cambridge University Press; 2009.
- De Wals P, Lefebvre B, Defay F, Deceuninck G, Boulianne N. Invasive pneumococcal diseases in birth cohorts vaccinated with PCV-7 and/or PHiD-CV in the province of Quebec, Canada. Vaccine. 2012;30:6416–20 .
- Palmu AA, Jokinen J, Borys D, Nieminen H, Ruokokoski E, Siira L, Effectiveness of the ten-valent pneumococcal Haemophilus influenzae protein D conjugate vaccine (PHiD-CV10) against invasive pneumococcal disease: a cluster randomised trial. Lancet. 2013;381:214–22 .
- Tregnaghi MW, Sáez-Llorens X, López P, Abate H, Smith E, Pósleman A, Evaluating the efficacy of 10-valent pneumococcal non-typeable Haemophilus influenzae protein-D conjugate vaccine (PHID-CV) against community-acquired pneumonia in Latin America. 29th Annual Meeting of the European Society for Paediatric Infectious Diseases; 2011 Jun 7–11; The Hague (the Netherlands): The Society; 2011. p. 864 [cited 2013 Feb 4]. http://www.kenes.com/espid2011/cd/PDF/O223.pdf?zoom_highlightsub=Tregnaghi#search=%22Tregnaghi%22
- Grijalva CG, Nuorti JP, Arbogast PG, Martin SW, Edwards KM, Griffin MR. Decline in pneumonia admissions after routine childhood immunisation with pneumococcal conjugate vaccine in the USA: a time-series analysis. Lancet. 2007;369:1179–86.
- Koshy E, Murray J, Bottle A, Sharland M, Saxena S. Impact of the seven-valent pneumococcal conjugate vaccination (PCV7) programme on childhood hospital admissions for bacterial pneumonia and empyema in England: national time-trends study, 1997–2008. Thorax. 2010;65:770–4.
- Zhou F, Kyaw MH, Shefer A, Winston CA, Nuorti JP. Health care utilization for pneumonia in young children after routine pneumococcal conjugate vaccine use in the United States. Arch Pediatr Adolesc Med. 2007;161:1162–8.
- Alfradique ME, Bonolo Pde F, Dourado I, Lima-Costa MF, Macinko J, Mendonca CS, Ambulatory care sensitive hospitalizations: elaboration of Brazilian list as a tool for measuring health system performance (Project ICSAP–Brazil) [in Portuguese]. Cad Saude Publica. 2009;25:1337–49.
- Aquino R, de Oliveira NF, Barreto ML. Impact of the family health program on infant mortality in Brazilian municipalities. Am J Public Health. 2009;99:87–93.
- Paim J, Travassos C, Almeida C, Bahia L, Macinko J. The Brazilian health system: history, advances, and challenges. Lancet. 2011;377:1778–97.
- Berezin EN, de Moraes JC, Hong T, Todd M, Seljan MP. Pneumonia hospitalization in Brazil from 2003 to 2007. Int J Infect Dis. 2012;16:e583–90.
- Macinko J, Lima-Costa MF. Horizontal equity in health care utilization in Brazil, 1998–2008. Int J Equity Health. 2012;11:33.
- Rodrigues LC, Smith PG. Use of the case–control approach in vaccine evaluation: efficacy and adverse effects. Epidemiol Rev. 1999;21:56–72.