Skip directly to site content Skip directly to page options Skip directly to A-Z link Skip directly to A-Z link Skip directly to A-Z link
Volume 10, Number 1—January 2004
Research

Evaluating Detection and Diagnostic Decision Support Systems for Bioterrorism Response

Dena M. Bravata*†Comments to Author , Vandana Sundaram*†‡, Kathryn M. McDonald*†, Wendy M. Smith*†, Herbert Szeto*†§, Mark D. Schleinitz¶#, and Douglas K. Owens*†‡
Author affiliations: *University of California San Francisco-Stanford Evidence-based Practice Center, Stanford, California, USA; †Stanford University School of Medicine, Stanford, California, USA; ‡VA Palo Alto Healthcare System, Palo Alto, California, USA; §Kaiser Permanente, Redwood City, California, USA; ¶Rhode Island Hospital, Providence, Rhode Island, USA; #Brown University School of Medicine, Providence, Rhode Island, USA

Main Article

Table 2

Evaluation data for detection systems for bioterrorism agentsa

System name Purpose Evaluation datab
Anthrax Sensor (7) A portable detection system for “highly sensitive detection of biological agents within seconds” (7). Reported to be capable of detecting endotoxins at a level that is “20 times lower than previously achieved by similar devices” (7).c
BioCapture (8) A portable collection system for use by first responders. Was compared to an All Glass Impinger (AGI) that collects samples into liquid and a Slit Sampler that impacts bacteria directly onto growth media and found to have a collection efficiency of 50%-80% relative to the AGI and 60%-125% relative to the Slit Sampler (8).c
Digital Smell/Electronic Nose (9) To detect and classify microorganisms according to the volatile gases given off during metabolism. An array of 15 sensors was able to correctly classify 68 of 90 colonies containing 0 or 1 of 5 test organisms and an uninoculated control; however, it registered 22 of 90 as false-positives (9).
Fluorescence-based array immuno-sensor (10) To provide simultaneous, antibody-based detection of bioactive analytes in clinical fluids. Bioterrorism agents intended to be detected include Staphylococcus enterotoxin B and F1 antigen from Yersinia pestis. It was unable to detect S. enterotoxin B levels (<125 ng/mL) in experimentally spiked urine, saliva, and blood products; sensitivity for F1 antigen from Y. pestis was reported at 25 ng/mL (10).
LightCycler; Ruggedized Advanced Pathogen Identification Device (RAPID) (11) LightCycler uses a PCR cycler for “real-time” quantification of DNA samples. RAPID is a rugged, portable system that uses LightCycler technology for field detection of bioterrorism agents. RAPID is reported by the manufacturer to be 99.9% specific (11). For each assay, the sensitivity is set to half the infective dose (for example, the infectious dose of foot and mouth disease is 10 virus particles; RAPID’s sensitivity is set to detect 5 virus particles [11]).c
MiniFlo (12) For rapid, portable detection of multiple biological agents using flow cytometry. Detected 87% of unknown biological agent simulants, including agents similar to anthrax and plague, with a false-positive rate of 0.4% (12). Bioterrorism agents identifiable: Y. pestis and Bacillus anthracis, as well as other viruses, bacteria and proteins (12 ).
Model 3312A Ultraviolet Aerodynamic Particle Sizer (UV-APS) and Fluorescence Aerodynamic Particle Sizer-2 (FLAPS-2) (13) To detect living organisms in aerosols and nonvolatile liquids. FLAPS-2 was able to detect 39 of 40 blind releases of stimulant aerosols (of particle ranging in size from 0.5 to 15 μm) at a distance of about 1 km with no false alarms during a 3-week period. In another trial, it was able to detect as few as 10 agent-containing particles per liter of air (13,4).
Sensitive Membrane Antigen Rapid Test (SMART) and the Antibody-based Lateral Flow Economical Recognition Ticket (ALERT) (14–17) A handheld antigen/antibody test for the rapid detection of bioterrorism agents. When field tested during the Gulf War, the SMART system had an “alarmingly” high false-positive rate thought secondary to contamination (14). SMART tests are reported per the manufacturer to have a 96% to 99% sensitivity and 94% to 99% specificity for Vibrio cholerae O139 and O1) (14–17)

aPCR; polymerase chain reaction.
bWhere possible, we report sensitivity and specificity data (and highlight them in bold); if the published reports did not provide these values directly but did provide sufficient data for them to be calculated, we performed these calculations.
cDenotes systems for which available evaluation data were from manufacturers’ Web sites only.

Main Article

References
  1. Hughes  JM, Gerberding  JL. Anthrax bioterrorism: lessons learned and future directions. Emerg Infect Dis. 2002;8:1013–4.PubMedGoogle Scholar
  2. Heller  MB, Bunning  ML, France  ME, Niemeyer  DM, Peruski  L, Naimi  T, Laboratory response to anthrax bioterrorism, New York City, 2001. Emerg Infect Dis. 2002;8:1096–102.PubMedGoogle Scholar
  3. McCullough  M. Anthrax hoaxes, false alarms taxing authorities nationwide. The Seattle Times. November 10, 2001;Nation & World.
  4. Perkins  BA, Popovic  T, Yeskey  K. Public health in the time of bioterrorism. Emerg Infect Dis. 2002;8:1015–8.PubMedGoogle Scholar
  5. Bravata  DM, McDonald  K, Owens  DK, Smith  W, Rydzak  C, Szeto  H, Bioterrorism preparedness and response: use of information technologies and decision support systems (Evidence Report/Technology Assessment No. 59). Rockville (MD): prepared by the UCSF-Stanford Evidence-based Practice Center under Contract No. 290-97-0013 for the Agency for Healthcare Research and Quality; 2002.
  6. F.Y. 2002-F.Y. 2006 plan for combating bioterrorism. Washington: U.S. Department of Health and Human Services; 2001.
  7. Henahan  S. Anthrax sensor. Access Excellence.com. [Accessed September 28, 2001]. Available from: URL: http://www.accessexcellence.com/WN/SUA12/anthrax298.html
  8. MesoSystems Products. MesoSystems Technology Inc. [Accessed October 29, 2001]. Available at: http://www.mesosystems.com
  9. Holmberg  M, Gustafsson  F, Hornsten  EG, Winquist  F, Nilsson  LE, Ljung  L, Bacteria classification based on feature extraction from sensor data. Biotechnol Tech. 1998;12:319–24. DOIGoogle Scholar
  10. Rowe  CA, Scruggs  SB, Feldstein  MJ, Golden  JP, Ligler  FS. An array immunosensor for simultaneous detection of clinical analytes. Anal Chem. 1999;71:433–9. DOIPubMedGoogle Scholar
  11. Idaho Technologies products. Idaho Technologies. [Accessed October 29, 2001]. Available from: URL: http://www.idahotech.com
  12. Milanovich  F. Reducing the threat of biological weapons. Lawrence Livermore National Laboratory, Science and Technology Review. [Accessed September 7, 2001]. Available from: URL: http://www.llnl.gov/str/Milan.html
  13. Commission on Life Sciences, National Research Council. Chemical and biological terrorism: research and development to improve civilian medical response. Washington: National Academy Press; 1999.
  14. Biological detection system technologies: technology and industrial base study: a primer on biological detection technologies. North American Technology and Industrial Base Organization; 2001.
  15. Rostker  B. Close-out report: biological warfare investigation. Washington: Department of Defense; 2000.
  16. Von Bredow  J, Myers  M, Wagner  D, Valdes  J, Loomis  L, Zamani  K. Agroterrorism: agricultural infrastructure vulnerability. Ann N Y Acad Sci. 1999;894:168–80. DOIPubMedGoogle Scholar
  17. New Horizons Diagnostics Corporation. New Horizons Diagnostics Corp. [Accessed August 22, 2001]. Available from: URL: http://www.nhdiag.com/
  18. Ticket  SMART. (biological agents). American School of Defense. [Accessed October 24, 2001]. Available from: URL: http://www.asod.org/id10.htm
  19. Centers for Disease Control and Prevention. Handheld immunoassays for detection of Bacillus anthracis spores. [Accessed October 25, 2001]. Available from: URL: http://www.bt.cdc.gov/DocumentsApp/Anthrax/10182001HealthAlertPM/10182001HealthAlertPM.asp
  20. Government Service Administration. GSA Policy Advisory: Guidelines for federal mail centers in the Washington, DC, Metropolitan Area for managing possible anthrax contamination. [Accessed March 26, 2003]. Available from: URL http://www.ostp.gov/html/GSAAnthraxGuidelines.html
  21. Knirsch  CA, Jain  NL, Pablos-Mendez  A, Friedman  C, Hripcsak  G. Respiratory isolation of tuberculosis patients using clinical guidelines and an automated clinical decision support system. Infect Control Hosp Epidemiol. 1998;19:94100.
  22. Hripcsak  G, Friedman  C, Alderson  PO, DuMouchel  W, Johnson  SB, Clayton  PD. Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med. 1995;122:681–8.PubMedGoogle Scholar
  23. Berger  SA, Blackman  U. Computer program for diagnosing and teaching geographic medicine. J Travel Med. 1995;2:199–203. DOIPubMedGoogle Scholar
  24. Brooks  GJ, Ashton  RE, Pethybridge  RJ. DERMIS: a computer system for assisting primary-care physicians with dermatological diagnosis. Br J Dermatol. 1992;127:614–9. DOIPubMedGoogle Scholar
  25. Smith  HR, Ashton  RE, Brooks  GJ. Initial use of a computer system for assisting dermatological diagnosis in general practice. Med Inform Internet Med. 2000;25:103–8. DOIPubMedGoogle Scholar
  26. Hammersley  JR, Cooney  K. Evaluating the utility of available different diagnosis systems. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1988:229–31.
  27. Cundell  DR, Silibovsky  RS, Sanders  R, Sztandera  LM. Using fuzzy sets to analyze putative correlates between age, blood type, gender and/or race with bacterial infection. Artif Intell Med. 2001;21:235–9. DOIPubMedGoogle Scholar
  28. Ross  JJ, Shapiro  DS. Evaluation of the computer program GIDEON (Global Infectious Disease and Epidemiology Network) for the diagnosis of fever in patients admitted to a medical service. Clin Infect Dis. 1998;26:766–7. DOIPubMedGoogle Scholar
  29. Murphy  GC, Friedman  CP, Elstein  AS, Wolf  FM, Miller  T, Miller  JG. The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. AMIA Proc Annu Fall Symp 1996:219–23.
  30. Berner  ES, Webster  GD, Shugerman  AA, Jackson  JR, Algina  J, Baker  AL, Performance of four computer-based diagnostic systems. N Engl J Med. 1994;330:1792–6. DOIPubMedGoogle Scholar
  31. Bouhaddou  O, Lambert  JG, Miller  S. Consumer health informatics: knowledge engineering and evaluation studies of medical HouseCall. Proc AMIA Symp 1998:612–6.
  32. El-Solh  AA, Hsiao  CB, Goodnough  S, Serghani  J, Grant  BJ. Predicting active pulmonary tuberculosis using an artificial neural network. Chest. 1999;116:968–73. DOIPubMedGoogle Scholar
  33. Verdaguer  A, Patak  A, Sancho  JJ, Sierra  C, Sanz  F. Validation of the medical expert system PNEUMON-IA. Comput Biomed Res. 1992;25:511–26. DOIPubMedGoogle Scholar
  34. Bankowitz  RA, McNeil  MA, Challinor  SM, Parker  RC, Kapoor  WN, Miller  RA. A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype. Ann Intern Med. 1989;110:824–32.PubMedGoogle Scholar
  35. Fiszman  M, Chapman  WW, Evans  SR, Haug  PJ. Automatic identification of pneumonia related concepts on chest x-ray reports. Proc AMIA Symp 1999:67–71.
  36. Chapman  WW, Haug  PJ. Comparing expert systems for identifying chest x-ray reports that support pneumonia. Proc AMIA Symp 1999:216–20.
  37. Carter  CN, Ronald  NC, Steele  JH, Young  E, Taylor  JP, Russell  LH, Knowledge-based patient screening for rare and emerging infectious/parasitic diseases: a case study of brucellosis and murine typhus. Emerg Infect Dis. 1997;3:73–6.PubMedGoogle Scholar
  38. Ashizawa  K, MacMahon  H, Ishida  T, Nakamura  K, Vyborny  CJ, Katsuragawa  S, Effect of an artificial neural network on radiologists' performance in the differential diagnosis of interstitial lung disease using chest radiographs. AJR Am J Roentgenol. 1999;172:1311–5.PubMedGoogle Scholar
  39. Monnier-Cholley  L, MacMahon  H, Katsuragawa  S, Morishita  J, Ishida  T, Doi  K. Computer-aided diagnosis for detection of interstitial opacities on chest radiographs. AJR Am J Roentgenol. 1998;171:1651–6.PubMedGoogle Scholar
  40. Sox  HC, Blatt  MA, Higgins  MC, Marton  KI. Medical decision making. Boston: Butterworth-Heinemann; 1988.

Main Article

Page created: December 21, 2010
Page updated: December 21, 2010
Page reviewed: December 21, 2010
The conclusions, findings, and opinions expressed by authors contributing to this journal do not necessarily reflect the official position of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors' affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above.
file_external