Volume 10, Number 1—January 2004
Evaluating Detection and Diagnostic Decision Support Systems for Bioterrorism Response
|System name||Purpose||Evaluation datab|
|Anthrax Sensor (7)||A portable detection system for “highly sensitive detection of biological agents within seconds” (7).||Reported to be capable of detecting endotoxins at a level that is “20 times lower than previously achieved by similar devices” (7).c|
|BioCapture (8)||A portable collection system for use by first responders.||Was compared to an All Glass Impinger (AGI) that collects samples into liquid and a Slit Sampler that impacts bacteria directly onto growth media and found to have a collection efficiency of 50%-80% relative to the AGI and 60%-125% relative to the Slit Sampler (8).c|
|Digital Smell/Electronic Nose (9)||To detect and classify microorganisms according to the volatile gases given off during metabolism.||An array of 15 sensors was able to correctly classify 68 of 90 colonies containing 0 or 1 of 5 test organisms and an uninoculated control; however, it registered 22 of 90 as false-positives (9).|
|Fluorescence-based array immuno-sensor (10)||To provide simultaneous, antibody-based detection of bioactive analytes in clinical fluids.||Bioterrorism agents intended to be detected include Staphylococcus enterotoxin B and F1 antigen from Yersinia pestis. It was unable to detect S. enterotoxin B levels (<125 ng/mL) in experimentally spiked urine, saliva, and blood products; sensitivity for F1 antigen from Y. pestis was reported at 25 ng/mL (10).|
|LightCycler; Ruggedized Advanced Pathogen Identification Device (RAPID) (11)||LightCycler uses a PCR cycler for “real-time” quantification of DNA samples. RAPID is a rugged, portable system that uses LightCycler technology for field detection of bioterrorism agents.||RAPID is reported by the manufacturer to be 99.9% specific (11). For each assay, the sensitivity is set to half the infective dose (for example, the infectious dose of foot and mouth disease is 10 virus particles; RAPID’s sensitivity is set to detect 5 virus particles ).c|
|MiniFlo (12)||For rapid, portable detection of multiple biological agents using flow cytometry.||Detected 87% of unknown biological agent simulants, including agents similar to anthrax and plague, with a false-positive rate of 0.4% (12). Bioterrorism agents identifiable: Y. pestis and Bacillus anthracis, as well as other viruses, bacteria and proteins (12 ).|
|Model 3312A Ultraviolet Aerodynamic Particle Sizer (UV-APS) and Fluorescence Aerodynamic Particle Sizer-2 (FLAPS-2) (13)||To detect living organisms in aerosols and nonvolatile liquids.||FLAPS-2 was able to detect 39 of 40 blind releases of stimulant aerosols (of particle ranging in size from 0.5 to 15 μm) at a distance of about 1 km with no false alarms during a 3-week period. In another trial, it was able to detect as few as 10 agent-containing particles per liter of air (13,4).|
|Sensitive Membrane Antigen Rapid Test (SMART) and the Antibody-based Lateral Flow Economical Recognition Ticket (ALERT) (14–17)||A handheld antigen/antibody test for the rapid detection of bioterrorism agents.||When field tested during the Gulf War, the SMART system had an “alarmingly” high false-positive rate thought secondary to contamination (14). SMART tests are reported per the manufacturer to have a 96% to 99% sensitivity and 94% to 99% specificity for Vibrio cholerae O139 and O1) (14–17)|
aPCR; polymerase chain reaction.
bWhere possible, we report sensitivity and specificity data (and highlight them in bold); if the published reports did not provide these values directly but did provide sufficient data for them to be calculated, we performed these calculations.
cDenotes systems for which available evaluation data were from manufacturers’ Web sites only.
- Hughes JM, Gerberding JL. Anthrax bioterrorism: lessons learned and future directions. Emerg Infect Dis. 2002;8:1013–4.
- Heller MB, Bunning ML, France ME, Niemeyer DM, Peruski L, Naimi T, Laboratory response to anthrax bioterrorism, New York City, 2001. Emerg Infect Dis. 2002;8:1096–102.
- McCullough M. Anthrax hoaxes, false alarms taxing authorities nationwide. The Seattle Times. November 10, 2001;Nation & World.
- Perkins BA, Popovic T, Yeskey K. Public health in the time of bioterrorism. Emerg Infect Dis. 2002;8:1015–8.
- Bravata DM, McDonald K, Owens DK, Smith W, Rydzak C, Szeto H, Bioterrorism preparedness and response: use of information technologies and decision support systems (Evidence Report/Technology Assessment No. 59). Rockville (MD): prepared by the UCSF-Stanford Evidence-based Practice Center under Contract No. 290-97-0013 for the Agency for Healthcare Research and Quality; 2002.
- F.Y. 2002-F.Y. 2006 plan for combating bioterrorism. Washington: U.S. Department of Health and Human Services; 2001.
- Henahan S. Anthrax sensor. Access Excellence.com. [Accessed September 28, 2001]. Available from: URL: http://www.accessexcellence.com/WN/SUA12/anthrax298.html
- MesoSystems Products. MesoSystems Technology Inc. [Accessed October 29, 2001]. Available at: http://www.mesosystems.com
- Holmberg M, Gustafsson F, Hornsten EG, Winquist F, Nilsson LE, Ljung L, Bacteria classification based on feature extraction from sensor data. Biotechnol Tech. 1998;12:319–24.
- Rowe CA, Scruggs SB, Feldstein MJ, Golden JP, Ligler FS. An array immunosensor for simultaneous detection of clinical analytes. Anal Chem. 1999;71:433–9.
- Idaho Technologies products. Idaho Technologies. [Accessed October 29, 2001]. Available from: URL: http://www.idahotech.com
- Milanovich F. Reducing the threat of biological weapons. Lawrence Livermore National Laboratory, Science and Technology Review. [Accessed September 7, 2001]. Available from: URL: http://www.llnl.gov/str/Milan.html
- Commission on Life Sciences, National Research Council. Chemical and biological terrorism: research and development to improve civilian medical response. Washington: National Academy Press; 1999.
- Biological detection system technologies: technology and industrial base study: a primer on biological detection technologies. North American Technology and Industrial Base Organization; 2001.
- Rostker B. Close-out report: biological warfare investigation. Washington: Department of Defense; 2000.
- Von Bredow J, Myers M, Wagner D, Valdes J, Loomis L, Zamani K. Agroterrorism: agricultural infrastructure vulnerability. Ann N Y Acad Sci. 1999;894:168–80.
- New Horizons Diagnostics Corporation. New Horizons Diagnostics Corp. [Accessed August 22, 2001]. Available from: URL: http://www.nhdiag.com/
- Ticket SMART. (biological agents). American School of Defense. [Accessed October 24, 2001]. Available from: URL: http://www.asod.org/id10.htm
- Centers for Disease Control and Prevention. Handheld immunoassays for detection of Bacillus anthracis spores. [Accessed October 25, 2001]. Available from: URL: http://www.bt.cdc.gov/DocumentsApp/Anthrax/10182001HealthAlertPM/10182001HealthAlertPM.asp
- Government Service Administration. GSA Policy Advisory: Guidelines for federal mail centers in the Washington, DC, Metropolitan Area for managing possible anthrax contamination. [Accessed March 26, 2003]. Available from: URL http://www.ostp.gov/html/GSAAnthraxGuidelines.html
- Knirsch CA, Jain NL, Pablos-Mendez A, Friedman C, Hripcsak G. Respiratory isolation of tuberculosis patients using clinical guidelines and an automated clinical decision support system. Infect Control Hosp Epidemiol. 1998;19:94100.
- Hripcsak G, Friedman C, Alderson PO, DuMouchel W, Johnson SB, Clayton PD. Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med. 1995;122:681–8.
- Berger SA, Blackman U. Computer program for diagnosing and teaching geographic medicine. J Travel Med. 1995;2:199–203.
- Brooks GJ, Ashton RE, Pethybridge RJ. DERMIS: a computer system for assisting primary-care physicians with dermatological diagnosis. Br J Dermatol. 1992;127:614–9.
- Smith HR, Ashton RE, Brooks GJ. Initial use of a computer system for assisting dermatological diagnosis in general practice. Med Inform Internet Med. 2000;25:103–8.
- Hammersley JR, Cooney K. Evaluating the utility of available different diagnosis systems. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1988:229–31.
- Cundell DR, Silibovsky RS, Sanders R, Sztandera LM. Using fuzzy sets to analyze putative correlates between age, blood type, gender and/or race with bacterial infection. Artif Intell Med. 2001;21:235–9.
- Ross JJ, Shapiro DS. Evaluation of the computer program GIDEON (Global Infectious Disease and Epidemiology Network) for the diagnosis of fever in patients admitted to a medical service. Clin Infect Dis. 1998;26:766–7.
- Murphy GC, Friedman CP, Elstein AS, Wolf FM, Miller T, Miller JG. The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. AMIA Proc Annu Fall Symp 1996:219–23.
- Berner ES, Webster GD, Shugerman AA, Jackson JR, Algina J, Baker AL, Performance of four computer-based diagnostic systems. N Engl J Med. 1994;330:1792–6.
- Bouhaddou O, Lambert JG, Miller S. Consumer health informatics: knowledge engineering and evaluation studies of medical HouseCall. Proc AMIA Symp 1998:612–6.
- El-Solh AA, Hsiao CB, Goodnough S, Serghani J, Grant BJ. Predicting active pulmonary tuberculosis using an artificial neural network. Chest. 1999;116:968–73.
- Verdaguer A, Patak A, Sancho JJ, Sierra C, Sanz F. Validation of the medical expert system PNEUMON-IA. Comput Biomed Res. 1992;25:511–26.
- Bankowitz RA, McNeil MA, Challinor SM, Parker RC, Kapoor WN, Miller RA. A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype. Ann Intern Med. 1989;110:824–32.
- Fiszman M, Chapman WW, Evans SR, Haug PJ. Automatic identification of pneumonia related concepts on chest x-ray reports. Proc AMIA Symp 1999:67–71.
- Chapman WW, Haug PJ. Comparing expert systems for identifying chest x-ray reports that support pneumonia. Proc AMIA Symp 1999:216–20.
- Carter CN, Ronald NC, Steele JH, Young E, Taylor JP, Russell LH, Knowledge-based patient screening for rare and emerging infectious/parasitic diseases: a case study of brucellosis and murine typhus. Emerg Infect Dis. 1997;3:73–6.
- Ashizawa K, MacMahon H, Ishida T, Nakamura K, Vyborny CJ, Katsuragawa S, Effect of an artificial neural network on radiologists' performance in the differential diagnosis of interstitial lung disease using chest radiographs. AJR Am J Roentgenol. 1999;172:1311–5.
- Monnier-Cholley L, MacMahon H, Katsuragawa S, Morishita J, Ishida T, Doi K. Computer-aided diagnosis for detection of interstitial opacities on chest radiographs. AJR Am J Roentgenol. 1998;171:1651–6.
- Sox HC, Blatt MA, Higgins MC, Marton KI. Medical decision making. Boston: Butterworth-Heinemann; 1988.