Infectious disease surveillance 2.0: Crawling the Net to detect outbreaks
Think of it as an early-warning system for health officials
Computerworld - While recent outbreaks of salmonella in the U.S. have made headlines, an automated real-time system that scours the Web for information about disease outbreaks spied early reports in New Mexico about suspicious gastrointestinal illnesses days before the U.S. Centers for Diseases Control and Prevention (CDC) issued an official report on the problem.
The system, called HealthMap, is a free data-mining tool that extracts, categorizes, filters and links 20,000 Web-based data sources such as news sites, blogs, e-mail lists and chat rooms to monitor emerging public health issues. HealthMap, which is profiled in the July issue of the journal Public Library of Science Medicine and is open to anyone, was developed in late 2006 by John Brownstein and Clark Freifeld. Both men work in the informatics program at Children's Hospital Boston.
The system's goal is to try to detect early outbreaks of diseases before they are spotted by traditional surveillance sources or the international public health community, Brownstein said. Often, reports of new occurrences of infectious disease surface on the Web well before they percolate to the attention of the existing public health reporting infrastructure, he said.
"There were media reports and chat room discussions -- things going on in the informal settings -- saying things about the SARS outbreak [in China] before traditional official channels were reporting on the outbreak," he noted.
As of July 8, the tool was tracking 226 reports of salmonella, 110 reports of avian influenza, 49 reports of Dengue fever, 28 reports of anthrax, 24 reports of West Nile virus and 21 cases of hemorrhagic fever worldwide.
The tool is especially vital to developing countries or other areas of the world that might not have traditional disease surveillance mechanisms in place, Brownstein said. "There was no real source that brought all information about outbreaks together," he said. "This is a way to bring all this information together in a very organized and synthesized way while filtering a lot of noise that might otherwise exist on the Web."
HealthMap scours Web sources every hour, seven days a week, Freifeld added. The pair use RSS feeds when available to access the data or screen-scraping techniques for Web sources not available in feeds. The data is then categorized by the type of disease being described and the location of the outbreak, Freifeld added.
Articles are then analyzed for duplication and content, with duplicate articles removed and those that include new information about a related topic added to a Google Map. On the map, alerts are color-coded for how much recent news about those outbreaks has been noted. HealthMap also includes pop-up windows on highlighted cities or states that provide links to all the news reports on an outbreak for that location.
- Big Data, Big Mess: Sound Risk Intelligence Through Complete Context This paper examines the insecurity of the small businesses in the supply chain and offers tips to close those backdoors into the enterprise.
- CIOs strive to harness Big Data while keeping an eye on the bottom line Read this whitepaper to learn how Red Hat Storage Server allows CIOs to confidently support business growth, manage cost and risk, capitalize on...
- Enterprise architects challenged to manage data explosion Read this whitepaper to find out how Red Hat Storage Server can allow enterprises to quickly and confidently deliver business applications that minimize...
- Software Asset Management: Ensuring Today's Assets Today's trends like BYOD and SaaS are new and exciting in terms of how they will help make our jobs more productive but...
- Live Webcast Charting Your Analytical Future - "Making predictive analytics part of your business processes" Webinar This session will show how predictive analytics can be used throughout the organization by anyone looking for answers and how organizations can make...
- Charting Your Analytical Future - "Making predictive analytics part of your business processes" Webinar This session will show how predictive analytics can be used throughout the organization by anyone looking for answers and how organizations can make...
- Capturing Data in Motion: Delivering Real-Time Insight from Data Streams This webcast will help organizations of all types and sizes learn about a technology and business strategy for tapping into the wealth of... All Business Intelligence/Analytics White Papers | Webcasts
Our new bimonthly Internet of Things newsletter helps you keep pace with the rapidly evolving technologies, trends and developments related to the IoT. Subscribe now and stay up to date!