Predictions Through Artificial Intelligence

Mining the Web to Predict Future Events

By Kira Radinsky and Eric Horvitz

ABSTRACT

We describe and evaluate methods for learning to forecast
forthcoming events of interest from a corpus containing 22
years of news stories. We consider the examples of identifying
signi_cant increases in the likelihood of disease outbreaks,
deaths, and riots in advance of the occurrence of
these events in the world. We provide details of methods
and studies, including the automated extraction and generalization
of sequences of events from news corpora and multiple
web resources. We evaluate the predictive power of the
approach on real-world events withheld from the system.

INTRODUCTION

Mark Twain famously said that \the past does not repeat
itself, but it rhymes." In the spirit of this reection, we develop
and test methods for leveraging large-scale digital histories
captured from 22 years of news reports from the New
York Times (NYT) archive to make real-time predictions
about the likelihoods of future human and natural events of
interest. We describe how we can learn to predict the future
by generalizing sets of speci_c transitions in sequences of
reported news events, extracted from a news archive spanning
the years 1986{2008. In addition to the news corpora,
we leverage data from freely available Web resources, including
Wikipedia, FreeBase, OpenCyc, and GeoNames, via
the LinkedData platform [6]. The goal is to build predictive
models that generalize from speci_c sets of sequences of
events to provide likelihoods of future outcomes, based on
patterns of evidence observed in near-term newsfeeds. We
propose the methods as a means of generating actionable
forecasts in advance of the occurrence of target events in
the world.

The methods we describe operate on newsfeeds and can
provide large numbers of predictions. We demonstrate the
predictive power of mining thousands of news stories to create
classiers for a range of prediction problems. We show
as examples forecasts on three prediction challenges: proactive
alerting on forthcoming disease outbreaks, deaths, and
riots. These event classes are interesting in serving as examples
of predictions that can serve as heralds for attention
for guiding interventions that may be able to change outcomes
for the better. We compare the predictive power of
the methods to several baselines and demonstrate precisions
of forecasts in these domains ranging from 70% to 90% with
a recall of 30% to 60%.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.

WSDM’13, February 4–8, 2012, Rome, Italy.
Copyright 2013 ACM 978-1-4503-1869-3/13/02

= = = = = = = = = = = = = = = = = = = = = = = = = = =
  http://research.microsoft.com/en-us/um/people/horvitz/future_news_wsdm.pdf