contextual anomaly detection in text data上下文在文本数据异常检测.pdfVIP

Algorithms 2012, 4, 469-489; doi:10.3390/a5040469 OPEN ACCESS algorithms ISSN 1999-4893 /journal/algorithms Article Contextual Anomaly Detection in Text Data Amogh Mahapatra *, Nisheeth Srivastava and Jaideep Srivastava Department of Computer Science, University of Minnesota, 200 Union St SE, Minneapolis 55455, USA; E-Mails: nsriva@ (N.S.); srivasta@ (J.S.) * Author to whom correspondence should be addressed; E-Mail: mahap022@; Tel.: +61-2669-2974. Received: 20 June 2012; in revised form: 10 October 2012 / Accepted: 11 October 2012 / Published: 19 October 2012 Abstract: We propose using side information to further inform anomaly detection algorithms of the semantic context of the text data they are analyzing, thereby considering both divergence from the statistical pattern seen in particular datasets and divergence seen from more general semantic expectations. Computational experiments show that our algorithm performs as expected on data that reflect real-world events with contextual ambiguity, while replicating conventional clustering on data that are either too specialized or generic to result in contextual information being actionable. These results suggest that our algorithm could potentially reduce false positive rates in existing anomaly detection systems. Keywords: context detection; topic modeling; anomaly detection 1. Introduction The ability to detect anomalies in streams of text data finds broad applicability in a number of web applications. For example, it can be used to detect the occ