INDRI - Overview and Model.ppt

An Overview of the Indri Search Engine Don MetzlerCenter for Intelligent Information RetrievalUniversity of Massachusetts, Amherst Outline Overview Retrieval Model System Architecture Evaluation Conclusions Zoology 101 Lemurs are primates found only in Madagascar 50 species (17 are endangered) Ring-tailed lemurs lemur catta Zoology 101 The indri is the largest type of lemur When first spotted the natives yelled “Indri! Indri!” Malagasy for Look! ?Over there! What is INDRI? INDRI is a “larger” version of the Lemur Toolkit Influences INQUERY [Callan, et. al. ’92] Inference network framework Structured query language Lemur [/] Language modeling (LM) toolkit Lucene [/lucene/docs/index.html] Popular off the shelf Java-based IR system Based on heuristic retrieval models No IR system currently combines all of these features Design Goals Robust retrieval model Inference net + language modeling [Metzler and Croft ’04] Powerful query language Extensions to INQUERY query language driven by requirements of QA, web search, and XML retrieval Designed to be as simple to use as possible, yet robust Off the shelf (Windows, *NIX, Mac platforms) Separate download, compatible with Lemur Simple to set up and use Fully functional API w/ language wrappers for Java, etc… Scalable Highly efficient code Distributed retrieval Comparing Collections Outline Overview Retrieval Model Model Query Language Applications System Architecture Evaluation Conclusions Document Representation Model Based on original inference network retrieval framework [Turtle and Croft ’91] Casts retrieval as inference in simple graphical model Extensions made to original model Incorporation of probabilities based on language modeling rather than tf.idf Multiple language models allowed in the network (one per indexed context) Model Model P( r | θ ) Probability of observing a term, phrase, or “concept” given a context language model ri nodes are binary Assume r ~ Bernoulli( θ ) “Model B” – [Metzler, Lavrenko, Croft ’