MachineTranslationPhraseAlignment.pptVIP

Stephan Vogel - Machine Translation Machine TranslationPhrase Alignment Stephan Vogel Spring Semester 2011 Overview Why Phrase Alignment? Phrase Pairs from Viterbi Alignment Heuristics Some Analysis Phrase Pair Extraction as Sentence Splitting Additional Phrase Pair Features Alignment Example One Chinese word aligned to multi-word English phrase In lexicon individual entries with ‘the’, ‘development’, ‘of’ Difficult to generate from words Main translation ‘development’ Test if insertions of ‘the’ and ‘of’ improves LM probability Easier to generate if we have phrase pairs available Why Phrase to Phrase Translation Captures n x m alignments Encapsulates context Local reordering Compensates segmentation errors How to get Phrase Translation Typically: Train word alignment model and extract phrase-to-phrase translations from Viterbi path IBM model 4 alignment HMM alignment Bilingual Bracketing Genuine phrase translation models Integrated segmentation and alignment (ISA) Phase Pair Extraction via full Sentence Alignment Notes: Often better results when training target to source for extraction of phrase translations due to asymmetry of alignment models Phrases are not fully integrated into the alignment model, they are extracted only after training is completed – how to assign probabilities? Phrase Pairs from Viterbi Path Train your favorite word alignment (IBMn, HMM, …) Calculate Viterbi path (i.e. path with highest probability or best score) The details …. Word Alignment Matrix Alignment probabilities according to lexicon Viterbi Path Calculate Viterbi path (i.e. path with highest probability) Phrases from Viterbi Path Read off source phrase – target phrase pairs Extraction of Phrases foreach source phrase length l { foreach start position j1 = 1 … J – l { foreach end position j2 = j1 + l – 1 { min_i = min{ a(j) : j = j1 … j2 } max_i = max{ a(j) : j = j1 … j2 } SourcePhrase = fj1 … fj2 TargetPhra