1、Event Extraction:Learning from CorporaPrepared by Ralph GrishmanBased on research and slides by Roman YangarberNYUFinding PatternsHow can we collect patterns?Supervised learningmark information to be extracted in textcollect information and context = specific patternsgeneralize patternsAnnotation qu

2、ite expensiveZipfian distribution of patterns means that annotation of consecutive text is inefficient the same pattern is annotated many timesUnsupervised learning?The intuition:if we collect documents DR relevant to the scenario, patterns relevant to the scenario will occur more frequently in DR t

3、han in the language as a whole(cf. sublanguage predicates in Harriss distributional analysis)Riloff 96Corpus manually divided into relevant and irrelevant documentsCollect patterns around each noun phraseScore patterns by R log Fwhere R = relevance rate= freq in relevant docs / overall freqSelect to

4、p-ranked patternsThese patterns each find one template slot;combining filled slots into templates is a separate taskExtending the Discovery ProcedureFinding relevant documents automaticallyYangarber use patterns to select documentsSudo use keywords and IR engineDefining larger patterns (covering sev

5、eral template slots)Yangarber clause structuresNobata; Sudo larger structuresAutomated Extraction Pattern DiscoveryGoal: find examples / patterns relevant to a given scenariowithout any corpus tagging (Yangarber 00)Method:identify a few seed patterns for scenarioretrieve documents containing pattern

6、sfind subject-verb-object pattern withhigh frequency in retrieved documentsrelatively high frequency in retrieved docs vs. other docsadd pattern to seed and repeat#1: pick seed patternSeed: #2: retrieve relevant documentsSeed: Fred retired. Harry was named president.Maki retired. Yuki was named pres

7、ident.Relevant documentsOtherdocuments#3: pick new patternSeed: appears in several relevant documents (top-ranked by Riloff metric)Fred retired. Harry was named president.Maki retired. Yuki was named president.#4: add new pattern to pattern setPattern set: Note: new patterns added with confidence 1E

8、xperimentTask: Management succession (as MUC-6)Source: Wall Street JournalTraining corpus: 6,000 articlesTest corpus:100 documents: MUC-6 formal training+ 150 documents judged manuallyPre-processingFor each document, find and classify names: person | location | organization | Parse document(regulari

9、ze passive, relative clauses, etc.)For each clause, collect a candidate pattern:tuple: heads of subject verb direct object object/subject complement locative and temporal modifiers Experiment: two seed patternsv-appoint = appoint, elect, promote, name v-resign = resign, depart, quit, step-down Run d

10、iscovery procedure for 80 iterationsEvaluationLook at discovered patternsnew patterns, missed in manual trainingDocument filteringSlot fillingDiscovered patternsEvaluation: Text FilteringHow effective are discovered patterns at selecting relevant documents?IR-styledocuments matching at least one patternHow effective are patterns within a complete IE system?MUC-style IE on MUC-6 corporaCaveat: filtered / aligned by handtraining