pp380-385.
HavingYourCakeandEatingItToo:AutonomyandInteractionin
aModelofSentenceProcessing
CollegeofComputing
GeorgiaInstituteofTechnologyAtlanta,Georgia30332-0280
KurtP.Eiselt∗
CollegeofComputing
GeorgiaInstituteofTechnologyAtlanta,Georgia30332-0280KaviMahesh∗JenniferK.Holbrook
DepartmentofPsychology
AlbionCollegeAlbion,Michigan492244991 guA 13 1v0208049/lg-pmc:viXraeiselt@cc.gatech.edu
mahesh@cc.gatech.edu
Abstract
Isthehumanlanguageunderstanderacollectionofmodularprocessesoperatingwithrelativeautonomy,orisitasingleintegratedprocess?Thisongoingde-batehaspolarizedthelanguageprocessingcommu-nity,withtwofundamentallydifferenttypesofmodelposited,andwitheachcampconcludingthattheotheriswrong.Onecampputsforthamodelwithseparateprocessorsanddistinctknowledgesourcestoexplainonebodyofdata,andtheotherproposesamodelwithasingleprocessorandahomogeneous,monolithicknowledgesourcetoexplaintheotherbodyofdata.Inthispaperwearguethatahybridapproachwhichcombinesaunifiedprocessorwithseparateknowledgesourcesprovidesanexplanationofbothbodiesofdata,andwedemonstratethefeasibilityofthisapproachwiththecomputationalmodelcalledCOMPERE.Webelievethatthisapproachbringsthelanguageprocess-ingcommunitysignificantlyclosertoofferinghuman-likelanguageprocessingsystems.
TheBigQuestions
Yearsofresearchbylinguists,psychologists,andartifi-cialintelligencespecialistshaveprovidedsignificantin-sightintotheworkingsofthehumanlanguageproces-sor.Still,fundamentalquestionsremainunanswered.Inparticular,thedebateovermodularprocessingver-susintegratedprocessingrageson,andexperimentaldataandcomputationalmodelsexisttosupportbothpositions.Furthermore,iftheintegratedprocessingpositioniscorrect,justwhatexactlyisintegrated?Andifthemodularpositionistherightone,whatarethedifferentmodules?Dotheyinteract,andifso,towhatextentandwhen?Orarethosemodulesentirelyautonomous?
Wrestlingwiththesequestionsinducesconsiderablefrustrationinresearchers.Thisfrustrationstemsnotonlyfromtheresearchcommunity’sapparentinabil-itytoanswerthemsatisfactorily,butalsofromtheoverwhelmingimportanceoftheanswersthemselves—theseanswers,onceuncovered,undoubtedlywillim-pactthinkinginallareasofartificialintelligenceand
jen@cedar.cic.net
&Bienkowski,1982),and,moreimportantly,itmadepsychologicalpredictionsabouttheretentionofuns-electedmeaningsthatwereexperimentallyvalidated(Eiselt&Holbrook,1991;Holbrook,1989).ATLASTprovidedanarchitectureofsentenceprocessingwhichwasalsousedtoexplainrecoveryfromerroneousde-cisionsinmakingpragmaticinferencesaswellasex-plainingindividualdifferencesinpragmaticinferences(Eiselt,1989;cf.Granger,Eiselt,&Holbrook,1983).Errorrecoveryinsemanticprocessinghadoc-casionallyarousedtheattentionofresearchersinconceptually-basednaturallanguageunderstanding,butthequestionsthatarosewereusuallydismissedasunimportantorsomethingwhichcouldbere-solvedasanafterthought(Birnbaum&Selfridge,1981;Lebowitz,1980;Lytinen,1984).Theseresearcherswerecontenttoassumethatthefirstinferencedeci-sionmadewasthecorrectone.Meanwhile,otherre-searchersinvestigatingsyntactically-basedapproacheshadlongsinceconcludedthatthemeansbywhicher-roneoussyntacticdecisionswereaccommodatedhadadramaticimpactonthearchitectureofthesyntac-ticprocessorbeingproposed.Forexample,theback-trackingmodelsembodiedthetheorythatonlyasin-glesyntacticinterpretationneedbemaintainedatanygiventime,solongastheprocessorcouldkeeptrackofitsdecisions,undothemwhenanerroneousdecisionwasdiscovered,andthenreinterprettheinput(e.g.,Woods,1973).Thelookaheadparserstriedtosidesteptheproblemsinherentinbacktrackingbypostponinganydecisionuntilenoughinputhadbeenprocessedtoguaranteeacorrectdecision,therebyavoidinger-roneousdecisionstosomeextent(e.g.,Marcus,1980).Anotherapproachtoavoidingerroneousdecisionswasofferedbyparallelparserswhichmaintainedallplausi-blesyntacticinterpretationsatthesametime(Kurtz-man,1985).ATLAST,however,wasamodelofseman-ticprocessinganddidnotaddresstheissueofrecoveryfromerroneoussyntacticdecisions,nordiditsubstan-tiallyaddresstheissueofsyntacticprocessingatall.Recently,Stowe(1991)presentedexperimentalev-idenceshowingthatindealingwithsyntacticambi-guity,thesentenceprocessoraccessesallpossiblesyn-tacticstructuressimultaneouslyand,ifthestructurepreferredforsyntacticreasonsconflictswiththestruc-turefavoredbythecurrentsemanticbias,thecom-petingstructuresaremaintainedandthedecisionisdelayed.Furthermore,theworksuggestsaninterac-tionofthevariousknowledgetypes,asinsomecasessemanticinformationinfluencesstructureassignmentortriggersreactivationofunselectedstructures.Thismodeloflimiteddelayeddecisioninsyntacticambigu-ityresolutionhadmuchincommonwiththeATLASTmodelofsemanticambiguityresolution.Bothmodelsproposedanearlycommitmentwherepossible.Bothmodelshadthecapabilitytopursuemultipleinterpre-tationsinparallelwhenambiguitymadeitnecessary.Bothmodelsexplainederrorrecoveryasanoperation
ofswitchingtoanotherinterpretationmaintainedinparallelbythesentenceprocessor.Finally,bothmod-elsmadedecisionsbyintegratingthepreferencesfromsyntaxandsemantics.
Oneexplanationforthishighdegreeofsimilaritybe-tweenthesyntacticandsemanticerrorrecoverymecha-nismsisthattherearetwoseparateprocessors,oneforsyntaxandoneforsemantics,eachwithitscorrespond-ingsourceoflinguisticknowledge,andeachdoingex-actlythesamething.Amoreeconomicalexplanation,however,isthatthereisonlyoneprocesswhichdealswithsyntacticandsemanticinformationinthesamemanner.Wehavechosentoexplorethelatterexplana-tion,asothershavedone,butwehavealsochosentomaintaintheseparateknowledgesourcesforreasonswhichwillbeexplainedbelow.(SeealsoHolbrook,Eiselt,&Mahesh,1992.)
OverviewofCOMPERE
Ournewmodelofsentenceprocessing,calledCOM-PERE(CognitiveModelofParsingandErrorRecov-ery),consistsofasingleunifiedprocessoperatingonindependentsourcesofsyntacticandsemanticknowl-edge.Thisismadepossiblebyauniformrepresenta-tionofbothtypesofknowledge.Theunifiedprocessappliesthesameoperationstothedifferenttypesofknowledge,andhasasinglecontrolstructurewhichperformstheoperationsonsyntacticandsemanticknowledgeintandem.Thispermitsarichinteractionbetweenthetwosourcesofknowledge,boththroughtransferofcontrolandthroughasharedrepresenta-tionoftheinterpretationsoftheinputtextbeingbuiltbytheunifiedprocess.
Anadvantageofrepresentingthedifferentkindsofknowledgeinthesameformisthattheboundariesbetweenthedifferenttypesofknowledgecanbeill-defined.Oftenitisdifficulttoclassifyapieceofknowl-edgeasbelongingtoaparticularclasssuchassyntac-ticorsemantic.Withauniformrepresentation,suchknowledgeliesinbetweenandcanbetreatedasbe-longingtoeitherclass.
Syntacticandsemanticknowledgearerepresentedinseparatenetworksinwhicheachnodeisastruc-turedrepresentationofalltheinformationpertainingtoasyntacticorsemanticcategoryorconcept.Alink,representedasaslot-fillerpairinthenode,specifiesaparentcategoryorconceptofwhichthenodecanbeapart,togetherwiththeconditionsunderwhichitcanbeboundtotheparent,andtheexpectationsthatarecertaintobefulfilledshouldthenodebeboundtotheparent.Inaddition,nodesineithernetworkarelinkedtocorrespondingnodesintheothernetworksothattheunifiedprocesscanbuildon-lineinterpretationsoftheinputsentenceinwhicheachsyntacticunithasacor-respondingrepresentationofitsthematicroleanditsmeaning.Inaddition,thereisalexiconaswellascer-tainotherminorheuristicandcontrolknowledgethatispartoftheprocess.(COMPERE’sarchitectureand
knowledgerepresentationaredisplayedgraphicallyinFigures1and2.)
Theunifiedprocessisabottom-up,early-commitmentparsingmechanismintegratedwithtop-downguidancethroughexpectations.TheoperatorsandthecontrolstructurethatconstitutetheunifiedprocessaredescribedbrieflyinthealgorithmshowninFigure3.
Syntactic ParseSemanticConceptual Tree Roles MeaningWordsLexicalEntriesLexiconUnifiedProcessSyntaxSemanticsConceptualKnowledgeFigure1:ArchitectureofCOMPERE.
\"moved\"wordword: \"moved\"category: VMOVE:sub-cat: (simple-past, Agent: (must-be animate) past-participle) Theme: ()meaning: MOVE To-Location: ()lexical-entryconceptual-nodesyntactic-nodesemantic-nodeNP:VP: (must-precede V)Active-SUBJ:S: (must-precede NIL) Agent: ((satisfies-event-role agent) (expect VP) (satisfies-filler-constraints agent))PP: (must-precede Prep) Non-Agent-SUBJ: .....Figure2:KnowledgeRepresentationinCOMPERE.1
——————————
1.Accesslexicalentriesofnextword.
2.Createinstancenodesforsyntacticcategory,mean-ing,and(primitive)thematicrole.
3.Computefeasiblebindingstoparentsforsyntacticinstancenodeandroleinstancenode.(Thisopera-tionchecksanyconditionstobesatisfiedtomakethebindingfeasible;italsotakesexistingexpectationsintoaccount.)
4.Ranksyntacticandsemanticfeasiblebindingsbytheirrespectivepreferencecriteria.
Combinefeasiblebindingsandselectthemostpre-ferredbinding.
5.Makethebindingbycreatingparentnodeinstancesandappropriatelinks,andgeneratinganyexpecta-tions.Createlinksbetweencorrespondinginstancesinsyntaxandtheirthematicrolesandmeanings.
6.Retainalternativebindingsforpossibleerrorrecov-ery.
7.Ifthereisnofeasiblebindingforanode,explorepreviouslyretainedalternativestorecoverfromerrors.8.Continuetobindtheparentnodestonodesfurtherupasfaraspossible(suchasuntiltheSnodeinsyntaxortheEventnodeinsemantics).——————————
Figure3:UnifiedProcess:Algorithm.
TheCOMPEREprototypehasbeenimplementedinCommonLISPonaSymbolicsLISPMachine.Atthistime,itsunifiedprocesscanperformon-lineinter-pretationsofitsinput,andcanrecoverfromerroneoussyntacticdecisionswhennecessary.COMPEREisabletoprocessrelativelycomplexsyntacticstructures,in-cludingrelativeclauses,andcanresolvetheassociatedstructuralambiguities.
Autonomyandinteractioneffectsfrom
oneprocess
COMPEREisabletoexhibitseeminglymodularpro-cessingbehaviorthatmatchestheresultsofexperi-mentsshowingtheautonomyofdifferentlevelsoflan-guageprocessing(e.g.,Forster,1979;Frazier,1987).Itisalsoabletodisplayseeminglyintegratedbehav-iorthatmatchestheresultsofexperimentsshowingsemanticinfluencesonsyntacticstructureassignment(e.g.,Crain&Steedman,1985;Tyler&Marslen-Wilson,1977).Forexample,considertheprocessingofthefollowingsentence:
(1)Thebugsmovedintothenewloungewerefoundquickly.
Thissentencehasalexicalsemanticambiguityatthesubjectnounbugsthatcouldmeaneitherinsects
orelectronicmicrophones.Inaddition,itisalsosyn-tacticallyambiguouslocallyattheverbmovedsincethereisnodistinctionbetweenitspast-tenseformanditspast-participleform.Inthesimplepastreadingofmoved,itwouldbethemainverbwiththecorrespond-inginterpretationthat“thebugsmovedthemselvesintothenewlounge.”Ontheotherhand,ifmovedisreadasaverbinitspast-participleform,itwouldbetheverbinareducedrelativeclausecorrespondingtothemeaning“thebugswhichweremovedbysome-bodyelseintothenewlounge....”Parsetreesforthetwostructuralinterpretationsandthecorrespondingthematic-roleassignmentsareshowninFigures4and5.2
SNPVPARTNVPPThebugsmovedMove: Agent: bug To-Loc: loungeintothenewloungeFigure4:GardenPath:Main-ClauseInterpretation.NullContext:Whensentence(1)ispresentedtoCOMPEREinanullsemanticcontext,onewherethereisnobiasforeithermeaningofthenounbugs,COM-PEREreadsaheadwithoutresolvingthelexicalambi-guityatthewordbugs.Whenitencountersthestruc-turalambiguityattheverbmoved,COMPEREdoesnothavethenecessaryinformationtodecidewhichofthetwostructuresinFigures4and5istheappropriateonetopursue.
However,COMPEREhasasyntacticpreferenceforthemain-verbinterpretationovertherelativeclauseone.Thoughthispreferencecanbeexplainedbytheminimalattachmentprinciple(Frazier,1987),COM-PEREoffersamoregeneralexplanation.Extrapolat-ingfromStowe’smodel,wehaveendowedCOMPEREwiththepervasivegoalofcompletinganincompleteitematanylevelofprocessing.Insyntacticprocess-ing,ithasagoaltocompletethesyntacticstructureofaunitsuchasaphrase,clause,orasentence.COM-PEREprefersthealternativewhichhelpscompletethecurrentstructure(calledtheSyntacticDefault)overonethataddsanoptionalconstituentleavingthein-
3
COMPERE’sprogramdoesnotresolvelexicalseman-ticambiguitiesatthistime.Wearecurrentlyrectifyingthisbyincorporatinglexicalambiguityresolutionstrate-giesfromourearliermodelATLAST(Eiselt,1989)inCOMPERE.
uponreadingthissentence.Thatis,thesentencepro-cessorisledupagardenpathandhastobacktrackwhenlaterinformationshowsthatitwasthewrongpathtotake.Thisbehaviorisnotinfluencedbyse-manticorconceptualpreferencesandcanbeperceivedasamodularbehavior.COMPERE’serrorrecoverymethodwasfirstdevelopedintheATLASTmodel(Eiselt,1987).Itwasalsoexperimentallyvalidated(Eiselt&Holbrook,1991).
Asaconsequenceofswitchingtothenewsyntac-ticinterpretation,COMPEREmakescorrespondingchangestothematicroleassignmentsandalso“unre-solves”thelexicalambiguity.Thereisnolongeranyreasontoeliminatetheelectronicbugmeaningsinceeitherkindofbugscanbemovedbyothers.
SemanticallyBiasingContext:Nowconsidersen-tence(1)inasemanticallybiasingcontextsuchastheonein(2).4
(2)TheAmericansbuiltanewwingtotheembassy.TheRussianspiesquicklytransferredthemicrophonestothenewwing.Thebugsmovedintothenewloungewerefoundquickly.
Thesemanticcontextin(2)resolvesthelexicalam-biguitybychoosingtheelectronicbugmeaning.ThisdecisionhelpsCOMPEREresolvethestructuralambi-guityattheverbmoved.Usingitsconceptualknowl-edge,representedasaselectionalrestriction,thatonlyanimateagentscanmovebythemselves,COMPEREdecidesthatmovedcannotbeamainverbandgoesdirectlytothereducedrelativeclauseinterpretation(Fig.5),therebyavoidingthegardenpath.Thisshowshowthesameunifiedprocessthatpreviouslyexhibitedmodularprocessingbehaviorcanalsoproduceinter-activeprocessingbehaviorwhensemanticinformationisavailable.SyntaxandsemanticsinteractinCOM-PEREtohelpresolveambiguitiesineachother.
COMPEREcanalsouseindependentsyntacticpref-erencesinothertypesofsentencessuchasthosewithprepositionalattachmentambiguities.TheCOM-PEREprototypethusdemonstratesthattherangeofbehaviorsthattheinteractivemodelsaccountfor(Crain&Steedman,1985;Tyler&Marslen-Wilson,1977),andthebehaviorsthatthe“firstanalysis”mod-elsaccountfor(Frazier,1987),canbeexplainedbyaunifiedmodelwithasingleprocessoroperatingonmul-tipleindependentsourcesofknowledge.
Comparativeevaluation
Thereiscertainlynothinguniqueaboutaunifiedpro-cessmodeloflanguageunderstanding—theintegrated
plementationcurrentlydivergesslightlyfromtheory.Thedivergenceappearsintheprocessitself:thethe-oreticalmodelhasasingleunifiedprocess,whiletheprototypecomputationalmodelconsistsoftwonearly-identicalprocesses—oneforsyntaxandoneforseman-tics.Thesetwoprocessesshareidenticalcontrolstruc-tures,buttheyareduplicatedbecausewehavenotyetcompletedthetaskofrepresentingthedifferenttypesofinformationinauniformformat.Somereadersmaytakethisasanindicationthatwearedoomedtofail-ure,buttheconnectionistmodelsmentionedearlierserveasexistenceproofsthatfindingauniformformatforrepresentingdifferenttypesoflinguisticknowledgeisbynomeansanimpossibletask.
Conclusion
Isthehumanlanguageunderstanderacollectionofmodularprocessesoperatingwithrelativeautonomy,orisitasingleintegratedprocess?Thisongoingde-batehaspolarizedthelanguageprocessingcommu-nity,withtwofundamentallydifferenttypesofmodelposited,andwitheachcampconcludingthattheotheriswrong.Onecampputsforthamodelwithseparateprocessorsanddistinctknowledgesourcestoexplainonebodyofdata,andtheotherproposesamodelwithasingleprocessorandahomogeneous,mono-lithicknowledgesourcetoexplaintheotherbodyofdata.Inthispaperwehavearguedthatahybridap-proachwhichcombinesaunifiedprocessorwithsep-arateknowledgesourcesprovidesanexplanationofbothbodiesofdata,andwehavedemonstratedthefeasibilityofthisapproachwiththecomputationalmodelcalledCOMPERE.Webelievethatthisap-proachbringsthelanguageprocessingcommunitysig-nificantlyclosertoofferinghuman-likelanguagepro-cessingsystems.
Acknowledgement:WewouldliketothankJustinPetersonforhiscommentsonthisworkandhishelpinfindinggoodexamples.
References
Birnbaum,L.,andSelfridge,M.1981.Conceptualanalysisofnaturallanguage.InSchank,R.C.,andRiesbeck,C.K.eds.Insidecomputerunderstanding:Fiveprogramsplusminiatures,318-353.Hillsdale,NJ:LawrenceErlbaum.
Caramazza,A.,andBerndt,R.S.1978.Semanticandsyntacticprocessesinaphasia:Areviewofthelitera-ture.PsychologicalBulletin85:898-918.
Cottrell,G.W.1985.Aconnectionistapproachtowordsensedisambiguation,TechnicalReport,154,ComputerScienceDepartment,UniversityofRochester.
Cottrell,G.W.,andSmall,S.L.1983.Aconnection-istschemeformodellingwordsensedisambiguation.CognitionandBrainTheory6:89-120.
Crain,S.,andSteedman,M.1985.Onnotbeingledupthegardenpath:Theuseofcontextbythepsycho-logicalsyntaxprocessor.InDowty,D.R.,Kartunnen,L.,andZwicky,A.M.eds.Naturallanguageparsing:Psychological,computational,andtheoreticalperspec-tives,320-358.Cambridge,England:CambridgeUni-versityPress.
Eiselt,K.P.1987.Recoveringfromerroneousinfer-ences.InProc.AAAI-87SixthNationalConferenceonArtificialIntelligence,540-544.SanMateo,CA:MorganKaufmann.
Eiselt,K.P.1989.Inferenceprocessinganderrorre-coveryinsentenceunderstanding,TechnicalReport,89-24,Ph.D.diss.,Dept.ofComputerScience,Uni-versityofCalifornia,Irvine.
Eiselt,K.P.,andHolbrook,J.K.1991.Towardaunifiedtheoryoflexicalerrorrecovery.InProc.oftheThirteenthAnnualConferenceoftheCognitiveScienceSociety,239-244.Hillsdale,NJ:LawrenceErlbaum.Forster,K.I.1979.Levelsofprocessingandthestruc-tureofthelanguageprocessor.InCooper,W.E.,andWalker,E.C.T.eds.Sentenceprocessing:Psy-cholinguisticstudiespresentedtoMerrillGarrett,27-85.Hillsdale,NJ:LawrenceErlbaum.
Frazier,L.1987.Theoriesofsentenceprocessing.InGarfield,J.L.ed.Modularityinknowledgerepresenta-tionandnatural-languageunderstanding.Cambridge,MA:MITPress.
Granger,R.H.,Eiselt,K.P.,andHolbrook,J.K.1983.STRATEGIST:Aprogramthatmodelsstrategy-drivenandcontent-driveninferencebehavior.InProc.oftheNationalConferenceonArtificialIntel-ligence,139-147.SanMateo,CA:MorganKaufmann.Holbrook,J.K.1989.Studiesofinferenceretentioninlexicalambiguityresolution.Ph.D.diss.,SchoolofSocialSciences,UniversityofCalifornia,Irvine.
Holbrook,J.K.,Eiselt,K.P.,andMahesh,K.1992.Aunifiedprocessmodelofsyntacticandsemanticerrorrecoveryinsentenceunderstanding.InProc.oftheFourteenthAnnualConferenceoftheCognitiveScienceSociety,195-200.Hillsdale,NJ:LawrenceErlbaum.Jurafsky,D.1992.Anon-linecomputationalmodelofhumansentenceinterpretation.InProc.oftheTenthNationalConferenceonArtificialIntelligence,302-308.SanMateo,CA:MorganKaufmann.
Kurtzman,H.S.1985.Studiesinsyntacticambiguityresolution.Ph.D.diss.,Dept.ofPsychology,Mas-sachusettsInstituteofTechnology.
Lebowitz,M.1980.Generalizationandmemoryinanintegratedunderstandingsystem,ResearchReport,186,Dept.ofComputerScience,YaleUniversity.
Lytinen,S.L.1984.Theorganizationofknowledgeinamulti-lingual,integratedparser,ResearchReport,YALEU/CSD/RR340,Dept.ofComputerScience,YaleUniversity.
Marcus,M.P.1980.Atheoryofsyntacticrecognitionfornaturallanguage.Cambridge,MA:MITPress.Pollack,J.B.1987.Onconnectionistmodelsofnaturallanguageprocessing,TechnicalReport,MCCS-87-100,ComputingResearchLaboratory,NewMexicoStateUniversity.
Riesbeck,C.K.,andMartin,C.E.1986.Towardscom-pletelyintegratedparsingandinferencing.InProc.oftheEighthAnnualConferenceoftheCognitiveScienceSociety,381-387.Hillsdale,NJ:LawrenceErlbaum.Seidenberg,M.S.,Tanenhaus,M.K.,Leiman,J.M.,andBienkowski,M.1982.Automaticaccessofthemeaningsofambiguouswordsincontext:Somelimi-tationsofknowledge-basedprocessing.CognitivePsy-chology14:489-537.
Stowe,L.A.1991.Ambiguityresolution:Behavioralevidenceforadelay.InProc.oftheThirteenthAnnualConferenceoftheCognitiveScienceSociety,257-262.Hillsdale,NJ:LawrenceErlbaum.
Taraban,R.,andMcClelland,J.L.1988.Constituentattachmentandthematicroleassignmentinsentenceprocessing:Influencesofcontent-basedexpectations.JournalofMemoryandLanguage27:597-632.
Tyler,L.K.,andMarslen-Wilson,W.D.1977.Theon-lineeffectsofsemanticcontextonsyntacticprocess-ing.JournalofVerbalLearningandVerbalBehavior16:683-692.
Waltz,D.L.,andPollack,J.B.1985.Massivelypar-allelparsing:Astronglyinteractivemodelofnaturallanguageinterpretation.CognitiveScience9:51-74.Woods,W.A.1973.Anexperimentalparsingsystemfortransitionnetworkgrammars.InRustin,R.ed.Naturallanguageprocessing.NewYork:AlgorithmicsPress.
因篇幅问题不能全部显示，请点此查看更多更全内容