Evaluating the Markov assumption in Markov Decision Processes for spoken dialogue management.
Saved in:
| Title: | Evaluating the Markov assumption in Markov Decision Processes for spoken dialogue management. |
|---|---|
| Authors: | Paek, Tim1 timpaek@microsoft.com, Chickering, David Maxwell1 max.chickering@microsoft.com |
| Source: | Language Resources & Evaluation. Feb2006, Vol. 40 Issue 1, p47-66. 20p. |
| Subjects: | Markov processes, Dialogue analysis, Interpersonal communication, Belief & doubt, Learning, Web browsers |
| Abstract: | The goal of dialogue management in a spoken dialogue system is to take actions based on observations and inferred beliefs. To ensure that the actions optimize the performance or robustness of the system, researchers have turned to reinforcement learning methods to learn policies for action selection. To derive an optimal policy from data, the dynamics of the system is often represented as a Markov Decision Process (MDP), which assumes that the state of the dialogue depends only on the previous state and action. In this article, we investigate whether constraining the state space by the Markov assumption, especially when the structure of the state space may be unknown, truly affords the highest reward. In simulation experiments conducted in the context of a dialogue system for interacting with a speech-enabled web browser, models under the Markov assumption did not perform as well as an alternative model which classifies the total reward with accumulating features. We discuss the implications of the study as well as its limitations. [ABSTRACT FROM AUTHOR] |
| Copyright of Language Resources & Evaluation is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 23218134 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Evaluating the Markov assumption in Markov Decision Processes for spoken dialogue management. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Paek%2C+Tim%22">Paek, Tim</searchLink><relatesTo>1</relatesTo><i> timpaek@microsoft.com</i><br /><searchLink fieldCode="AR" term="%22Chickering%2C+David+Maxwell%22">Chickering, David Maxwell</searchLink><relatesTo>1</relatesTo><i> max.chickering@microsoft.com</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Language+Resources+%26+Evaluation%22">Language Resources & Evaluation</searchLink>. Feb2006, Vol. 40 Issue 1, p47-66. 20p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Markov+processes%22">Markov processes</searchLink><br /><searchLink fieldCode="DE" term="%22Dialogue+analysis%22">Dialogue analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Interpersonal+communication%22">Interpersonal communication</searchLink><br /><searchLink fieldCode="DE" term="%22Belief+%26+doubt%22">Belief & doubt</searchLink><br /><searchLink fieldCode="DE" term="%22Learning%22">Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Web+browsers%22">Web browsers</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: The goal of dialogue management in a spoken dialogue system is to take actions based on observations and inferred beliefs. To ensure that the actions optimize the performance or robustness of the system, researchers have turned to reinforcement learning methods to learn policies for action selection. To derive an optimal policy from data, the dynamics of the system is often represented as a Markov Decision Process (MDP), which assumes that the state of the dialogue depends only on the previous state and action. In this article, we investigate whether constraining the state space by the Markov assumption, especially when the structure of the state space may be unknown, truly affords the highest reward. In simulation experiments conducted in the context of a dialogue system for interacting with a speech-enabled web browser, models under the Markov assumption did not perform as well as an alternative model which classifies the total reward with accumulating features. We discuss the implications of the study as well as its limitations. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Language Resources & Evaluation is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=23218134 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10579-006-9008-2 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 20 StartPage: 47 Subjects: – SubjectFull: Markov processes Type: general – SubjectFull: Dialogue analysis Type: general – SubjectFull: Interpersonal communication Type: general – SubjectFull: Belief & doubt Type: general – SubjectFull: Learning Type: general – SubjectFull: Web browsers Type: general Titles: – TitleFull: Evaluating the Markov assumption in Markov Decision Processes for spoken dialogue management. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Paek, Tim – PersonEntity: Name: NameFull: Chickering, David Maxwell IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 02 Text: Feb2006 Type: published Y: 2006 Identifiers: – Type: issn-print Value: 1574020X Numbering: – Type: volume Value: 40 – Type: issue Value: 1 Titles: – TitleFull: Language Resources & Evaluation Type: main |
| ResultId | 1 |
