View in EDS

Manipulation Action Understanding for Observation and Execution

Saved in:
Bibliographic Details
Title: Manipulation Action Understanding for Observation and Execution
Language: English
Authors: Yang, Yezhou
Source: ProQuest LLC. 2015Ph.D. Dissertation, University of Maryland, College Park.
Availability: ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Peer Reviewed: N
Page Count: 211
Publication Date: 2015
Document Type: Dissertations/Theses - Doctoral Dissertations
Descriptors: Observation, Object Manipulation, Perceptual Motor Coordination, Semantics, Grammar, Robotics, Experiments, Artificial Intelligence, Cognitive Processes, Thinking Skills, Perceptual Motor Learning, Learning Modalities, Psychomotor Skills, Psychomotor Objectives, Transformational Generative Grammar, Generative Grammar, Syntax
ISBN: 978-1-339-47342-0
Abstract: Modern intelligent agents will need to learn the actions that humans perform. They will need to recognize these actions when they see them and they will need to perform these actions themselves. We want to propose a cognitive system that interprets human manipulation actions from perceptual information (image and depth data) and consists of perceptual modules and reasoning modules that are in interaction with each other. The contributions of this work are given along two core problems at the heart of action understanding: a.) the grounding of relevant information about actions in perception (the perception-action integration problem), and b.) the organization of perceptual and high-level symbolic information for interpreting the actions (the sequencing problem). At the high level, actions are represented with the Manipulation Action Context-free Grammar (MACFG) , a syntactic grammar and associated parsing algorithms, which organizes actions as a sequence of sub-events. Each sub-event is described by the hand (as well as grasp type), movements (actions) and the objects and tools involved, and the relevant information about these quantities is obtained from biological-inspired perception modules. These modules track the hands and objects and recognize the hand grasp, actions, segmentation, and action consequences. Furthermore, a probabilistic semantic parsing framework based on CCG (Combinatory Categorial Grammar) theory is adopted to model the semantic meaning of human manipulation actions. Additionally, the lesson from the findings on mirror neurons is that the two processes of interpreting visually observed action and generating actions, should share the same underlying cognitive process. Recent studies have shown that grammatical structures underlie the representation of manipulation actions, which are used both to understand and to execute these actions. Analogically, understanding manipulation actions is like understanding language, while executing them is like generating language. Experiments on two tasks, 1) a robot observing people performing manipulation actions, and 2) a robot then executing manipulation actions accordingly, are presented to validate the formalism. The technical parts of this thesis are devoted to the experimental setting of task (1), while the task (2) is given as a live demonstration. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
Abstractor: As Provided
Entry Date: 2016
Access URL: https://gateway.proquest.com/openurl?url_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqm&rft_dat=xri:pqdiss:10011530
Accession Number: ED566619
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED566619
AccessLevel: 3
PubType: Dissertation/ Thesis
PubTypeId: dissertation
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Manipulation Action Understanding for Observation and Execution
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Yang%2C+Yezhou%22">Yang, Yezhou</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22ProQuest+LLC%22"><i>ProQuest LLC</i></searchLink>. 2015Ph.D. Dissertation, University of Maryland, College Park.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 211
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2015
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Dissertations/Theses - Doctoral Dissertations
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Observation%22">Observation</searchLink><br /><searchLink fieldCode="DE" term="%22Object+Manipulation%22">Object Manipulation</searchLink><br /><searchLink fieldCode="DE" term="%22Perceptual+Motor+Coordination%22">Perceptual Motor Coordination</searchLink><br /><searchLink fieldCode="DE" term="%22Semantics%22">Semantics</searchLink><br /><searchLink fieldCode="DE" term="%22Grammar%22">Grammar</searchLink><br /><searchLink fieldCode="DE" term="%22Robotics%22">Robotics</searchLink><br /><searchLink fieldCode="DE" term="%22Experiments%22">Experiments</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Processes%22">Cognitive Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Thinking+Skills%22">Thinking Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Perceptual+Motor+Learning%22">Perceptual Motor Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Modalities%22">Learning Modalities</searchLink><br /><searchLink fieldCode="DE" term="%22Psychomotor+Skills%22">Psychomotor Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Psychomotor+Objectives%22">Psychomotor Objectives</searchLink><br /><searchLink fieldCode="DE" term="%22Transformational+Generative+Grammar%22">Transformational Generative Grammar</searchLink><br /><searchLink fieldCode="DE" term="%22Generative+Grammar%22">Generative Grammar</searchLink><br /><searchLink fieldCode="DE" term="%22Syntax%22">Syntax</searchLink>
– Name: ISBN
  Label: ISBN
  Group: ISBN
  Data: 978-1-339-47342-0
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Modern intelligent agents will need to learn the actions that humans perform. They will need to recognize these actions when they see them and they will need to perform these actions themselves. We want to propose a cognitive system that interprets human manipulation actions from perceptual information (image and depth data) and consists of perceptual modules and reasoning modules that are in interaction with each other. The contributions of this work are given along two core problems at the heart of action understanding: a.) the grounding of relevant information about actions in perception (the perception-action integration problem), and b.) the organization of perceptual and high-level symbolic information for interpreting the actions (the sequencing problem). At the high level, actions are represented with the Manipulation Action Context-free Grammar (MACFG) , a syntactic grammar and associated parsing algorithms, which organizes actions as a sequence of sub-events. Each sub-event is described by the hand (as well as grasp type), movements (actions) and the objects and tools involved, and the relevant information about these quantities is obtained from biological-inspired perception modules. These modules track the hands and objects and recognize the hand grasp, actions, segmentation, and action consequences. Furthermore, a probabilistic semantic parsing framework based on CCG (Combinatory Categorial Grammar) theory is adopted to model the semantic meaning of human manipulation actions. Additionally, the lesson from the findings on mirror neurons is that the two processes of interpreting visually observed action and generating actions, should share the same underlying cognitive process. Recent studies have shown that grammatical structures underlie the representation of manipulation actions, which are used both to understand and to execute these actions. Analogically, understanding manipulation actions is like understanding language, while executing them is like generating language. Experiments on two tasks, 1) a robot observing people performing manipulation actions, and 2) a robot then executing manipulation actions accordingly, are presented to validate the formalism. The technical parts of this thesis are devoted to the experimental setting of task (1), while the task (2) is given as a live demonstration. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2016
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://gateway.proquest.com/openurl?url_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqm&rft_dat=xri:pqdiss:10011530" linkWindow="_blank">http://gateway.proquest.com/openurl?url_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqm&rft_dat=xri:pqdiss:10011530</link>
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED566619
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED566619
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 211
    Subjects:
      – SubjectFull: Observation
        Type: general
      – SubjectFull: Object Manipulation
        Type: general
      – SubjectFull: Perceptual Motor Coordination
        Type: general
      – SubjectFull: Semantics
        Type: general
      – SubjectFull: Grammar
        Type: general
      – SubjectFull: Robotics
        Type: general
      – SubjectFull: Experiments
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Cognitive Processes
        Type: general
      – SubjectFull: Thinking Skills
        Type: general
      – SubjectFull: Perceptual Motor Learning
        Type: general
      – SubjectFull: Learning Modalities
        Type: general
      – SubjectFull: Psychomotor Skills
        Type: general
      – SubjectFull: Psychomotor Objectives
        Type: general
      – SubjectFull: Transformational Generative Grammar
        Type: general
      – SubjectFull: Generative Grammar
        Type: general
      – SubjectFull: Syntax
        Type: general
    Titles:
      – TitleFull: Manipulation Action Understanding for Observation and Execution
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yang, Yezhou
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2015
          Identifiers:
            – Type: isbn-print
              Value: 978-1-339-47342-0
          Titles:
            – TitleFull: ProQuest LLC
              Type: main
ResultId 1