View in EDS

Two-handed gesture recognition and fusion with speech to command a robot.

Saved in:
Bibliographic Details
Title: Two-handed gesture recognition and fusion with speech to command a robot.
Authors: Burger, B. bburger@laas.fr, Ferrané, I. ferrane@irit.fr, Lerasle, F. lerasle@laas.fr, Infantes, G.1 guillaume.infantes@onera.fr
Source: Autonomous Robots. Feb2012, Vol. 32 Issue 2, p129-147. 19p.
Subjects: Autonomous robots, Fusion (Phase transformation), Speech perception, Algorithms, Motion
Abstract: Assistance is currently a pivotal research area in robotics, with huge societal potential. Since assistant robots directly interact with people, finding natural and easy-to-use user interfaces is of fundamental importance. This paper describes a flexible multimodal interface based on speech and gesture modalities in order to control our mobile robot named Jido. The vision system uses a stereo head mounted on a pan-tilt unit and a bank of collaborative particle filters devoted to the upper human body extremities to track and recognize pointing/symbolic mono but also bi-manual gestures. Such framework constitutes our first contribution, as it is shown, to give proper handling of natural artifacts (self-occlusion, camera out of view field, hand deformation) when performing 3D gestures using one or the other hand even both. A speech recognition and understanding system based on the Julius engine is also developed and embedded in order to process deictic and anaphoric utterances. The second contribution deals with a probabilistic and multi-hypothesis interpreter framework to fuse results from speech and gesture components. Such interpreter is shown to improve the classification rates of multimodal commands compared to using either modality alone. Finally, we report on successful live experiments in human-centered settings. Results are reported in the context of an interactive manipulation task, where users specify local motion commands to Jido and perform safe object exchanges. [ABSTRACT FROM AUTHOR]
Copyright of Autonomous Robots is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 70532272
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Two-handed gesture recognition and fusion with speech to command a robot.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Burger%2C+B%2E%22">Burger, B.</searchLink><i> bburger@laas.fr</i><br /><searchLink fieldCode="AR" term="%22Ferrané%2C+I%2E%22">Ferrané, I.</searchLink><i> ferrane@irit.fr</i><br /><searchLink fieldCode="AR" term="%22Lerasle%2C+F%2E%22">Lerasle, F.</searchLink><i> lerasle@laas.fr</i><br /><searchLink fieldCode="AR" term="%22Infantes%2C+G%2E%22">Infantes, G.</searchLink><relatesTo>1</relatesTo><i> guillaume.infantes@onera.fr</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Autonomous+Robots%22">Autonomous Robots</searchLink>. Feb2012, Vol. 32 Issue 2, p129-147. 19p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Autonomous+robots%22">Autonomous robots</searchLink><br /><searchLink fieldCode="DE" term="%22Fusion+%28Phase+transformation%29%22">Fusion (Phase transformation)</searchLink><br /><searchLink fieldCode="DE" term="%22Speech+perception%22">Speech perception</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Motion%22">Motion</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Assistance is currently a pivotal research area in robotics, with huge societal potential. Since assistant robots directly interact with people, finding natural and easy-to-use user interfaces is of fundamental importance. This paper describes a flexible multimodal interface based on speech and gesture modalities in order to control our mobile robot named Jido. The vision system uses a stereo head mounted on a pan-tilt unit and a bank of collaborative particle filters devoted to the upper human body extremities to track and recognize pointing/symbolic mono but also bi-manual gestures. Such framework constitutes our first contribution, as it is shown, to give proper handling of natural artifacts (self-occlusion, camera out of view field, hand deformation) when performing 3D gestures using one or the other hand even both. A speech recognition and understanding system based on the Julius engine is also developed and embedded in order to process deictic and anaphoric utterances. The second contribution deals with a probabilistic and multi-hypothesis interpreter framework to fuse results from speech and gesture components. Such interpreter is shown to improve the classification rates of multimodal commands compared to using either modality alone. Finally, we report on successful live experiments in human-centered settings. Results are reported in the context of an interactive manipulation task, where users specify local motion commands to Jido and perform safe object exchanges. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Autonomous Robots is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=70532272
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10514-011-9263-y
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 19
        StartPage: 129
    Subjects:
      – SubjectFull: Autonomous robots
        Type: general
      – SubjectFull: Fusion (Phase transformation)
        Type: general
      – SubjectFull: Speech perception
        Type: general
      – SubjectFull: Algorithms
        Type: general
      – SubjectFull: Motion
        Type: general
    Titles:
      – TitleFull: Two-handed gesture recognition and fusion with speech to command a robot.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Burger, B.
      – PersonEntity:
          Name:
            NameFull: Ferrané, I.
      – PersonEntity:
          Name:
            NameFull: Lerasle, F.
      – PersonEntity:
          Name:
            NameFull: Infantes, G.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Text: Feb2012
              Type: published
              Y: 2012
          Identifiers:
            – Type: issn-print
              Value: 09295593
          Numbering:
            – Type: volume
              Value: 32
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Autonomous Robots
              Type: main
ResultId 1