View in EDS

Enhanced Attention Tracking With Multi-Branch Network for Egocentric Activity Recognition.

Saved in:
Bibliographic Details
Title: Enhanced Attention Tracking With Multi-Branch Network for Egocentric Activity Recognition.
Authors: Liu, Tianshan1 tianshan.liu@connect.polyu.hk, Lam, Kin-Man1 enkmlam@polyu.edu.hk, Zhao, Rui1 rick10.zhao@connect.polyu.hk, Kong, Jun2 kongjun@jiangnan.edu.cn
Source: IEEE Transactions on Circuits & Systems for Video Technology. Jun2022, Vol. 32 Issue 6, p3587-3602. 16p.
Subjects: Machine learning, Feature extraction
Abstract: The emergence of wearable devices has opened up new potentials for egocentric activity recognition. Although some methods integrate attention mechanisms into deep neural networks to capture fine-grained human-object interactions in a weak-supervision manner, they either ignore exploiting the temporal consistency or generate attention based on considering appearance cues only. To address these limitations, in this paper, we propose an enhanced attention-tracking method, combined with multi-branch network (EAT-MBNet), for egocentric activity recognition. Specifically, we propose class-aware attention maps (CAAMs) by employing a self-attention-based module to refine the class activation maps (CAMs). Our proposed method can enhance the semantic dependency between the activity categories and the feature maps. To highlight the discriminative features from the regions of interest across frames, we propose a flow-guided attention-tracking (F-AT) module, by simultaneously leveraging historical attention and motion patterns. Furthermore, we propose a cross-modality modeling branch based on an interactive GRU module, which captures the time-synchronized long-term relationships between the appearance and motion branches. Experimental results on four egocentric activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Circuits & Systems for Video Technology is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 157258444
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Enhanced Attention Tracking With Multi-Branch Network for Egocentric Activity Recognition.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Liu%2C+Tianshan%22">Liu, Tianshan</searchLink><relatesTo>1</relatesTo><i> tianshan.liu@connect.polyu.hk</i><br /><searchLink fieldCode="AR" term="%22Lam%2C+Kin-Man%22">Lam, Kin-Man</searchLink><relatesTo>1</relatesTo><i> enkmlam@polyu.edu.hk</i><br /><searchLink fieldCode="AR" term="%22Zhao%2C+Rui%22">Zhao, Rui</searchLink><relatesTo>1</relatesTo><i> rick10.zhao@connect.polyu.hk</i><br /><searchLink fieldCode="AR" term="%22Kong%2C+Jun%22">Kong, Jun</searchLink><relatesTo>2</relatesTo><i> kongjun@jiangnan.edu.cn</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22IEEE+Transactions+on+Circuits+%26+Systems+for+Video+Technology%22">IEEE Transactions on Circuits & Systems for Video Technology</searchLink>. Jun2022, Vol. 32 Issue 6, p3587-3602. 16p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Feature+extraction%22">Feature extraction</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The emergence of wearable devices has opened up new potentials for egocentric activity recognition. Although some methods integrate attention mechanisms into deep neural networks to capture fine-grained human-object interactions in a weak-supervision manner, they either ignore exploiting the temporal consistency or generate attention based on considering appearance cues only. To address these limitations, in this paper, we propose an enhanced attention-tracking method, combined with multi-branch network (EAT-MBNet), for egocentric activity recognition. Specifically, we propose class-aware attention maps (CAAMs) by employing a self-attention-based module to refine the class activation maps (CAMs). Our proposed method can enhance the semantic dependency between the activity categories and the feature maps. To highlight the discriminative features from the regions of interest across frames, we propose a flow-guided attention-tracking (F-AT) module, by simultaneously leveraging historical attention and motion patterns. Furthermore, we propose a cross-modality modeling branch based on an interactive GRU module, which captures the time-synchronized long-term relationships between the appearance and motion branches. Experimental results on four egocentric activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of IEEE Transactions on Circuits & Systems for Video Technology is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=157258444
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/TCSVT.2021.3104651
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 3587
    Subjects:
      – SubjectFull: Machine learning
        Type: general
      – SubjectFull: Feature extraction
        Type: general
    Titles:
      – TitleFull: Enhanced Attention Tracking With Multi-Branch Network for Egocentric Activity Recognition.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Liu, Tianshan
      – PersonEntity:
          Name:
            NameFull: Lam, Kin-Man
      – PersonEntity:
          Name:
            NameFull: Zhao, Rui
      – PersonEntity:
          Name:
            NameFull: Kong, Jun
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Text: Jun2022
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 10518215
          Numbering:
            – Type: volume
              Value: 32
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: IEEE Transactions on Circuits & Systems for Video Technology
              Type: main
ResultId 1