View in EDS

Generalisation of recursive doubling for AllReduce: Now with simulation.

Saved in:
Bibliographic Details
Title: Generalisation of recursive doubling for AllReduce: Now with simulation.
Authors: Ruefenacht, Martin1 m.ruefenacht@ed.ac.uk, Bull, Mark1 m.bull@ed.ac.uk, Booth, Stephen1 s.booth@ed.ac.uk
Source: Parallel Computing. Nov2017, Vol. 69, p24-44. 21p.
Subjects: Computer simulation, Cray computers, Data pipelining, Computer software execution, Recursive functions
Abstract: The performance of AllReduce is crucial at scale. The recursive doubling with pairwise exchange algorithm theoretically achieves O (log 2   N ) scaling for short messages with N peers, but is limited by improvements in network latency. A multi-way exchange can be implemented using message pipelining, which is easier to improve than latency. Using our method, recursive multiplying, we show reductions in execution time of between 8% and 40% of AllReduce on a Cray XC30 over recursive doubling. Using a custom simulator we further explore the dynamics of recursive multiplying. [ABSTRACT FROM AUTHOR]
Copyright of Parallel Computing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 125858998
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Generalisation of recursive doubling for AllReduce: Now with simulation.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ruefenacht%2C+Martin%22">Ruefenacht, Martin</searchLink><relatesTo>1</relatesTo><i> m.ruefenacht@ed.ac.uk</i><br /><searchLink fieldCode="AR" term="%22Bull%2C+Mark%22">Bull, Mark</searchLink><relatesTo>1</relatesTo><i> m.bull@ed.ac.uk</i><br /><searchLink fieldCode="AR" term="%22Booth%2C+Stephen%22">Booth, Stephen</searchLink><relatesTo>1</relatesTo><i> s.booth@ed.ac.uk</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Parallel+Computing%22">Parallel Computing</searchLink>. Nov2017, Vol. 69, p24-44. 21p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+simulation%22">Computer simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Cray+computers%22">Cray computers</searchLink><br /><searchLink fieldCode="DE" term="%22Data+pipelining%22">Data pipelining</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+software+execution%22">Computer software execution</searchLink><br /><searchLink fieldCode="DE" term="%22Recursive+functions%22">Recursive functions</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The performance of AllReduce is crucial at scale. The recursive doubling with pairwise exchange algorithm theoretically achieves O (log 2   N ) scaling for short messages with N peers, but is limited by improvements in network latency. A multi-way exchange can be implemented using message pipelining, which is easier to improve than latency. Using our method, recursive multiplying, we show reductions in execution time of between 8% and 40% of AllReduce on a Cray XC30 over recursive doubling. Using a custom simulator we further explore the dynamics of recursive multiplying. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Parallel Computing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=125858998
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.parco.2017.08.004
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 21
        StartPage: 24
    Subjects:
      – SubjectFull: Computer simulation
        Type: general
      – SubjectFull: Cray computers
        Type: general
      – SubjectFull: Data pipelining
        Type: general
      – SubjectFull: Computer software execution
        Type: general
      – SubjectFull: Recursive functions
        Type: general
    Titles:
      – TitleFull: Generalisation of recursive doubling for AllReduce: Now with simulation.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ruefenacht, Martin
      – PersonEntity:
          Name:
            NameFull: Bull, Mark
      – PersonEntity:
          Name:
            NameFull: Booth, Stephen
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Text: Nov2017
              Type: published
              Y: 2017
          Identifiers:
            – Type: issn-print
              Value: 01678191
          Numbering:
            – Type: volume
              Value: 69
          Titles:
            – TitleFull: Parallel Computing
              Type: main
ResultId 1