Maximizing Communication-Computation Overlap Through Automatic Parallelization and Run-time Tuning of Non-blocking Collective Operations.
Saved in:
| Title: | Maximizing Communication-Computation Overlap Through Automatic Parallelization and Run-time Tuning of Non-blocking Collective Operations. |
|---|---|
| Authors: | Barigou, Youcef1 youcef.barigou@gmail.com, Gabriel, Edgar1 gabriel@central.uh.edu |
| Source: | International Journal of Parallel Programming. Dec2017, Vol. 45 Issue 6, p1390-1416. 27p. |
| Subjects: | Computer systems, Loop tiling (Computer science), Frequency tuning, Communication, Application program interfaces |
| Abstract: | Non-blocking collective communication operations extend the concept of collective operations by offering the additional benefit of being able to overlap communication and computation. They are often considered key building blocks for scaling applications to very large process counts. Yet, using non-blocking collective operations in real-world applications is non-trivial. Application codes often have to be restructured significantly in order to maximize the communication-computation overlap. This paper presents an approach to maximize the communication-computation overlap for hybrid OpenMP/MPI applications. The work leverages automatic parallelization by extending the ability of an existing tool to utilize non-blocking collective operations. It further integrates run-time auto-tuning techniques of non-blocking collective operations, optimizing both, the algorithms used for the non-blocking collective operations as well as location and frequency of accompanying progress function calls. Four application benchmarks were used to demonstrate the efficiency and versatility of the approach on two different platforms. The results indicate significant performance improvements in virtually all test scenarios. The resulting parallel applications achieved a performance improvement of up to 43% compared to the version using blocking communication operations, and up to 95% of the maximum theoretical communication-computation overlap identified for each scenario. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Parallel Programming is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 125257205 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Maximizing Communication-Computation Overlap Through Automatic Parallelization and Run-time Tuning of Non-blocking Collective Operations. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Barigou%2C+Youcef%22">Barigou, Youcef</searchLink><relatesTo>1</relatesTo><i> youcef.barigou@gmail.com</i><br /><searchLink fieldCode="AR" term="%22Gabriel%2C+Edgar%22">Gabriel, Edgar</searchLink><relatesTo>1</relatesTo><i> gabriel@central.uh.edu</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Parallel+Programming%22">International Journal of Parallel Programming</searchLink>. Dec2017, Vol. 45 Issue 6, p1390-1416. 27p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+systems%22">Computer systems</searchLink><br /><searchLink fieldCode="DE" term="%22Loop+tiling+%28Computer+science%29%22">Loop tiling (Computer science)</searchLink><br /><searchLink fieldCode="DE" term="%22Frequency+tuning%22">Frequency tuning</searchLink><br /><searchLink fieldCode="DE" term="%22Communication%22">Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Application+program+interfaces%22">Application program interfaces</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Non-blocking collective communication operations extend the concept of collective operations by offering the additional benefit of being able to overlap communication and computation. They are often considered key building blocks for scaling applications to very large process counts. Yet, using non-blocking collective operations in real-world applications is non-trivial. Application codes often have to be restructured significantly in order to maximize the communication-computation overlap. This paper presents an approach to maximize the communication-computation overlap for hybrid OpenMP/MPI applications. The work leverages automatic parallelization by extending the ability of an existing tool to utilize non-blocking collective operations. It further integrates run-time auto-tuning techniques of non-blocking collective operations, optimizing both, the algorithms used for the non-blocking collective operations as well as location and frequency of accompanying progress function calls. Four application benchmarks were used to demonstrate the efficiency and versatility of the approach on two different platforms. The results indicate significant performance improvements in virtually all test scenarios. The resulting parallel applications achieved a performance improvement of up to 43% compared to the version using blocking communication operations, and up to 95% of the maximum theoretical communication-computation overlap identified for each scenario. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of International Journal of Parallel Programming is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=125257205 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10766-016-0477-7 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 27 StartPage: 1390 Subjects: – SubjectFull: Computer systems Type: general – SubjectFull: Loop tiling (Computer science) Type: general – SubjectFull: Frequency tuning Type: general – SubjectFull: Communication Type: general – SubjectFull: Application program interfaces Type: general Titles: – TitleFull: Maximizing Communication-Computation Overlap Through Automatic Parallelization and Run-time Tuning of Non-blocking Collective Operations. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Barigou, Youcef – PersonEntity: Name: NameFull: Gabriel, Edgar IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 12 Text: Dec2017 Type: published Y: 2017 Identifiers: – Type: issn-print Value: 08857458 Numbering: – Type: volume Value: 45 – Type: issue Value: 6 Titles: – TitleFull: International Journal of Parallel Programming Type: main |
| ResultId | 1 |
