View in EDS

Scalable Design and Implementations for MPI Parallel Overlapping I/O.

Saved in:
Bibliographic Details
Title: Scalable Design and Implementations for MPI Parallel Overlapping I/O.
Authors: Wei-keng Liao1 wkliao@ece.northwestern.edu, Coloma, Kenin1 kcoloma@ece.northwestern.edu, Choudhary, Alok1 choudherj@ece.northwestern.edu, Ward, Lee2 lee@sandia.gov, Russell, Eric2 edrusse@sandia.gov, Pundit, Neil2 pundit@sandia.gov
Source: IEEE Transactions on Parallel & Distributed Systems. Nov2006, Vol. 17 Issue 11, p1264-1276. 13p. 5 Black and White Photographs, 10 Diagrams, 1 Chart, 2 Graphs.
Subjects: Cache memory, POSIX (Computer software standard), Interface circuits, Computer storage devices, Computer operating systems, Computer science
Abstract: We investigate the Message Passing Interface Input/Output (MPI I/O) implementation issues for two overlapping access patterns: the overlaps among processes within a single I/O operation and the overlaps across a sequence of I/O operations. The former case considers whether I/O atomicity can be obtained in the overlapping regions. The latter focuses on the file consistency problem on parallel machines with client-side file caching enabled. Traditional solutions for both overlapping I/O problems use whole file or byte-range file locking to ensure exclusive access to the overlapping regions and bypass the file system cache. Unfortunately, not only can file locking serialize I/O, but it can also increase the aggregate communication overhead between clients and I/O servers. For atomicity, we first differentiate MPI's requirements from the Portable Operating System Interface (POSIX) standard and propose two scalable approaches, graph coloring and process-rank ordering, which can resolve access conflicts and maintain I/O parallelism. For solving the file consistency problem across multiple I/O operations, we propose a method called Persistent File Domains, which tackles cache coherency with additional information and coordination to guarantee safe cache access without using file locks. [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Parallel & Distributed Systems is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:We investigate the Message Passing Interface Input/Output (MPI I/O) implementation issues for two overlapping access patterns: the overlaps among processes within a single I/O operation and the overlaps across a sequence of I/O operations. The former case considers whether I/O atomicity can be obtained in the overlapping regions. The latter focuses on the file consistency problem on parallel machines with client-side file caching enabled. Traditional solutions for both overlapping I/O problems use whole file or byte-range file locking to ensure exclusive access to the overlapping regions and bypass the file system cache. Unfortunately, not only can file locking serialize I/O, but it can also increase the aggregate communication overhead between clients and I/O servers. For atomicity, we first differentiate MPI's requirements from the Portable Operating System Interface (POSIX) standard and propose two scalable approaches, graph coloring and process-rank ordering, which can resolve access conflicts and maintain I/O parallelism. For solving the file consistency problem across multiple I/O operations, we propose a method called Persistent File Domains, which tackles cache coherency with additional information and coordination to guarantee safe cache access without using file locks. [ABSTRACT FROM AUTHOR]
ISSN:10459219
DOI:10.1109/TPDS.2006.163