View in EDS

The Evolution of Global Scale Filesystems for Scientific Software Distribution.

Saved in:
Bibliographic Details
Title: The Evolution of Global Scale Filesystems for Scientific Software Distribution.
Authors: Blomer, Jakob1, Buncic, Predrag1, Meusel, Rene1, Ganis, Gerardo1, Sfiligoi, Igor2, Thain, Douglas3
Source: Computing in Science & Engineering. Nov2015, Vol. 17 Issue 6, p61-71. 11p.
Subjects: Scientific software, System analysis software, Scientific computing, Information storage & retrieval systems, Computer systems
Abstract: Delivering complex software across a worldwide distributed system is a major challenge in high-throughput scientific computing. The problem arises at different scales for many scientific communities that use grids, clouds, and distributed clusters to satisfy their computing needs. For high-energy physics (HEP) collaborations dealing with large amounts of data that rely on hundreds of thousands of cores spread around the world for data processing, the challenge is particularly acute. To serve the needs of the HEP community, several iterations were made to create a scalable, user-level filesystem that delivers software worldwide on a daily basis. The implementation was designed in 2006 to serve the needs of one experiment running on thousands of machines. Since that time, this idea evolved into a new production global-scale filesystem serving the needs of multiple science communities on hundreds of thousands of machines around the world. [ABSTRACT FROM PUBLISHER]
Copyright of Computing in Science & Engineering is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Delivering complex software across a worldwide distributed system is a major challenge in high-throughput scientific computing. The problem arises at different scales for many scientific communities that use grids, clouds, and distributed clusters to satisfy their computing needs. For high-energy physics (HEP) collaborations dealing with large amounts of data that rely on hundreds of thousands of cores spread around the world for data processing, the challenge is particularly acute. To serve the needs of the HEP community, several iterations were made to create a scalable, user-level filesystem that delivers software worldwide on a daily basis. The implementation was designed in 2006 to serve the needs of one experiment running on thousands of machines. Since that time, this idea evolved into a new production global-scale filesystem serving the needs of multiple science communities on hundreds of thousands of machines around the world. [ABSTRACT FROM PUBLISHER]
ISSN:15219615
DOI:10.1109/MCSE.2015.111