Wright, S. A. orcid.org/0000-0001-7133-8533, Hammond, S. D., Pennycook, S. J. et al. (3 more authors) (2012) LDPLFS:Improving I/O performance without application modification. In: Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012. 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012, 21-25 May 2012 , CHN , pp. 1352-1359.
Abstract
Input/Output (I/O) operations can represent a significant proportion of run-time when large scientific applications are run in parallel and at scale. In order to address the growing divergence between processing speeds and I/O performance, the Parallel Log-structured File System (PLFS) has been developed by EMC Corporation and the Los Alamos National Laboratory (LANL) to improve the performance of parallel file activities. Currently, PLFS requires the use of either (i) the FUSE Linux Kernel module, (ii) a modified MPI library with a customised ROMIO MPI-IO library, or (iii) an application rewrite to utilise the PLFS API directly. In this paper we present an alternative method of utilising PLFS in applications. This method employs a dynamic library to intercept the low-level POSIX operations and retarget them to use the equivalents offered by PLFS. We demonstrate our implementation of this approach, named LDPLFS, on a set of standard UNIX tools, as well on as a set of standard parallel I/O intensive mini-applications. The results demonstrate almost equivalent performance to a modified build of ROMIO and improvements over the FUSE-based approach. Furthermore, through our experiments we demonstrate decreased performance in PLFS when ran at scale on the Lustre file system.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © IEEE, 2012. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details |
Keywords: | Data Storage Systems,File Systems,High Performance Computing,I/O |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 04 Oct 2018 09:20 |
Last Modified: | 06 Jan 2025 00:23 |
Published Version: | https://doi.org/10.1109/IPDPSW.2012.172 |
Status: | Published |
Identification Number: | 10.1109/IPDPSW.2012.172 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:136615 |
Download
Filename: LDPLFS_improving_performance_without_application_modification_Wright_2012.pdf
Description: LDPLFS-improving--performance-without-application-modification-Wright-2012