Qiu, S, You, L and Wang, Z orcid.org/0000-0001-6157-0662 (2022) Optimizing Sparse Matrix Multiplications for Graph Neural Networks. In: Li, X and Chandrasekaran, S, (eds.) Languages and Compilers for Parallel Computing. 34th International Workshop, LCPC 2021, 13-14 Oct 2021, Virtual. Lecture Notes in Computer Science, 1318 . Springer , pp. 101-117. ISBN 978-3-030-99371-9
Abstract
Graph neural networks (GNNs) are emerging as a powerful technique for modeling graph structures. Due to the sparsity of real-world graph data, GNN performance is limited by extensive sparse matrix multiplication (SpMM) operations involved in computation. While the right sparse matrix storage format varies across input data, existing deep learning frameworks employ a single, static storage format, leaving much room for improvement. This paper investigates how the choice of sparse matrix storage formats affect the GNN performance. We observe that choosing a suitable sparse matrix storage format can significantly improve the GNN training performance, but the right format depends on the input workloads and can change as the GNN iterates over the input graph. We then develop a predictive model to dynamically choose a sparse matrix storage format to be used by a GNN layer based on the input matrices. Our model is first trained offline using training matrix samples, and the trained model can be applied to any input matrix and GNN kernels with SpMM computation. We implement our approach on top of PyTorch and apply it to 5 representative GNN models running on a multi-core CPU using real-life and synthetic datasets. Experimental results show that our approach gives an average speedup of 1.17x (up to 3x) for GNN running time.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2022 Springer Nature Switzerland AG. This is an author produced version of a conference paper published in Languages and Compilers for Parallel Computing. Uploaded in accordance with the publisher's self-archiving policy. |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Funding Information: | Funder Grant number Alibaba.com Singapore E-Commerce Private Not Known |
Depositing User: | Symplectic Publications |
Date Deposited: | 01 Oct 2021 08:27 |
Last Modified: | 18 Oct 2023 03:41 |
Status: | Published |
Publisher: | Springer |
Series Name: | Lecture Notes in Computer Science |
Identification Number: | 10.1007/978-3-030-99372-6_7 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:178630 |