View in EDS

vTensor-Based GPU Memory Management for Edge Deep Learning Training.

Saved in:
Bibliographic Details
Title: vTensor-Based GPU Memory Management for Edge Deep Learning Training.
Authors: Gao, He-Ran1,2 (AUTHOR) gaoheran19@otcaix.iscas.ac.cn, Luo, Diao-Han1,2 (AUTHOR) luodiaohan21@otcaix.iscas.ac.cn, Wu, Yue-Wen1 (AUTHOR) wuyuewen11@otcaix.iscas.ac.cn, Wu, Heng1,3,4 (AUTHOR) wuheng@iscas.ac.cn, Zhang, Wen-Bo1,3,4 (AUTHOR) zhangwenbo@otcaix.iscas.ac.cn
Source: Journal of Computer Science & Technology (10009000). Nov2025, Vol. 40 Issue 6, p1608-1625. 18p.
Subjects: Edge computing, Computer memory management, Virtual storage (Computer science), Resource allocation
Abstract: Supporting real-time and privacy-preserving learning at the edge is emerging as a critical trend, bringing forth substantial challenges for deep learning (DL) training in the context of limited GPU (graphic processing unit) memory. Recent work has sought to address the limitations by swapping tensors between GPU memory and CPU memory. Unfortunately, their tensor-based memory management encounters additional overhead since the swapped tensors do not align with the actual memory demands, resulting in decreased throughput. This paper introduces a vTensor-based memory management approach designed to mitigate memory swapping overhead. Virtualized tensors, dubbed vTensors, are used to finely align memory swapping amounts with real-time memory demands. Firstly, we introduce an abstraction layer that virtualizes coarse-grained tensors to multiple finer-grained vTensors. Secondly, we propose the Layered Graph Model (LGM) for analyzing vTensor mappings, which produces a memory swapping plan leveraged in the subsequent DL training iterations. Evaluations conducted on typical edge deep learning models illustrate that our approach surpasses prior work with a 15.60% increase in DL training throughput. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Computer Science & Technology (10009000) is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Supporting real-time and privacy-preserving learning at the edge is emerging as a critical trend, bringing forth substantial challenges for deep learning (DL) training in the context of limited GPU (graphic processing unit) memory. Recent work has sought to address the limitations by swapping tensors between GPU memory and CPU memory. Unfortunately, their tensor-based memory management encounters additional overhead since the swapped tensors do not align with the actual memory demands, resulting in decreased throughput. This paper introduces a vTensor-based memory management approach designed to mitigate memory swapping overhead. Virtualized tensors, dubbed vTensors, are used to finely align memory swapping amounts with real-time memory demands. Firstly, we introduce an abstraction layer that virtualizes coarse-grained tensors to multiple finer-grained vTensors. Secondly, we propose the Layered Graph Model (LGM) for analyzing vTensor mappings, which produces a memory swapping plan leveraged in the subsequent DL training iterations. Evaluations conducted on typical edge deep learning models illustrate that our approach surpasses prior work with a 15.60% increase in DL training throughput. [ABSTRACT FROM AUTHOR]
ISSN:10009000
DOI:10.1007/s11390-025-4788-2