View in EDS

vTensor-Based GPU Memory Management for Edge Deep Learning Training.

Saved in:
Bibliographic Details
Title: vTensor-Based GPU Memory Management for Edge Deep Learning Training.
Authors: Gao, He-Ran1,2 (AUTHOR) gaoheran19@otcaix.iscas.ac.cn, Luo, Diao-Han1,2 (AUTHOR) luodiaohan21@otcaix.iscas.ac.cn, Wu, Yue-Wen1 (AUTHOR) wuyuewen11@otcaix.iscas.ac.cn, Wu, Heng1,3,4 (AUTHOR) wuheng@iscas.ac.cn, Zhang, Wen-Bo1,3,4 (AUTHOR) zhangwenbo@otcaix.iscas.ac.cn
Source: Journal of Computer Science & Technology (10009000). Nov2025, Vol. 40 Issue 6, p1608-1625. 18p.
Subjects: Edge computing, Computer memory management, Virtual storage (Computer science), Resource allocation
Abstract: Supporting real-time and privacy-preserving learning at the edge is emerging as a critical trend, bringing forth substantial challenges for deep learning (DL) training in the context of limited GPU (graphic processing unit) memory. Recent work has sought to address the limitations by swapping tensors between GPU memory and CPU memory. Unfortunately, their tensor-based memory management encounters additional overhead since the swapped tensors do not align with the actual memory demands, resulting in decreased throughput. This paper introduces a vTensor-based memory management approach designed to mitigate memory swapping overhead. Virtualized tensors, dubbed vTensors, are used to finely align memory swapping amounts with real-time memory demands. Firstly, we introduce an abstraction layer that virtualizes coarse-grained tensors to multiple finer-grained vTensors. Secondly, we propose the Layered Graph Model (LGM) for analyzing vTensor mappings, which produces a memory swapping plan leveraged in the subsequent DL training iterations. Evaluations conducted on typical edge deep learning models illustrate that our approach surpasses prior work with a 15.60% increase in DL training throughput. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Computer Science & Technology (10009000) is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 190855173
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: vTensor-Based GPU Memory Management for Edge Deep Learning Training.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Gao%2C+He-Ran%22">Gao, He-Ran</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> gaoheran19@otcaix.iscas.ac.cn</i><br /><searchLink fieldCode="AR" term="%22Luo%2C+Diao-Han%22">Luo, Diao-Han</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> luodiaohan21@otcaix.iscas.ac.cn</i><br /><searchLink fieldCode="AR" term="%22Wu%2C+Yue-Wen%22">Wu, Yue-Wen</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> wuyuewen11@otcaix.iscas.ac.cn</i><br /><searchLink fieldCode="AR" term="%22Wu%2C+Heng%22">Wu, Heng</searchLink><relatesTo>1,3,4</relatesTo> (AUTHOR)<i> wuheng@iscas.ac.cn</i><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Wen-Bo%22">Zhang, Wen-Bo</searchLink><relatesTo>1,3,4</relatesTo> (AUTHOR)<i> zhangwenbo@otcaix.iscas.ac.cn</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Computer+Science+%26+Technology+%2810009000%29%22">Journal of Computer Science & Technology (10009000)</searchLink>. Nov2025, Vol. 40 Issue 6, p1608-1625. 18p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Edge+computing%22">Edge computing</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+memory+management%22">Computer memory management</searchLink><br /><searchLink fieldCode="DE" term="%22Virtual+storage+%28Computer+science%29%22">Virtual storage (Computer science)</searchLink><br /><searchLink fieldCode="DE" term="%22Resource+allocation%22">Resource allocation</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Supporting real-time and privacy-preserving learning at the edge is emerging as a critical trend, bringing forth substantial challenges for deep learning (DL) training in the context of limited GPU (graphic processing unit) memory. Recent work has sought to address the limitations by swapping tensors between GPU memory and CPU memory. Unfortunately, their tensor-based memory management encounters additional overhead since the swapped tensors do not align with the actual memory demands, resulting in decreased throughput. This paper introduces a vTensor-based memory management approach designed to mitigate memory swapping overhead. Virtualized tensors, dubbed vTensors, are used to finely align memory swapping amounts with real-time memory demands. Firstly, we introduce an abstraction layer that virtualizes coarse-grained tensors to multiple finer-grained vTensors. Secondly, we propose the Layered Graph Model (LGM) for analyzing vTensor mappings, which produces a memory swapping plan leveraged in the subsequent DL training iterations. Evaluations conducted on typical edge deep learning models illustrate that our approach surpasses prior work with a 15.60% increase in DL training throughput. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Computer Science & Technology (10009000) is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=190855173
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11390-025-4788-2
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 1608
    Subjects:
      – SubjectFull: Edge computing
        Type: general
      – SubjectFull: Computer memory management
        Type: general
      – SubjectFull: Virtual storage (Computer science)
        Type: general
      – SubjectFull: Resource allocation
        Type: general
    Titles:
      – TitleFull: vTensor-Based GPU Memory Management for Edge Deep Learning Training.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Gao, He-Ran
      – PersonEntity:
          Name:
            NameFull: Luo, Diao-Han
      – PersonEntity:
          Name:
            NameFull: Wu, Yue-Wen
      – PersonEntity:
          Name:
            NameFull: Wu, Heng
      – PersonEntity:
          Name:
            NameFull: Zhang, Wen-Bo
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Text: Nov2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 10009000
          Numbering:
            – Type: volume
              Value: 40
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Journal of Computer Science & Technology (10009000)
              Type: main
ResultId 1