View in EDS PDF Full Text

面向带宽受限型DSP的高效大语言模型推理方法.

Saved in:
Bibliographic Details
Title: 面向带宽受限型DSP的高效大语言模型推理方法.
Alternate Title: An efficient large language model inference method for bandwidth-constrained digital signal processors.
Authors: 陈 阳1,2 yang_chen@nudt.edu.cn, 杨 希1,2, 苏华友1,2, 陈抗抗1,2
Source: Computer Engineering & Science / Jisuanji Gongcheng yu Kexue. Apr2026, Vol. 48 Issue 4, p599-607. 9p.
Subjects: Digital signal processing, Bandwidths, Computer performance, Language models, Matrix multiplications, Mathematical optimization
Abstract (English): With the rise of large language models (LLMs), the parameter scale of neural network models has grown exponentially, reaching the order of hundreds of billions or even trillions, posing immense challenges to the computing power and bandwidth of computational devices for model inference tasks. To achieve high-performance LLMs inference on low-bandwidth devices, this study focuses on bandwidth-constrained, long-vector digital signal processor (DSP) architectures, designing and implementing efficient LLMs inference methods. It proposes a tensor shape-aware low-precision matrix multiplication method that fully leverages the DSP's computational capabilities while reducing memory access pressure. Additionally, it introduces a data dependency-based operator fusion method to minimize the transmission of intermediate temporary data and employs a deferred operator execution method to enhance the core execution efficiency of DSP devices. Experimental results demonstrate that this approach effectively improves the inference per-formance of large models on bandwidth-constrained DSP devices. Compared to conventional implementations, the optimized inference method achieves a speedup ranging from 1.4 to 2.3 times. Furthermore, when compared to multi-core ARM CPUs and Intel􀆿 Xeon􀆿 GoldCPUs with higher memory bandwidth, the LLMs inference performance achieves speedups of 2.5 times and 1.5 times, respectively, under the same number of cores. [ABSTRACT FROM AUTHOR]
Abstract (Chinese): 随着大语言模型的兴起,神经网络模型的参数规模呈指数级增长并达到千/万亿量级,模型的 推理任务对计算设备的算力和带宽提出了巨大挑战。为实现低带宽设备上的高性能LLMs推理,针对带 宽受限、长向量数字信号处理器体系结构,设计并实现高效的LLMs推理方法,提出基于张量形状感知的 低精度矩阵乘方法,充分利用DSP的计算能力和降低访存压力的能力;提出基于数据依赖关系的算子融 合方法减少中间临时数据的传输;使用延迟算子执行方法提升DSP设备内核执行效率。实验表明,该方 法能够有效提升大模型在带宽受限DSP设备上的推理性能,优化后的推理方法相较于普通实现能够实现 1.4~2.3倍的加速比;相较于内存带宽更高的多核ARM CPU 以及Intel􀆿 Xeon􀆿 Gold CPU,同等核心数 量下LLMs推理性能的加速比分别达到2.5倍和1.2倍以上。. [ABSTRACT FROM AUTHOR]
Copyright of Computer Engineering & Science / Jisuanji Gongcheng yu Kexue is the property of Computer Engineering & Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 193831502
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: 面向带宽受限型DSP的高效大语言模型推理方法.
– Name: TitleAlt
  Label: Alternate Title
  Group: TiAlt
  Data: An efficient large language model inference method for bandwidth-constrained digital signal processors.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22陈+阳%22">陈 阳</searchLink><relatesTo>1,2</relatesTo><i> yang_chen@nudt.edu.cn</i><br /><searchLink fieldCode="AR" term="%22杨+希%22">杨 希</searchLink><relatesTo>1,2</relatesTo><br /><searchLink fieldCode="AR" term="%22苏华友%22">苏华友</searchLink><relatesTo>1,2</relatesTo><br /><searchLink fieldCode="AR" term="%22陈抗抗%22">陈抗抗</searchLink><relatesTo>1,2</relatesTo>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Computer+Engineering+%26+Science+%2F+Jisuanji+Gongcheng+yu+Kexue%22">Computer Engineering & Science / Jisuanji Gongcheng yu Kexue</searchLink>. Apr2026, Vol. 48 Issue 4, p599-607. 9p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Digital+signal+processing%22">Digital signal processing</searchLink><br /><searchLink fieldCode="DE" term="%22Bandwidths%22">Bandwidths</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+performance%22">Computer performance</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Matrix+multiplications%22">Matrix multiplications</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+optimization%22">Mathematical optimization</searchLink>
– Name: Abstract
  Label: Abstract (English)
  Group: Ab
  Data: With the rise of large language models (LLMs), the parameter scale of neural network models has grown exponentially, reaching the order of hundreds of billions or even trillions, posing immense challenges to the computing power and bandwidth of computational devices for model inference tasks. To achieve high-performance LLMs inference on low-bandwidth devices, this study focuses on bandwidth-constrained, long-vector digital signal processor (DSP) architectures, designing and implementing efficient LLMs inference methods. It proposes a tensor shape-aware low-precision matrix multiplication method that fully leverages the DSP's computational capabilities while reducing memory access pressure. Additionally, it introduces a data dependency-based operator fusion method to minimize the transmission of intermediate temporary data and employs a deferred operator execution method to enhance the core execution efficiency of DSP devices. Experimental results demonstrate that this approach effectively improves the inference per-formance of large models on bandwidth-constrained DSP devices. Compared to conventional implementations, the optimized inference method achieves a speedup ranging from 1.4 to 2.3 times. Furthermore, when compared to multi-core ARM CPUs and Intel􀆿 Xeon􀆿 GoldCPUs with higher memory bandwidth, the LLMs inference performance achieves speedups of 2.5 times and 1.5 times, respectively, under the same number of cores. [ABSTRACT FROM AUTHOR]
– Name: Abstract
  Label: Abstract (Chinese)
  Group: Ab
  Data: 随着大语言模型的兴起,神经网络模型的参数规模呈指数级增长并达到千/万亿量级,模型的 推理任务对计算设备的算力和带宽提出了巨大挑战。为实现低带宽设备上的高性能LLMs推理,针对带 宽受限、长向量数字信号处理器体系结构,设计并实现高效的LLMs推理方法,提出基于张量形状感知的 低精度矩阵乘方法,充分利用DSP的计算能力和降低访存压力的能力;提出基于数据依赖关系的算子融 合方法减少中间临时数据的传输;使用延迟算子执行方法提升DSP设备内核执行效率。实验表明,该方 法能够有效提升大模型在带宽受限DSP设备上的推理性能,优化后的推理方法相较于普通实现能够实现 1.4~2.3倍的加速比;相较于内存带宽更高的多核ARM CPU 以及Intel􀆿 Xeon􀆿 Gold CPU,同等核心数 量下LLMs推理性能的加速比分别达到2.5倍和1.2倍以上。. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Computer Engineering & Science / Jisuanji Gongcheng yu Kexue is the property of Computer Engineering & Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=193831502
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.3969/j.issn.1007-130X.2026.04.004
    Languages:
      – Code: chi
        Text: Chinese
    PhysicalDescription:
      Pagination:
        PageCount: 9
        StartPage: 599
    Subjects:
      – SubjectFull: Digital signal processing
        Type: general
      – SubjectFull: Bandwidths
        Type: general
      – SubjectFull: Computer performance
        Type: general
      – SubjectFull: Language models
        Type: general
      – SubjectFull: Matrix multiplications
        Type: general
      – SubjectFull: Mathematical optimization
        Type: general
    Titles:
      – TitleFull: 面向带宽受限型DSP的高效大语言模型推理方法.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: 陈 阳
      – PersonEntity:
          Name:
            NameFull: 杨 希
      – PersonEntity:
          Name:
            NameFull: 苏华友
      – PersonEntity:
          Name:
            NameFull: 陈抗抗
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Text: Apr2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 1007130X
          Numbering:
            – Type: volume
              Value: 48
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Computer Engineering & Science / Jisuanji Gongcheng yu Kexue
              Type: main
ResultId 1