View in EDS

Res-DNN: A Residue Number System-Based DNN Accelerator Unit.

Saved in:
Bibliographic Details
Title: Res-DNN: A Residue Number System-Based DNN Accelerator Unit.
Authors: Samimi, Nasim1 (AUTHOR) nasim.samimi@ut.ac.ir, Kamal, Mehdi1 (AUTHOR) mehdikamal@ut.ac.ir, Afzali-Kusha, Ali1 (AUTHOR) afzali@ut.ac.ir, Pedram, Massoud2 (AUTHOR) pedram@usc.edu
Source: IEEE Transactions on Circuits & Systems. Part I: Regular Papers. Feb2020, Vol. 67 Issue 2, p658-671. 14p.
Subjects: Binary number system, Number systems, Energy consumption
Abstract: In this article, a technique, based on using Residue Number System (RNS) is suggested to improve the energy efficiency of Deep Neural Networks (DNNs). In the DNN architecture, which is fully RNS-based, only weights and the primary inputs in the main memory are in the binary number system (BNS). The architecture, which is called Res-DNN, offers a high energy saving while requiring higher bit count for data to handle the overflow compared to that of a BNS one. Scaling techniques in the processing elements are employed in the RNS-based computations to make the computation bit widths the same as the BNS bit width. In this architecture, the MAX pooling and ReLU activation function are implemented in the RNS format. To lower the memory usage and required memory bandwidth, we suggest a Huffman-based coding. Additionally, for accessing the weights stored in the main memory, to obtain further energy reduction, we propose a structural modification to the memory hierarchy where a lower level register file is added to the data path of these accesses. The effectiveness of the proposed architecture is evaluated under seven state-of-the-art DNNs with the datasets of ImageNet and CIFAR-10. The obtained results show that Res-DNN leads to $2.5\times $ lower energy for computations and an average of 30% overall energy reduction compared to those of the binary counterpart. [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Circuits & Systems. Part I: Regular Papers is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 141515272
AccessLevel: 6
PubType: Periodical
PubTypeId: serialPeriodical
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Res-DNN: A Residue Number System-Based DNN Accelerator Unit.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Samimi%2C+Nasim%22">Samimi, Nasim</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> nasim.samimi@ut.ac.ir</i><br /><searchLink fieldCode="AR" term="%22Kamal%2C+Mehdi%22">Kamal, Mehdi</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> mehdikamal@ut.ac.ir</i><br /><searchLink fieldCode="AR" term="%22Afzali-Kusha%2C+Ali%22">Afzali-Kusha, Ali</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> afzali@ut.ac.ir</i><br /><searchLink fieldCode="AR" term="%22Pedram%2C+Massoud%22">Pedram, Massoud</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> pedram@usc.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22IEEE+Transactions+on+Circuits+%26+Systems%2E+Part+I%3A+Regular+Papers%22">IEEE Transactions on Circuits & Systems. Part I: Regular Papers</searchLink>. Feb2020, Vol. 67 Issue 2, p658-671. 14p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Binary+number+system%22">Binary number system</searchLink><br /><searchLink fieldCode="DE" term="%22Number+systems%22">Number systems</searchLink><br /><searchLink fieldCode="DE" term="%22Energy+consumption%22">Energy consumption</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In this article, a technique, based on using Residue Number System (RNS) is suggested to improve the energy efficiency of Deep Neural Networks (DNNs). In the DNN architecture, which is fully RNS-based, only weights and the primary inputs in the main memory are in the binary number system (BNS). The architecture, which is called Res-DNN, offers a high energy saving while requiring higher bit count for data to handle the overflow compared to that of a BNS one. Scaling techniques in the processing elements are employed in the RNS-based computations to make the computation bit widths the same as the BNS bit width. In this architecture, the MAX pooling and ReLU activation function are implemented in the RNS format. To lower the memory usage and required memory bandwidth, we suggest a Huffman-based coding. Additionally, for accessing the weights stored in the main memory, to obtain further energy reduction, we propose a structural modification to the memory hierarchy where a lower level register file is added to the data path of these accesses. The effectiveness of the proposed architecture is evaluated under seven state-of-the-art DNNs with the datasets of ImageNet and CIFAR-10. The obtained results show that Res-DNN leads to $2.5\times $ lower energy for computations and an average of 30% overall energy reduction compared to those of the binary counterpart. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of IEEE Transactions on Circuits & Systems. Part I: Regular Papers is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=141515272
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/TCSI.2019.2951083
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 658
    Subjects:
      – SubjectFull: Binary number system
        Type: general
      – SubjectFull: Number systems
        Type: general
      – SubjectFull: Energy consumption
        Type: general
    Titles:
      – TitleFull: Res-DNN: A Residue Number System-Based DNN Accelerator Unit.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Samimi, Nasim
      – PersonEntity:
          Name:
            NameFull: Kamal, Mehdi
      – PersonEntity:
          Name:
            NameFull: Afzali-Kusha, Ali
      – PersonEntity:
          Name:
            NameFull: Pedram, Massoud
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Text: Feb2020
              Type: published
              Y: 2020
          Identifiers:
            – Type: issn-print
              Value: 15498328
          Numbering:
            – Type: volume
              Value: 67
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: IEEE Transactions on Circuits & Systems. Part I: Regular Papers
              Type: main
ResultId 1