View in EDS

DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs.

Saved in:
Bibliographic Details
Title: DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs.
Authors: Burrello, Alessio1 (AUTHOR) alessio.burrello@unibo.it, Garofalo, Angelo1 (AUTHOR) angelo.garofalo@unibo.it, Bruschi, Nazareno1 (AUTHOR) nazareno.bruschi@unibo.it, Tagliavini, Giuseppe2 (AUTHOR) giuseppe.tagliavini@unibo.it, Rossi, Davide1 (AUTHOR) davide.rossi@unibo.it, Conti, Francesco1 (AUTHOR) f.conti@unibo.it
Source: IEEE Transactions on Computers. Aug2021, Vol. 70 Issue 8, p1253-1268. 16p.
Subjects: Explicit memory, Internet of things, Constraint programming, Energy consumption, Microelectromechanical systems, Cache memory, Kernel operating systems
Abstract: The deployment of Deep Neural Networks (DNNs) on end-nodes at the extreme edge of the Internet-of-Things is a critical enabler to support pervasive Deep Learning-enhanced applications. Low-Cost MCU-based end-nodes have limited on-chip memory and often replace caches with scratchpads, to reduce area overheads and increase energy efficiency – requiring explicit DMA-based memory transfers between different levels of the memory hierarchy. Mapping modern DNNs on these systems requires aggressive topology-dependent tiling and double-buffering. In this work, we propose DORY (Deployment Oriented to memoRY) – an automatic tool to deploy DNNs on low cost MCUs with typically less than 1MB of on-chip SRAM memory. DORY abstracts tiling as a Constraint Programming (CP) problem: it maximizes L1 memory utilization under the topological constraints imposed by each DNN layer. Then, it generates ANSI C code to orchestrate off- and on-chip transfers and computation phases. Furthermore, to maximize speed, DORY augments the CP formulation with heuristics promoting performance-effective tile sizes. As a case study for DORY, we target GreenWaves Technologies GAP8, one of the most advanced parallel ultra-low power MCU-class devices on the market. On this device, DORY achieves up to 2.5× better MAC/cycle than the GreenWaves proprietary software solution and 18.1× better than the state-of-the-art result on an STM32-H743 MCU on single layers. Using our tool, GAP-8 can perform end-to-end inference of a 1.0-MobileNet-128 network consuming just 63 pJ/MAC on average @ 4.3 fps – 15.4× better than an STM32-H743. We release all our developments – the DORY framework, the optimized backend kernels, and the related heuristics – as open-source software. [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Computers is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 152968808
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Burrello%2C+Alessio%22">Burrello, Alessio</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> alessio.burrello@unibo.it</i><br /><searchLink fieldCode="AR" term="%22Garofalo%2C+Angelo%22">Garofalo, Angelo</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> angelo.garofalo@unibo.it</i><br /><searchLink fieldCode="AR" term="%22Bruschi%2C+Nazareno%22">Bruschi, Nazareno</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> nazareno.bruschi@unibo.it</i><br /><searchLink fieldCode="AR" term="%22Tagliavini%2C+Giuseppe%22">Tagliavini, Giuseppe</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> giuseppe.tagliavini@unibo.it</i><br /><searchLink fieldCode="AR" term="%22Rossi%2C+Davide%22">Rossi, Davide</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> davide.rossi@unibo.it</i><br /><searchLink fieldCode="AR" term="%22Conti%2C+Francesco%22">Conti, Francesco</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> f.conti@unibo.it</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22IEEE+Transactions+on+Computers%22">IEEE Transactions on Computers</searchLink>. Aug2021, Vol. 70 Issue 8, p1253-1268. 16p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Explicit+memory%22">Explicit memory</searchLink><br /><searchLink fieldCode="DE" term="%22Internet+of+things%22">Internet of things</searchLink><br /><searchLink fieldCode="DE" term="%22Constraint+programming%22">Constraint programming</searchLink><br /><searchLink fieldCode="DE" term="%22Energy+consumption%22">Energy consumption</searchLink><br /><searchLink fieldCode="DE" term="%22Microelectromechanical+systems%22">Microelectromechanical systems</searchLink><br /><searchLink fieldCode="DE" term="%22Cache+memory%22">Cache memory</searchLink><br /><searchLink fieldCode="DE" term="%22Kernel+operating+systems%22">Kernel operating systems</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The deployment of Deep Neural Networks (DNNs) on end-nodes at the extreme edge of the Internet-of-Things is a critical enabler to support pervasive Deep Learning-enhanced applications. Low-Cost MCU-based end-nodes have limited on-chip memory and often replace caches with scratchpads, to reduce area overheads and increase energy efficiency – requiring explicit DMA-based memory transfers between different levels of the memory hierarchy. Mapping modern DNNs on these systems requires aggressive topology-dependent tiling and double-buffering. In this work, we propose DORY (Deployment Oriented to memoRY) – an automatic tool to deploy DNNs on low cost MCUs with typically less than 1MB of on-chip SRAM memory. DORY abstracts tiling as a Constraint Programming (CP) problem: it maximizes L1 memory utilization under the topological constraints imposed by each DNN layer. Then, it generates ANSI C code to orchestrate off- and on-chip transfers and computation phases. Furthermore, to maximize speed, DORY augments the CP formulation with heuristics promoting performance-effective tile sizes. As a case study for DORY, we target GreenWaves Technologies GAP8, one of the most advanced parallel ultra-low power MCU-class devices on the market. On this device, DORY achieves up to 2.5× better MAC/cycle than the GreenWaves proprietary software solution and 18.1× better than the state-of-the-art result on an STM32-H743 MCU on single layers. Using our tool, GAP-8 can perform end-to-end inference of a 1.0-MobileNet-128 network consuming just 63 pJ/MAC on average @ 4.3 fps – 15.4× better than an STM32-H743. We release all our developments – the DORY framework, the optimized backend kernels, and the related heuristics – as open-source software. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of IEEE Transactions on Computers is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=152968808
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/TC.2021.3066883
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 1253
    Subjects:
      – SubjectFull: Explicit memory
        Type: general
      – SubjectFull: Internet of things
        Type: general
      – SubjectFull: Constraint programming
        Type: general
      – SubjectFull: Energy consumption
        Type: general
      – SubjectFull: Microelectromechanical systems
        Type: general
      – SubjectFull: Cache memory
        Type: general
      – SubjectFull: Kernel operating systems
        Type: general
    Titles:
      – TitleFull: DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Burrello, Alessio
      – PersonEntity:
          Name:
            NameFull: Garofalo, Angelo
      – PersonEntity:
          Name:
            NameFull: Bruschi, Nazareno
      – PersonEntity:
          Name:
            NameFull: Tagliavini, Giuseppe
      – PersonEntity:
          Name:
            NameFull: Rossi, Davide
      – PersonEntity:
          Name:
            NameFull: Conti, Francesco
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Text: Aug2021
              Type: published
              Y: 2021
          Identifiers:
            – Type: issn-print
              Value: 00189340
          Numbering:
            – Type: volume
              Value: 70
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: IEEE Transactions on Computers
              Type: main
ResultId 1