View in EDS

4Ward: A relayering strategy for efficient training of arbitrarily complex directed acyclic graphs.

Saved in:
Bibliographic Details
Title: 4Ward: A relayering strategy for efficient training of arbitrarily complex directed acyclic graphs.
Authors: Boccato, Tommaso1 (AUTHOR) tommaso.boccato@uniroma2.it, Ferrante, Matteo1 (AUTHOR), Duggento, Andrea1 (AUTHOR), Toschi, Nicola1,2 (AUTHOR)
Source: Neurocomputing. Feb2024, Vol. 568, pN.PAG-N.PAG. 1p.
Subjects: Directed acyclic graphs, Multilayer perceptrons, Python programming language, Sequential learning, Deep learning
Abstract: Thanks to their ease of implementation, multilayer perceptrons (MLPs) have become ubiquitous in deep learning applications. The graph underlying an MLP is indeed multipartite, i.e. each layer of neurons only connects to neurons belonging to the adjacent layer. In contrast, in vivo brain connectomes at the level of individual synapses suggest that biological neuronal networks are characterized by scale-free degree distributions or exponentially truncated power law strength distributions, hinting at potentially novel avenues for the exploitation of evolution-derived neuronal networks. In this paper, we present "4Ward", a method and Python library capable of generating flexible and efficient neural networks (NNs) from arbitrarily complex directed acyclic graphs. 4Ward is inspired by layering algorithms drawn from the graph drawing discipline to implement efficient forward passes, and provides significant time gains in computational experiments with various Erdős-Rényi graphs. 4Ward not only overcomes the sequential nature of the learning matrix method, by parallelizing the computation of activations, but also addresses the scalability issues encountered in the current state-of-the-art and provides the designer with freedom to customize weight initialization and activation functions. Our algorithm can be of aid for any investigator seeking to exploit complex topologies in a NN design framework at the microscale. [ABSTRACT FROM AUTHOR]
Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 174318305
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: 4Ward: A relayering strategy for efficient training of arbitrarily complex directed acyclic graphs.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Boccato%2C+Tommaso%22">Boccato, Tommaso</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> tommaso.boccato@uniroma2.it</i><br /><searchLink fieldCode="AR" term="%22Ferrante%2C+Matteo%22">Ferrante, Matteo</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Duggento%2C+Andrea%22">Duggento, Andrea</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Toschi%2C+Nicola%22">Toschi, Nicola</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Neurocomputing%22">Neurocomputing</searchLink>. Feb2024, Vol. 568, pN.PAG-N.PAG. 1p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Directed+acyclic+graphs%22">Directed acyclic graphs</searchLink><br /><searchLink fieldCode="DE" term="%22Multilayer+perceptrons%22">Multilayer perceptrons</searchLink><br /><searchLink fieldCode="DE" term="%22Python+programming+language%22">Python programming language</searchLink><br /><searchLink fieldCode="DE" term="%22Sequential+learning%22">Sequential learning</searchLink><br /><searchLink fieldCode="DE" term="%22Deep+learning%22">Deep learning</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Thanks to their ease of implementation, multilayer perceptrons (MLPs) have become ubiquitous in deep learning applications. The graph underlying an MLP is indeed multipartite, i.e. each layer of neurons only connects to neurons belonging to the adjacent layer. In contrast, in vivo brain connectomes at the level of individual synapses suggest that biological neuronal networks are characterized by scale-free degree distributions or exponentially truncated power law strength distributions, hinting at potentially novel avenues for the exploitation of evolution-derived neuronal networks. In this paper, we present "4Ward", a method and Python library capable of generating flexible and efficient neural networks (NNs) from arbitrarily complex directed acyclic graphs. 4Ward is inspired by layering algorithms drawn from the graph drawing discipline to implement efficient forward passes, and provides significant time gains in computational experiments with various Erdős-Rényi graphs. 4Ward not only overcomes the sequential nature of the learning matrix method, by parallelizing the computation of activations, but also addresses the scalability issues encountered in the current state-of-the-art and provides the designer with freedom to customize weight initialization and activation functions. Our algorithm can be of aid for any investigator seeking to exploit complex topologies in a NN design framework at the microscale. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=174318305
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.neucom.2023.127058
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 1
        StartPage: N.PAG
    Subjects:
      – SubjectFull: Directed acyclic graphs
        Type: general
      – SubjectFull: Multilayer perceptrons
        Type: general
      – SubjectFull: Python programming language
        Type: general
      – SubjectFull: Sequential learning
        Type: general
      – SubjectFull: Deep learning
        Type: general
    Titles:
      – TitleFull: 4Ward: A relayering strategy for efficient training of arbitrarily complex directed acyclic graphs.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Boccato, Tommaso
      – PersonEntity:
          Name:
            NameFull: Ferrante, Matteo
      – PersonEntity:
          Name:
            NameFull: Duggento, Andrea
      – PersonEntity:
          Name:
            NameFull: Toschi, Nicola
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Text: Feb2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 09252312
          Numbering:
            – Type: volume
              Value: 568
          Titles:
            – TitleFull: Neurocomputing
              Type: main
ResultId 1