LDS-Inspired Residual Networks.
Saved in:
| Title: | LDS-Inspired Residual Networks. |
|---|---|
| Authors: | Dimou, Anastasios1 (AUTHOR) dimou@iti.gr, Ataloglou, Dimitrios2 (AUTHOR) ataloglou@iti.gr, Dimitropoulos, Kosmas2 (AUTHOR) dimitrop@iti.gr, Alvarez, Federico1 (AUTHOR) fag@gatv.ssr.upm.es, Daras, Petros2 (AUTHOR) daras@iti.gr |
| Source: | IEEE Transactions on Circuits & Systems for Video Technology. Aug2019, Vol. 29 Issue 8, p2363-2375. 13p. |
| Subjects: | Artificial neural networks, Linear dynamical systems, Deep learning, Concept mapping, Learning communities |
| Abstract: | Residual networks (ResNets) have introduced a milestone for the deep learning community due to their outstanding performance in diverse applications. They enable efficient training of increasingly deep networks, reducing the training difficulty and error. The main intuition behind them is that, instead of mapping the input information, they are mapping a residual part of it. Since the original work, a lot of extensions have been proposed to improve information mapping. In this paper, a novel extension of the residual block is proposed inspired by linear dynamical systems (LDSs), called LDS-ResNet. Specifically, a new module is presented that improves mapping of residual information by transforming it in a hidden state and then mapping it back to the desired feature space using convolutional layers. The proposed module is utilized to construct multi-branch residual blocks for convolutional neural networks. An exploration of possible architectural choices is presented and evaluated. Experimental results show that LDS-ResNet outperforms the original ResNet in image classification and object detection tasks on public datasets such as CIFAR-10/100, ImageNet, VOC, and MOT2017. Moreover, its performance boost is complementary to other extensions of the original network such as pre-activation and bottleneck, as well as stochastic training and Squeeze-Excitation. [ABSTRACT FROM AUTHOR] |
| Copyright of IEEE Transactions on Circuits & Systems for Video Technology is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 137987681 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: LDS-Inspired Residual Networks. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Dimou%2C+Anastasios%22">Dimou, Anastasios</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> dimou@iti.gr</i><br /><searchLink fieldCode="AR" term="%22Ataloglou%2C+Dimitrios%22">Ataloglou, Dimitrios</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> ataloglou@iti.gr</i><br /><searchLink fieldCode="AR" term="%22Dimitropoulos%2C+Kosmas%22">Dimitropoulos, Kosmas</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> dimitrop@iti.gr</i><br /><searchLink fieldCode="AR" term="%22Alvarez%2C+Federico%22">Alvarez, Federico</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> fag@gatv.ssr.upm.es</i><br /><searchLink fieldCode="AR" term="%22Daras%2C+Petros%22">Daras, Petros</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> daras@iti.gr</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22IEEE+Transactions+on+Circuits+%26+Systems+for+Video+Technology%22">IEEE Transactions on Circuits & Systems for Video Technology</searchLink>. Aug2019, Vol. 29 Issue 8, p2363-2375. 13p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Artificial+neural+networks%22">Artificial neural networks</searchLink><br /><searchLink fieldCode="DE" term="%22Linear+dynamical+systems%22">Linear dynamical systems</searchLink><br /><searchLink fieldCode="DE" term="%22Deep+learning%22">Deep learning</searchLink><br /><searchLink fieldCode="DE" term="%22Concept+mapping%22">Concept mapping</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+communities%22">Learning communities</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Residual networks (ResNets) have introduced a milestone for the deep learning community due to their outstanding performance in diverse applications. They enable efficient training of increasingly deep networks, reducing the training difficulty and error. The main intuition behind them is that, instead of mapping the input information, they are mapping a residual part of it. Since the original work, a lot of extensions have been proposed to improve information mapping. In this paper, a novel extension of the residual block is proposed inspired by linear dynamical systems (LDSs), called LDS-ResNet. Specifically, a new module is presented that improves mapping of residual information by transforming it in a hidden state and then mapping it back to the desired feature space using convolutional layers. The proposed module is utilized to construct multi-branch residual blocks for convolutional neural networks. An exploration of possible architectural choices is presented and evaluated. Experimental results show that LDS-ResNet outperforms the original ResNet in image classification and object detection tasks on public datasets such as CIFAR-10/100, ImageNet, VOC, and MOT2017. Moreover, its performance boost is complementary to other extensions of the original network such as pre-activation and bottleneck, as well as stochastic training and Squeeze-Excitation. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of IEEE Transactions on Circuits & Systems for Video Technology is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=137987681 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1109/TCSVT.2018.2869680 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 2363 Subjects: – SubjectFull: Artificial neural networks Type: general – SubjectFull: Linear dynamical systems Type: general – SubjectFull: Deep learning Type: general – SubjectFull: Concept mapping Type: general – SubjectFull: Learning communities Type: general Titles: – TitleFull: LDS-Inspired Residual Networks. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Dimou, Anastasios – PersonEntity: Name: NameFull: Ataloglou, Dimitrios – PersonEntity: Name: NameFull: Dimitropoulos, Kosmas – PersonEntity: Name: NameFull: Alvarez, Federico – PersonEntity: Name: NameFull: Daras, Petros IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Text: Aug2019 Type: published Y: 2019 Identifiers: – Type: issn-print Value: 10518215 Numbering: – Type: volume Value: 29 – Type: issue Value: 8 Titles: – TitleFull: IEEE Transactions on Circuits & Systems for Video Technology Type: main |
| ResultId | 1 |
