Federated learning by employing knowledge distillation on edge devices with limited hardware resources.
Saved in:
| Title: | Federated learning by employing knowledge distillation on edge devices with limited hardware resources. |
|---|---|
| Authors: | Tanghatari, Ehsan1 (AUTHOR) tanghatari.e@usc.edu, Kamal, Mehdi1,2 (AUTHOR) mehdi.kamal@usc.edu, Afzali-Kusha, Ali1 (AUTHOR) afzali@usc.edu, Pedram, Massoud2 (AUTHOR) pedram@usc.edu |
| Source: | Neurocomputing. Apr2023, Vol. 531, p87-99. 13p. |
| Subjects: | Artificial neural networks, Knowledge transfer, Power resources |
| Abstract: | This paper presents a federated learning approach based on utilizing computational resources of the IoT edge devices for training deep neural networks. In this approach, the edge devices and the cloud server collaborate in the training phase while preserving the privacy of the edge device data. Owing to the limited computational power and resources available to the edge devices, instead of the original neural network (NN), we suggest to use a smaller NN generated using a proposed heuristic method. In the proposed approach, the smaller model, which is trained on the edge device, is generated from the main NN model. By the exploiting Knowledge Distillation (K D) approach, the learned knowledge in the server and the edge devices can be exchanged, leading to lower required computation on the server and preserving data privacy of the edge devices. Also, to reduce the knowledge transfer overhead on the communication links between the server and the edge devices, a method for selecting the most valuable data to transfer the knowledge is introduced. The effectiveness of this method is assessed by comparing it to state-of-the-art methods. The results show that the proposed method lowers the communication traffic by up to 250 × and increases the learning accuracy by an average of 8.9 % in the cloud compared to the prior K D -based distributed training approaches in CIFAR-10 dataset. [ABSTRACT FROM AUTHOR] |
| Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 162438582 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Federated learning by employing knowledge distillation on edge devices with limited hardware resources. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Tanghatari%2C+Ehsan%22">Tanghatari, Ehsan</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> tanghatari.e@usc.edu</i><br /><searchLink fieldCode="AR" term="%22Kamal%2C+Mehdi%22">Kamal, Mehdi</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> mehdi.kamal@usc.edu</i><br /><searchLink fieldCode="AR" term="%22Afzali-Kusha%2C+Ali%22">Afzali-Kusha, Ali</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> afzali@usc.edu</i><br /><searchLink fieldCode="AR" term="%22Pedram%2C+Massoud%22">Pedram, Massoud</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> pedram@usc.edu</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Neurocomputing%22">Neurocomputing</searchLink>. Apr2023, Vol. 531, p87-99. 13p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Artificial+neural+networks%22">Artificial neural networks</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+transfer%22">Knowledge transfer</searchLink><br /><searchLink fieldCode="DE" term="%22Power+resources%22">Power resources</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: This paper presents a federated learning approach based on utilizing computational resources of the IoT edge devices for training deep neural networks. In this approach, the edge devices and the cloud server collaborate in the training phase while preserving the privacy of the edge device data. Owing to the limited computational power and resources available to the edge devices, instead of the original neural network (NN), we suggest to use a smaller NN generated using a proposed heuristic method. In the proposed approach, the smaller model, which is trained on the edge device, is generated from the main NN model. By the exploiting Knowledge Distillation (K D) approach, the learned knowledge in the server and the edge devices can be exchanged, leading to lower required computation on the server and preserving data privacy of the edge devices. Also, to reduce the knowledge transfer overhead on the communication links between the server and the edge devices, a method for selecting the most valuable data to transfer the knowledge is introduced. The effectiveness of this method is assessed by comparing it to state-of-the-art methods. The results show that the proposed method lowers the communication traffic by up to 250 × and increases the learning accuracy by an average of 8.9 % in the cloud compared to the prior K D -based distributed training approaches in CIFAR-10 dataset. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=162438582 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.neucom.2023.02.011 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 87 Subjects: – SubjectFull: Artificial neural networks Type: general – SubjectFull: Knowledge transfer Type: general – SubjectFull: Power resources Type: general Titles: – TitleFull: Federated learning by employing knowledge distillation on edge devices with limited hardware resources. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Tanghatari, Ehsan – PersonEntity: Name: NameFull: Kamal, Mehdi – PersonEntity: Name: NameFull: Afzali-Kusha, Ali – PersonEntity: Name: NameFull: Pedram, Massoud IsPartOfRelationships: – BibEntity: Dates: – D: 28 M: 04 Text: Apr2023 Type: published Y: 2023 Identifiers: – Type: issn-print Value: 09252312 Numbering: – Type: volume Value: 531 Titles: – TitleFull: Neurocomputing Type: main |
| ResultId | 1 |
