Migrating federated learning to centralized learning with the leverage of unlabeled data.
Saved in:
| Title: | Migrating federated learning to centralized learning with the leverage of unlabeled data. |
|---|---|
| Authors: | Wang, Xiaoya1 (AUTHOR), Zhu, Tianqing2 (AUTHOR) tianqing.zhu@ieee.org, Ren, Wei1 (AUTHOR), Zhang, Dongmei1 (AUTHOR), Xiong, Ping3 (AUTHOR) |
| Source: | Knowledge & Information Systems. Sep2023, Vol. 65 Issue 9, p3725-3752. 28p. |
| Subjects: | Supervised learning, Secure Sockets Layer (Computer network protocol), Information sharing, Data distribution |
| Abstract: | Federated learning carries out cooperative training without local data sharing; the obtained global model performs generally better than independent local models. Benefiting from the free data sharing, federated learning preserves the privacy of local users. However, the performance of the global model might be degraded if diverse clients hold non-IID training data. This is because the different distributions of local data lead to weight divergence of local models. In this paper, we introduce a novel teacher–student framework to alleviate the negative impact of non-IID data. On the one hand, we maintain the advantage of the federated learning on the privacy-preserving, and on the other hand, we take the advantage of the centralized learning on the accuracy. We use unlabeled data and global models as teachers to generate a pseudo-labeled dataset, which can significantly improve the performance of the global model. At the same time, the global model as a teacher provides more accurate pseudo-labels. In addition, we perform a model rollback to mitigate the impact of latent noise labels and data imbalance in the pseudo-labeled dataset. Extensive experiments have verified that our teacher ensemble performs a more robust training. The empirical study verifies that the reliance on the centralized pseudo-labeled data enables the global model almost immune to non-IID data. [ABSTRACT FROM AUTHOR] |
| Copyright of Knowledge & Information Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 164875600 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Migrating federated learning to centralized learning with the leverage of unlabeled data. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Wang%2C+Xiaoya%22">Wang, Xiaoya</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Zhu%2C+Tianqing%22">Zhu, Tianqing</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> tianqing.zhu@ieee.org</i><br /><searchLink fieldCode="AR" term="%22Ren%2C+Wei%22">Ren, Wei</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Zhang%2C+Dongmei%22">Zhang, Dongmei</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Xiong%2C+Ping%22">Xiong, Ping</searchLink><relatesTo>3</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Knowledge+%26+Information+Systems%22">Knowledge & Information Systems</searchLink>. Sep2023, Vol. 65 Issue 9, p3725-3752. 28p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Supervised+learning%22">Supervised learning</searchLink><br /><searchLink fieldCode="DE" term="%22Secure+Sockets+Layer+%28Computer+network+protocol%29%22">Secure Sockets Layer (Computer network protocol)</searchLink><br /><searchLink fieldCode="DE" term="%22Information+sharing%22">Information sharing</searchLink><br /><searchLink fieldCode="DE" term="%22Data+distribution%22">Data distribution</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Federated learning carries out cooperative training without local data sharing; the obtained global model performs generally better than independent local models. Benefiting from the free data sharing, federated learning preserves the privacy of local users. However, the performance of the global model might be degraded if diverse clients hold non-IID training data. This is because the different distributions of local data lead to weight divergence of local models. In this paper, we introduce a novel teacher–student framework to alleviate the negative impact of non-IID data. On the one hand, we maintain the advantage of the federated learning on the privacy-preserving, and on the other hand, we take the advantage of the centralized learning on the accuracy. We use unlabeled data and global models as teachers to generate a pseudo-labeled dataset, which can significantly improve the performance of the global model. At the same time, the global model as a teacher provides more accurate pseudo-labels. In addition, we perform a model rollback to mitigate the impact of latent noise labels and data imbalance in the pseudo-labeled dataset. Extensive experiments have verified that our teacher ensemble performs a more robust training. The empirical study verifies that the reliance on the centralized pseudo-labeled data enables the global model almost immune to non-IID data. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Knowledge & Information Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=164875600 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10115-023-01869-8 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 28 StartPage: 3725 Subjects: – SubjectFull: Supervised learning Type: general – SubjectFull: Secure Sockets Layer (Computer network protocol) Type: general – SubjectFull: Information sharing Type: general – SubjectFull: Data distribution Type: general Titles: – TitleFull: Migrating federated learning to centralized learning with the leverage of unlabeled data. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Wang, Xiaoya – PersonEntity: Name: NameFull: Zhu, Tianqing – PersonEntity: Name: NameFull: Ren, Wei – PersonEntity: Name: NameFull: Zhang, Dongmei – PersonEntity: Name: NameFull: Xiong, Ping IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 09 Text: Sep2023 Type: published Y: 2023 Identifiers: – Type: issn-print Value: 02191377 Numbering: – Type: volume Value: 65 – Type: issue Value: 9 Titles: – TitleFull: Knowledge & Information Systems Type: main |
| ResultId | 1 |
