Brain-Inspired Continual Learning: Rethinking the Role of Features in the Stability-Plasticity Dilemma
Saved in:
| Title: | Brain-Inspired Continual Learning: Rethinking the Role of Features in the Stability-Plasticity Dilemma |
|---|---|
| Authors: | Khan, Hikmat |
| Committee Members: | Nidhal Carla Bouaynaya, Ph.D; Ravi Prakash Ramachandran, Ph.D; Robi Polikar, Ph.D |
| Summary: | Continual learning (CL) enables deep learning models to learn new tasks sequentially while preserving performance on previously learned tasks, akin to the human's ability to accumulate knowledge over time. However, existing approaches to CL face the challenge of catastrophic forgetting, which occurs when a model's performance on previously learned tasks declines after learning the new task. In this dissertation, we focus on the crucial role of input data features in determining the robustness of CL models to mitigate catastrophic forgetting. We propose a framework to create CL-robustified versions of standard datasets using a pre-trained Oracle CL model. Our experiments show that the CL model trained on CL-robust features mitigates catastrophic forgetting. We then introduce a novel approach inspired by neuroscience called robust rehearsal, which distills CL-robust samples without needing a pre-trained Oracle model or pre-distilled CL-robust sample for training the CL model. We demonstrate that rehearsal-based CL approaches mitigate catastrophic forgetting using CIFAR10, CIFAR100, and Federal Aviation Administration (FAA) provided real-world helicopter attitude datasets. Moreover, we propose an approach to address the observed issue of overfitting rehearsal memory through adversarial diversification. This approach increases the complexity of rehearsal samples, reduces memory overfitting, and maintains their effectiveness throughout sequential learning. Finally, we conducted an extensive study to further elucidate the crucial role of features in shaping the model's overall characteristics, specifically its robustness and mitigating catastrophic forgetting. |
| URL: | https://rdw.rowan.edu/etd/3212 |
| Database: | OpenDissertations |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ddu DbLabel: OpenDissertations An: ddu.oai.rdw.rowan.edu.etd.4218 AccessLevel: 6 PubType: Dissertation/ Thesis PubTypeId: dissertation PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Brain-Inspired Continual Learning: Rethinking the Role of Features in the Stability-Plasticity Dilemma – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Khan%2C+Hikmat%22">Khan, Hikmat</searchLink> – Name: Author Label: Committee Members Group: Au Data: <searchLink fieldCode="CO" term="%22Nidhal+Carla+Bouaynaya%2C+Ph%2ED%22">Nidhal Carla Bouaynaya, Ph.D</searchLink>; <searchLink fieldCode="CO" term="%22Ravi+Prakash+Ramachandran%2C+Ph%2ED%22">Ravi Prakash Ramachandran, Ph.D</searchLink>; <searchLink fieldCode="CO" term="%22Robi+Polikar%2C+Ph%2ED%22">Robi Polikar, Ph.D</searchLink> – Name: Abstract Label: Summary Group: Ab Data: Continual learning (CL) enables deep learning models to learn new tasks sequentially while preserving performance on previously learned tasks, akin to the human's ability to accumulate knowledge over time. However, existing approaches to CL face the challenge of catastrophic forgetting, which occurs when a model's performance on previously learned tasks declines after learning the new task. In this dissertation, we focus on the crucial role of input data features in determining the robustness of CL models to mitigate catastrophic forgetting. We propose a framework to create CL-robustified versions of standard datasets using a pre-trained Oracle CL model. Our experiments show that the CL model trained on CL-robust features mitigates catastrophic forgetting. We then introduce a novel approach inspired by neuroscience called robust rehearsal, which distills CL-robust samples without needing a pre-trained Oracle model or pre-distilled CL-robust sample for training the CL model. We demonstrate that rehearsal-based CL approaches mitigate catastrophic forgetting using CIFAR10, CIFAR100, and Federal Aviation Administration (FAA) provided real-world helicopter attitude datasets. Moreover, we propose an approach to address the observed issue of overfitting rehearsal memory through adversarial diversification. This approach increases the complexity of rehearsal samples, reduces memory overfitting, and maintains their effectiveness throughout sequential learning. Finally, we conducted an extensive study to further elucidate the crucial role of features in shaping the model's overall characteristics, specifically its robustness and mitigating catastrophic forgetting. – Name: URL Label: URL Group: URL Data: <link linkTarget="URL" linkTerm="https://rdw.rowan.edu/etd/3212" linkWindow="_blank">https://rdw.rowan.edu/etd/3212</link> |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ddu&AN=ddu.oai.rdw.rowan.edu.etd.4218 |
| RecordInfo | BibRecord: BibEntity: Languages: – Code: eng Text: English Subjects: – SubjectFull: Artificial Intelligence; Catastrophic Forgetting; Continual Learning; Deep Learning; Machine Learning; Neuroscience Type: general – SubjectFull: Deep learning (Machine learning) Type: general – SubjectFull: Electrical and Computer Engineering Type: general – SubjectFull: Physical Sciences and Mathematics Type: general Titles: – TitleFull: Brain-Inspired Continual Learning: Rethinking the Role of Features in the Stability-Plasticity Dilemma Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Khan, Hikmat IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Type: published Y: 2024 |
| ResultId | 1 |
