Brain-Inspired Continual Learning: Rethinking the Role of Features in the Stability-Plasticity Dilemma
Saved in:
| Title: | Brain-Inspired Continual Learning: Rethinking the Role of Features in the Stability-Plasticity Dilemma |
|---|---|
| Authors: | Khan, Hikmat |
| Committee Members: | Nidhal Carla Bouaynaya, Ph.D; Ravi Prakash Ramachandran, Ph.D; Robi Polikar, Ph.D |
| Summary: | Continual learning (CL) enables deep learning models to learn new tasks sequentially while preserving performance on previously learned tasks, akin to the human's ability to accumulate knowledge over time. However, existing approaches to CL face the challenge of catastrophic forgetting, which occurs when a model's performance on previously learned tasks declines after learning the new task. In this dissertation, we focus on the crucial role of input data features in determining the robustness of CL models to mitigate catastrophic forgetting. We propose a framework to create CL-robustified versions of standard datasets using a pre-trained Oracle CL model. Our experiments show that the CL model trained on CL-robust features mitigates catastrophic forgetting. We then introduce a novel approach inspired by neuroscience called robust rehearsal, which distills CL-robust samples without needing a pre-trained Oracle model or pre-distilled CL-robust sample for training the CL model. We demonstrate that rehearsal-based CL approaches mitigate catastrophic forgetting using CIFAR10, CIFAR100, and Federal Aviation Administration (FAA) provided real-world helicopter attitude datasets. Moreover, we propose an approach to address the observed issue of overfitting rehearsal memory through adversarial diversification. This approach increases the complexity of rehearsal samples, reduces memory overfitting, and maintains their effectiveness throughout sequential learning. Finally, we conducted an extensive study to further elucidate the crucial role of features in shaping the model's overall characteristics, specifically its robustness and mitigating catastrophic forgetting. |
| URL: | https://rdw.rowan.edu/etd/3212 |
| Database: | OpenDissertations |
| Abstract: | Continual learning (CL) enables deep learning models to learn new tasks sequentially while preserving performance on previously learned tasks, akin to the human's ability to accumulate knowledge over time. However, existing approaches to CL face the challenge of catastrophic forgetting, which occurs when a model's performance on previously learned tasks declines after learning the new task. In this dissertation, we focus on the crucial role of input data features in determining the robustness of CL models to mitigate catastrophic forgetting. We propose a framework to create CL-robustified versions of standard datasets using a pre-trained Oracle CL model. Our experiments show that the CL model trained on CL-robust features mitigates catastrophic forgetting. We then introduce a novel approach inspired by neuroscience called robust rehearsal, which distills CL-robust samples without needing a pre-trained Oracle model or pre-distilled CL-robust sample for training the CL model. We demonstrate that rehearsal-based CL approaches mitigate catastrophic forgetting using CIFAR10, CIFAR100, and Federal Aviation Administration (FAA) provided real-world helicopter attitude datasets. Moreover, we propose an approach to address the observed issue of overfitting rehearsal memory through adversarial diversification. This approach increases the complexity of rehearsal samples, reduces memory overfitting, and maintains their effectiveness throughout sequential learning. Finally, we conducted an extensive study to further elucidate the crucial role of features in shaping the model's overall characteristics, specifically its robustness and mitigating catastrophic forgetting. |
|---|
