Efficient personalized image memorability via gaze-guided semantic cloning distillation.
Saved in:
| Title: | Efficient personalized image memorability via gaze-guided semantic cloning distillation. |
|---|---|
| Authors: | Ghosh, Indrajeet1 (AUTHOR) vkw4ze@virginia.edu, Anwar, Mohammad Saeid2 (AUTHOR) saeid.anwar@umbc.edu, Jayarajah, Kasthuri3 (AUTHOR) kasthuri.jayarajah@njit.edu, Roy, Nirmalya2 (AUTHOR) nroy@umbc.edu |
| Source: | Knowledge & Information Systems. 7/7/2026, Vol. 68 Issue 1, p1-26. 26p. |
| Subjects: | Visual memory, Knowledge transfer, Deep learning, Selectivity (Psychology), Edge computing, Imitative behavior, Human-machine systems |
| Abstract: | Image memorability (IM) estimation typically relies on learning generic semantic features from large-scale datasets; however, memorability is intrinsically individual-dependent and personalized visual viewing behavior, but overlooking this variability can undermine model performance in downstream cognitive and human–machine interaction tasks, leading toward suboptimal performance. To address this, we propose PerMem, a unified framework that integrates knowledge distillation (KD) and imitation learning (IL) to jointly learn generic and personalized salient representations for memorability estimation by leveraging both image content and gaze-derived heatmaps. PerMem employs a teacher network built upon a pre-trained ResNet-50 backbone, followed by an encoder–decoder architecture coupled with spatial and channel attention mechanisms to generate generic saliency-aware memorability maps. A lightweight, attention-guided student encoder–decoder is then optimized through a composite imitation-guided distillation process, where knowledge is distilled from the teacher while simultaneously imitating user-specific gaze fixation heatmaps. Through this joint training process, the student network learns to produce personalized memorability estimates while achieving substantial reductions in computational complexity. We validate PerMem on two public IM estimation datasets (LaMem and SUN) and a in-house WoM dataset comprising of 45 participants (UMBC IRB #670) engaged in visual search and navigation tasks that reflect individualized visual attention patterns. PerMem outperforms nine state-of-the-art IM models by capturing coarse-to-fine saliency and adapting to individual attention, achieving ≈ 6% improvement in memorability prediction. Lastly, we evaluate PerMem on heterogeneous embedded edge devices, including Jetson Nano, Jetson Xavier NX, and Raspberry Pi, demonstrating efficient on-device inference with consistent reductions in memory usage (25.6–33.4%), power consumption (19.0–25.0%), and inference latency (34.9–39.8%) relative to the teacher model, highlighting its practical robustness for resource-constrained, human-centric applications. [ABSTRACT FROM AUTHOR] |
| Copyright of Knowledge & Information Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 195151189 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Efficient personalized image memorability via gaze-guided semantic cloning distillation. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ghosh%2C+Indrajeet%22">Ghosh, Indrajeet</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> vkw4ze@virginia.edu</i><br /><searchLink fieldCode="AR" term="%22Anwar%2C+Mohammad+Saeid%22">Anwar, Mohammad Saeid</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> saeid.anwar@umbc.edu</i><br /><searchLink fieldCode="AR" term="%22Jayarajah%2C+Kasthuri%22">Jayarajah, Kasthuri</searchLink><relatesTo>3</relatesTo> (AUTHOR)<i> kasthuri.jayarajah@njit.edu</i><br /><searchLink fieldCode="AR" term="%22Roy%2C+Nirmalya%22">Roy, Nirmalya</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> nroy@umbc.edu</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Knowledge+%26+Information+Systems%22">Knowledge & Information Systems</searchLink>. 7/7/2026, Vol. 68 Issue 1, p1-26. 26p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Visual+memory%22">Visual memory</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+transfer%22">Knowledge transfer</searchLink><br /><searchLink fieldCode="DE" term="%22Deep+learning%22">Deep learning</searchLink><br /><searchLink fieldCode="DE" term="%22Selectivity+%28Psychology%29%22">Selectivity (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Edge+computing%22">Edge computing</searchLink><br /><searchLink fieldCode="DE" term="%22Imitative+behavior%22">Imitative behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Human-machine+systems%22">Human-machine systems</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Image memorability (IM) estimation typically relies on learning generic semantic features from large-scale datasets; however, memorability is intrinsically individual-dependent and personalized visual viewing behavior, but overlooking this variability can undermine model performance in downstream cognitive and human–machine interaction tasks, leading toward suboptimal performance. To address this, we propose PerMem, a unified framework that integrates knowledge distillation (KD) and imitation learning (IL) to jointly learn generic and personalized salient representations for memorability estimation by leveraging both image content and gaze-derived heatmaps. PerMem employs a teacher network built upon a pre-trained ResNet-50 backbone, followed by an encoder–decoder architecture coupled with spatial and channel attention mechanisms to generate generic saliency-aware memorability maps. A lightweight, attention-guided student encoder–decoder is then optimized through a composite imitation-guided distillation process, where knowledge is distilled from the teacher while simultaneously imitating user-specific gaze fixation heatmaps. Through this joint training process, the student network learns to produce personalized memorability estimates while achieving substantial reductions in computational complexity. We validate PerMem on two public IM estimation datasets (LaMem and SUN) and a in-house WoM dataset comprising of 45 participants (UMBC IRB #670) engaged in visual search and navigation tasks that reflect individualized visual attention patterns. PerMem outperforms nine state-of-the-art IM models by capturing coarse-to-fine saliency and adapting to individual attention, achieving ≈ 6% improvement in memorability prediction. Lastly, we evaluate PerMem on heterogeneous embedded edge devices, including Jetson Nano, Jetson Xavier NX, and Raspberry Pi, demonstrating efficient on-device inference with consistent reductions in memory usage (25.6–33.4%), power consumption (19.0–25.0%), and inference latency (34.9–39.8%) relative to the teacher model, highlighting its practical robustness for resource-constrained, human-centric applications. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Knowledge & Information Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=195151189 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10115-026-02835-w Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 26 StartPage: 1 Subjects: – SubjectFull: Visual memory Type: general – SubjectFull: Knowledge transfer Type: general – SubjectFull: Deep learning Type: general – SubjectFull: Selectivity (Psychology) Type: general – SubjectFull: Edge computing Type: general – SubjectFull: Imitative behavior Type: general – SubjectFull: Human-machine systems Type: general Titles: – TitleFull: Efficient personalized image memorability via gaze-guided semantic cloning distillation. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ghosh, Indrajeet – PersonEntity: Name: NameFull: Anwar, Mohammad Saeid – PersonEntity: Name: NameFull: Jayarajah, Kasthuri – PersonEntity: Name: NameFull: Roy, Nirmalya IsPartOfRelationships: – BibEntity: Dates: – D: 07 M: 07 Text: 7/7/2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 02191377 Numbering: – Type: volume Value: 68 – Type: issue Value: 1 Titles: – TitleFull: Knowledge & Information Systems Type: main |
| ResultId | 1 |
