Heterogeneous Forgetting Rates and Greedy Allocation in Slot-Based Memory Networks Promotes Signal Retention.
Saved in:
| Title: | Heterogeneous Forgetting Rates and Greedy Allocation in Slot-Based Memory Networks Promotes Signal Retention. |
|---|---|
| Authors: | Jones, BethAnna1 (AUTHOR) bethannajones@wustl.edu, Snyder, Lawrence2 (AUTHOR) lsnyder@wustl.edu, Ching, ShiNung1 (AUTHOR) shinung@wustl.edu |
| Source: | Neural Computation. May2024, Vol. 36 Issue 5, p1022-1040. 19p. |
| Subjects: | Sensory memory, Short-term memory, Online shopping, Resource allocation, Stimulus & response (Psychology) |
| Abstract: | A key question in the neuroscience of memory encoding pertains to the mechanisms by which afferent stimuli are allocated within memory networks. This issue is especially pronounced in the domain of working memory, where capacity is finite. Presumably the brain must embed some "policy" by which to allocate these mnemonic resources in an online manner in order to maximally represent and store afferent information for as long as possible and without interference from subsequent stimuli. Here, we engage this question through a top-down theoretical modeling framework. We formally optimize a gating mechanism that projects afferent stimuli onto a finite number of memory slots within a recurrent network architecture. In the absence of external input, the activity in each slot attenuates over time (i.e., a process of gradual forgetting). It turns out that the optimal gating policy consists of a direct projection from sensory activity to memory slots, alongside an activity-dependent lateral inhibition. Interestingly, allocating resources myopically (greedily with respect to the current stimulus) leads to efficient utilization of slots over time. In other words, later-arriving stimuli are distributed across slots in such a way that the network state is minimally shifted and so prior signals are minimally "overwritten." Further, networks with heterogeneity in the timescales of their forgetting rates retain stimuli better than those that are more homogeneous. Our results suggest how online, recurrent networks working on temporally localized objectives without high-level supervision can nonetheless implement efficient allocation of memory resources over time. [ABSTRACT FROM AUTHOR] |
| Copyright of Neural Computation is the property of MIT Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 176806056 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Heterogeneous Forgetting Rates and Greedy Allocation in Slot-Based Memory Networks Promotes Signal Retention. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Jones%2C+BethAnna%22">Jones, BethAnna</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> bethannajones@wustl.edu</i><br /><searchLink fieldCode="AR" term="%22Snyder%2C+Lawrence%22">Snyder, Lawrence</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> lsnyder@wustl.edu</i><br /><searchLink fieldCode="AR" term="%22Ching%2C+ShiNung%22">Ching, ShiNung</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> shinung@wustl.edu</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Neural+Computation%22">Neural Computation</searchLink>. May2024, Vol. 36 Issue 5, p1022-1040. 19p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Sensory+memory%22">Sensory memory</searchLink><br /><searchLink fieldCode="DE" term="%22Short-term+memory%22">Short-term memory</searchLink><br /><searchLink fieldCode="DE" term="%22Online+shopping%22">Online shopping</searchLink><br /><searchLink fieldCode="DE" term="%22Resource+allocation%22">Resource allocation</searchLink><br /><searchLink fieldCode="DE" term="%22Stimulus+%26+response+%28Psychology%29%22">Stimulus & response (Psychology)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: A key question in the neuroscience of memory encoding pertains to the mechanisms by which afferent stimuli are allocated within memory networks. This issue is especially pronounced in the domain of working memory, where capacity is finite. Presumably the brain must embed some "policy" by which to allocate these mnemonic resources in an online manner in order to maximally represent and store afferent information for as long as possible and without interference from subsequent stimuli. Here, we engage this question through a top-down theoretical modeling framework. We formally optimize a gating mechanism that projects afferent stimuli onto a finite number of memory slots within a recurrent network architecture. In the absence of external input, the activity in each slot attenuates over time (i.e., a process of gradual forgetting). It turns out that the optimal gating policy consists of a direct projection from sensory activity to memory slots, alongside an activity-dependent lateral inhibition. Interestingly, allocating resources myopically (greedily with respect to the current stimulus) leads to efficient utilization of slots over time. In other words, later-arriving stimuli are distributed across slots in such a way that the network state is minimally shifted and so prior signals are minimally "overwritten." Further, networks with heterogeneity in the timescales of their forgetting rates retain stimuli better than those that are more homogeneous. Our results suggest how online, recurrent networks working on temporally localized objectives without high-level supervision can nonetheless implement efficient allocation of memory resources over time. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Neural Computation is the property of MIT Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=176806056 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1162/neco_a_01655 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 19 StartPage: 1022 Subjects: – SubjectFull: Sensory memory Type: general – SubjectFull: Short-term memory Type: general – SubjectFull: Online shopping Type: general – SubjectFull: Resource allocation Type: general – SubjectFull: Stimulus & response (Psychology) Type: general Titles: – TitleFull: Heterogeneous Forgetting Rates and Greedy Allocation in Slot-Based Memory Networks Promotes Signal Retention. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Jones, BethAnna – PersonEntity: Name: NameFull: Snyder, Lawrence – PersonEntity: Name: NameFull: Ching, ShiNung IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 08997667 Numbering: – Type: volume Value: 36 – Type: issue Value: 5 Titles: – TitleFull: Neural Computation Type: main |
| ResultId | 1 |
