Memory hierarchy in AI systems: dual contexts, estimation modelling, and optimisation strategies.

Saved in:

Bibliographic Details
Title:	Memory hierarchy in AI systems: dual contexts, estimation modelling, and optimisation strategies.
Authors:	Brahma, Manmohan¹ manmohan.brahma@gmail.com, Patra, G. K.^1,2
Source:	Current Science (00113891). 7/10/2026, Vol. 131 Issue 1, p64-76. 13p.
Subjects:	Computer memory management, Estimation theory, Artificial intelligence, Systems design
Abstract:	The rapid growth of large-scale artificial intelligence (AI) models has placed memory management at the centre of both hardware and software design challenges. The present study investigates and introduces the Memory Capacity Estimation Model (MCEM): a parametric, device-agnostic, and precision-aware analytical framework for predicting peak memory utilisation across central processing units, graphics processing units, and neural processing units. MCEM formalises a dual-context memory taxonomy that distinguishes training (bandwidth-bound, writedominated) from inference (latency-sensitive, key-value cache-dominated) based on dataflow characteristics, temporal write/read ratios, and reuse distance. Validated against open benchmarks from MLPerf, Hugging Face, and advanced micro devices Instinct MI300X hardware reports, MCEM achieves estimation errors of 0.73%-2.7% across five diverse model-device configurations. The framework is shown to reduce memory overprovisioning by 30%-40% relative to naive sizing, with implications for sustainable, cost-effective AI deployment at cloud, high-performance computing, and edge scales. [ABSTRACT FROM AUTHOR]
	Copyright of Current Science (00113891) is the property of Indian Academy of Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	The rapid growth of large-scale artificial intelligence (AI) models has placed memory management at the centre of both hardware and software design challenges. The present study investigates and introduces the Memory Capacity Estimation Model (MCEM): a parametric, device-agnostic, and precision-aware analytical framework for predicting peak memory utilisation across central processing units, graphics processing units, and neural processing units. MCEM formalises a dual-context memory taxonomy that distinguishes training (bandwidth-bound, writedominated) from inference (latency-sensitive, key-value cache-dominated) based on dataflow characteristics, temporal write/read ratios, and reuse distance. Validated against open benchmarks from MLPerf, Hugging Face, and advanced micro devices Instinct MI300X hardware reports, MCEM achieves estimation errors of 0.73%-2.7% across five diverse model-device configurations. The framework is shown to reduce memory overprovisioning by 30%-40% relative to naive sizing, with implications for sustainable, cost-effective AI deployment at cloud, high-performance computing, and edge scales. [ABSTRACT FROM AUTHOR]
ISSN:	00113891
DOI:	10.18520/cs/v131/i1/64-76