AI-driven log reduction and storage optimization for security operations.

Saved in:

Bibliographic Details
Title:	AI-driven log reduction and storage optimization for security operations.
Authors:	Chalaemwongwan, Nutthakorn¹ nutthakorn.ch@kmitl.ac.th
Source:	International Journal of Electrical & Computer Engineering (2088-8708). Jun2026, Vol. 16 Issue 3, p1417-1424. 8p.
Subjects:	Anomaly detection (Computer security), Storage, Security management, Machine learning
Abstract:	In this study, we present an AI-driven framework that integrates semantic log reduction with compliance-aware storage optimization, specifically designed for security operations center (SOC) and managed security service provider (MSSP) environments. Traditional approaches such as uniform compression, keyword filtering, and static tiering often either miss critical anomalies or preserve redundant noise, leading to excessive storage use, slower search performance, and analyst fatigue. The proposed framework addresses these challenges by combining three components: semantic reduction of repetitive entries, anomaly-focused retention supported by self-supervised models, and adaptive tiering aligned with regulatory requirements. Evaluations on HDFS, BGL, CICIDS2017, and Suricata datasets achieved 70%-80% log reduction, 55%-65% storage savings, recall rates above 95%, and a one-third reduction in query latency. These results demonstrate that pre-index reduction, together with anomaly- and compliance-aware retention, offers a scalable and regulator-ready solution for operational security environments. [ABSTRACT FROM AUTHOR]
	Copyright of International Journal of Electrical & Computer Engineering (2088-8708) is the property of Institute of Advanced Engineering & Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	In this study, we present an AI-driven framework that integrates semantic log reduction with compliance-aware storage optimization, specifically designed for security operations center (SOC) and managed security service provider (MSSP) environments. Traditional approaches such as uniform compression, keyword filtering, and static tiering often either miss critical anomalies or preserve redundant noise, leading to excessive storage use, slower search performance, and analyst fatigue. The proposed framework addresses these challenges by combining three components: semantic reduction of repetitive entries, anomaly-focused retention supported by self-supervised models, and adaptive tiering aligned with regulatory requirements. Evaluations on HDFS, BGL, CICIDS2017, and Suricata datasets achieved 70%-80% log reduction, 55%-65% storage savings, recall rates above 95%, and a one-third reduction in query latency. These results demonstrate that pre-index reduction, together with anomaly- and compliance-aware retention, offers a scalable and regulator-ready solution for operational security environments. [ABSTRACT FROM AUTHOR]
ISSN:	20888708
DOI:	10.11591/ijece.v16i3.pp1417-1424