View in EDS

Stream ETL framework for twitter-based sentiment analysis: Leveraging big data technologies.

Saved in:
Bibliographic Details
Title: Stream ETL framework for twitter-based sentiment analysis: Leveraging big data technologies.
Authors: Ismail, Azlan1,2 (AUTHOR) azlanismail@uitm.edu.my, Sazali, Faris Haziq1 (AUTHOR) 2021118083@student.uitm.edu.my, Agos Jawaddi, Siti Nuraishah1 (AUTHOR) 2021388277@student.uitm.edu.my, Mutalib, Sofianita1 (AUTHOR) sofianita@uitm.edu.my
Source: Expert Systems with Applications. Feb2025, Vol. 261, pN.PAG-N.PAG. 1p.
Subjects: X Corp., Sentiment analysis, Electronic data processing, Public opinion, Data warehousing, Scalability
Abstract: Twitter has emerged as a rich source of real-time data, providing valuable insights across various domains such as healthcare and business. Sentiment analysis is crucial in understanding public reactions and sentiments expressed on Twitter, empowering organizations to make informed decisions. However, efficiently analyzing sentiment from social media data, presents a challenge for real-time streaming. Conventional Extract-Transform-Load (ETL) methods, inadequate for this challenge, limit their applicability in processing vast volumes of Twitter data. Big data technologies like Kafka, Spark, Hadoop HDFS, Hive, and HBase have become indispensable in addressing this challenge. To address this, we propose a stream ETL framework for Twitter-based sentiment analysis, leveraging Kafka, Spark, Cassandra, HBase, Hive, and HDFS. Our framework enables data stream processing, bias detection and correction, sentiment-based analysis, and visualization of tweets' geospatial distribution. We present a set of use case studies to illustrate the applicability of the proposed framework comprises of sentiment classification, and bias detection and correction capability. We also present a comparative study to demonstrate the performance of data streaming processing, analysis, and visualization that has been implemented using multiple big data technologies under different parameter settings. Experimental results demonstrate the framework scalability and trade-off factors of the data stream processing pipeline in the execution of big data processing tasks. • A framework with bias detection and correction, sentiment analysis and visualization • Integration of stream ETL process with big data computing and storage technologies • Case studies and experiments showcasing framework usage and performance evaluation [ABSTRACT FROM AUTHOR]
Copyright of Expert Systems with Applications is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Twitter has emerged as a rich source of real-time data, providing valuable insights across various domains such as healthcare and business. Sentiment analysis is crucial in understanding public reactions and sentiments expressed on Twitter, empowering organizations to make informed decisions. However, efficiently analyzing sentiment from social media data, presents a challenge for real-time streaming. Conventional Extract-Transform-Load (ETL) methods, inadequate for this challenge, limit their applicability in processing vast volumes of Twitter data. Big data technologies like Kafka, Spark, Hadoop HDFS, Hive, and HBase have become indispensable in addressing this challenge. To address this, we propose a stream ETL framework for Twitter-based sentiment analysis, leveraging Kafka, Spark, Cassandra, HBase, Hive, and HDFS. Our framework enables data stream processing, bias detection and correction, sentiment-based analysis, and visualization of tweets' geospatial distribution. We present a set of use case studies to illustrate the applicability of the proposed framework comprises of sentiment classification, and bias detection and correction capability. We also present a comparative study to demonstrate the performance of data streaming processing, analysis, and visualization that has been implemented using multiple big data technologies under different parameter settings. Experimental results demonstrate the framework scalability and trade-off factors of the data stream processing pipeline in the execution of big data processing tasks. • A framework with bias detection and correction, sentiment analysis and visualization • Integration of stream ETL process with big data computing and storage technologies • Case studies and experiments showcasing framework usage and performance evaluation [ABSTRACT FROM AUTHOR]
ISSN:09574174
DOI:10.1016/j.eswa.2024.125523