Stream ETL framework for twitter-based sentiment analysis: Leveraging big data technologies.
Saved in:
| Title: | Stream ETL framework for twitter-based sentiment analysis: Leveraging big data technologies. |
|---|---|
| Authors: | Ismail, Azlan1,2 (AUTHOR) azlanismail@uitm.edu.my, Sazali, Faris Haziq1 (AUTHOR) 2021118083@student.uitm.edu.my, Agos Jawaddi, Siti Nuraishah1 (AUTHOR) 2021388277@student.uitm.edu.my, Mutalib, Sofianita1 (AUTHOR) sofianita@uitm.edu.my |
| Source: | Expert Systems with Applications. Feb2025, Vol. 261, pN.PAG-N.PAG. 1p. |
| Subjects: | X Corp., Sentiment analysis, Electronic data processing, Public opinion, Data warehousing, Scalability |
| Abstract: | Twitter has emerged as a rich source of real-time data, providing valuable insights across various domains such as healthcare and business. Sentiment analysis is crucial in understanding public reactions and sentiments expressed on Twitter, empowering organizations to make informed decisions. However, efficiently analyzing sentiment from social media data, presents a challenge for real-time streaming. Conventional Extract-Transform-Load (ETL) methods, inadequate for this challenge, limit their applicability in processing vast volumes of Twitter data. Big data technologies like Kafka, Spark, Hadoop HDFS, Hive, and HBase have become indispensable in addressing this challenge. To address this, we propose a stream ETL framework for Twitter-based sentiment analysis, leveraging Kafka, Spark, Cassandra, HBase, Hive, and HDFS. Our framework enables data stream processing, bias detection and correction, sentiment-based analysis, and visualization of tweets' geospatial distribution. We present a set of use case studies to illustrate the applicability of the proposed framework comprises of sentiment classification, and bias detection and correction capability. We also present a comparative study to demonstrate the performance of data streaming processing, analysis, and visualization that has been implemented using multiple big data technologies under different parameter settings. Experimental results demonstrate the framework scalability and trade-off factors of the data stream processing pipeline in the execution of big data processing tasks. • A framework with bias detection and correction, sentiment analysis and visualization • Integration of stream ETL process with big data computing and storage technologies • Case studies and experiments showcasing framework usage and performance evaluation [ABSTRACT FROM AUTHOR] |
| Copyright of Expert Systems with Applications is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 181196770 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Stream ETL framework for twitter-based sentiment analysis: Leveraging big data technologies. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ismail%2C+Azlan%22">Ismail, Azlan</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> azlanismail@uitm.edu.my</i><br /><searchLink fieldCode="AR" term="%22Sazali%2C+Faris+Haziq%22">Sazali, Faris Haziq</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> 2021118083@student.uitm.edu.my</i><br /><searchLink fieldCode="AR" term="%22Agos+Jawaddi%2C+Siti+Nuraishah%22">Agos Jawaddi, Siti Nuraishah</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> 2021388277@student.uitm.edu.my</i><br /><searchLink fieldCode="AR" term="%22Mutalib%2C+Sofianita%22">Mutalib, Sofianita</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> sofianita@uitm.edu.my</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Expert+Systems+with+Applications%22">Expert Systems with Applications</searchLink>. Feb2025, Vol. 261, pN.PAG-N.PAG. 1p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22X+Corp%2E%22">X Corp.</searchLink><br /><searchLink fieldCode="DE" term="%22Sentiment+analysis%22">Sentiment analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+data+processing%22">Electronic data processing</searchLink><br /><searchLink fieldCode="DE" term="%22Public+opinion%22">Public opinion</searchLink><br /><searchLink fieldCode="DE" term="%22Data+warehousing%22">Data warehousing</searchLink><br /><searchLink fieldCode="DE" term="%22Scalability%22">Scalability</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Twitter has emerged as a rich source of real-time data, providing valuable insights across various domains such as healthcare and business. Sentiment analysis is crucial in understanding public reactions and sentiments expressed on Twitter, empowering organizations to make informed decisions. However, efficiently analyzing sentiment from social media data, presents a challenge for real-time streaming. Conventional Extract-Transform-Load (ETL) methods, inadequate for this challenge, limit their applicability in processing vast volumes of Twitter data. Big data technologies like Kafka, Spark, Hadoop HDFS, Hive, and HBase have become indispensable in addressing this challenge. To address this, we propose a stream ETL framework for Twitter-based sentiment analysis, leveraging Kafka, Spark, Cassandra, HBase, Hive, and HDFS. Our framework enables data stream processing, bias detection and correction, sentiment-based analysis, and visualization of tweets' geospatial distribution. We present a set of use case studies to illustrate the applicability of the proposed framework comprises of sentiment classification, and bias detection and correction capability. We also present a comparative study to demonstrate the performance of data streaming processing, analysis, and visualization that has been implemented using multiple big data technologies under different parameter settings. Experimental results demonstrate the framework scalability and trade-off factors of the data stream processing pipeline in the execution of big data processing tasks. • A framework with bias detection and correction, sentiment analysis and visualization • Integration of stream ETL process with big data computing and storage technologies • Case studies and experiments showcasing framework usage and performance evaluation [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Expert Systems with Applications is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=181196770 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.eswa.2024.125523 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 1 StartPage: N.PAG Subjects: – SubjectFull: X Corp. Type: general – SubjectFull: Sentiment analysis Type: general – SubjectFull: Electronic data processing Type: general – SubjectFull: Public opinion Type: general – SubjectFull: Data warehousing Type: general – SubjectFull: Scalability Type: general Titles: – TitleFull: Stream ETL framework for twitter-based sentiment analysis: Leveraging big data technologies. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ismail, Azlan – PersonEntity: Name: NameFull: Sazali, Faris Haziq – PersonEntity: Name: NameFull: Agos Jawaddi, Siti Nuraishah – PersonEntity: Name: NameFull: Mutalib, Sofianita IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 02 Text: Feb2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 09574174 Numbering: – Type: volume Value: 261 Titles: – TitleFull: Expert Systems with Applications Type: main |
| ResultId | 1 |
