OPTIMIZING HADOOP DATA LOCALITY: PERFORMANCE ENHANCEMENT STRATEGIES IN HETEROGENEOUS COMPUTING ENVIRONMENTS.
Saved in:
| Title: | OPTIMIZING HADOOP DATA LOCALITY: PERFORMANCE ENHANCEMENT STRATEGIES IN HETEROGENEOUS COMPUTING ENVIRONMENTS. |
|---|---|
| Authors: | SI-YEONG KIM1 siyeong23@jnu.ac.kr, TAI-HOON KIM1 taihoonn@chonnam.ac.kr |
| Source: | Scalable Computing: Practice & Experience. Nov2024, Vol. 25 Issue 6, p4558-4575. 18p. |
| Subjects: | Machine learning, Heterogeneous distributed computing, Heterogeneous computing, Distributed computing, Electronic data processing |
| Abstract: | As organizations increasingly harness big data for analytics and decision-making, the efficient processing of massive datasets becomes paramount. Hadoop, a widely adopted distributed computing framework, excels in processing large-scale data. However, its performance is contingent on effective data locality, which becomes challenging in heterogeneous computing environments comprising diverse hardware resources. This research addresses the imperative of enhancing Hadoop's data locality performance in heterogeneous computing environments. The study explores strategies to optimize data placement and task scheduling, considering the diverse characteristics of nodes within the infrastructure. Through a comprehensive analysis of Hadoop's data locality algorithms and their impact on performance, this work proposes novel approaches to mitigate challenges associated with disparate hardware capabilities. Weighted Extreme Learning Machine Technique (Weighted ELM) with the Firefly Algorithm (WELM-FF) is used in the proposed work. The integration of Weighted Extreme Learning Machine (WELM) with the Firefly Algorithm holds promise for enhancing machine learning models in the context of large-scale data processing. The research employs a combination of theoretical analysis and practical experiments to evaluate the effectiveness of the proposed enhancements. Factors such as network latency, disk I/O, and CPU capabilities are taken into account to develop a holistic framework for improving data locality and, consequently, overall Hadoop performance. The findings presented in this study contribute valuable insights to the field of distributed computing, offering practical recommendations for organizations seeking to maximize the efficiency of their Hadoop deployments in heterogeneous computing environments. By addressing the intricacies of data locality, this research strives to enhance the scalability and performance of Hadoop clusters, thereby facilitating more effective utilization of big data resources. [ABSTRACT FROM AUTHOR] |
| Copyright of Scalable Computing: Practice & Experience is the property of Scalable Computing: Practice & Experience and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 180063248 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: OPTIMIZING HADOOP DATA LOCALITY: PERFORMANCE ENHANCEMENT STRATEGIES IN HETEROGENEOUS COMPUTING ENVIRONMENTS. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22SI-YEONG+KIM%22">SI-YEONG KIM</searchLink><relatesTo>1</relatesTo><i> siyeong23@jnu.ac.kr</i><br /><searchLink fieldCode="AR" term="%22TAI-HOON+KIM%22">TAI-HOON KIM</searchLink><relatesTo>1</relatesTo><i> taihoonn@chonnam.ac.kr</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Scalable+Computing%3A+Practice+%26+Experience%22">Scalable Computing: Practice & Experience</searchLink>. Nov2024, Vol. 25 Issue 6, p4558-4575. 18p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Heterogeneous+distributed+computing%22">Heterogeneous distributed computing</searchLink><br /><searchLink fieldCode="DE" term="%22Heterogeneous+computing%22">Heterogeneous computing</searchLink><br /><searchLink fieldCode="DE" term="%22Distributed+computing%22">Distributed computing</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+data+processing%22">Electronic data processing</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: As organizations increasingly harness big data for analytics and decision-making, the efficient processing of massive datasets becomes paramount. Hadoop, a widely adopted distributed computing framework, excels in processing large-scale data. However, its performance is contingent on effective data locality, which becomes challenging in heterogeneous computing environments comprising diverse hardware resources. This research addresses the imperative of enhancing Hadoop's data locality performance in heterogeneous computing environments. The study explores strategies to optimize data placement and task scheduling, considering the diverse characteristics of nodes within the infrastructure. Through a comprehensive analysis of Hadoop's data locality algorithms and their impact on performance, this work proposes novel approaches to mitigate challenges associated with disparate hardware capabilities. Weighted Extreme Learning Machine Technique (Weighted ELM) with the Firefly Algorithm (WELM-FF) is used in the proposed work. The integration of Weighted Extreme Learning Machine (WELM) with the Firefly Algorithm holds promise for enhancing machine learning models in the context of large-scale data processing. The research employs a combination of theoretical analysis and practical experiments to evaluate the effectiveness of the proposed enhancements. Factors such as network latency, disk I/O, and CPU capabilities are taken into account to develop a holistic framework for improving data locality and, consequently, overall Hadoop performance. The findings presented in this study contribute valuable insights to the field of distributed computing, offering practical recommendations for organizations seeking to maximize the efficiency of their Hadoop deployments in heterogeneous computing environments. By addressing the intricacies of data locality, this research strives to enhance the scalability and performance of Hadoop clusters, thereby facilitating more effective utilization of big data resources. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Scalable Computing: Practice & Experience is the property of Scalable Computing: Practice & Experience and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=180063248 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.12694/scpe.v25i6.3294 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 18 StartPage: 4558 Subjects: – SubjectFull: Machine learning Type: general – SubjectFull: Heterogeneous distributed computing Type: general – SubjectFull: Heterogeneous computing Type: general – SubjectFull: Distributed computing Type: general – SubjectFull: Electronic data processing Type: general Titles: – TitleFull: OPTIMIZING HADOOP DATA LOCALITY: PERFORMANCE ENHANCEMENT STRATEGIES IN HETEROGENEOUS COMPUTING ENVIRONMENTS. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: SI-YEONG KIM – PersonEntity: Name: NameFull: TAI-HOON KIM IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 11 Text: Nov2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 18951767 Numbering: – Type: volume Value: 25 – Type: issue Value: 6 Titles: – TitleFull: Scalable Computing: Practice & Experience Type: main |
| ResultId | 1 |
