Performance optimization of computing task scheduling based on the Hadoop big data platform.
Saved in:
| Title: | Performance optimization of computing task scheduling based on the Hadoop big data platform. |
|---|---|
| Authors: | Li, Yang1 (AUTHOR) liyang_walker@stu.xaut.edu.cn, Hei, Xinhong1 (AUTHOR) heixinhong@xaut.edu.cn |
| Source: | Neural Computing & Applications. May2025, Vol. 37 Issue 13, p8181-8192. 12p. |
| Subjects: | Distributed computing, Systems software, Electronic data processing, Computing platforms, Big data |
| Abstract: | Hadoop, a distributed computing framework that can efficiently process large-scale datasets, has been used by an increasing number of organizations as the basic computing framework to build cloud computing platforms. Improving its execution efficiency is a hot research direction in the industry, and the scheduling problem is a key factor affecting the execution efficiency of Hadoop. It is very important to identify its shortcomings and improve them. This paper examines and analyses the optimization of computing task scheduling performance based on the Hadoop big data platform. This paper first analyses Hadoop big data processing. Hadoop has high scalability. Computing nodes can be added at any time, and they can participate in cluster work through simple configuration. The paper discusses the improvement in the Hadoop resource scheduling algorithm. The task scheduling algorithm in the Hadoop-based data task localization proposed in this paper is compared with the default algorithm used in the Hadoop task scheduling algorithm. The former shows better local data in all four jobs, there are more data localization tasks, and the expected goal is achieved. The effectiveness of the algorithm is verified, and the performance is improved by 30%. [ABSTRACT FROM AUTHOR] |
| Copyright of Neural Computing & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 184671163 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Performance optimization of computing task scheduling based on the Hadoop big data platform. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Li%2C+Yang%22">Li, Yang</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> liyang_walker@stu.xaut.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Hei%2C+Xinhong%22">Hei, Xinhong</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> heixinhong@xaut.edu.cn</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Neural+Computing+%26+Applications%22">Neural Computing & Applications</searchLink>. May2025, Vol. 37 Issue 13, p8181-8192. 12p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Distributed+computing%22">Distributed computing</searchLink><br /><searchLink fieldCode="DE" term="%22Systems+software%22">Systems software</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+data+processing%22">Electronic data processing</searchLink><br /><searchLink fieldCode="DE" term="%22Computing+platforms%22">Computing platforms</searchLink><br /><searchLink fieldCode="DE" term="%22Big+data%22">Big data</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Hadoop, a distributed computing framework that can efficiently process large-scale datasets, has been used by an increasing number of organizations as the basic computing framework to build cloud computing platforms. Improving its execution efficiency is a hot research direction in the industry, and the scheduling problem is a key factor affecting the execution efficiency of Hadoop. It is very important to identify its shortcomings and improve them. This paper examines and analyses the optimization of computing task scheduling performance based on the Hadoop big data platform. This paper first analyses Hadoop big data processing. Hadoop has high scalability. Computing nodes can be added at any time, and they can participate in cluster work through simple configuration. The paper discusses the improvement in the Hadoop resource scheduling algorithm. The task scheduling algorithm in the Hadoop-based data task localization proposed in this paper is compared with the default algorithm used in the Hadoop task scheduling algorithm. The former shows better local data in all four jobs, there are more data localization tasks, and the expected goal is achieved. The effectiveness of the algorithm is verified, and the performance is improved by 30%. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Neural Computing & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=184671163 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s00521-022-08114-3 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 12 StartPage: 8181 Subjects: – SubjectFull: Distributed computing Type: general – SubjectFull: Systems software Type: general – SubjectFull: Electronic data processing Type: general – SubjectFull: Computing platforms Type: general – SubjectFull: Big data Type: general Titles: – TitleFull: Performance optimization of computing task scheduling based on the Hadoop big data platform. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Li, Yang – PersonEntity: Name: NameFull: Hei, Xinhong IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 09410643 Numbering: – Type: volume Value: 37 – Type: issue Value: 13 Titles: – TitleFull: Neural Computing & Applications Type: main |
| ResultId | 1 |
