Job scheduler for streaming applications in heterogeneous distributed processing systems.
Saved in:
| Title: | Job scheduler for streaming applications in heterogeneous distributed processing systems. |
|---|---|
| Authors: | Al-Sinayyid, Ali1 (AUTHOR) ali.s@siu.edu, Zhu, Michelle2 (AUTHOR) |
| Source: | Journal of Supercomputing. Dec2020, Vol. 76 Issue 12, p9609-9628. 20p. |
| Subjects: | Distributed computing, Heterogeneous distributed computing, Bottlenecks (Manufacturing), Directed acyclic graphs, Databases, Algorithms |
| Abstract: | In this study, we investigated the problem of scheduling streaming applications on a heterogeneous cluster environment and, based on our previous work, developed the maximum throughput scheduler algorithm (MT-Scheduler) for streaming applications. The proposed algorithm uses a dynamic programming technique to efficiently map the application topology onto the heterogeneous distributed system based on computing and data transfer requirements, while also taking into account the capacity of the underlying cluster resources. The proposed approach maximizes the system throughput by identifying and minimizing the time incurred at the computing/transfer bottleneck. The MT-Scheduler supports scheduling applications structured as a directed acyclic graph. We conducted experiments using three Storm microbenchmark topologies in both simulation and real Apache Storm environments. In terms of the performance evaluation, we compared the proposed MT-Scheduler with the simulated round robin and the default Storm scheduler algorithms. The results indicated that the MT-Scheduler outperforms the default round robin approach in terms of both the average system latency and throughput. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Supercomputing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
Be the first to leave a comment!
