View in EDS HTML Full Text PDF Full Text

Job scheduler for streaming applications in heterogeneous distributed processing systems.

Saved in:
Bibliographic Details
Title: Job scheduler for streaming applications in heterogeneous distributed processing systems.
Authors: Al-Sinayyid, Ali1 (AUTHOR) ali.s@siu.edu, Zhu, Michelle2 (AUTHOR)
Source: Journal of Supercomputing. Dec2020, Vol. 76 Issue 12, p9609-9628. 20p.
Subjects: Distributed computing, Heterogeneous distributed computing, Bottlenecks (Manufacturing), Directed acyclic graphs, Databases, Algorithms
Abstract: In this study, we investigated the problem of scheduling streaming applications on a heterogeneous cluster environment and, based on our previous work, developed the maximum throughput scheduler algorithm (MT-Scheduler) for streaming applications. The proposed algorithm uses a dynamic programming technique to efficiently map the application topology onto the heterogeneous distributed system based on computing and data transfer requirements, while also taking into account the capacity of the underlying cluster resources. The proposed approach maximizes the system throughput by identifying and minimizing the time incurred at the computing/transfer bottleneck. The MT-Scheduler supports scheduling applications structured as a directed acyclic graph. We conducted experiments using three Storm microbenchmark topologies in both simulation and real Apache Storm environments. In terms of the performance evaluation, we compared the proposed MT-Scheduler with the simulated round robin and the default Storm scheduler algorithms. The results indicated that the MT-Scheduler outperforms the default round robin approach in terms of both the average system latency and throughput. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Supercomputing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 146367666
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Job scheduler for streaming applications in heterogeneous distributed processing systems.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Al-Sinayyid%2C+Ali%22">Al-Sinayyid, Ali</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> ali.s@siu.edu</i><br /><searchLink fieldCode="AR" term="%22Zhu%2C+Michelle%22">Zhu, Michelle</searchLink><relatesTo>2</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Supercomputing%22">Journal of Supercomputing</searchLink>. Dec2020, Vol. 76 Issue 12, p9609-9628. 20p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Distributed+computing%22">Distributed computing</searchLink><br /><searchLink fieldCode="DE" term="%22Heterogeneous+distributed+computing%22">Heterogeneous distributed computing</searchLink><br /><searchLink fieldCode="DE" term="%22Bottlenecks+%28Manufacturing%29%22">Bottlenecks (Manufacturing)</searchLink><br /><searchLink fieldCode="DE" term="%22Directed+acyclic+graphs%22">Directed acyclic graphs</searchLink><br /><searchLink fieldCode="DE" term="%22Databases%22">Databases</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In this study, we investigated the problem of scheduling streaming applications on a heterogeneous cluster environment and, based on our previous work, developed the maximum throughput scheduler algorithm (MT-Scheduler) for streaming applications. The proposed algorithm uses a dynamic programming technique to efficiently map the application topology onto the heterogeneous distributed system based on computing and data transfer requirements, while also taking into account the capacity of the underlying cluster resources. The proposed approach maximizes the system throughput by identifying and minimizing the time incurred at the computing/transfer bottleneck. The MT-Scheduler supports scheduling applications structured as a directed acyclic graph. We conducted experiments using three Storm microbenchmark topologies in both simulation and real Apache Storm environments. In terms of the performance evaluation, we compared the proposed MT-Scheduler with the simulated round robin and the default Storm scheduler algorithms. The results indicated that the MT-Scheduler outperforms the default round robin approach in terms of both the average system latency and throughput. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Supercomputing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=146367666
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11227-020-03223-z
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 9609
    Subjects:
      – SubjectFull: Distributed computing
        Type: general
      – SubjectFull: Heterogeneous distributed computing
        Type: general
      – SubjectFull: Bottlenecks (Manufacturing)
        Type: general
      – SubjectFull: Directed acyclic graphs
        Type: general
      – SubjectFull: Databases
        Type: general
      – SubjectFull: Algorithms
        Type: general
    Titles:
      – TitleFull: Job scheduler for streaming applications in heterogeneous distributed processing systems.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Al-Sinayyid, Ali
      – PersonEntity:
          Name:
            NameFull: Zhu, Michelle
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Text: Dec2020
              Type: published
              Y: 2020
          Identifiers:
            – Type: issn-print
              Value: 09208542
          Numbering:
            – Type: volume
              Value: 76
            – Type: issue
              Value: 12
          Titles:
            – TitleFull: Journal of Supercomputing
              Type: main
ResultId 1