View in EDS

SEIZE: Runtime Inspection for Parallel Dataflow Systems.

Saved in:
Bibliographic Details
Title: SEIZE: Runtime Inspection for Parallel Dataflow Systems.
Authors: Li, Youfu1 youfuli@cs.ucla.edu, Interlandi, Matteo2 mainterl@microsoft.com, Psallidas, Fotis2 fotis.psallidas@microsoft.com, Wang, Wei1 weiwang@cs.ucla.edu, Zaniolo, Carlo1 zaniolo@cs.ucla.edu
Source: IEEE Transactions on Parallel & Distributed Systems. Apr2021, Vol. 32 Issue 4, p842-854. 13p.
Subjects: Computer workstation clusters, Task analysis
Abstract: Many Data-Intensive Scalable Computing (DISC) Systems provide easy-to-use functional APIs, and efficient scheduling and execution strategies allowing users to build concise data-parallel programs. In these systems, data transformations are concealed by exposed APIs, and intermediate execution states are masked under dataflow transitions. Consequently, many crucial features and optimizations (e.g., debugging, data provenance, runtime skew detection), which require runtime datafow states, are not well-supported. Inspired by our experience in implementing features and optimizations over DISC systems, we present $\mathsf {SEIZE}$ SEIZE , a unified framework that enables dataflow inspection—wiretapping the data-path with listening logic—in MapReduce-style programming model. We generalize our lessons learned by providing a set of primitives defining dataflow inspection, orchestration options for different inspection granularities, and operator decomposition and dataflow punctuation strategy for dataflow intervention. We demonstrate the generality and flexibility of the approach by deploying $\mathsf {SEIZE}$ SEIZE in both Apache Spark and Apache Flink, and by implementing a prototype runtime query optimizer for Spark. Our experiments show that, the overhead introduced by the inspection logic is most of the time negligible (less than 5 percent in Spark and 10 percent in Flink). [ABSTRACT FROM AUTHOR]
Copyright of IEEE Transactions on Parallel & Distributed Systems is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 147041357
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: SEIZE: Runtime Inspection for Parallel Dataflow Systems.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Li%2C+Youfu%22">Li, Youfu</searchLink><relatesTo>1</relatesTo><i> youfuli@cs.ucla.edu</i><br /><searchLink fieldCode="AR" term="%22Interlandi%2C+Matteo%22">Interlandi, Matteo</searchLink><relatesTo>2</relatesTo><i> mainterl@microsoft.com</i><br /><searchLink fieldCode="AR" term="%22Psallidas%2C+Fotis%22">Psallidas, Fotis</searchLink><relatesTo>2</relatesTo><i> fotis.psallidas@microsoft.com</i><br /><searchLink fieldCode="AR" term="%22Wang%2C+Wei%22">Wang, Wei</searchLink><relatesTo>1</relatesTo><i> weiwang@cs.ucla.edu</i><br /><searchLink fieldCode="AR" term="%22Zaniolo%2C+Carlo%22">Zaniolo, Carlo</searchLink><relatesTo>1</relatesTo><i> zaniolo@cs.ucla.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22IEEE+Transactions+on+Parallel+%26+Distributed+Systems%22">IEEE Transactions on Parallel & Distributed Systems</searchLink>. Apr2021, Vol. 32 Issue 4, p842-854. 13p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+workstation+clusters%22">Computer workstation clusters</searchLink><br /><searchLink fieldCode="DE" term="%22Task+analysis%22">Task analysis</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Many Data-Intensive Scalable Computing (DISC) Systems provide easy-to-use functional APIs, and efficient scheduling and execution strategies allowing users to build concise data-parallel programs. In these systems, data transformations are concealed by exposed APIs, and intermediate execution states are masked under dataflow transitions. Consequently, many crucial features and optimizations (e.g., debugging, data provenance, runtime skew detection), which require runtime datafow states, are not well-supported. Inspired by our experience in implementing features and optimizations over DISC systems, we present $\mathsf {SEIZE}$ SEIZE , a unified framework that enables dataflow inspection—wiretapping the data-path with listening logic—in MapReduce-style programming model. We generalize our lessons learned by providing a set of primitives defining dataflow inspection, orchestration options for different inspection granularities, and operator decomposition and dataflow punctuation strategy for dataflow intervention. We demonstrate the generality and flexibility of the approach by deploying $\mathsf {SEIZE}$ SEIZE in both Apache Spark and Apache Flink, and by implementing a prototype runtime query optimizer for Spark. Our experiments show that, the overhead introduced by the inspection logic is most of the time negligible (less than 5 percent in Spark and 10 percent in Flink). [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of IEEE Transactions on Parallel & Distributed Systems is the property of IEEE and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=147041357
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/TPDS.2020.3035170
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 842
    Subjects:
      – SubjectFull: Computer workstation clusters
        Type: general
      – SubjectFull: Task analysis
        Type: general
    Titles:
      – TitleFull: SEIZE: Runtime Inspection for Parallel Dataflow Systems.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Li, Youfu
      – PersonEntity:
          Name:
            NameFull: Interlandi, Matteo
      – PersonEntity:
          Name:
            NameFull: Psallidas, Fotis
      – PersonEntity:
          Name:
            NameFull: Wang, Wei
      – PersonEntity:
          Name:
            NameFull: Zaniolo, Carlo
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Text: Apr2021
              Type: published
              Y: 2021
          Identifiers:
            – Type: issn-print
              Value: 10459219
          Numbering:
            – Type: volume
              Value: 32
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: IEEE Transactions on Parallel & Distributed Systems
              Type: main
ResultId 1