View in EDS

More Powerful A/B Testing Using Auxiliary Data and Deep Learning

Saved in:
Bibliographic Details
Title: More Powerful A/B Testing Using Auxiliary Data and Deep Learning
Language: English
Authors: Sales, Adam C., Prihar, Ethan, Gagnon-Bartsch, Johann, Gurung, Ashish, Heffernan, Neil T.
Source: Grantee Submission. 2022.
Peer Reviewed: Y
Page Count: 4
Publication Date: 2022
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305D210031
Document Type: Speeches/Meeting Papers
Reports - Research
Descriptors: Research Methodology, Educational Experiments, Causal Models, Computation, Electronic Learning, Statistical Inference, Data Analysis, Accuracy
DOI: 10.1007/978-3-031-11647-6_107
Abstract: Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and unbiased without additional assumptions, regardless of the deep-learning model's quality. In this dataset, incorporating auxiliary data improves precision consistently and, in some cases, substantially.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2023
Accession Number: ED627066
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED627066
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: More Powerful A/B Testing Using Auxiliary Data and Deep Learning
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Sales%2C+Adam+C%2E%22">Sales, Adam C.</searchLink><br /><searchLink fieldCode="AR" term="%22Prihar%2C+Ethan%22">Prihar, Ethan</searchLink><br /><searchLink fieldCode="AR" term="%22Gagnon-Bartsch%2C+Johann%22">Gagnon-Bartsch, Johann</searchLink><br /><searchLink fieldCode="AR" term="%22Gurung%2C+Ashish%22">Gurung, Ashish</searchLink><br /><searchLink fieldCode="AR" term="%22Heffernan%2C+Neil+T%2E%22">Heffernan, Neil T.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2022.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 4
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305D210031
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Research+Methodology%22">Research Methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Experiments%22">Educational Experiments</searchLink><br /><searchLink fieldCode="DE" term="%22Causal+Models%22">Causal Models</searchLink><br /><searchLink fieldCode="DE" term="%22Computation%22">Computation</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Learning%22">Electronic Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Inference%22">Statistical Inference</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/978-3-031-11647-6_107
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and unbiased without additional assumptions, regardless of the deep-learning model's quality. In this dataset, incorporating auxiliary data improves precision consistently and, in some cases, substantially.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED627066
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED627066
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/978-3-031-11647-6_107
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 4
    Subjects:
      – SubjectFull: Research Methodology
        Type: general
      – SubjectFull: Educational Experiments
        Type: general
      – SubjectFull: Causal Models
        Type: general
      – SubjectFull: Computation
        Type: general
      – SubjectFull: Electronic Learning
        Type: general
      – SubjectFull: Statistical Inference
        Type: general
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Accuracy
        Type: general
    Titles:
      – TitleFull: More Powerful A/B Testing Using Auxiliary Data and Deep Learning
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Sales, Adam C.
      – PersonEntity:
          Name:
            NameFull: Prihar, Ethan
      – PersonEntity:
          Name:
            NameFull: Gagnon-Bartsch, Johann
      – PersonEntity:
          Name:
            NameFull: Gurung, Ashish
      – PersonEntity:
          Name:
            NameFull: Heffernan, Neil T.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 25
              M: 07
              Type: published
              Y: 2022
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1