Deep Learning-Based Causal Inference for Large-Scale Combinatorial Experiments: Theory and Empirical Evidence.
Saved in:
| Authors: | Ye, Zikun1 (AUTHOR) zikunye@uw.edu, Zhang, Zhiqi2 (AUTHOR) z.zhiqi@wustl.edu, Zhang, Dennis J.2 (AUTHOR) denniszhang@wustl.edu, Zhang, Heng3 (AUTHOR) hengzhang24@asu.edu, Zhang, Renyu4 (AUTHOR) philipzhang@cuhk.edu.hk |
|---|---|
| Source: | Management Science (INFORMS). Jul2026, Vol. 72 Issue 7, p5611-5634. 24p. |
| Subject Terms: | *Digital platforms, *Multivariable testing, Deep learning, Causal inference |
| Abstract: | Large-scale online platforms launch hundreds of randomized experiments (also known as A/B tests) every day to iterate their operations and marketing strategies. The combinations of these treatments are typically not exhaustively tested, which triggers an important question of both academic and practical interest. Without observing the outcomes of all treatment combinations, how does one estimate the causal effect of any treatment combination and identify the optimal treatment combination? We develop a novel framework combining deep learning and doubly robust estimation to estimate the causal effect of any treatment combination for each user on the platform when observing only a small subset of treatment combinations. Our proposed framework (called debiased deep learning (DeDL)) exploits Neyman orthogonality and combines interpretable and flexible structural layers in deep learning. We show theoretically that this framework yields efficient, consistent, and asymptotically normal estimators under mild assumptions, thus allowing for identifying the best treatment combination when observing only a few combinations. To empirically validate our method, we collaborated with a large-scale video-sharing platform and implemented our framework for three experiments involving three treatments, where each combination of treatments is tested. When observing only a subset of treatment combinations, our DeDL approach significantly outperforms other benchmarks to accurately estimate and infer the average treatment effect of any treatment combination and to identify the optimal treatment combination. This paper was accepted by Vivek Farias, data science. Funding: R. Zhang is grateful for financial support from the Hong Kong Research Grants Council General Research Fund [Grants 14502722, 14503224, and 14504123]. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2024.04625. [ABSTRACT FROM AUTHOR] |
| Database: | Entrepreneurial Studies Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: ent DbLabel: Entrepreneurial Studies Source An: 195097789 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ye%2C+Zikun%22">Ye, Zikun</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> zikunye@uw.edu</i><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Zhiqi%22">Zhang, Zhiqi</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> z.zhiqi@wustl.edu</i><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Dennis+J%2E%22">Zhang, Dennis J.</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> denniszhang@wustl.edu</i><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Heng%22">Zhang, Heng</searchLink><relatesTo>3</relatesTo> (AUTHOR)<i> hengzhang24@asu.edu</i><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Renyu%22">Zhang, Renyu</searchLink><relatesTo>4</relatesTo> (AUTHOR)<i> philipzhang@cuhk.edu.hk</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Management+Science+%28INFORMS%29%22">Management Science (INFORMS)</searchLink>. Jul2026, Vol. 72 Issue 7, p5611-5634. 24p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Digital+platforms%22">Digital platforms</searchLink><br />*<searchLink fieldCode="DE" term="%22Multivariable+testing%22">Multivariable testing</searchLink><br /><searchLink fieldCode="DE" term="%22Deep+learning%22">Deep learning</searchLink><br /><searchLink fieldCode="DE" term="%22Causal+inference%22">Causal inference</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Large-scale online platforms launch hundreds of randomized experiments (also known as A/B tests) every day to iterate their operations and marketing strategies. The combinations of these treatments are typically not exhaustively tested, which triggers an important question of both academic and practical interest. Without observing the outcomes of all treatment combinations, how does one estimate the causal effect of any treatment combination and identify the optimal treatment combination? We develop a novel framework combining deep learning and doubly robust estimation to estimate the causal effect of any treatment combination for each user on the platform when observing only a small subset of treatment combinations. Our proposed framework (called debiased deep learning (DeDL)) exploits Neyman orthogonality and combines interpretable and flexible structural layers in deep learning. We show theoretically that this framework yields efficient, consistent, and asymptotically normal estimators under mild assumptions, thus allowing for identifying the best treatment combination when observing only a few combinations. To empirically validate our method, we collaborated with a large-scale video-sharing platform and implemented our framework for three experiments involving three treatments, where each combination of treatments is tested. When observing only a subset of treatment combinations, our DeDL approach significantly outperforms other benchmarks to accurately estimate and infer the average treatment effect of any treatment combination and to identify the optimal treatment combination. This paper was accepted by Vivek Farias, data science. Funding: R. Zhang is grateful for financial support from the Hong Kong Research Grants Council General Research Fund [Grants 14502722, 14503224, and 14504123]. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2024.04625. [ABSTRACT FROM AUTHOR] |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ent&AN=195097789 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1287/mnsc.2024.04625 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 24 StartPage: 5611 Subjects: – SubjectFull: Digital platforms Type: general – SubjectFull: Multivariable testing Type: general – SubjectFull: Deep learning Type: general – SubjectFull: Causal inference Type: general Titles: – TitleFull: Deep Learning-Based Causal Inference for Large-Scale Combinatorial Experiments: Theory and Empirical Evidence. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ye, Zikun – PersonEntity: Name: NameFull: Zhang, Zhiqi – PersonEntity: Name: NameFull: Zhang, Dennis J. – PersonEntity: Name: NameFull: Zhang, Heng – PersonEntity: Name: NameFull: Zhang, Renyu IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 07 Text: Jul2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 00251909 Numbering: – Type: volume Value: 72 – Type: issue Value: 7 Titles: – TitleFull: Management Science (INFORMS) Type: main |
| ResultId | 1 |
