Generative Multimodal Models for Social Science: An Application with Satellite and Streetscape Imagery
Saved in:
| Title: | Generative Multimodal Models for Social Science: An Application with Satellite and Streetscape Imagery |
|---|---|
| Language: | English |
| Authors: | Tina Law (ORCID |
| Source: | Sociological Methods & Research. 2025 54(3):889-932. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 44 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Descriptive |
| Descriptors: | Artificial Intelligence, Visual Aids, Open Source Technology, Social Science Research, Neighborhood Integration, Data Analysis, Research Problems |
| DOI: | 10.1177/00491241251339673 |
| ISSN: | 0049-1241 1552-8294 |
| Abstract: | Although there is growing social science research examining how generative AI models can be effectively and systematically applied to text-based tasks, whether and how these models can be used to analyze images remain open questions. In this article, we introduce a framework for analyzing images with generative multimodal models, which consists of three core tasks: curation, discovery, and measurement and inference. We demonstrate this framework with an empirical application that uses OpenAI's GPT-4o model to analyze satellite and streetscape images (n = 1,101) to identify built environment features that contribute to contemporary residential segregation in U.S. cities. We find that when GPT-4o is provided with well-defined image labels, the model labels images with high validity compared to expert labels. We conclude with thoughts for other use cases and discuss how social scientists can work collaboratively to ensure that image analysis with generative multimodal models is rigorous, reproducible, ethical, and sustainable. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1475739 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Although there is growing social science research examining how generative AI models can be effectively and systematically applied to text-based tasks, whether and how these models can be used to analyze images remain open questions. In this article, we introduce a framework for analyzing images with generative multimodal models, which consists of three core tasks: curation, discovery, and measurement and inference. We demonstrate this framework with an empirical application that uses OpenAI's GPT-4o model to analyze satellite and streetscape images (n = 1,101) to identify built environment features that contribute to contemporary residential segregation in U.S. cities. We find that when GPT-4o is provided with well-defined image labels, the model labels images with high validity compared to expert labels. We conclude with thoughts for other use cases and discuss how social scientists can work collaboratively to ensure that image analysis with generative multimodal models is rigorous, reproducible, ethical, and sustainable. |
|---|---|
| ISSN: | 0049-1241 1552-8294 |
| DOI: | 10.1177/00491241251339673 |
