From Codebooks to Promptbooks: Extracting Information from Text with Generative Large Language Models
Saved in:
| Title: | From Codebooks to Promptbooks: Extracting Information from Text with Generative Large Language Models |
|---|---|
| Language: | English |
| Authors: | Oscar Stuhler (ORCID |
| Source: | Sociological Methods & Research. 2025 54(3):794-848. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 55 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Evaluative |
| Descriptors: | Artificial Intelligence, Sociology, Social Science Research, Natural Language Processing, Information Processing, Biographies, Open Source Technology, Prompting, Cues |
| DOI: | 10.1177/00491241251336794 |
| ISSN: | 0049-1241 1552-8294 |
| Abstract: | Generative AI (GenAI) is quickly becoming a valuable tool for sociological research. Already, sociologists employ GenAI for tasks like classifying text and simulating human agents. We point to another major use case: the extraction of structured information from unstructured text. Information Extraction (IE) is an established branch of Natural Language Processing, but leveraging the affordances of this paradigm has thus far required familiarity with specialized models. GenAI changes this by allowing researchers to define their own IE tasks and execute them via targeted prompts. This article explores the potential of open-source large language models for IE by extracting and encoding biographical information (e.g., age, occupation, origin) from a corpus of newspaper obituaries. As we proceed, we discuss how sociologists can develop and evaluate prompt architectures for such tasks, turning codebooks into "promptbooks." We also evaluate models of different sizes and prompting techniques. Our analysis showcases the potential of GenAI as a flexible and accessible tool for IE while also underscoring risks like non-random error patterns that can bias downstream analyses. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1475720 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Generative AI (GenAI) is quickly becoming a valuable tool for sociological research. Already, sociologists employ GenAI for tasks like classifying text and simulating human agents. We point to another major use case: the extraction of structured information from unstructured text. Information Extraction (IE) is an established branch of Natural Language Processing, but leveraging the affordances of this paradigm has thus far required familiarity with specialized models. GenAI changes this by allowing researchers to define their own IE tasks and execute them via targeted prompts. This article explores the potential of open-source large language models for IE by extracting and encoding biographical information (e.g., age, occupation, origin) from a corpus of newspaper obituaries. As we proceed, we discuss how sociologists can develop and evaluate prompt architectures for such tasks, turning codebooks into "promptbooks." We also evaluate models of different sizes and prompting techniques. Our analysis showcases the potential of GenAI as a flexible and accessible tool for IE while also underscoring risks like non-random error patterns that can bias downstream analyses. |
|---|---|
| ISSN: | 0049-1241 1552-8294 |
| DOI: | 10.1177/00491241251336794 |
