The Effects of Open-Ended Probes on Closed Survey Questions in Web Surveys
Saved in:
| Title: | The Effects of Open-Ended Probes on Closed Survey Questions in Web Surveys |
|---|---|
| Language: | English |
| Authors: | Patricia Hadler (ORCID |
| Source: | Sociological Methods & Research. 2025 54(1):106-139. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 34 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Online Surveys, Discovery Processes, Test Items, Data Collection, Cost Effectiveness, Questionnaires, Response Rates (Questionnaires), Cognitive Processes, Foreign Countries, Regional Characteristics, Age, Sex, Sampling, Research Methodology |
| Geographic Terms: | Germany, East Germany |
| DOI: | 10.1177/00491241231176846 |
| ISSN: | 0049-1241 1552-8294 |
| Abstract: | Probes are follow-ups to survey questions used to gain insights on respondents' understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued for embedding more open-ended probes in large-scale web surveys. However, there are concerns that this may cause reactivity and impact survey data. The study presents a randomized experiment in which identical survey questions were run with and without open-ended probes. Embedding open-ended probes resulted in higher levels of survey break off, as well as increased backtracking and answer changes to previous questions. In most cases, there was no impact of open-ended probes on the cognitive processing of and response to survey questions. Implications for embedding open-ended probes into web surveys are discussed. |
| Abstractor: | As Provided |
| Notes: | https://doi.org/10.6084/m9.figshare.22499557.v1 |
| Entry Date: | 2025 |
| Accession Number: | EJ1458200 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Probes are follow-ups to survey questions used to gain insights on respondents' understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued for embedding more open-ended probes in large-scale web surveys. However, there are concerns that this may cause reactivity and impact survey data. The study presents a randomized experiment in which identical survey questions were run with and without open-ended probes. Embedding open-ended probes resulted in higher levels of survey break off, as well as increased backtracking and answer changes to previous questions. In most cases, there was no impact of open-ended probes on the cognitive processing of and response to survey questions. Implications for embedding open-ended probes into web surveys are discussed. |
|---|---|
| ISSN: | 0049-1241 1552-8294 |
| DOI: | 10.1177/00491241231176846 |
