View in EDS HTML Full Text PDF Full Text

Effects of Objective and Perceived Burden on Response Quality in Web Surveys

Saved in:
Bibliographic Details
Title: Effects of Objective and Perceived Burden on Response Quality in Web Surveys
Language: English
Authors: Tanja Kunz (ORCID 0000-0001-8460-2583), Tobias Gummer (ORCID 0000-0001-6469-7802)
Source: International Journal of Social Research Methodology. 2025 28(4):385-395.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 11
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Foreign Countries, Online Surveys, Response Rates (Questionnaires), Questionnaires, Response Style (Tests), Sequential Approach, Serial Ordering, Responsibility, Test Validity, Research Problems
Geographic Terms: Germany
DOI: 10.1080/13645579.2024.2393795
ISSN: 1364-5579
1464-5300
Abstract: Respondent burden is considered a decisive factor affecting response quality in web surveys. To investigate the objective and perceived burden on web survey respondents and its effects on response quality, we conducted a web survey among members of a German online access panel using a questionnaire with a completely randomized question order and measures of perceived burden at four time points. We found that perceived burden increased only slightly during the survey, suggesting that objective burden--as measured by the position of a question in the questionnaire--has a modest impact on the level of perceived burden. Although the position of a question in the questionnaire also had little effect on response quality, higher perceived burden consistently resulted in lower response quality.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1495786
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGM_EGCsS3oHS7FZ6Hx5zCBAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDKnPQ8tcl09nVNlzbwIBEICBmwm2hoA1oZDjVTSzGhWdUuMPm08gFpxqnaIHL0BuO9W8zwL6vAzbG1YmSfNWqmRAPra6z0EY9DU4ZN7ozm99dZJAEqPqNF1nuUvdU945vch9Xmtdj5xVFErgyDZGLO-EUa40nI_Osl3VabvDUxCVtWBpNp-hCA9xjj4F1-cGWvo-1NWL7W9H5u8b8Vbauv6f-ET3OInirAUFsZMf
Text:
  Availability: 1
  Value: <anid>AN0185986528;9eb01jul.25;2025Jun19.01:57;v2.2.500</anid> <title id="AN0185986528-1">Effects of objective and perceived burden on response quality in web surveys </title> <p>Respondent burden is considered a decisive factor affecting response quality in web surveys. To investigate the objective and perceived burden on web survey respondents and its effects on response quality, we conducted a web survey among members of a German online access panel using a questionnaire with a completely randomized question order and measures of perceived burden at four time points. We found that perceived burden increased only slightly during the survey, suggesting that objective burden―as measured by the position of a question in the questionnaire―has a modest impact on the level of perceived burden. Although the position of a question in the questionnaire also had little effect on response quality, higher perceived burden consistently resulted in lower response quality.</p> <p>Keywords: Web survey length; question order; respondent burden; response quality; satisficing</p> <hd id="AN0185986528-2">Introduction</hd> <p>Response quality in web surveys depends on respondents thoroughly processing the four steps of the cognitive response process, namely: 'comprehension of the item, retrieval of relevant information, use of that information to make required judgments, and selection and reporting of an answer' (Tourangeau et al., [<reflink idref="bib33" id="ref1">33</reflink>], p. 7). As interviewers are not available in web surveys to provide assistance and motivation, respondents who are unable or unmotivated to conscientiously perform all four steps may omit one or more steps and take cognitive shortcuts. This response behavior, termed 'satisficing' (Krosnick, [<reflink idref="bib20" id="ref2">20</reflink>], p. 215), negatively affects the quality of the responses. Forms of satisficing response behavior include, for example, leaving questions unanswered (Beatty & Herrmann, [<reflink idref="bib3" id="ref3">3</reflink>]); choosing the 'don't know' response option (Krosnick et al., [<reflink idref="bib21" id="ref4">21</reflink>]); selecting the same response option for all items of an item battery (i.e. straightlining) or speeding through the questionnaire to keep the cognitive effort low and to move quickly through the questionnaire (Zhang & Conrad, [<reflink idref="bib43" id="ref5">43</reflink>]). Respondents' motivation often wanes as the survey progresses because 'they are likely to become increasingly fatigued, disinterested, impatient, and distracted' (Krosnick, [<reflink idref="bib20" id="ref6">20</reflink>], p. 214), which increases the risk of satisficing. Thus, respondent burden may increase with each additional question the respondents answer, lowering their motivation to strive for complete and accurate responses (Galesic, [<reflink idref="bib11" id="ref7">11</reflink>]; Galesic & Bosnjak, [<reflink idref="bib12" id="ref8">12</reflink>]).</p> <p>Bradburn ([<reflink idref="bib5" id="ref9">5</reflink>]) was one of the first to emphasize that to maintain respondents' willingness to provide complete and accurate responses, it is important to ensure that they do not find the survey burdensome. Although there is no prevailing definition of respondent burden, theoretical models agree that it is not determined only by the length of the survey or the time required to complete it but also depends on whether respondents <emph>perceive</emph> the survey as burdensome (for an overview of different models of respondent burden, see Haraldsen, [<reflink idref="bib19" id="ref10">19</reflink>]; Yan & Williams, [<reflink idref="bib40" id="ref11">40</reflink>]). Research even suggests that objective survey characteristics (e.g. number of questions, content of questions, layout of the questionnaire) have little direct influence on respondent burden, and that respondents' subjective perceptions of the survey characteristics are the primary influencing factor (Galesic, [<reflink idref="bib11" id="ref12">11</reflink>]; Haraldsen, [<reflink idref="bib19" id="ref13">19</reflink>]; Yan & Williams, [<reflink idref="bib40" id="ref14">40</reflink>]; Yan et al., [<reflink idref="bib37" id="ref15">37</reflink>]). Therefore, a distinction is made between measures of objective burden, such as survey length, the number of questions or web pages, and the time respondents need to complete the questionnaire (e.g. Cook et al., [<reflink idref="bib7" id="ref16">7</reflink>]; Galesic & Bosnjak, [<reflink idref="bib12" id="ref17">12</reflink>]; Haraldsen, [<reflink idref="bib19" id="ref18">19</reflink>]; Rolstad et al., [<reflink idref="bib28" id="ref19">28</reflink>]), and perceived burden, which is unique to each respondent. Perceived burden is usually measured based on a self-reported assessment of specific survey characteristics, such as whether the questionnaire was 'interesting,' 'important,' 'easy or difficult to answer,' or 'too long' (e.g. Sharp & Frankel, [<reflink idref="bib31" id="ref20">31</reflink>]). Alternatively, respondents are asked how 'burdensome' they found answering individual questions (Galesic, [<reflink idref="bib11" id="ref21">11</reflink>]) or the survey as a whole (e.g. Dale & Haraldsen, [<reflink idref="bib8" id="ref22">8</reflink>]; Eisele et al., [<reflink idref="bib10" id="ref23">10</reflink>]; Yan et al., [<reflink idref="bib37" id="ref24">37</reflink>]).</p> <p>Previous research on the impact of objective burden on response quality in web surveys has yielded mixed findings depending on the indicators considered. Some studies have found more item nonresponse in the later parts of a questionnaire (Miller & Lambert, [<reflink idref="bib23" id="ref25">23</reflink>]), whereas others have found no effect of question positioning on item nonresponse (Andreadis & Kartsounidou, [<reflink idref="bib2" id="ref26">2</reflink>]; Galesic & Bosnjak, [<reflink idref="bib12" id="ref27">12</reflink>]; Neuert, [<reflink idref="bib24" id="ref28">24</reflink>]). In addition, studies have found more 'don't know' answers in longer or later parts of questionnaires (Deutskens et al., [<reflink idref="bib9" id="ref29">9</reflink>]; Wang et al., [<reflink idref="bib35" id="ref30">35</reflink>]) and more 'neither/nor' answers in longer questionnaires (Andreadis & Kartsounidou, [<reflink idref="bib2" id="ref31">2</reflink>]). Furthermore, whereas some studies have found less differentiated responses to item batteries later in a questionnaire (Galesic & Bosnjak, [<reflink idref="bib12" id="ref32">12</reflink>]; Neuert, [<reflink idref="bib24" id="ref33">24</reflink>]), others have found no differences in this regard (Andreadis & Kartsounidou, [<reflink idref="bib2" id="ref34">2</reflink>]; Bowling et al., [<reflink idref="bib4" id="ref35">4</reflink>]; Ganassali, [<reflink idref="bib13" id="ref36">13</reflink>]; Gibson & Bowling, [<reflink idref="bib14" id="ref37">14</reflink>]). Nor have differences been found in the internal consistency reliability, validity, or factor structure of item batteries based on their position in a questionnaire (Bowling et al., [<reflink idref="bib4" id="ref38">4</reflink>]). Some studies have found shorter responses to open-ended questions in later parts of or in longer questionnaires (Andreadis & Kartsounidou, [<reflink idref="bib2" id="ref39">2</reflink>]; Galesic & Bosnjak, [<reflink idref="bib12" id="ref40">12</reflink>]), whereas others have found the opposite (Ganassali, [<reflink idref="bib13" id="ref41">13</reflink>]) or no difference (Schmidt et al., [<reflink idref="bib29" id="ref42">29</reflink>]). Although most studies have found shorter response times for questions positioned later in the questionnaire (Bowling et al., [<reflink idref="bib4" id="ref43">4</reflink>]; Galesic & Bosnjak, [<reflink idref="bib12" id="ref44">12</reflink>]; Neuert, [<reflink idref="bib24" id="ref45">24</reflink>]; Schmidt et al., [<reflink idref="bib29" id="ref46">29</reflink>]; Wang et al., [<reflink idref="bib35" id="ref47">35</reflink>]; Yan & Tourangeau, [<reflink idref="bib39" id="ref48">39</reflink>]), some have found no differences at all (Andreadis & Kartsounidou, [<reflink idref="bib2" id="ref49">2</reflink>]). Regarding the relationship between objective and perceived burden in web surveys, Galesic ([<reflink idref="bib11" id="ref50">11</reflink>]) found a slight increase in perceived burden as the survey progressed: The more blocks of questions respondents answered, the higher their self-reported perceived burden. Similarly, Eisele et al. ([<reflink idref="bib10" id="ref51">10</reflink>]) found that respondents who answered a long questionnaire reported higher perceived burden than those receiving a shorter one.</p> <p>In the present study, we address four limitations of previous research on the relationship between respondent burden and response quality in web surveys. First, most studies have focused on objective burden. To our knowledge, no studies have examined the effects of perceived burden on response quality, and few studies have examined the relationship between objective and perceived burden. To address this limitation, we examine this relationship and the effects of objective and perceived burden on response quality.</p> <p>Second, most studies have manipulated objective burden by only changing the position of individual questions (i.e. by placing a question at the beginning or end of the questionnaire), randomly ordering blocks of thematically related questions, or increasing the length of the questionnaire (Andreadis & Kartsounidou, [<reflink idref="bib2" id="ref52">2</reflink>]; Bowling et al., [<reflink idref="bib4" id="ref53">4</reflink>]; Deutskens et al., [<reflink idref="bib9" id="ref54">9</reflink>]; Galesic, [<reflink idref="bib11" id="ref55">11</reflink>]; Galesic & Bosnjak, [<reflink idref="bib12" id="ref56">12</reflink>]; Ganassali, [<reflink idref="bib13" id="ref57">13</reflink>]; Gibson & Bowling, [<reflink idref="bib14" id="ref58">14</reflink>]; Neuert, [<reflink idref="bib24" id="ref59">24</reflink>]; Rolstad et al., [<reflink idref="bib28" id="ref60">28</reflink>]; Sharp & Frankel, [<reflink idref="bib31" id="ref61">31</reflink>]; Yan & Tourangeau, [<reflink idref="bib39" id="ref62">39</reflink>]; Yan et al., [<reflink idref="bib36" id="ref63">36</reflink>]). However, without completely randomizing the order of all questions, the effect of question position may be confounded by the order of other questions and their characteristics (e.g. content, format, complexity; Smyth et al., [<reflink idref="bib32" id="ref64">32</reflink>]). Further, as Yang and Toth ([<reflink idref="bib41" id="ref65">41</reflink>]) noted, 'it is important to account for other survey features or respondent characteristics when assessing the relationship between perceived burden and the objective measures of burden because these can affect a respondent's experience of burden' (p. 1130). Because it is impossible to control all survey features or respondent characteristics in a web survey, we completely randomized the order of all questions (i.e. each respondent received the questions in a different order) to obtain data that are as free as possible from confounding effects due to a fixed question order.</p> <p>The third limitation of previous studies is that they have focused on just a few response quality indicators. However, given the complexity and variety of the response behaviors that have to be identified, a comprehensive set of indicators is necessary (Greszki et al., [<reflink idref="bib15" id="ref66">15</reflink>]; Leiner, [<reflink idref="bib22" id="ref67">22</reflink>]). In addition, the interpretation of quality indicators is not always straightforward and unambiguous. For example, 'don't know' answers and short response times per se cannot be considered an expression of poor response quality (e.g. Krosnick et al., [<reflink idref="bib21" id="ref68">21</reflink>]; Leiner, [<reflink idref="bib22" id="ref69">22</reflink>]; Turner et al., [<reflink idref="bib34" id="ref70">34</reflink>]; Yan et al., [<reflink idref="bib38" id="ref71">38</reflink>]). Therefore, in our study, we used a comprehensive set of quality indicators and different question formats, such as single- or multiple-choice questions, rating scales, and open-ended questions (for a complete list of questions of the main questionnaire, see Table A1 in the Appendix).</p> <p>Fourth, most previous studies have measured perceived burden with a single question at the end of the questionnaire. To address this limitation, we measured perceived burden at four points in the questionnaire, allowing us to investigate changes in perceived burden while completing the questionnaire and the effects of perceived burden on response quality, independent of question characteristics.</p> <p>Our study aims to answer the following research questions:</p> <hd id="AN0185986528-3">RQ1:</hd> <p>What is the association between objective and perceived burden?</p> <hd id="AN0185986528-4">RQ2:</hd> <p>How does the level of objective burden impact response quality?</p> <hd id="AN0185986528-5">RQ3:</hd> <p>How does the level of perceived burden impact response quality?</p> <hd id="AN0185986528-6">Data and methods</hd> <p></p> <hd id="AN0185986528-7">Participants</hd> <p>We conducted a web survey in November and December 2018 among members of a large German online access panel provided by Bilendi & respondi. The target population comprised individuals aged 18–69 years living in households with Internet access in Germany. A quota sample based on gender, age, and educational attainment was used. We invited 2,704 panelists to participate, 311 of whom were screened out due to age restrictions or because the quotas were full. The participation rate (American Association for Public Opinion Research AAPOR, [<reflink idref="bib1" id="ref72">1</reflink>]) was 91.6% (<emph>n</emph> = 2,193), and the overall breakoff rate (Callegaro & DiSogra, [<reflink idref="bib6" id="ref73">6</reflink>]) was 4.4% (<emph>n</emph> = 101). Of those who completed the survey, 49.6% were female, the average age was 44 years (<emph>Mdn</emph> = 45), and 33.0% had a higher education entrance qualification. Panelists were told it would take 20–25 minutes to complete the survey, for which they received an incentive (panel points). To optimize the survey for mobile devices, we used a responsive questionnaire design where the layout dynamically adapted to different screen sizes. The mean completion time was 23.7 minutes (<emph>Mdn</emph> = 20.7); 21.5% of the respondents completed the web survey by smartphone, and 6.7% did so on a tablet device.</p> <hd id="AN0185986528-8">Questionnaire</hd> <p>The questionnaire consisted of three parts: (a) screening questions on the respondent's gender, age, and educational attainment level; (b) the main questionnaire; and (c) survey assessment questions (see Figure A1 in the Appendix). Responses to all questions were voluntary, and respondents could skip any questions they did not want to answer, except the screening questions.</p> <p>The main questionnaire comprised 28 web pages (see Table A1 in the Appendix), with one question presented per page (except for two knowledge questions presented on one page; item batteries were treated as one question). All web pages were presented to each respondent in random order. In addition, the main questionnaire was divided into four blocks of seven web pages each. After each block (i.e. after the 7th, 14th, 21st, and 28th web pages), we asked identical follow-up questions on different aspects of the response task, including our perceived burden measures (i.e. self-reports on difficulty, disengagement, fatigue). The instructions introducing the follow-up questions after the first block read: 'Please evaluate the questions you have been asked so far based on the following statements.' The instructions introducing the follow-up questions after the second, third, and fourth blocks read: 'Now please evaluate only the questions you were asked in the current question block (since the last evaluation).'</p> <p>We opted for block-wise evaluation for two reasons. First, respondents' perceived burden can be expected to fluctuate as they proceed through the different types of questions in a questionnaire, either leading to a monotonic increase or decrease in perceived burden or to a non-monotonic pattern with peaks and troughs (Read, [<reflink idref="bib25" id="ref74">25</reflink>]; Yan & Williams, [<reflink idref="bib40" id="ref75">40</reflink>]). To better capture these changes, respondents were asked to focus their evaluation on the question block just answered. Second, because of short-term memory and limited memory capacity, the most recently processed information is usually more easily accessible to respondents (Zaller & Feldman, [<reflink idref="bib42" id="ref76">42</reflink>]). Respondents were therefore asked to limit their evaluation to a manageable number of questions they had just answered and could remember well.</p> <hd id="AN0185986528-9">Measures of respondent burden</hd> <p>We operationalized objective burden based on the position of the question in the questionnaire. We created an independent variable indicating whether the question was positioned in Block #1, #2, #3, or #4. We measured perceived burden using respondents' self-reported difficulty ('The questions were difficult to answer.'), disengagement ('I got easily distracted while answering the questions.'), and fatigue ('Answering the questions was tiring.') (Krosnick, [<reflink idref="bib20" id="ref77">20</reflink>]; Sharp & Frankel, [<reflink idref="bib31" id="ref78">31</reflink>]). The three items were answered on a 5-point scale ranging from 1 (<emph>not at all</emph>) to 5 (<emph>very much</emph>). They were highly correlated (Pearson correlations: <emph>r</emph><subs>a–b</subs> =.635; <emph>r</emph><subs>a–c</subs> =.692; <emph>r</emph><subs>b–c</subs> =.657; non-parametric Spearman correlations: <emph>rs</emph><subs>a–b</subs> =.605; <emph>rs</emph><subs>a–c</subs> =.699; <emph>rs</emph><subs>b–c</subs> =.663); principal component analysis extracted one component (Cronbach's alpha =.853). We calculated a perceived burden index across the three items and the four measurement time points. The mean index takes values between 1 and 5. Using the mean (1.60) as a cutoff, we calculated a binary variable indicating for each respondent whether the level of perceived burden was 'below average' (<emph>n</emph> = 1,306) or 'above average' (<emph>n</emph> = 887).</p> <hd id="AN0185986528-10">Indicators of response quality</hd> <p>We assessed response quality based on a set of eight indicators.</p> <hd id="AN0185986528-11">No answer</hd> <p>Based on all questions in the main questionnaire, we counted the number of questions with 'no answer,' and created a binary variable where 1 means at least one question was left unanswered by a respondent.</p> <hd id="AN0185986528-12">'Don't know' answer</hd> <p>We created a continuous variable indicating the proportion of 'don't know' answers out of all questions with a 'don't know' option. The indicator can take values from 0 to 100, where a value of 50 means that half of the questions with a 'don't know' option were answered with 'don't know' by a respondent.</p> <hd id="AN0185986528-13">Correct response failure</hd> <p>Knowledge questions have factually correct responses that a respondent either knows, guesses, or can look up, for example, on the Internet. Gummer et al. ([<reflink idref="bib17" id="ref79">17</reflink>]) found that in the probability-based German Internet Panel about 50% of the respondents looked up answers to knowledge questions. Whereas looking up answers increases the burden of the cognitive response process (Gummer & Kunz, [<reflink idref="bib16" id="ref80">16</reflink>]), reducing the burden by not looking up the answer and/or superficially processing the question is likely to result in an incorrect answer. Therefore, we created a binary variable for each of the two knowledge questions, where 1 represents an incorrect answer by the respondent (including 'don't know' responses).</p> <hd id="AN0185986528-14">Straightlining</hd> <p>We created a binary variable for each of the two item batteries in the questionnaire (13 and 20 items, respectively), where 1 means that a respondent answered all items in the battery with the same response option (respondents who answered less than half of the items in the battery were excluded from analyses).</p> <hd id="AN0185986528-15">Item attention check failure</hd> <p>An instructed response item asking respondents to select a specific response option (e.g. 'Please select "rather disagree".') can be used as an attention check for item batteries to identify respondents who do not pay enough attention to the item content or ignore it altogether (e.g. Gummer et al., [<reflink idref="bib18" id="ref81">18</reflink>]). We created a binary variable based on the instructed response item, 'To check the questionnaire, please answer "rather agree".' which was one of six randomly ordered items in a battery. A value of 1 means that a respondent failed the item attention check by <emph>not</emph> selecting 'rather agree.'</p> <hd id="AN0185986528-16">Instruction attention check failure</hd> <p>Like instructed response items, instructions asking respondents to engage in a particular response behavior (e.g. skip a question, select a specific answer or a specific number of response options) can be used as an attention check to identify respondents who do not read an instruction (e.g. Shamon & Berning, [<reflink idref="bib30" id="ref82">30</reflink>]). We created a binary variable based on a check-all-that-apply question with 15 response options and the instruction, 'Please select from the following list the 5 aspects that are most important to you personally.' A value of 1 indicates that a respondent failed the instruction attention check by <emph>not</emph> selecting five response options (including 0 selected response options).</p> <hd id="AN0185986528-17">Number of topics below average</hd> <p>Based on two narrative open-ended questions, we counted the number of topics mentioned. We created a binary variable for each question, with 1 indicating that a respondent's answer included a below-average number of topics (including 0 topics mentioned).</p> <hd id="AN0185986528-18">Speeding</hd> <p>Using page-wise response times for all 28 web pages of the main questionnaire, we created a continuous variable indicating the proportion of web pages on which speeding was detected. The indicator can take values from 0 to 100, where 50 means that speeding was detected on half of the web pages answered by a respondent. We defined the speeding threshold following Greszki et al. ([<reflink idref="bib15" id="ref83">15</reflink>]), who identified responses to a question as too fast if they were given more than 50% faster than the median response time for that question.</p> <hd id="AN0185986528-19">Results</hd> <p></p> <hd id="AN0185986528-20">What is the association between objective and perceived burden? (RQ1)</hd> <p>In general, our three measures of perceived burden had very low mean values, indicating low self-reported difficulty, disengagement, and fatigue. Table 1 shows the perceived burden level at the four measurement time points.</p> <p>Table 1. Results on the level of perceived burden during survey completion.</p> <p> <ephtml> <table><thead><tr><td /><td><italic>n</italic></td><td>Block #1</td><td>Block #2</td><td>Block #3</td><td>Block #4</td><td><italic>p</italic></td><td>η<sub>p</sub><sup>2</sup></td></tr></thead><tbody><tr><td><italic>Difficulty</italic></td><td>2,146</td><td>1.54<sup>bcd</sup></td><td>1.60<sup>a</sup></td><td>1.61<sup>a</sup></td><td>1.61<sup>a</sup></td><td><.001</td><td>.004</td></tr><tr><td><italic>Disengagement</italic></td><td>2,144</td><td>1.58<sup>bcd</sup></td><td>1.50<sup>a</sup></td><td>1.50<sup>a</sup></td><td>1.51<sup>a</sup></td><td><.001</td><td>.005</td></tr><tr><td><italic>Fatigue</italic></td><td>2,152</td><td>1.65<sup>cd</sup></td><td>1.67<sup>cd</sup></td><td>1.71<sup>ab</sup></td><td>1.74<sup>ab</sup></td><td><.001</td><td>.006</td></tr></tbody></table> </ephtml> </p> <p>1 The items were answered on a 5-point scale ranging from 1 (<emph>not at all</emph>) to 5 (<emph>very much</emph>). Superscripts a, b, c, and d refer to post hoc tests (Bonferroni corrected) indicating a significant difference (<emph>p</emph> <.05 or less) between any two of the four block-wise question positions—that is, compared with Block #1 (a), Block #2 (b), Block #3 (c), Block #4 (d).</p> <p>Repeated measures ANOVAs with Huynh-Feldt correction (correcting for violations of sphericity, ε >.75) revealed that our three perceived burden measures were not rated equally throughout the survey (also shown using non-parametric Friedman tests). Bonferroni post hoc tests showed that the perceived difficulty of answering the questions increased slightly over the course of the survey, with significantly higher mean values in the second, third, and fourth question blocks compared with the first. The feeling of fatigue also increased slightly throughout the survey, with significantly higher mean values in the third and fourth question blocks compared with the first two blocks. By contrast, self-reported disengagement decreased as the survey progressed, as indicated by significantly lower mean values in Blocks #2, #3, and #4 compared with Block #1. At first glance, this was somewhat surprising, as we would also have expected disengagement to be higher in later blocks. However, given the very small differences in the mean values, we do not want to overinterpret this finding. Bonferroni post hoc tests following non-parametric Friedman tests revealed similar differences between the question blocks for two of our perceived burden measures, difficulty and disengagement. For our third measure, fatigue, only the fourth question block had a significantly higher median than the first two blocks. Although parametric and non-parametric tests revealed some significant differences between the four measurement time points, the mean differences and effect sizes were very small.</p> <hd id="AN0185986528-21">How does the level of objective burden (RQ2) and perceived burden (RQ3) impact response quali...</hd> <p>Table 2 summarizes the results of the analyses of the effects of objective and perceived burden on response quality using Pearson's chi-squared tests for categorical dependent variables and ANOVAs for continuous dependent variables.</p> <p>Table 2. Results of the analyses of the impact of objective and perceived burden on response quality.</p> <p> <ephtml> <table><thead><tr><td /><td /><td>Objective burden</td><td /><td /><td>Perceived burden</td><td /><td /></tr><tr><td /><td><italic>n</italic></td><td>Block #1</td><td>Block #2</td><td>Block #3</td><td>Block #4</td><td><italic>p</italic><sup>3</sup></td><td>Effect size<sup>4</sup></td><td>Below average</td><td>Above average</td><td><italic>p</italic><sup>3</sup></td><td>Effect size<sup>4</sup></td></tr></thead><tbody><tr><td>No answer<sup>2</sup> (%)</td><td>8,760</td><td>22.4</td><td>20.8</td><td>24.2</td><td>22.5</td><td>.061</td><td>-</td><td>19.9</td><td>26.3</td><td><.001</td><td>.075</td></tr><tr><td>Don't know<sup>1,2</sup> (<italic>M</italic>)</td><td>8,642</td><td>7.4</td><td>7.7</td><td>8.1</td><td>8.5</td><td>.203</td><td>-</td><td>6.5</td><td>10.0</td><td><.001</td><td>.010</td></tr><tr><td>Correct response failure (%)</td><td /><td /><td /><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> Question 1</td><td>2,176</td><td>37.0</td><td>38.2</td><td>39.9</td><td>38.6</td><td>.805</td><td>-</td><td>36.4</td><td>41.4</td><td>.018</td><td>.051</td></tr><tr><td> Question 2</td><td>2,177</td><td>44.0</td><td>44.0</td><td>45.9</td><td>47.4</td><td>.605</td><td>-</td><td>43.0</td><td>48.8</td><td>.007</td><td>.057</td></tr><tr><td>Straightlining (%)</td><td /><td /><td /><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> Question 1</td><td>2,187</td><td>3.2</td><td>2.6</td><td>3.1</td><td>3.9</td><td>.648</td><td>-</td><td>1.4</td><td>5.9</td><td><.001</td><td>.125</td></tr><tr><td> Question 2</td><td>2,189</td><td>2.9</td><td>3.4</td><td>3.4</td><td>3.8</td><td>.891</td><td>-</td><td>0.9</td><td>7.0</td><td><.001</td><td>.165</td></tr><tr><td>Item attention check failure (%)</td><td>2,181</td><td>8.9</td><td>9.0</td><td>8.5<sup>d</sup></td><td>13.6<sup>c</sup></td><td>.014</td><td>.070</td><td>4.6</td><td>18.0</td><td><.001</td><td>.218</td></tr><tr><td>Instruction attention check failure (%)</td><td>2,192</td><td>18.8</td><td>20.2</td><td>22.8</td><td>19.8</td><td>.388</td><td>-</td><td>13.6</td><td>30.3</td><td><.001</td><td>.204</td></tr><tr><td>Number of topics below average (%)</td><td /><td /><td /><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> Question 1</td><td>2,186</td><td>48.5</td><td>46.7</td><td>48.5</td><td>48.2</td><td>.921</td><td>-</td><td>42.8</td><td>55.6</td><td><.001</td><td>.126</td></tr><tr><td> Question 2</td><td>2,189</td><td>45.1</td><td>51.2</td><td>50.9</td><td>53.1</td><td>.052</td><td>-</td><td>45.2</td><td>57.2</td><td><.001</td><td>.117</td></tr><tr><td>Speeding<sup>1,2</sup> (<italic>M</italic>)</td><td>8,760</td><td>8.7<sup>cd</sup></td><td>10.3<sup>d</sup></td><td>11.6<sup>a</sup></td><td>12.6<sup>ab</sup></td><td><.001</td><td>.005</td><td>6.9</td><td>16.6</td><td><.001</td><td>.052</td></tr></tbody></table> </ephtml> </p> <p>2 Pearson's chi-square tests were used for categorical dependent variables unless otherwise specified. <sups>1</sups>ANOVAs were applied for continuous dependent variables. <sups>2</sups>Analysis is based on long-format data. <sups>3</sups><emph>p</emph> values (two-tailed) from Pearson's chi-square analysis for categorical variables and from ANOVA (<emph>F</emph> tests) for continuous variables with Bonferroni correction. Superscripts a, b, c, and d refer to post hoc tests indicating a significant difference (<emph>p</emph> <.05 or less) between any two of the four block-wise question positions—that is, compared with Block #1 (a), Block #2 (b), Block #3 (c), Block #4 (d). <sups>4</sups>Effect sizes refer to Cramer's <emph>V</emph> for categorical variables and partial eta squared (η<subs>p</subs><sups>2</sups>) for continuous variables.</p> <p>For objective burden, a significant effect was found for only two of the eight response quality indicators. Item attention check failure was most likely in the last part of the survey, with a significantly higher proportion of respondents not answering the instructed response item as requested with 'rather agree' when the item was asked in the fourth question block compared with the third block (13.6% vs. 8.5%). Speeding increased throughout the survey, as indicated by a higher proportion of web pages with speeding responses in the third question block compared with the first block (11.6% vs. 8.7%) and in the fourth question block compared with the first and second blocks (12.6% vs. 8.7% and 10.3%).</p> <p>For perceived burden, we found significant differences for all eight response quality indicators, depending on whether perceived burden was below or above average. Regarding the completeness of respondents' answers, we found that the proportions of respondents who provided either 'no answer' or a 'don't know' answer were significantly higher for respondents with above-average perceived burden than for respondents with below-average perceived burden (26.3% vs. 19.9% and 10.0% vs. 6.5%, respectively). For the two knowledge questions, we found a significantly higher proportion of respondents who answered the questions incorrectly among those with above-average perceived burden compared with those with below-average perceived burden (41.4% vs. 36.4% and 48.8% vs. 43.0%, respectively). For the two item batteries, we found a significantly higher proportion of respondents who straightlined (i.e. answered all items in a battery with the same response option) among the group of respondents who reported an above-average perceived burden level (5.9% vs. 1.4% and 7.0% vs. 0.9%, respectively). The proportion of respondents who failed the item attention check and the instruction attention check was significantly higher among respondents with above-average perceived burden than among those with below-average perceived burden (18.0% vs. 4.6% and 30.3% vs. 13.6%, respectively). In the instruction attention check, respondents were instructed to select five response options. Even when we excluded the cases with fewer than five response options because it could be assumed that falling below five represented the 'true answer' and did not indicate a lack of attention to the instruction, we found that significantly more respondents with above-average perceived burden failed the instruction attention check than did respondents with below-average perceived burden (9.9% vs. 5.5%). For both open-ended questions, we found that a significantly higher proportion of respondents who reported above-average perceived burden mentioned fewer topics than did those who reported below-average perceived burden (55.6% vs. 42.8% and 57.2% vs. 45.2%, respectively). Finally, the proportion of web pages where speeding was detected was significantly higher among respondents with above-average perceived burden than among those with below-average perceived burden (16.6% vs. 6.9%).</p> <p>Robustness checks based on regression analyses using our perceived burden index as a continuous independent variable and controlling for objective burden, gender, age, educational attainment, and device used to complete the survey confirmed our results (see Table A2 in the Appendix). Again, even when we controlled for objective burden and important respondent characteristics, we found a significant effect of perceived burden for all eight indicators of response quality: Higher perceived burden led to more 'no answers' and 'don't know' answers, more failed knowledge questions and item and instructional attention checks, fewer topics mentioned, and more straightlining and speeding.</p> <hd id="AN0185986528-22">Discussion</hd> <p>In the present study, we investigated the relationship between respondents' objective and perceived burden and the effects of both components on response quality in a web survey. The effects of objective and perceived burden were assessed independently of the content, format, and difficulty of individual questions and the overall question order by using a questionnaire with a completely random order of questions. In addition, we measured self-reported perceived burden at four time points to detect changes over the course of the questionnaire. Furthermore, we used a comprehensive set of indicators to evaluate response quality across various question formats.</p> <p>Our results showed that perceived burden increased only slightly during survey completion, a finding consistent with Galesic ([<reflink idref="bib11" id="ref84">11</reflink>]). Thus, a higher number of questions answered by respondents resulted in only a slight increase in self-reported difficulty and fatigue. Given this slight increase in perceived burden during the survey, we concluded that objective burden – as measured by the position of a question in the questionnaire – has only minor effects on the level of perceived burden (RQ1). In addition, we found only minor effects of objective burden on response quality, with significant effects for only two of the eight indicators. Thus, response quality indicators were largely unaffected by question position, except for item attention check failure and speeding, which were more likely in later parts of the questionnaire (RQ2). By contrast, we found consistent effects of perceived burden on response quality for all eight indicators. Thus, higher perceived burden is associated with lower response quality (RQ3).</p> <p>Our study provides several insights for survey researchers and practitioners. First, based on our findings, objective burden has only minor effects on response quality. Survey designers often advocate reducing the number of questions to reduce the respondent burden, which presents other challenges. Thus, from a practical point of view, it is good news that even long web surveys of over 20 minutes are conceivable without significant losses in response quality.</p> <p>Second, our results indicate that, for response quality, the level of perceived burden is a more decisive factor than the level of objective burden. To better understand and prevent low-quality responses in web surveys, survey designers should pay more attention to respondents' subjective perception of burden during the development and pretesting of survey instruments (Yan et al., [<reflink idref="bib37" id="ref85">37</reflink>]).</p> <p>Third, our results show that respondents' perceived burden changed only slightly during the questionnaire, which contradicts the common belief that perceived burden increases as the survey progresses. It is therefore crucial to minimize the initial perceived burden and to ensure that respondents do not perceive the survey as burdensome from the start. To achieve this, we need to understand which respondents find the survey burdensome from the outset. Are they respondents who react only to the third reminder, have confidentiality concerns, or have little interest in the topic?</p> <p>Fourth, it is important to measure not only whether and which respondents find the survey burdensome but also to identify why they find it burdensome – that is, the perceived causes of the burden (Dale & Haraldsen, [<reflink idref="bib8" id="ref86">8</reflink>]). Knowing the reasons can provide initial clues as to how to reduce the perceived burden. For example, do these respondents have an issue with the content, language, design, or layout of the questionnaire or with the survey mode?</p> <p>Fifth, identifying respondents who find a survey burdensome from the start and understanding the reasons for their perception of burden can help us apply targeted strategies to minimize perceived burden (e.g. by improving the mobile optimization of questionnaires or simplifying the language of the questions). It also enables the use of tailored measures in adaptive designs or panel studies, where we have more information about respondents' preferences (e.g. by inviting respondents to participate in their preferred survey mode or providing a questionnaire in plain language).</p> <p>The present study has some limitations that present opportunities for future research. First, although perceived burden appears to be particularly important for response quality, there is no established instrument for measuring perceived burden at multiple time points within the questionnaire. We used self-reports of difficulty, disengagement, and fatigue and asked respondents to rate their perceived burden block-wise by including only the last few questions they answered. We encourage future studies to further develop our instrument and test whether additional aspects are relevant for measuring perceived burden. There is also the question of whether it is better to measure cumulative perceived burden (i.e. asking respondents to include all previously answered questions in the evaluation) or block-wise perceived burden. In the latter case, it must be ensured that respondents can remember the most recently answered questions well, while excluding others from their evaluation.</p> <p>Second, the current study is based on a web survey with a mean completion time of 24 minutes. Although we consider the survey duration to be quite long, clearly exceeding the recommended optimal length of 10–15 minutes (Revilla & Höhne, [<reflink idref="bib26" id="ref87">26</reflink>]; Revilla & Ochoa, [<reflink idref="bib27" id="ref88">27</reflink>]), even longer web surveys are not uncommon in social science practice. Therefore, we see merit in replicating our study using even longer web surveys – for example, 40 minutes or more – to investigate whether there is a threshold above which objective burden significantly affects perceived burden and response quality.</p> <p>Third, although we went further than previous studies on respondent burden by using a comprehensive set of question formats that enabled us to analyze a variety of response quality indicators, even more burdensome questions (e.g. sensitive questions, social network questions, or questions that require detailed retrospective recall) than those used in our study are quite common in social science practice. Future studies should therefore test other questions and indicators to assess different aspects of response quality, thus helping to test the generalizability of our results to other social science surveys and to further clarify the impact of respondent burden on response quality depending on specific question characteristics (e.g. content, type, format).</p> <p>And finally, our results are based on a web survey conducted in a German online access panel. Further surveys in other countries and with other samples, such as probability-based samples, are recommended to ensure the generalizability of our conclusions. These replications could provide valuable insights into the role of respondent burden in different contexts and for specific subgroups of respondents.</p> <hd id="AN0185986528-23">Disclosure statement</hd> <p>No potential conflict of interest was reported by the author(s).</p> <hd id="AN0185986528-24">Ethics declaration</hd> <p>Ethical approval was not required because our study is noninterventional (i.e. a survey). However, we obtained informed consent from the respondents.</p> <hd id="AN0185986528-25">Supplementary material</hd> <p>Supplemental data for this article can be accessed online at https://doi.org/10.1080/13645579.2024.2393795</p> <ref id="AN0185986528-26"> <title> References </title> <blist> <bibl id="bib1" idref="ref72" type="bt">1</bibl> <bibtext> American Association for Public Opinion Research (AAPOR). (2023). Standard definitions. Final dispositions of case codes and outcome rates for surveys (10th ed.). https://<ulink href="http://www.aapor.org/AAPOR%5fMain/media/publications/Standard-Definitions20169theditionfinal.pdf">www.aapor.org/AAPOR%5fMain/media/publications/Standard-Definitions20169theditionfinal.pdf</ulink></bibtext> </blist> <blist> <bibl id="bib2" idref="ref26" type="bt">2</bibl> <bibtext> Andreadis, I., & Kartsounidou, E. (2020). The impact of splitting a long online questionnaire on data quality. Survey Research Methods, 14 (1). https://doi.org/10.18148/srm/2020.v14i1.7294</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Beatty, P. C., & Herrmann, D. (2002). To answer or not to answer: Decision processes related to survey item nonresponse. In R. M. Groves, D. A. Dillmann, J. L. Eltinge, & R. J. A. Little (Eds.), Survey nonresponse (pp. 71 – 86). Wiley.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref35" type="bt">4</bibl> <bibtext> Bowling, N. A., Gibson, A. M., & DeSimone, J. A. (2022). Stop with the questions already! Does data quality suffer for scales positioned near the end of a lengthy questionnaire? Journal of Business & Psychology, 37 (5), 1099 – 1116. https://doi.org/10.1007/s10869-021-09787-8</bibtext> </blist> <blist> <bibl id="bib5" idref="ref9" type="bt">5</bibl> <bibtext> Bradburn, N. (1978). Respondent burden. JSM Proceedings, Survey Research Methods Section, American Statistical Association (pp. 35 – 40). <ulink href="http://www.asasrms.org/Proceedings/papers/1978%5f007.pdf">http://www.asasrms.org/Proceedings/papers/1978%5f007.pdf</ulink></bibtext> </blist> <blist> <bibl id="bib6" idref="ref73" type="bt">6</bibl> <bibtext> Callegaro, M., & DiSogra, C. (2008). Computing response metrics for online panels. Public Opinion Quarterly, 72 (5), 1008 – 1032. https://doi.org/10.1093/poq/nfn065</bibtext> </blist> <blist> <bibl id="bib7" idref="ref16" type="bt">7</bibl> <bibtext> Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60 (6), 821 – 836. https://doi.org/10.1177/00131640021970934</bibtext> </blist> <blist> <bibl id="bib8" idref="ref22" type="bt">8</bibl> <bibtext> Dale, T., & Haraldsen, G. (Eds.). (2007). Handbook for monitoring and evaluating business survey response burden. European Commission, Eurostat. https://ec.europa.eu/eurostat/documents/64157/4374310/12-HANDBOOK-FOR-MONITORING-AND-EVALUATING-BUSINESS-SURVEY-RESONSE-BURDEN.pdf</bibtext> </blist> <blist> <bibl id="bib9" idref="ref29" type="bt">9</bibl> <bibtext> Deutskens, E., de Ruyter, K., Wetzels, M., & Oosterfeld, P. (2004). Response rate and response quality of internet-based surveys: An experimental study. Marketing Letters, 15 (1), 21 – 36. https://doi.org/10.1023/B:MARK.0000021968.86465.00</bibtext> </blist> <blist> <bibtext> Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2022). The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. Assessment, 29 (2), 136 – 151. https://doi.org/10.1177/1073191120957102</bibtext> </blist> <blist> <bibtext> Galesic, M. (2006). Dropouts on the web: Effects of interest and burden experienced during an online survey. Journal of Official Statistics, 22 (2), 313 – 328.</bibtext> </blist> <blist> <bibtext> Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73 (2), 349 – 360. https://doi.org/10.1093/poq/nfp031</bibtext> </blist> <blist> <bibtext> Ganassali, S. (2008). The influence of the design of web survey questionnaires on the quality of responses. Survey Research Methods, 2 (1), 21 – 32. https://doi.org/10.18148/srm/2008.v2i1.598</bibtext> </blist> <blist> <bibtext> Gibson, A. M., & Bowling, N. A. (2020). The effects of questionnaire length and behavioral consequences on careless responding. European Journal of Psychological Assessment, 36 (2), 410 – 420. https://doi.org/10.1027/1015-5759/a000526</bibtext> </blist> <blist> <bibtext> Greszki, R., Meyer, M., & Schoen, H. (2014). The impact of speeding on data quality in nonprobability and freshly recruited probability-based online panels. In M. Callegaro, R. P. Baker, J. G. Bethlehem, A. S. Göritz, J. A. Krosnick, & P. J. Lavrakas (Eds.), Online panel research. A data quality perspective (pp. 238 – 262). Wiley.</bibtext> </blist> <blist> <bibtext> Gummer, T., & Kunz, T. (2022). Relying on external information sources when answering knowledge questions in web surveys. Sociological Methods & Research, 51 (2), 816 – 836. https://doi.org/10.1177/0049124119882470</bibtext> </blist> <blist> <bibtext> Gummer, T., Kunz, T., Rettig, T., & Höhne, J. K. (2023). How to detect and influence looking up answers to political knowledge questions in web surveys. Public Opinion Quarterly, 87 (SI), 507 – 541. https://doi.org/10.1093/poq/nfad027</bibtext> </blist> <blist> <bibtext> Gummer, T., Roßmann, J., & Silber, H. (2021). Using instructed response items as attention checks in web surveys: Properties and implementation. Sociological Methods & Research, 50 (1), 238 – 264. https://doi.org/10.1177/0049124118769083</bibtext> </blist> <blist> <bibtext> Haraldsen, G. (2004). Identifying and reducing response burdens in internet business surveys. Journal of Official Statistics, 20 (2), 393 – 410.</bibtext> </blist> <blist> <bibtext> Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5 (3), 213 – 236. https://doi.org/10.1002/acp.2350050305</bibtext> </blist> <blist> <bibtext> Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, M. W., Kopp, R. J., Mitchell, R. C., Presser, S., Ruud, P. A., Smith, K. V., Moody, W. R., Green, M. C., & Conaway, M. (2002). The impact of "no opinion" response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66 (3), 371 – 403. https://doi.org/10.1086/341394</bibtext> </blist> <blist> <bibtext> Leiner, D. J. (2019). Too fast, too straight, too weird: Non-reactive indicators for meaningless data in internet surveys. Survey Research Methods, 13 (3), 229 – 248. https://doi.org/10.18148/srm/2019.v13i3.7403</bibtext> </blist> <blist> <bibtext> Miller, A. L., & Lambert, A. D. (2014). Open-ended survey questions: Item nonresponse nightmare or qualitative data dream? Survey Practice, 7 (5), 1 – 11. https://doi.org/10.29115/SP-2014-0024</bibtext> </blist> <blist> <bibtext> Neuert, C. E. (2024). The effect of question positioning on data quality in web surveys. Sociological Methods & Research, 53 (1), 279 – 295. https://doi.org/10.1177/0049124120986207</bibtext> </blist> <blist> <bibtext> Read, B. (2019). Respondent burden in a mobile app: Evidence from a shopping receipt scanning study. Survey Research Methods, 13 (1), 45 – 71. https://doi.org/10.18148/srm/2019.v1i1.7379</bibtext> </blist> <blist> <bibtext> Revilla, M. A., & Höhne, J. K. (2020). How long do respondents think online surveys should be? New evidence from two online panels in Germany. International Journal of Market Research, 62 (5), 538 – 545. https://doi.org/10.1177/1470/7853/2094/3049</bibtext> </blist> <blist> <bibtext> Revilla, M. A., & Ochoa, C. (2017). Ideal and maximum length for a web survey. International Journal of Market Research, 59 (5), 557 – 565. https://journals.sagepub.com/doi/ 10.2501/IJMR-2017-039</bibtext> </blist> <blist> <bibtext> Rolstad, S., Adler, J., & Rydén, A. (2011). Response burden and questionnaire length: Is shorter better? A review and meta-analysis. Value in Health, 14 (8), 1101 – 1108. https://doi.org/10.1016/j.jval.2011.06.003</bibtext> </blist> <blist> <bibtext> Schmidt, K., Gummer, T., & Roßmann, J. (2020). Effects of respondent and survey characteristics on the response quality of an open-ended attitude question in web surveys. Methods, Data, Analyses, 14 (1), 3 – 34. https://doi.org/10.12758/mda.2019.05</bibtext> </blist> <blist> <bibtext> Shamon, H., & Berning, C. C. (2020). Attention check items and instructions in online surveys with incentivized and non-incentivized samples: Boon or bane for data quality? Survey Research Methods, 14 (1), 55 – 77. https://doi.org/10.2139/ssrn.3549789</bibtext> </blist> <blist> <bibtext> Sharp, L. M., & Frankel, J. (1983). Respondent burden: A test of some common assumptions. Public Opinion Quarterly, 47 (1), 36 – 53. https://doi.org/10.1086/268765</bibtext> </blist> <blist> <bibtext> Smyth, J. D., Dillman, D. A., & Christian, L. M. (2007). Context effects in internet surveys. New issues and evidence. In A. N. Joinson, K. Y. A. McKenna, T. Postmes, & U.-D. Reips (Eds.), The oxford handbook of internet psychology (pp. 429 – 445). Oxford University Press.</bibtext> </blist> <blist> <bibtext> Tourangeau, R., Rips, L., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press.</bibtext> </blist> <blist> <bibtext> Turner, G., Sturgis, P., & Martin, D. (2015). Can response latencies be used to detect survey satisficing on cognitively demanding questions? Journal of Survey Statistics and Methodology, 3 (1), 89 – 108. https://doi.org/10.1093/jssam/smu022</bibtext> </blist> <blist> <bibtext> Wang, M., McCutcheon, A. L., & Allen, L. (2015, May 14–17). Grids and online panels: A comparison of grid location and device type from a survey quality perspective [paper presentation]. 70th AAPOR Annual Conference, Hollywood, FL.</bibtext> </blist> <blist> <bibtext> Yan, T., Conrad, F. G., Tourangeau, R., & Couper, M. P. (2010). Should I stay or should I go: The effects of progress feedback, promised task duration, and length of questionnaire on completing web surveys. International Journal of Public Opinion Research, 23 (2), 131 – 147. https://doi.org/10.1093/ijpor/edq046</bibtext> </blist> <blist> <bibtext> Yan, T., Fricker, S., & Tsai, S. (2020). Response burden: What is it and what predicts it? In P. C. Beatty, D. Collins, L. Kaye, J.-L. Padilla, G. B. Willis, & A. Wilmot (Eds.), Advances in questionnaire design, development, evaluation and testing (pp. 193 – 212). John Wiley & Sons, Inc. https://doi.org/10.1002/9781119263685.ch8</bibtext> </blist> <blist> <bibtext> Yan, T., Ryan, L., Becker, S. E., & Smith, J. (2015). Assessing quality of answers to a global subjective well-being question through response times. Survey Research Methods, 9 (2), 101 – 109. https://doi.org/10.18148/srm/2015.v9i2.6107</bibtext> </blist> <blist> <bibtext> Yan, T., & Tourangeau, R. (2008). Fast times and easy questions: The effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology, 22 (1), 51 – 68. https://doi.org/10.1002/acp.1331</bibtext> </blist> <blist> <bibtext> Yan, T., & Williams, D. (2022). Response burden – Review and conceptual framework. Journal of Official Statistics, 38 (4), 939 – 961. https://doi.org/10.2478/jos-2022-0041</bibtext> </blist> <blist> <bibtext> Yang, D. K., & Toth, D. S. (2022). Analyzing the association of objective burden measures to perceived burden with regression trees. Journal of Official Statistics, 38 (4), 1125 – 1144. https://doi.org/10.2478/jos-2022-0048</bibtext> </blist> <blist> <bibtext> Zaller, J., & Feldman, S. (1992). A simple theory of the survey response: Answering questions versus revealing preferences. American Journal of Political Science, 36 (3), 579 – 616. https://doi.org/10.2307/2111583</bibtext> </blist> <blist> <bibtext> Zhang, C., & Conrad, F. G. (2013). Speeding in web surveys: The tendency to answer very fast and its association with straightlining. Survey Research Methods, 8 (2), 127 – 135. https://doi.org/10.18148/srm/2014.v8i2.5453</bibtext> </blist> </ref> <aug> <p>By Tanja Kunz and Tobias Gummer</p> <p>Reported by Author; Author</p> <p></p> <p>Tanja Kunz is a senior researcher at GESIS – Leibniz Institute for the Social Sciences in Mannheim, Germany. Her research interests include designing and implementing mixed-mode survey data collections, focusing on questionnaire design and data quality. tanja.kunz@gesis.org. ORCID: https://orcid.org/0000-0001-8460-2583. Postal address: GESIS – Leibniz Institute for the Social Sciences, B6.4-5, 68159 Mannheim, Germany.</p> <p>Tobias Gummer is a senior researcher and team leader at GESIS – Leibniz Institute for the Social Sciences in Mannheim, Germany, and a professor at the University of Mannheim. His methodological research interests include survey design, data quality, nonresponse error, response behavior, and prevention and correction methods for biases. tobias.gummer@gesis.org. ORCID: https://orcid.org/0000-0001-6469-7802. Postal address: GESIS – Leibniz Institute for the Social Sciences, B6.4-5, 68159 Mannheim, Germany.</p> </aug> <nolink nlid="nl1" bibid="bib33" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib20" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib21" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib43" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib11" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib12" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib19" firstref="ref10"></nolink> <nolink nlid="nl8" bibid="bib40" firstref="ref11"></nolink> <nolink nlid="nl9" bibid="bib37" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib28" firstref="ref19"></nolink> <nolink nlid="nl11" bibid="bib31" firstref="ref20"></nolink> <nolink nlid="nl12" bibid="bib10" firstref="ref23"></nolink> <nolink nlid="nl13" bibid="bib23" firstref="ref25"></nolink> <nolink nlid="nl14" bibid="bib24" firstref="ref28"></nolink> <nolink nlid="nl15" bibid="bib35" firstref="ref30"></nolink> <nolink nlid="nl16" bibid="bib13" firstref="ref36"></nolink> <nolink nlid="nl17" bibid="bib14" firstref="ref37"></nolink> <nolink nlid="nl18" bibid="bib29" firstref="ref42"></nolink> <nolink nlid="nl19" bibid="bib39" firstref="ref48"></nolink> <nolink nlid="nl20" bibid="bib36" firstref="ref63"></nolink> <nolink nlid="nl21" bibid="bib32" firstref="ref64"></nolink> <nolink nlid="nl22" bibid="bib41" firstref="ref65"></nolink> <nolink nlid="nl23" bibid="bib15" firstref="ref66"></nolink> <nolink nlid="nl24" bibid="bib22" firstref="ref67"></nolink> <nolink nlid="nl25" bibid="bib34" firstref="ref70"></nolink> <nolink nlid="nl26" bibid="bib38" firstref="ref71"></nolink> <nolink nlid="nl27" bibid="bib25" firstref="ref74"></nolink> <nolink nlid="nl28" bibid="bib42" firstref="ref76"></nolink> <nolink nlid="nl29" bibid="bib17" firstref="ref79"></nolink> <nolink nlid="nl30" bibid="bib16" firstref="ref80"></nolink> <nolink nlid="nl31" bibid="bib18" firstref="ref81"></nolink> <nolink nlid="nl32" bibid="bib30" firstref="ref82"></nolink> <nolink nlid="nl33" bibid="bib26" firstref="ref87"></nolink> <nolink nlid="nl34" bibid="bib27" firstref="ref88"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1495786
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Effects of Objective and Perceived Burden on Response Quality in Web Surveys
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Tanja+Kunz%22">Tanja Kunz</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-8460-2583">0000-0001-8460-2583</externalLink>)<br /><searchLink fieldCode="AR" term="%22Tobias+Gummer%22">Tobias Gummer</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-6469-7802">0000-0001-6469-7802</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Social+Research+Methodology%22"><i>International Journal of Social Research Methodology</i></searchLink>. 2025 28(4):385-395.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Online+Surveys%22">Online Surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Rates+%28Questionnaires%29%22">Response Rates (Questionnaires)</searchLink><br /><searchLink fieldCode="DE" term="%22Questionnaires%22">Questionnaires</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Style+%28Tests%29%22">Response Style (Tests)</searchLink><br /><searchLink fieldCode="DE" term="%22Sequential+Approach%22">Sequential Approach</searchLink><br /><searchLink fieldCode="DE" term="%22Serial+Ordering%22">Serial Ordering</searchLink><br /><searchLink fieldCode="DE" term="%22Responsibility%22">Responsibility</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Problems%22">Research Problems</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Germany%22">Germany</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/13645579.2024.2393795
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1364-5579<br />1464-5300
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Respondent burden is considered a decisive factor affecting response quality in web surveys. To investigate the objective and perceived burden on web survey respondents and its effects on response quality, we conducted a web survey among members of a German online access panel using a questionnaire with a completely randomized question order and measures of perceived burden at four time points. We found that perceived burden increased only slightly during the survey, suggesting that objective burden--as measured by the position of a question in the questionnaire--has a modest impact on the level of perceived burden. Although the position of a question in the questionnaire also had little effect on response quality, higher perceived burden consistently resulted in lower response quality.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1495786
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1495786
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/13645579.2024.2393795
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 385
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Online Surveys
        Type: general
      – SubjectFull: Response Rates (Questionnaires)
        Type: general
      – SubjectFull: Questionnaires
        Type: general
      – SubjectFull: Response Style (Tests)
        Type: general
      – SubjectFull: Sequential Approach
        Type: general
      – SubjectFull: Serial Ordering
        Type: general
      – SubjectFull: Responsibility
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Research Problems
        Type: general
      – SubjectFull: Germany
        Type: general
    Titles:
      – TitleFull: Effects of Objective and Perceived Burden on Response Quality in Web Surveys
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Tanja Kunz
      – PersonEntity:
          Name:
            NameFull: Tobias Gummer
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1364-5579
            – Type: issn-electronic
              Value: 1464-5300
          Numbering:
            – Type: volume
              Value: 28
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: International Journal of Social Research Methodology
              Type: main
ResultId 1