View in EDS HTML Full Text PDF Full Text

The Factorial Survey: The Impact of the Presentation Format of Vignettes on Answer Behavior and Processing Time

Saved in:
Bibliographic Details
Title: The Factorial Survey: The Impact of the Presentation Format of Vignettes on Answer Behavior and Processing Time
Language: English
Authors: Shamon, Hawal (ORCID 0000-0001-6304-9301), Dülmer, Hermann, Giza, Adam
Source: Sociological Methods & Research. Feb 2022 51(1):396-438.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Peer Reviewed: Y
Page Count: 43
Publication Date: 2022
Document Type: Journal Articles
Reports - Research
Descriptors: Vignettes, Surveys, Response Style (Tests), Reaction Time, Foreign Countries, Adults, Age Differences, Educational Attainment
Geographic Terms: Germany
DOI: 10.1177/0049124119852382
ISSN: 0049-1241
Abstract: The factorial survey is an experimental design in which the researcher constructs varying descriptions of situations or individual persons (vignettes), which will be judged by respondents with regard to a particular aspect. Some researchers present vignettes in text format as short stories, others present the central information of vignettes in a tabular format. To date, only a few sentences have been published, by Auspurg and Hinz, on the impact of the presentation format (text vs. table) on the answer behavior of students. Empirically, no differences were found between either format. Based on an Internet experiment conducted with a quota sample, we find evidence that ordinary tabular formats outperform text vignettes in terms of total vignette nonresponse but not when it comes to processing time. The former result especially applies in the case of less well-educated people. We further find that tabular format does not perform worse than text format regarding response inconsistency.
Abstractor: As Provided
Entry Date: 2022
Accession Number: EJ1325868
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGhi2yg_QuEQwKOJ6FZ3uzkAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBRUBqaAlitjBLUgHwIBEICBmzH20-n7NxWYYlL0mjl7gmaPLZx390b2rSe0Snh-xbZOhW946Rp2CMa5G1PH9aL6nBNugjUkTzTovL0aNmjEKSIUVp-vLnikZAyLEyFv6aQCIBPd6idVLTL_V-mWs-Fm5sNZkJHm5uQya41MF7Cnu4vxxhuelzV2p_TSuU3giQQuIYnqrhS51ZHknHOjG3ce2QXtrvo7l95js3ok
Text:
  Availability: 1
  Value: <anid>AN0154953735;som01feb.22;2022Feb01.05:47;v2.2.500</anid> <title id="AN0154953735-1">The Factorial Survey: The Impact of the Presentation Format of Vignettes on Answer Behavior and Processing Time </title> <p>The factorial survey is an experimental design in which the researcher constructs varying descriptions of situations or individual persons (vignettes), which will be judged by respondents with regard to a particular aspect. Some researchers present vignettes in text format as short stories, others present the central information of vignettes in a tabular format. To date, only a few sentences have been published, by Auspurg and Hinz, on the impact of the presentation format (text vs. table) on the answer behavior of students. Empirically, no differences were found between either format. Based on an Internet experiment conducted with a quota sample, we find evidence that ordinary tabular formats outperform text vignettes in terms of total vignette nonresponse but not when it comes to processing time. The former result especially applies in the case of less well-educated people. We further find that tabular format does not perform worse than text format regarding response inconsistency.</p> <p>Keywords: factorial survey; satisficing; text vignette; table vignette; response time; nonresponse; cognitive scheme; design; methodological study</p> <p>The question–answer process for standardized interviews starts with reading or listening to a question. It ends when an answer is given by a respondent. The survey response process consists of separate steps that can be arranged into four main categories (cf. [<reflink idref="bib52" id="ref1">52</reflink>]): (<reflink idref="bib1" id="ref2">1</reflink>) understanding a question, (<reflink idref="bib2" id="ref3">2</reflink>) retrieving from memory information subjectively perceived as relevant, (<reflink idref="bib3" id="ref4">3</reflink>) forming a judgment based on the retrieved information, and (<reflink idref="bib4" id="ref5">4</reflink>) communicating the judgment to the interviewer or researcher, which in the case of a closed answer scale has to be adapted to the given answer format.</p> <p>In this survey response process, the second step (i.e., retrieval of relevant information) forms a crucial basis for a stable judgment. Its importance becomes clear by elaborating on the retrieval step in case of attitude questions. When asked for their attitudes, respondents are sometimes confronted with a topic that they have rarely or even never been concerned about before. Under these conditions, it is rather unlikely that a preconceived attitude that can be easily retrieved from memory exists ([<reflink idref="bib37" id="ref6">37</reflink>]; [<reflink idref="bib52" id="ref7">52</reflink>]). When respondents have no preconceived attitude, it can be assumed that they arrive at an answer by, for instance, using available information from diverse sources ([<reflink idref="bib37" id="ref8">37</reflink>]; [<reflink idref="bib42" id="ref9">42</reflink>]). [<reflink idref="bib52" id="ref10">52</reflink>] highlight three compatible ways in which respondents can come to an answer when they are asked for their attitude. Firstly, without detailed knowledge of a topic, people can base their judgment on vague impressions or existing stereotypes. Secondly, they can align their judgment with their own value orientations. Finally, in order to reach a judgment, respondents can also search for concrete information that is personally considered to be relevant.[<reflink idref="bib4" id="ref11">4</reflink>] Based on these considerations, for conventional questions about attitudes that are asked of the same respondents repeatedly across time, a high variability can be expected in the answers ([<reflink idref="bib13" id="ref12">13</reflink>]; [<reflink idref="bib54" id="ref13">54</reflink>]). The less a respondent has a preconceived attitude toward an issue or object, and the more the person's attitude starts to form within the interview situation, the more plausible this expectation becomes. Moreover, these elaborations indicate that respondents may significantly differ with regard to the subjective basis of their judgment (hereafter called <emph>reference frame</emph>).</p> <p>Instead of measuring judgments directly via general survey questions, judgments can be measured indirectly by using a factorial survey where the survey response process does not require information retrieval from memory. The factorial survey ([<reflink idref="bib5" id="ref14">5</reflink>]; [<reflink idref="bib24" id="ref15">24</reflink>]; [<reflink idref="bib40" id="ref16">40</reflink>]) consists of a number of descriptions of various hypothetical situations, objects, or persons (vignettes) that have to be answered by the respondents with reference to a particular aspect the researcher is interested in (for instance, normative judgments, subjective beliefs, or intended actions; cf. [<reflink idref="bib24" id="ref17">24</reflink>]:344). The descriptions in a vignette are designed as an experimental setting. As such, each vignette includes several dimensions (also called factors) and one of several possible levels for each individual dimension. A factorial survey about fair levels of earnings for salesmen, for instance, may include the dimensions of gender, parenthood, job experience, and level of effort at work for a fictitious vignette person. In this case, a single vignette would inform a respondent about the combination of the levels (personal attributes/characteristics) of a fictitious vignette person for these four dimensions (e.g., female, two children, five years' job experience, and high effort at work). The task of the respondents is to judge concrete vignette descriptions as a whole without being forced to indicate the influence of each individual vignette characteristic explicitly. An advantage of the factorial survey is that judging concrete vignettes comes much closer to judgments made in real life than answering comparably general, mostly rather abstract questions as usually used for survey research (cf. [<reflink idref="bib9" id="ref18">9</reflink>]:304). The detailed descriptions also lead to a higher standardization (cf. also [<reflink idref="bib5" id="ref19">5</reflink>]:4, 7). This helps to minimize the problem of respondents' different subjective reference frames because in order to answer a question, respondents no longer have to retrieve relevant information from their memory (step 2 of [<reflink idref="bib52" id="ref20">52</reflink>]) but to take in information from a standardized reference frame (cf. [<reflink idref="bib46" id="ref21">46</reflink>]). Taken together, these facts might explain why in the factorial survey a respondent's answer can be expected to be ascertained with higher reliability and higher validity[<reflink idref="bib5" id="ref22">5</reflink>] than is possible with more general single questions more typical of survey research (cf. [<reflink idref="bib2" id="ref23">2</reflink>]:93).</p> <p>Although more recently factorial surveys have become increasingly popular among social scientists, only a small number of the published studies based on population or college samples ([<reflink idref="bib5" id="ref24">5</reflink>]:2) examined the effect of methodological issues of design complexity on respondents' cognitive load for different administration modes ([<reflink idref="bib6" id="ref25">6</reflink>]; [<reflink idref="bib8" id="ref26">8</reflink>]; [<reflink idref="bib43" id="ref27">43</reflink>]; [<reflink idref="bib51" id="ref28">51</reflink>]). Overall, these studies support the use of the factorial survey by showing that manifestations of cognitive overburden—such as response inconsistencies, fade-out of vignette dimensions, fatigue effects, and vignette dimension order effects—can be avoided by choosing an appropriate level of design complexity.[<reflink idref="bib6" id="ref29">6</reflink>] One area where no published studies exist up to now is the question of whether the <emph>presentation format of vignettes</emph> has any impact on the cognitive burden and, thus, on the answer behavior of respondents (cf. also [<reflink idref="bib5" id="ref30">5</reflink>]:71). Research on this topic is important because information intake is central to vignette studies. Basically, vignettes can be presented in a <emph>text format</emph> (running text) or in a <emph>tabular format</emph> (cf. [<reflink idref="bib24" id="ref31">24</reflink>]:411-12).</p> <p>Besides <emph>varying information</emph>, some factorial surveys might also consider to include <emph>fixed background information</emph> (only one level for a dimension). Fixed background information refers to those dimensions that are from a theoretical point of view assumed to be relevant for respondents' answer behavior, but whose examination is not central to a researcher's substantial interest in a particular study. However, although the researcher is neither substantively interested in fixed background information nor able to estimate its impact on the answer behavior, fixed background information is sometimes needed to standardize the vignettes sufficiently. Research suggests that people's short-term memory is limited to store "seven, plus or minus two" items of information at a time ([<reflink idref="bib34" id="ref32">34</reflink>]). Therefore, presenting fixed background information in the introduction of a factorial survey, as proposed by [<reflink idref="bib5" id="ref33">5</reflink>]:19), might not be sufficient to prevent from biases in the effect sizes, particularly in case of complex factorial survey designs. Specifying additionally fixed background information in a visually separated space at each vignette has the advantage that the information is always available and, if needed, can be read anew. In this way, possible ambiguities that otherwise might exist (and which might bias the results) are removed from the vignettes. Another advantage of using fixed background information is that in this way, the complexity of the factorial survey is reduced.</p> <p>In order to facilitate information intake, the researcher might decide to present the varying information for text formats, for instance, by underlining or by using a tabular format. A rationale for using a text format, according to [<reflink idref="bib5" id="ref34">5</reflink>]:70), is that short stories might facilitate the understanding of the described situation. It might help respondents to put themselves in the situation or to empathize with the described person. The initial research of [<reflink idref="bib5" id="ref35">5</reflink>]:71) with university students, however, suggests that tabular vignettes, if not overtly complex, result in similar evaluations to text vignettes. Although the authors found no substantial differences for highly educated students, important differences might not become visible until people with different educational backgrounds participate in a factorial survey. From past research, it is well known that the more syllables a word contains and the higher the number of words that are used in a sentence, the more difficult it becomes for people to understand the sentence (cf. [<reflink idref="bib19" id="ref36">19</reflink>]). This aspect is especially relevant for less well-educated people with poor reading skills, whereas people with good reading skills are better able to understand semantically complex sentences (cf. [<reflink idref="bib25" id="ref37">25</reflink>]). By reducing the number of words and by placing the relevant information at exactly the same location on a vignette (a predefined column), table formats contribute to make information more accessible to respondents (cf. also [<reflink idref="bib21" id="ref38">21</reflink>]:2396). This might help respondents to extract the potentially relevant information easier and, by giving a better overview over the varying information, may contribute to a better understanding even for respondents with lower cognitive skills.[<reflink idref="bib7" id="ref39">7</reflink>] In a direct comparison with an unsupported text format (varying information not highlighted, for instance, by underlining), the tabular format performed better than the text format regarding predicting real-world behavior, that is, the tabular format showed the higher external validity (cf. [<reflink idref="bib21" id="ref40">21</reflink>]). Nevertheless, the tabular format might not be practical for every research question.[<reflink idref="bib8" id="ref41">8</reflink>]</p> <p>The aim of this study is to investigate the impact of different presentation formats (text vs. tabular) on the answer behavior as well as on the processing time of respondents. We examine the formats' impacts on the probability of the respondent refusing to participate in the factorial survey, on the probability of the respondent breaking off their participation in the factorial survey, on the probability of unanswered vignettes, on the number of faded-out vignette dimensions, on coefficient weights of vignette dimensions, and on respondents' response inconsistency as well as the impact on the processing time taken by the respondent. The examined answer behaviors—refusal, break off, and vignette nonresponse—may be understood as an expression of a respondent's strong satisficing strategy, while number of faded-out vignette dimensions, differences in coefficient weights, and response inconsistency point to the potential impact of the presentation format on respondents' choice of a weak satisficing strategy for reducing the cognitive demands of an interview. Less well-educated people and older people might be especially prone to such strategies. If these groups are systematically affected by a higher rate of invalid or missing answers, then the data quality will suffer from a systematic bias.</p> <p>The topic of this study is important, firstly, because a longer processing time increases the costs of a survey. Secondly, having better knowledge about the impact of the different formats on the cognitive demand of the survey makes it possible to reduce the percentage of missing values, which in turn contributes to better data quality (higher statistical power, lower risk of potential bias; cf. also [<reflink idref="bib41" id="ref42">41</reflink>]; [<reflink idref="bib48" id="ref43">48</reflink>]). Our survey is based on the substantive ideas of a factorial survey conducted by [<reflink idref="bib47" id="ref44">47</reflink>]. The participants for our study were recruited via quota sampling from a German online access panel. The quotas were generated by crossing age, gender, and education. The 498 respondents who participated in the online survey were randomly assigned to one of the three experimental groups. In the following, we will give an overview of the theoretical frame of the study. Thereafter, we will present a modified cognitive scheme for the "question–answer process" in factorial surveys, which was introduced by [<reflink idref="bib46" id="ref45">46</reflink>]. Based on this theoretical framework, we will derive our hypotheses. Finally, we will introduce our research strategy, test the hypotheses, discuss the results, and draw some conclusions.</p> <hd id="AN0154953735-2">Theoretical Framework</hd> <p></p> <hd id="AN0154953735-3">Cognitive Schemes for Factorial Surveys and Answer Behavior Categorization</hd> <p>Participating in a survey results in costs, which arise, among other things, from cognitive efforts that are required for answering questions. Based on the anticipated costs as well as the subjectively expected net utility, a respondent will decide either to participate or not to participate in a survey (cf. [<reflink idref="bib14" id="ref46">14</reflink>]). People who decided to participate, however, can (a) work through the cognitive steps carefully and comprehensively (<emph>optimizing</emph>), (b) try to reduce the costs by completing the cognitive steps involved in answering questions with relatively little care (<emph>weak form of satisficing</emph>), or (c) try to reduce the costs by omitting at least one of the required cognitive steps (<emph>strong form of satisficing</emph>; cf. [<reflink idref="bib26" id="ref47">26</reflink>]).[<reflink idref="bib9" id="ref48">9</reflink>] The probability of using a satisficing strategy depends on the requirements of a task/question as well as on the skills and the motivation of the person responding to a respective question. [<reflink idref="bib26" id="ref49">26</reflink>] assumes that the more the time has passed, the lower the motivation of a respondent to complete all cognitive steps carefully. [<reflink idref="bib27" id="ref50">27</reflink>] showed that a respondent's ability and motivation as well as the position of the question in the survey determine strong forms of satisficing that manifest themselves in the absence of a respondent's answer to a survey item.</p> <p>In factorial surveys, participants are presented with a single description (one vignette per respondent factorial surveys) or a set of varying descriptions of hypothetical situations or persons (multiple vignettes per respondent factorial surveys). They are asked to evaluate the presented vignette(s) with regard to a particular aspect the researcher is interested in. In this context, the question and answer process proposed by [<reflink idref="bib52" id="ref51">52</reflink>] cannot be used without modifications. [<reflink idref="bib46" id="ref52">46</reflink>] proposed a modified cognitive scheme for factorial surveys. According to this scheme, instead of understanding a question, the first step in the question–answer process is information intake. Besides the varying information, which constitutes the core of the experimental design, some vignette studies also present fixed background information for each individual vignette. From the respondent's perspective, the (fixed) background information from the first vignette only has to be stored in the memory once and then retrieved every time a subsequent vignette description is evaluated. Varying information on the other hand has to be absorbed from each individual vignette. Without this information intake, varying vignette characteristics cannot have any impact on judgment behavior. In general, whenever fixed background information is used in a factorial survey, it is recommended that respondents are informed about the fixed vignette characteristics in the introduction, presented before the factorial survey starts.</p> <p>After information intake, reading and understanding the vignette question follows as a second step.[<reflink idref="bib10" id="ref53">10</reflink>] Like survey questions, vignette questions have to be formulated in a way that allows the respondents to unambiguously understand the semantic and pragmatic sense of the question.[<reflink idref="bib11" id="ref54">11</reflink>] In the third step, respondents have to form a judgment. Since respondents are free to decide which of the presented information they want to take into account and how strongly they want to weight the respective information, the information presented in the vignettes can be understood as the objective basis for judgment formation. In this sense, it is possible that respondents consciously or unconsciously ignore, partially or even completely in their subjective reference frame, some of the objectively given vignette information in the judgment formation step because they perceive it as irrelevant or because they want to reduce cognitive effort in answering the vignette questions. In the subsequent fourth step, the judgment has to be communicated to the interviewer. For factorial surveys, open or closed answer formats can be used. In contrast to open answer formats that require the interviewee to form a precise answer to a judgment task while allowing him or her to precisely communicate his or her optimal judgment, closed answer formats require the selection of the best fitting answer category out of an answer scale predefined by the researcher before the judgment can be communicated ([<reflink idref="bib50" id="ref55">50</reflink>]).</p> <p>Based on the theoretical considerations, the information in a vignette provides respondents with all the information needed to form a judgment. Hence, a respondent omitting to make a judgment reflects the application of a strong satisficing strategy rather than a valid indication that he or she has no judgment at all (cf. [<reflink idref="bib27" id="ref56">27</reflink>]:379). In one vignette per respondent factorial surveys, it is only possible ex post to distinguish between respondents who judged and those who did not judge the single vignette, either by skipping a question or by choosing a potentially offered "don't know" option. The analysis of the answer behavior in multiple vignettes per respondent factorial surveys allows for a much more nuanced analysis of the strong form of satisficing. Here, we can distinguish ex post between (a) participants who applied a strong form of satisficing (<emph>refusal</emph>) to all presented vignettes, (b) participants who started with an optimizing strategy or with a weak form of satisficing and changed to a strong form of satisficing (<emph>break-off</emph>) during the factorial survey, and (c) respondents who alternately apply a weak form of satisficing or optimizing and a strong form of satisficing (<emph>vignette nonresponse</emph>).</p> <p>Beside answer behaviors rooted in strong forms of satisficing, multiple vignettes per respondent factorial surveys allow also for a nuanced analysis of weak forms of satisficing. Multiple vignettes per respondent factorial surveys allow ex post to distinguish between respondents who reduce the cognitive costs of completing the cognitive steps involved in answering questions (a) by reducing the number of vignette dimensions considered in their subjective reference frame right from the beginning of the factorial survey (<emph>consequent dimension reduction</emph>)[<reflink idref="bib12" id="ref57">12</reflink>], (b) by changing their subjective reference frame from vignette to vignette (<emph>partial dimension reduction</emph>), and (c) by paying less attention on responding consistently to the vignette dimensions presented on the multiple vignettes to them (<emph>response inconsistency</emph>; cf. also [<reflink idref="bib6" id="ref58">6</reflink>]; [<reflink idref="bib43" id="ref59">43</reflink>]; [<reflink idref="bib51" id="ref60">51</reflink>]).[<reflink idref="bib13" id="ref61">13</reflink>]</p> <p>An advantage of using our nuanced classification scheme is that not only does it allow the total amount of vignette nonresponse to be analyzed but also makes it possible to identify the most important sources of different forms of strong and weak satisficing behavior across different presentation formats.</p> <hd id="AN0154953735-4">The Impact of the Presentation Format on Answer Behavior</hd> <p>Presentation formats of factorial surveys (tabular vs. text vignettes) differ in the degree to which they prestructure the varying information on the vignettes. By choosing an optimal presentation format, information intake by respondents will be facilitated, which contributes to valid answer behavior. A presentation format that supports the information intake of the varying information (<emph>supportive presentation format</emph>) should reduce a respondent's expected as well as real costs in terms of cognitive effort and time spent participating in a survey. In so doing, a supportive presentation format should help to lessen a possible decrease in respondents' motivation over the time spent doing the survey. Therefore, our general expectation is that the likelihood of choosing a strong satisficing strategy—whether intentional or not—will be reduced by using a supportive presentation format. In a similar vein, we expect that the use of a supportive presentation format would reduce the likelihood of choosing a weak satisficing strategy and, hence, contribute to preventing consequent dimension reduction, partial dimension reduction, and response inconsistencies. Moreover, we expect that a supportive presentation format allows for faster information intake, such that it reduces the processing time of respondents who complete all cognitive steps, that is, without applying a strong satisficing strategy.</p> <p>In text formats, the information intake of respondents can be facilitated by highlighting the varying information through the use of underlining or bold letters, for instance. In this way, the varying information is already emphasized (prestructured) and the text format becomes supportive, while a text format without highlighted varying information is a nonsupportive one. Using a tabular format for conducting a factorial survey is always a supportive format: In this case, the varying information is already spotlighted (prestructured) by presenting it in a specific column in each vignette (<emph>ordinary tabular presentation format</emph>). The support given by a tabular format is probably more structured than when highlighted varying information is used in text vignettes. Compared to text vignettes, where varying information is highlighted but spread throughout the vignette text, table vignettes display all relevant information, further organized in columns and rows, that probably allows respondents to get a faster and better overview of all varying information. This is generally the advantage of using tables. Based on these theoretical considerations, we will now formulate specific hypotheses about text and tabular vignette formats.</p> <p></p> <ulist> <item> <bold> Hypothesis 1a: </bold> Tabular vignette formats and supportive text formats are expected to outperform nonsupportive text formats in terms of refusals, break-offs, vignette nonresponse total vignette nonresponse, consequent dimension reduction, partial dimension reduction, and response inconsistency (<emph>nonsupportive text format–satisficing hypothesis</emph>).</item> </ulist> <p>Since tabular vignette formats organize the varying information in a specific vignette area, the support for information intake is seen as stronger for tabular vignettes than for text vignettes where varying information is only highlighted in a block of text. Hence, we have the following expectation:</p> <p></p> <ulist> <item> <bold> Hypothesis 1b: </bold> Tabular vignette formats are expected to outperform text vignettes with regard to refusals, break-offs, vignette nonresponse total vignette nonresponse, consequent dimension reduction, partial dimension reduction, and response inconsistency (<emph>tabular vs. text format–satisficing hypothesis</emph>).</item> </ulist> <p>Facilitating information intake should allow for a faster information intake and thus reduce the processing time of respondents who complete all cognitive steps. In accordance with Hypotheses 1a and 1b, we expect the following hypotheses concerning the processing time:</p> <p></p> <ulist> <item> <bold> Hypothesis 2a: </bold> Tabular vignette formats and supportive text formats are expected to outperform nonsupportive text formats regarding the processing time (<emph>nonsupportive text format–time hypothesis</emph>).</item> </ulist> <p>Since tabular vignette formats organize the varying information in a specific vignette area, the support for information intake is seen as stronger for tabular vignettes than for text vignettes where varying information is only highlighted in a block of text. Hence, we have the following expectation:</p> <p></p> <ulist> <item> <bold> Hypothesis 2b: </bold> Tabular vignette formats are expected to be superior to text vignettes with regard to processing time (<emph>tabular vs. text format–time hypothesis</emph>).</item> </ulist> <p>The task of picking out relevant information is assumed to be more difficult and therefore more costly as well as less motivating, the lower the reading competence of a respondent. Based on these considerations, in our <emph>presentation format-education hypothesis</emph>, we expect that comparably less well-educated respondents, that is, people with relatively low reading competence,[<reflink idref="bib14" id="ref62">14</reflink>] will profit most from supportive presentation formats:</p> <p></p> <ulist> <item> <bold> Hypothesis 3: </bold> The less well-educated a respondent is, the higher the likelihood a satisficing strategy will be chosen. The expected difference between the less well-educated and the more highly educated, however, should be smaller for a supportive presentation format than for a nonsupportive presentation format.</item> </ulist> <p>From psychological research, it is well known (cf. [<reflink idref="bib22" id="ref63">22</reflink>]) that due to cognitive aging, information intake generally takes longer for older people. Therefore, answering survey questions, ceteris paribus, should be more time-consuming and for this reason also more costly for older than for younger respondents. Based on this knowledge, we will also test in our <emph>presentation format–age hypothesis</emph> whether older people profit more from supportive presentation formats than younger people do:</p> <p></p> <ulist> <item> <bold> Hypothesis 4: </bold> Older people are more likely to choose a satisficing strategy than younger people. This difference, however, should be smaller for a supportive presentation format than for a nonsupportive presentation format.</item> </ulist> <hd id="AN0154953735-5">Operationalizations and Research Design</hd> <p></p> <hd id="AN0154953735-6">Vignette Study</hd> <p>The present study is based on a factorial survey carried out in 2012 by [<reflink idref="bib47" id="ref64">47</reflink>]. During the Internet-based survey, each participant had to evaluate a set of 16 vignettes, each describing a fictitious industrial sales representative, 35 years old, married to a spouse who is not gainfully employed, and with a monthly gross salary of €3,000. The five underlying dimensions (employee's occupation, employee's age, employee's marital status, spouse's occupational status, and employee's gross salary) were identical in each of the vignettes (fixed background information). These five dimensions were necessary to sufficiently specify the hypothetical situations and in this way to rule out different interpretations of the situation across our respondents. The vignettes varied with regard to six dimensions of substantial interest: gender (male, female), parenthood (no or two children), job experience (5 or 10 years), level of effort at work (low or high), regional average salary (€2,622, €3,009, €3,450), and regional pay inequality (range of gross salary €2,300–4,100 or €1,600–5,400).[<reflink idref="bib15" id="ref65">15</reflink>]Table 1 gives an overview of all six dimensions, their levels, and the coding of the levels.</p> <p>Graph</p> <p>Table 1. Vignette Dimensions, Vignette Characteristics and Coding of the Vignette Characteristics.</p> <p> <ephtml> <table><thead><tr><th>Factors or Dimensions Describing the Fictitious Vignette Person</th><th>Levels (Vignette Characteristics) for the Dimensions</th><th>Dummy Coding (0 = Reference Category)</th></tr></thead><tbody><tr><td colspan="3">The person H. O. (industrial sales representative, aged 35, married, spouse does not work) has further characteristics:</td></tr><tr><td rowspan="2"> Gender</td><td>Male</td><td>0</td></tr><tr><td>Female</td><td>1</td></tr><tr><td rowspan="2"> Own children</td><td>No children</td><td>0</td></tr><tr><td>2 Children</td><td>1</td></tr><tr><td rowspan="2"> Job experience</td><td>5 Years</td><td>0</td></tr><tr><td>10 Years</td><td>1</td></tr><tr><td> Effort at work</td><td>Low</td><td>0</td></tr><tr><td /><td>High</td><td>1</td></tr><tr><td>Factors or Dimensions Describing the Region Where the Fictitious Vignette Person Lives</td><td>Levels (Vignette Characteristics) for the Dimensions</td><td>Dummy Coding (0 = Reference Category)</td></tr><tr><td colspan="3">In the region in which H. O. works the following holds for industrial sales representatives:</td></tr><tr><td rowspan="3"> Average gross salary</td><td>€2,622</td><td>1</td></tr><tr><td>€3,009</td><td>0</td></tr><tr><td>€3,450</td><td>1</td></tr><tr><td rowspan="2"> Pay inequality (range of gross salary)</td><td>Between €2,300 and €4,100</td><td>0</td></tr><tr><td>Between €1,600 and €5,400</td><td>1</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>: The initial letters of the names of the fictitious vignette persons were generated randomly in order to increase the impression that different persons were described. The initials, however, do not represent a substantial dimension (which might have been the case if full names had been used instead).</p> <p>The fully crossed vignette universe of our factorial survey consists of 96 <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo stretchy="false">(</mo><mo>=</mo><mn>2</mn><mo>⋅</mo><mn>2</mn><mo>⋅</mo><mn>2</mn><mo>⋅</mo><mn>2</mn><mo>⋅</mo><mn>3</mn><mo>⋅</mo><mn>2</mn><mo stretchy="false">)</mo></mrow></math> </ephtml> vignettes. Among these vignettes, there were no implausible combinations. An interaction between one of the dichotomous and the trichotomous variables was expected. While [<reflink idref="bib12" id="ref66">12</reflink>] expose household as well as college sample respondents in personal interviews to 110 vignettes, [<reflink idref="bib40" id="ref67">40</reflink>]:41-42) fear that participants might refuse even to start answering a factorial survey when each respondent is expected to judge more than 25 vignettes in a mail survey. [<reflink idref="bib9" id="ref68">9</reflink>]:291) assume that for a representative survey, the number of vignettes per respondent should not exceed the range of 10–20 vignettes. Methodological research on this issue supports these suggestions by recommending not to use more than 20 vignettes per respondent for factorial surveys in general population samples ([<reflink idref="bib43" id="ref69">43</reflink>]).[<reflink idref="bib16" id="ref70">16</reflink>] Within the given limit for a reasonable set size (number of vignettes to be judged by a single respondent), no suitable fractional factorial design ([<reflink idref="bib2" id="ref71">2</reflink>]; [<reflink idref="bib20" id="ref72">20</reflink>]) could be found. Instead, a D-efficient design ([<reflink idref="bib30" id="ref73">30</reflink>]; cf. also [<reflink idref="bib15" id="ref74">15</reflink>], 2016) was generated by using the computer program SAS, Version 9.4. If all vignette variables are standardized orthogonally contrast coded ([<reflink idref="bib29" id="ref75">29</reflink>]:74), then D-efficiency is scaled to the range from 0 to 100 ([<reflink idref="bib30" id="ref76">30</reflink>]:547, 549). A D-efficiency of 100 will only be reached by balanced orthogonal designs. "Balanced" means that for each vignette dimension (including interaction terms that are expected to be important and for this reason are part of the design), the chosen levels within a vignette set appear with equal frequency; "orthogonal" means that variables of different dimensions (including important interaction terms) are uncorrelated ([<reflink idref="bib30" id="ref77">30</reflink>]; cf. also [<reflink idref="bib16" id="ref78">16</reflink>]). In our case, a local maximum for D-efficiency was reached for a set size of 16 vignettes per respondent. The D-efficiency for the selected quota design is 96.6591 (for further details, see [<reflink idref="bib47" id="ref79">47</reflink>]).</p> <hd id="AN0154953735-7">Experimental Settings</hd> <p>To test our hypotheses concerning the impact of different presentation formats for vignettes, three experimental settings were created: two for text vignettes (<emph>settings 1 and 2</emph>) and one for tabular vignettes (<emph>setting 3</emph>). The difference between settings 1 and 2 is that varying information was underlined for setting 2 in order to facilitate information intake (supportive presentation format) but not for setting 1 (nonsupportive presentation format), while in setting 3 varying information was presented in a column (<emph>ordinary tabular presentation format</emph>). Hence, with respect to information intake, a rank order is expected for the three settings: <emph>The higher the setting number, the easier information intake should be.</emph> In all settings, an introductory text was shown prior to presenting the 16 vignettes. Table 2 shows the introductory text, while Tables 3 and 4 depict an example vignette for different formats (a text and a tabular format, respectively).</p> <p>Graph</p> <p>Table 2. Introduction to the Factorial Survey Shown in Each Setting.</p> <p> <ephtml> <table><tbody><tr><td>On the following pages, we will present <italic>16 individuals</italic> from 16 different regions to you. Every person is <italic>35 years old</italic> and earns <italic>€3,000 per month gross</italic> as an <italic>industrial sales representative</italic>. Furthermore, all individuals are happily <italic>married</italic> to a <italic>partner who does not work</italic>.</td></tr><tr><td>However, the 16 people differ with regard to other personal characteristics. Furthermore, the working conditions in the regions in which each person lives are also different.</td></tr><tr><td>For each person, we will ask you to indicate what a fair salary should be.</td></tr></tbody></table> </ephtml> </p> <p>Graph</p> <p>Table 3. Example Text Vignette Setting 2.</p> <p> <ephtml> <table><tbody><tr><td>The person H. O. is an industrial sales representative, aged 35 years and married. His spouse does not work. <underline>Mr.</underline> H. O. has <underline>2 children</underline>. He has <underline>10 years</underline> of work experience and is well known for the <underline>high</underline> level of effort he puts in at work.</td></tr><tr><td>In the region in which H. O. works, the highest gross salary among industrial sales representatives is <underline>€4,100</underline>. On average, they earn <underline>€3,009</underline>. The lowest gross salary is <underline>€2,300</underline>.</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note</emph>: For setting 1, no underlining was used.</p> <p>Graph</p> <p>Table 4. Example Tabular Vignette Setting 3.</p> <p> <ephtml> <table><thead><tr><th colspan="2">The person H. O. (industrial sales representative, aged 35 years, married, spouse does not work) has the further characteristics:</th></tr><tr><th> Gender</th><th>Male</th></tr></thead><tbody><tr><td> Own children</td><td>2</td></tr><tr><td> Job experience</td><td>10 years</td></tr><tr><td> Effort at work</td><td>High</td></tr><tr><td colspan="2">In the region in which H. O. works, the following holds for industrial sales representatives:</td></tr><tr><td> Highest gross salary</td><td>€4,100</td></tr><tr><td> Average gross salary</td><td>€3,009</td></tr><tr><td> Lowest gross salary</td><td>€2,300</td></tr></tbody></table> </ephtml> </p> <p>Participants of each of these three settings were asked to fill in for each of the described fictitious vignette persons the gross salary which, in their opinion, would be a just earning. The answers had to be entered in an open text field.[<reflink idref="bib17" id="ref80">17</reflink>] Open answer formats have been shown to produce more missing data than closed answer formats ([<reflink idref="bib44" id="ref81">44</reflink>], [<reflink idref="bib45" id="ref82">45</reflink>]; [<reflink idref="bib53" id="ref83">53</reflink>]) and, hence, are more suitable for nonresponse analyses. Table 5 displays the open answer format. The entry for the gross salary reflects a participant's normative judgment—that is a respondent's notion of how things should be (cf. [<reflink idref="bib23" id="ref84">23</reflink>]; [<reflink idref="bib47" id="ref85">47</reflink>])—and can be called "just salary."</p> <p>Graph</p> <p>Table 5. Open Answer Format.</p> <p> <ephtml> <table><tbody><tr><td colspan="2">The described person, H. O., earns €3,000 gross a month. How much should H. O., in your opinion, <italic>justly earn</italic>?</td></tr><tr><td>□ Euros a month.</td><td>□ Don't know</td></tr></tbody></table> </ephtml> </p> <hd id="AN0154953735-8">Data</hd> <p>All participants of the present Internet-based study that was conducted at the end of September and the beginning of October 2014 are members of an online access panel. The target population consisted of persons aged 20–69 who were living in Germany in 2014. For sampling the respondents, a combined quota scheme for age, gender, and education was applied. Quotas are based on the German Census of 2011. For the completion of the survey, participants were paid a small monetary incentive (€2) by the polling agency. All in all, 498 persons who were assigned randomly to one of the three experimental settings participated in the study. In order to prevent a potential bias caused by systematic order effects, the order of the 16 vignettes of the D-efficient design was randomized for each respondent (cf. [<reflink idref="bib5" id="ref86">5</reflink>]:72-73; [<reflink idref="bib24" id="ref87">24</reflink>]:343).</p> <hd id="AN0154953735-9">Operationalizations and Method for Analyzing the Data</hd> <p>The operationalization of different strong and weak satisficing strategies follows our theoretical distinction between <emph>refusals</emph>, <emph>break-offs, vignette nonresponse, total vignette nonresponse, consequent dimension reduction</emph>, <emph>partial dimension reduction</emph>, and <emph>response inconsistency</emph>.</p> <hd id="AN0154953735-10">Refusals</hd> <p>In factorial surveys, we observe different answer patterns that can be categorized as refusals. Besides participants who evaluate none of the vignettes (implicit refusal in the original sense[<reflink idref="bib18" id="ref88">18</reflink>]), some of the respondents may show no variation in their answers to different vignettes (invalid constant answer behavior),[<reflink idref="bib19" id="ref89">19</reflink>] while other respondents answer all 16 vignettes with "don't know."[<reflink idref="bib20" id="ref90">20</reflink>] What these three answer patterns have in common is that respondents omit at least one of the four cognitive steps of our modified cognitive scheme for vignette studies from the first to the last vignette of the factorial survey, which is why these three answer patterns can be traced back to a strong form of satisficing strategy (cf. [<reflink idref="bib26" id="ref91">26</reflink>]), that is, to a respondent's nonparticipation in the factorial survey. Furthermore, in each of these three response patterns, respondents do not provide any explanation as to why they do not participate meaningful in the factorial survey. For this reason, all participants who evaluated none of the vignettes (either implicit refusals in the original sense or "don't know") or showed no variability in their answers (invalid constant answer behavior) over the 16 vignettes have been coded as refusals (<emph>refusal =</emph> 1) on a 0–1 coded dummy variable on the respondent level. Furthermore, we screened the answers of respondents whose standard deviation for the just earning was lower than €150. In cases in which we had good reasons to assume that the observed variation was caused by a simple typing error (e.g., a respondent answered to 15 vignettes €3,000 and to one vignette €300), we categorized these answers as invalid constant answer behaviors.[<reflink idref="bib21" id="ref92">21</reflink>]</p> <hd id="AN0154953735-11">Break-off</hd> <p>Another form of answer behavior can be traced back to a decreasing willingness to complete all four steps of the modified cognitive scheme. Such a decline in motivation could be caused by a high cognitive demand of the task, so that participants become increasingly fatigued or even bored. As a result, respondents might simplify the task at a certain point by switching to a strong form of satisficing: They either completely stop answering the questions or they stop their valid answer behavior by switching to an invalid constant answer behavior. Respondents with such answer patterns were coded on a respondent-level 0–1 dummy variable as stopping the factorial survey (<emph>break-off =</emph> 1) if at least the first vignette was answered and stopping occurred after the first or at the latest before the 16th vignette.</p> <hd id="AN0154953735-12">Vignette nonresponse</hd> <p>A further answer behavior consists in alternately judging a vignette and not judging a vignette. Skipping vignettes indicates a strong form of satisficing for the unjudged vignettes. For the evaluated vignettes, it remains unclear whether the respective respondent applied a weak form of satisficing or whether he or she followed the required cognitive steps carefully. In such cases, all judged vignettes might be included in the analysis. To analyze the impact of the presentation format and/or answer format on the probability of an invalid answer being given, a 0–1 coded dummy variable was computed. Vignettes of respondents who neither refused to answer the factorial survey nor broke off their participation were coded as 1 on a vignette-level dummy if a vignette was answered with "don't know" or was left unanswered (<emph>vignette nonresponse =</emph> 1).</p> <hd id="AN0154953735-13">Total vignette nonresponse</hd> <p>Certainly, a distinction between refusals, breaking-offs, and vignette nonresponse identifies the main sources of invalid answer behavior. Nonetheless, what is actually most important to researchers in the end is the total loss of information. Therefore, a further vignette-level 0–1 coded dummy variable was computed for vignettes that were judged with an invalid answer ("don't know" or unanswered), in this case, however, independently of whether the respondent was one of the participants who refused or stopped answering the factorial survey (<emph>total vignette nonresponse =</emph> 1).</p> <p>For analyzing refusals and break-offs, we estimated a respective logistic regression at the respondent level. The same was done at the vignette level for vignette nonresponse and total vignette nonresponse, respectively.[<reflink idref="bib22" id="ref93">22</reflink>] For analyzing the weak satisficing strategies and the total processing time, we restricted our sample to respondents who never used a strong satisficing strategy when they answered the factorial survey (<emph>non-strong-satisficing</emph>) because it is plausible to assume that the different answer behaviors encompassing a strong form of satisficing are differently correlated with the settings as well as with weak satisficing and processing time. Hence, the restriction to non-strong-satisficing participants is expected to increase the homogeneity of the analysis sample and, thus, the meaningfulness of our results regarding the following indicators:</p> <hd id="AN0154953735-14">Consequent dimension reduction</hd> <p>When respondents fade out a certain vignette dimension from the first to the last vignette, the respective vignette dimension has no impact on their answers (i.e., the effect size of the ignored vignette dimension is virtually 0). In order to determine the existence of consequent dimension reduction, we regressed for each respondent the answers to the 16 vignettes on the vignette dimensions using ordinary least squares (OLS) and saved the resulting respondent-specific weights of the vignette dimensions in a data file.[<reflink idref="bib23" id="ref94">23</reflink>] Subsequently, we coded a respondent-level dummy variable for each vignette dimension as 1 if the respective vignette dimension was faded out by the respondent (<emph>consequent dimension reduction =</emph> 1) and analyzed it in a logistic regression as a dependent variable. As mentioned above, consequent dimension reduction might take place due to justified reasons (information is unimportant to the respondent) or as an attempt to reduce cognitive efforts. By comparing the outcome of <emph>consequent dimension reduction</emph> across different formats, these dummy variables allow analyzing at least in relative terms whether or not a format is more prone to fade-outs as a consequence of a strong satisficing strategy regarding the number of persons who follow the strategy (<emph>person-related scope</emph>). To examine the intensity with which respondents apply this strategy, we additionally counted for each respondent the total number of coefficients with an effect size of zero (<emph>intensity-related scope</emph>) and used it as dependent variable in an OLS regression. A higher number of zero-sized coefficients suggest a higher intensity with which the strategy was performed.</p> <hd id="AN0154953735-15">Partial dimension reduction</hd> <p>When respondents consider all vignette dimensions in their subjective reference frame at the first vignette(s) of the factorial survey and start fading out a vignette dimension at the subsequent vignettes, then the effect size of the respective vignette dimension will be larger than zero in absolute terms but attenuated compared to a situation where respondents optimize on all presented vignettes. As it is difficult to distinguish between small nonzero effects as a consequence of an optimizing strategy and attenuated nonzero effects as a consequence of a partial dimension reduction strategy, we refrained from defining a cutoff value that distinguishes between the weak satisficing and the optimizing strategy. Instead, we estimated for each vignette dimension an OLS regression with respondents' coefficients for the respective vignette dimension as dependent variable.</p> <hd id="AN0154953735-16">Response inconsistency</hd> <p>For measuring response inconsistency, studies in the context of factorial surveys (cf., e.g., [<reflink idref="bib43" id="ref95">43</reflink>]; [<reflink idref="bib51" id="ref96">51</reflink>]) focused on the unexplained variance of estimation models regarding respondents' (valid) answers to vignettes. We followed the operationalization applied by [<reflink idref="bib43" id="ref97">43</reflink>]:94). On the basis of non-strong-satisficing respondents, we estimated a random intercept multilevel regression with respondents' answers to the 16 vignettes as dependent variable and the 6 vignette dimensions as predictors. In a subsequent step, we determined the squared residuals for each valid answer as an indicator for response inconsistency and used it as a dependent variable in a random intercept multilevel model for hierarchical data. Higher squared residuals reflect higher unexplained variances and, hence, higher response inconsistencies.</p> <hd id="AN0154953735-17">Total processing time</hd> <p>Para data were used in order to analyze the impact of the presentation format on the <emph>total processing time</emph> needed by respondents to judge the vignettes. Since one vignette was presented per page of the questionnaire, we were able to use the para data on the time spent per page to compute the total time a respondent needed to answer all 16 vignettes. To explain respondents' total processing time, this variable was used as dependent variable in an OLS regression.[<reflink idref="bib24" id="ref98">24</reflink>]</p> <p>In each of our models, we included the dummy variables for each respective experimental setting. We also controlled for respondents' gender (0 = male, 1 = female), age groups (age groups 20–29 years, 30–39 years, 40–49 years, 50–59 years, 60 years and older; the first age category serves as reference for the other 0–1 dummy coded age groups), and for the highest level of education. The coding for education is based on a slightly modified International Standard Classification of Education (ISCED)-97 classification, which was developed for comparison purposes within the framework of the European Values Study 2008/2009 (cf. [<reflink idref="bib17" id="ref99">17</reflink>]). By selecting ISCED 1 and 2 as reference, we used four 0–1 coded dummy variables (ISCED 3, ISCED 5, ISCED 6, and ISCED 7 and 8) for capturing the impact of education.[<reflink idref="bib25" id="ref100">25</reflink>]</p> <p>Subsequent to the estimation of each model, we computed the average adjusted predictions (AAP) for each experimental setting on the basis of the estimated model. For the calculation of AAP, several predictions are calculated for each respondent. For each prediction and each respondent, the value of the variable of interest (e.g., the setting variable) is varied according to the number of its levels, while all other independent model variables are left as they are ([<reflink idref="bib55" id="ref101">55</reflink>]). That is, in the case of a factor variable with three levels, we obtain three predictions for each respondent. The average(s) over each of these three predictions constitute the AAP. In comparison to estimations of representative values of the other independent model variables, this procedure has the advantage that the resulting predictions are not only restricted to specific values but are based on empirical values ([<reflink idref="bib55" id="ref102">55</reflink>]). In comparison to estimations of mean values of the other independent model variables, this procedure also has the advantage that the use of meaningless values can be ruled out (e.g., 0.42 for a dummy variable for female, if 42 percent of the respondents in the analysis sample are female; [<reflink idref="bib55" id="ref103">55</reflink>]). In a further step, we compared pairwise the resulting average adjusted probabilities and processing times, respectively, by using the Sidak test. The Sidak test is a test for pairwise multiple comparisons that is based on a <emph>t</emph> statistic and accounts for the multiple comparisons in the significance level ([<reflink idref="bib1" id="ref104">1</reflink>]).</p> <hd id="AN0154953735-18">Empirical Results</hd> <p></p> <hd id="AN0154953735-19">Strong Satisficing Behavior—Descriptive Results</hd> <p>Respondents were randomly assigned to one of our three experimental settings. Column 2 of Table 6 shows the distribution of the 498 participants across the resulting samples. All three settings include exactly 166 respondents. According to Table 6, 74 respondents (column 3) evaluated no vignette at all, and 17 persons (column 4) showed no variation in their answer behavior, so that in sum, 91 participants were counted as refusals. A further 29 participants stopped answering the factorial survey between the first and the last vignette (break-off). Hence, by far, the most common answer pattern for all refusals, including breaks-offs, was that of evaluating no vignette at all (74:120 = 61.7 percent). Setting 1, where nonsupportive text vignettes were used, performs worst (in total 31.3 percent refusals including break-offs), while setting 3, where tabular vignette formats were used, performs best (in total, 17.5 percent refusals including break-offs).</p> <p>Graph</p> <p>Table 6. Sample Sizes and Answer Behavior (Respondent Level).</p> <p> <ephtml> <table><thead><tr><th rowspan="2">Experimental Setting</th><th rowspan="2">Sample Size (<italic>n</italic>)</th><th>No Vignette Evaluated (Implicit Refusal, Don't Know)</th><th>Invalid Constant Answer Behavior</th><th>Refusals</th><th>Break-off</th><th>Total</th></tr><tr><th>(1)</th><th>(2)</th><th>(1) + (2)</th><th>(3)</th><th>(1) + (2) + (3)</th></tr></thead><tbody><tr><td>1</td><td>166</td><td>28</td><td>8</td><td>36 (21.7%)</td><td>16 (9.6%)</td><td>52 (31.3%)</td></tr><tr><td>2</td><td>166</td><td>28</td><td>5</td><td>33 (19.9%)</td><td>6 (3.6%)</td><td>39 (23.5%)</td></tr><tr><td>3</td><td>166</td><td>18</td><td>4</td><td>22 (13.3%)</td><td>7 (4.2%)</td><td>29 (17.5%)</td></tr><tr><td>Sum</td><td>498</td><td>74</td><td>17</td><td>91 (18.3%)</td><td>29 (5.8%)</td><td>120 (24.1%)</td></tr></tbody></table> </ephtml> </p> <p>3 <emph>Note</emph>: The reported row percentages are computed on the basis of a respective sample size. Setting 1: text vignettes, no underlining; setting 2: text vignette, varying information underlined; setting 3: tabular vignettes.</p> <p>While the results in Table 6 refer to the respondent level, similar analyses have also been carried out for the vignette level. The gross sample size of vignettes that were assigned to the three experimental settings was 7,968 (cf. Table 7). However, the gross sample of vignettes was reduced by 1,456 vignettes (18.3 percent of the gross sample of vignettes, cf. Table 7) due to the refusals (<emph>n</emph> = 91, cf. Table 6) and by 298 vignettes (3.7 percent of the gross sample of vignettes, cf. Table 7) due to the break-offs (<emph>n</emph> = 29, cf. Table 6). Moreover, vignette nonresponse decreased the number of validly judged vignettes by 278 vignettes (3.5 percent). So, total vignette nonresponse decreased the percentage of validly judged vignettes in total by 2,032 vignettes (25.5 percent). In line with Hypothesis 1b, experimental setting 3 (tabular vignette format) performs best regarding total vignette nonresponse, whereas setting 2 (supportive text format) performs worst with respect to vignette nonresponse. The percentage of unusable vignettes, however, is also quite low for setting 2 (4.3 percent). With respect to the total vignette nonresponse, setting 1 again performs worst (29.7 percent total loss of vignettes), which so far corroborates Hypothesis 1a. All in all, the rank order among the first three settings for total vignette nonresponse is setting 3, 2, 1 and, hence, in line with our expectation that tabular vignette formats outperform text formats (Hypothesis 1b).</p> <p>Graph</p> <p>Table 7. Sample Sizes and Answer Behavior (Vignette Level).</p> <p> <ephtml> <table><thead><tr><th rowspan="3">ES</th><th colspan="3">Gross Sample Size (<italic>n</italic> = 498)</th><th colspan="4">Net Sample Size After Excluding Refusals and Break-offs (<italic>n</italic> = 378)</th></tr><tr><th>Gross Sample of Vignettes</th><th>Refusals</th><th>Break-off</th><th>"Don't Know"</th><th>No Answer</th><th>Vignette Nonresponse</th><th>Total Vignette Nonresponse</th></tr><tr><th>(<italic>m</italic>)</th><th>(1) + (2)</th><th>(3)</th><th>(4)</th><th>(5)</th><th>(4) + (5)</th><th>(1) + (2) + (3) + (4) + (5)</th></tr></thead><tbody><tr><td>1</td><td>2,656</td><td>576 (21.7%)</td><td>146 (5.5%)</td><td>54</td><td>14</td><td>68 (2.6%)</td><td>790 (29.7%)</td></tr><tr><td>2</td><td>2,656</td><td>528 (19.9%)</td><td>69 (2.6%)</td><td>78</td><td>37</td><td>115 (4.3%)</td><td>712 (26.8%)</td></tr><tr><td>3</td><td>2,656</td><td>352 (13.3%)</td><td>83 (3.1%)</td><td>77</td><td>18</td><td>95 (3.6%)</td><td>530 (20.0%)</td></tr><tr><td>Sum</td><td>7,968</td><td>1,456 (18.3%)</td><td>298 (3.7%)</td><td>209</td><td>69</td><td>278 (3.5%)</td><td>2,032 (25.5%)</td></tr></tbody></table> </ephtml> </p> <p>4 <emph>Note</emph>: Gross person sample refers to all respondents, net person sample to the gross person sample excluding refusals (<emph>n</emph> = 91) and break-offs (<emph>n</emph> = 29); the gross sample of vignettes is calculated by multiplying the group size of an experimental setting by the set size (16 vignettes per respondent); the reported row percentages are based on the gross sample of vignettes assigned to each experimental setting; in contrast to "vignette nonresponse," "total vignette nonresponse" includes participants who refused or broke off the factorial survey (last column of Table 6). ES = experimental setting.</p> <hd id="AN0154953735-20">Strong Satisficing Behavior—Impact of Experimental Settings</hd> <p>Based on a logistic regression model in which we controlled for age, education, and gender of our respondents, for each separate setting, we predicted the average adjusted probabilities (cf. [<reflink idref="bib55" id="ref105">55</reflink>]) for refusals, break-offs, vignette nonresponse, and total vignette nonresponse. Analyses for refusals and break-offs are estimated at the respondent level, and analyses for vignette nonresponse and total vignette nonresponse are estimated at the vignette level. Table 8 presents the results, rank ordered by performance, separately for each answer behavior.</p> <p>Graph</p> <p>Table 8. Average Adjusted Predictions (AAP) for Different Answer Pattern Rooted in Strong Satisficing.</p> <p> <ephtml> <table><thead><tr><th rowspan="2">Rank</th><th colspan="2">Refusals: <italic>i</italic> = 91 (18.3%)</th><th colspan="2">Break-off: <italic>i</italic> = 29 (5.8%)</th><th colspan="2">Vignette Nonresponse: <italic>i</italic> = 278 (3.5%)</th><th colspan="2">Total Vignette Nonresponse: <italic>i</italic> = 2,032 (25.5%)</th></tr><tr><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th></tr></thead><tbody><tr><td>1</td><td>3</td><td>.133**</td><td>2</td><td>.033**</td><td>1</td><td>.025**</td><td>3</td><td>.201**</td></tr><tr><td>2</td><td>2</td><td>.198**</td><td>3</td><td>.046**</td><td>3</td><td>.035**</td><td>2</td><td>.266**</td></tr><tr><td>3</td><td>1</td><td>.216**</td><td>1</td><td>.097**</td><td>2</td><td>.045**</td><td>1</td><td>.297**</td></tr><tr><td /><td align="center" colspan="2">Respondent level (<italic>n</italic> = 498)</td><td align="center" colspan="2">Respondent level (<italic>n</italic> = 498)</td><td align="center" colspan="2">Vignette level (<italic>m</italic> = 7,968)</td><td align="center" colspan="2">Vignette level (<italic>m</italic> = 7,968)</td></tr></tbody></table> </ephtml> </p> <ulist> <item>5 <emph>Note</emph>: Significance levels refer to differences from zero and are tested one tailed; <emph>i</emph> = number of incidences; ES = experimental setting.</item> <item>6 *<emph>p</emph> <.05. **<emph>p</emph> <.01.</item> </ulist> <p>The main source of nonresponses at the respondent level is refusals (18.3 percent). Break-offs and vignette nonresponse are comparably very rare (5.8 percent and 3.5 percent, respectively). Since refusals make up the larger part of the total vignette nonresponse and both categories (refusals and total nonresponse) show the highest differences between different settings, they will be the main focus of our interpretation. However, the final decision concerning the best performing setting has to be based solely on the total vignette nonresponse, which acts as a summary measure that identifies the vignettes that can be used for the substantial analyses.</p> <p>In line with our expectation (Hypothesis 1b), setting 3 (tabulate format) produces on average the lowest refusal rate. Here, the average for the age, education, and gender adjusted probability of a refusal is 13.3 percent. This probability is 8.3 percent points lower than the average adjusted probability for the worst performing experimental setting, which in this case is setting 1 (AAP = 21.6 percent). For refusals and total vignette nonresponse, setting 3 is consistently followed by setting 2 (rank 2) and setting 1 (rank 3). So far, for the main sources of nonresponse, these results are in accordance with our expectation Hypothesis 1a (nonsupportive text vignettes perform worst) and Hypothesis 1b (tabular vignettes outperform text vignettes). The results for break-offs and vignette nonresponse are somewhat different: Text vignettes with underlined varying information (setting 2) outperform tabular vignettes when it comes to break-offs, whereas even nonsupportive text vignettes (setting 1) show a lower average adjusted predicted probability for vignette nonresponse than tabular vignettes do. Compared to the main sources of nonresponse (refusals and total vignette nonresponse), for these two answer patterns (break-off and vignette nonresponse), the differences in the average adjusted predicted probability are relatively small and for this reason of minor importance.</p> <p>In a further step, we used a Sidak multiple comparison test to see whether the AAP differed significantly between the settings (cf. Table 9). In setting 3 (ordinary tabular vignettes), for instance, the average adjusted probability for refusals is 8.3 percentage points smaller than setting 1 (nonsupportive text vignettes). The difference, however, is not significant. The same also applies to the other two observed differences in the AAP for refusals. With respect to the break-offs, only the difference between setting 2 (supportive text vignette) and setting 1 (nonsupportive text vignette) turns out to be significant (AAP difference = −.064, <emph>p</emph> < 0.05), corroborating the <emph>nonsupportive text format–satisficing hypothesis</emph> regarding the text vignette comparison (Hypothesis 1a).</p> <p>Graph</p> <p>Table 9. Results of Sidak Test–based Multiple Pairwise Comparisons of Average Adjusted Predictions for Different Nonresponse Pattern of Different Experimental Settings.</p> <p> <ephtml> <table><thead><tr><th align="center" colspan="3">Refusals</th><th colspan="3">Break-off</th></tr></thead><tbody><tr><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td></tr><tr><td>2</td><td>−.018</td><td /><td>2</td><td>−.064*</td><td /></tr><tr><td>3</td><td>−.083</td><td>−.065</td><td>3</td><td>−.051</td><td>0.01</td></tr><tr><td align="center" colspan="3">Vignette Nonresponse</td><td colspan="3">Total Vignette Nonresponse</td></tr><tr><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td></tr><tr><td>2</td><td>.020**</td><td /><td>2</td><td>−.030*</td><td /></tr><tr><td>3</td><td>.010*</td><td>−.011</td><td>3</td><td>−.095**</td><td>−.065**</td></tr></tbody></table> </ephtml> </p> <ulist> <item>7 <emph>Note</emph>: Directed hypotheses tested with a one-tailed test (only in cases where the expected pattern fitted the observed one); a negative sign indicates that the setting in a row outperforms the setting in a column.</item> <item>8 *<emph>p</emph> <.05. **<emph>p</emph> <.01.</item> </ulist> <p>We now turn from the analyses on the respondent level to the vignette level. The results for vignette nonresponse clearly contradict our nonsupportive text format–satisficing hypothesis (Hypothesis 1a) and our tabular versus text format-satisficing hypothesis (Hypothesis 1b): Setting 1 (nonsupportive text format) turns out to significantly outperform settings 2 and 3, while setting 3 does not perform significantly better than setting 2. However, regarding total vignette nonresponse, the assumptions stated in Hypotheses 1a and 1b, respectively, are clearly confirmed: The tabulate format shows a significantly lower predicted average adjusted probability for total vignette nonresponse than setting 1 (nonsupportive text format) and setting 2 (supportive text format), while at the same time, the supportive text format (setting 2) performs significantly better than the nonsupportive text format (setting 1). Hence, of all settings, setting 3 (ordinary tabulate format) turns out to be the best performing format, significantly outperforming all other settings. A final comparison of the total vignette nonresponse with the refusals, which make up the most part of the total vignette nonresponse, shows that the observed patterns for both are similar. Hence, one of the reasons why for refusals even relatively high differences in the AAP turned out to be insignificant is that the significance tests for the refusals are based on the number of respondents, whereas the significance tests for the total vignette nonresponse are based on the much higher number of vignettes.</p> <hd id="AN0154953735-21">Strong Satisficing Behavior—Impact of Education and Age on Strong Satisficing Behavior Across...</hd> <p>To test our <emph>presentation format-education hypothesis</emph> (Hypothesis 3) and <emph>our presentation format-age hypothesis</emph> (Hypothesis 4) in our models for refusals, break-offs, vignette-nonresponse, and total vignette nonresponse, we included interaction effects between the settings on the one hand and both education and age on the other hand. In the following, we will restrict our presentation to the models where significant differences between different settings were found as captured by interaction terms.</p> <p>Table 10 shows the AAP for total vignette nonresponse. The results are based on a logistic estimation in which we controlled for age, education, and gender and where interaction effects between age and setting, and between education and setting, were included. The AAP's of the interaction model differ only slightly from the AAP's of the logistic regression that only controlled for the main effects of age, education, and gender (cf. Table 8, column 5).</p> <p>Graph</p> <p>Table 10. Average Adjusted Predictions (AAP) for Total Vignette Nonresponse Based on Interaction Models.</p> <p> <ephtml> <table><thead><tr><th rowspan="2">Rank</th><th colspan="2">Total Vignette Nonresponse (<italic>i</italic> = 2,032)</th></tr><tr><th>ES</th><th>AAP</th></tr></thead><tbody><tr><td>1</td><td>3</td><td>.203**</td></tr><tr><td>2</td><td>2</td><td>.264**</td></tr><tr><td>3</td><td>1</td><td>.298**</td></tr><tr><td /><td align="center" colspan="2">Vignette level (<italic>m</italic> = 7,968)</td></tr></tbody></table> </ephtml> </p> <ulist> <item>9 <emph>Note</emph>: Hypotheses tested with a one-tailed test. ES = experimental setting.</item> <item>10 *<emph>p</emph> <.05. **<emph>p</emph> <.01.</item> </ulist> <p>For our three different settings, Figure 1 shows the total vignette nonresponse probabilities for the group of less well-educated persons (ISCED 1 and 2) compared to the group of highly educated persons (ISCED 7 and 8). In each of the settings 1, 2, and 3, the nonresponse probability is significantly higher for the less well-educated group compared to the highly educated group (cf. Sidak test, Table 11, column 2). Regarding our <emph>presentation format-education hypothesis</emph> (Hypothesis 3), we find that the differences between both educational groups decrease from setting 1 to setting 3. This result corroborates Hypothesis 3.</p> <p>Graph: Figure 1. The impact of education on total vignette nonresponse across different settings. Thirty-seven persons with International Standard Classification of Education (ISCED) 1 + 2 and 103 persons with ISCED 7 + 8 were randomly distributed across the three settings; predictions based on logistic model (m = 7,968); dependent variable: 1 = unit nonresponse; controlled for sex and age; confidence interval for level 95%, adjusted for multiple comparisons.</p> <p>Graph</p> <p>Table 11. Results of the Sidak Test by Setting for Total Vignette Nonresponse by Education and Age.</p> <p> <ephtml> <table><thead><tr><th rowspan="2">Experimental Setting</th><th>Education</th><th>Age</th></tr><tr><th>ISCED 7 and 8 versus ISCED 1 and 2</th><th>60 Years and Older Versus 20–29 Years</th></tr></thead><tbody><tr><td>1</td><td>−.408**</td><td>−.034</td></tr><tr><td>2</td><td>−.244**</td><td>−.185**</td></tr><tr><td>3</td><td>−.130**</td><td>−.051</td></tr></tbody></table> </ephtml> </p> <ulist> <item>11 <emph>Note</emph>: Hypotheses for education tested with a one-tailed test.</item> <item>12 *<emph>p</emph> <.05. **<emph>p</emph> <.01.</item> </ulist> <p>Figure 2 shows the total vignette nonresponse probabilities for the group of young persons (20–29 years) compared to the group of older persons (60 years and older). In settings 1, 2, and 3, the older group shows lower total vignette nonresponse rates than the younger group. However, the Sidak test (Table 11, column 3) reveals that the difference in total vignette nonresponse rates between the two age groups is significant only for setting 2. Hence, we do not find evidence for our <emph>presentation format–age hypothesis</emph> (Hypothesis 4).</p> <p>Graph: Figure 2. The impact of age on total vignette nonresponse across different settings. Ninety-one persons aged 20–29 years and 87 persons aged 60 years and older were randomly distributed across the three settings; predictions based on logistic model (m = 7,968); dependent variable: 1 = unit nonresponse; controlled for sex and education; confidence interval for level 95%, adjusted for multiple comparisons.</p> <hd id="AN0154953735-22">Weak Satisficing Behavior and Processing Time—Impact of Experimental Settings</hd> <p>To increase the meaningfulness of our results (cf. Operationalizations and Method for Analyzing the Data subsection), the analysis of weak satisficing behavior and processing time is based on the 322 non-strong-satisficing respondents who answered all 16 vignettes in a formally valid manner.</p> <hd id="AN0154953735-23">Consequent dimension reduction</hd> <p>Table 12 (cf. column 2) shows that 84 respondents (85.7 percent) in setting 1 considered all vignette dimensions in their answers (setting 2: 88.5 percent, setting 3: 90.8 percent), while 14 respondents (14.3 percent) applied a consequent dimension reduction strategy by consequently fading out at least one of the dimensions presented on the vignettes (setting 2: 11.5 percent, setting 3: 9.2 percent). In setting 1, 38 (5.5 percent) of the 686 (= 98 × 7) estimated coefficients were equal to 0, while 21 of the 728 (= 104 × 7) coefficients in setting 2 and 21 of the 840 (= 120 × 7) coefficients in setting 3, respectively, were equal to 0. Hence, in line with Hypotheses 1a and 1b, setting 1 performs worst regarding the person-related scope of the consequent dimension reduction strategy as well as in terms of its intensity. Corroborating Hypothesis 1b, setting 3 (tabular format) outperforms setting 2 (supportive text format) in terms of the person-related scope of the consequent dimension reduction strategy as well as in terms of the intensity of this strategy. Taken together, these results suggest that for the vast majority of respondents who answered all 16 vignettes, the subjective reference frame corresponded to the objective reference frame that we designed to examine the impact factors of ideas on just earnings.</p> <p>Graph</p> <p>Table 12. Consequent Dimension Reduction.</p> <p> <ephtml> <table><thead><tr><th rowspan="2" /><th rowspan="2" /><th colspan="2">Person-related Scope (<italic>n</italic> = 322)</th><th rowspan="2" /><th colspan="2">Intensity-related Scope (<italic>w</italic> = 2254)</th></tr><tr><th>A</th><th>B</th><th>C</th><th>D</th></tr></thead><tbody><tr><td>Setting 1 (<italic>n</italic> = 98)</td><td><italic>n</italic> (%)</td><td>84 (85.71)</td><td>14 (14.29)</td><td><italic>w</italic> (%)</td><td>648 (94.46)</td><td>38 (5.54)</td></tr><tr><td>Setting 2 (<italic>n</italic> = 104)</td><td><italic>n</italic> (%)</td><td>92 (88.46)</td><td>12 (11.54)</td><td><italic>w</italic> (%)</td><td>707 (97.12)</td><td>21 (2.88)</td></tr><tr><td>Setting 3 (<italic>n</italic> = 120)</td><td><italic>n</italic> (%)</td><td>109 (90.83)</td><td>11 (9.17)</td><td><italic>w</italic> (%)</td><td>819 (97.50)</td><td>21 (2.50)</td></tr><tr><td>Total</td><td><italic>n</italic> (%)</td><td>285 (88.51)</td><td>37 (11.49)</td><td><italic>w</italic> (%)</td><td>2,174 (96.45)</td><td>80 (3.55)</td></tr></tbody></table> </ephtml> </p> <p>13 <emph>Note</emph>: <emph>n</emph> = number of non-strong-satisficing respondents; <emph>w</emph> = number of respondent-specific coefficients; A = respondents who did not consequently fade-out any of the vignette dimensions; B = respondents who consequently faded-out at least one vignette dimension; C = coefficients of size not equaling zero; D = coefficients of size zero.</p> <p>Table 13 shows the average adjusted probability for applying a consequent dimension reduction strategy (cf. Table 13, person-related scope) and the average adjusted number of fade outs per respondent (cf. Table 13, intensity). The results are based, respectively, on a logistic regression and an OLS regression in which we took age, education, and gender into account. As expected in Hypothesis 1b, setting 3 (tabular format) performs best regarding both issues of the consequent dimension reduction strategy; setting 2 performs better than setting 1, corroborating Hypothesis 1a. The average adjusted probability to apply this weak form of satisficing amounts to 13.9 percent in setting 1 and 9.2 percent in setting 3; at the same time, respondents in setting 1 on average faded out more dimensions (.377) than respondents in setting 3 (.183). However, the difference is not significant in statistical terms, as the results of the Sidak test show in Table 14. None of the examined presentation formats are particularly more prone to consequent dimension reduction.</p> <p>Graph</p> <p>Table 13. Average Adjusted Predictions (AAP) for Different Answer Pattern Rooted in Weak Satisficing.</p> <p> <ephtml> <table><thead><tr><th rowspan="3">Rank</th><th colspan="4">Consequent Dimension Reduction</th><th colspan="14">Partial Dimension Reduction</th><th /><th /><th /><th /></tr><tr><th colspan="2">Person-related Scope <italic>n</italic> = 37 (11.5%)</th><th colspan="2">Intensity-related Scope <italic>i</italic> = 80 (3.6%)</th><th colspan="2">Vignette Dimension Gender</th><th colspan="2">Vignette Dimension Parenthood</th><th colspan="2">Vignette Dimension Job Experience</th><th colspan="2">Vignette Dimension Effort at Work</th><th colspan="2">Vignette Dimension Pay Inequality</th><th colspan="2">Vignette Dimension Low Average Salary</th><th colspan="2">Vignette Dimension High Average Salary</th><th colspan="2">Response Inconsistency</th><th colspan="2">Processing Time for all 16 Vignettes (in Seconds)</th></tr><tr><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th><th>ES</th><th>AAP</th></tr></thead><tbody><tr><td>1</td><td>3</td><td>.092**</td><td>3</td><td>.183**</td><td>2</td><td>43.50</td><td>3</td><td>229.90**</td><td>3</td><td>128.76**</td><td>1</td><td>850.68**</td><td>1</td><td>139.13**</td><td>1</td><td>−65.02**</td><td>1</td><td>122.72**</td><td>3</td><td>453,163**</td><td>3</td><td>334.5**</td></tr><tr><td>2</td><td>2</td><td>.118**</td><td>2</td><td>.203**</td><td>3</td><td>41.57</td><td>1</td><td>171.59**</td><td>2</td><td>98.69**</td><td>2</td><td>774.18**</td><td>3</td><td>73.57**</td><td>2</td><td>−37.23*</td><td>3</td><td>119.10**</td><td>1</td><td>417,526**</td><td>1</td><td>377.1**</td></tr><tr><td>3</td><td>1</td><td>.139**</td><td>1</td><td>.377**</td><td>1</td><td>−16.99</td><td>2</td><td>127.78**</td><td>1</td><td>81.56**</td><td>3</td><td>728.60**</td><td>2</td><td>16.23</td><td>3</td><td>−31.89</td><td>2</td><td>105.17**</td><td>2</td><td>347,868**</td><td>2</td><td>383.1**</td></tr><tr><td /><td align="center" colspan="4">Respondent level (<italic>n</italic> = 322)</td><td align="center" colspan="14">Respondent level (<italic>n</italic> = 322)</td><td align="center" colspan="4">Respondent level (<italic>n</italic> = 322a)</td></tr></tbody></table> </ephtml> </p> <ulist> <item>14 <sups>a</sups>With respect to the processing time, nine outliers were identified and therefore excluded from analyses (cf. footnote 21).</item> <item>15 <emph>Note</emph>: Significance levels refer to differences from zero and are tested one tailed, except for partial dimension direction where no directed hypotheses (positive or negative relationship) were formulated. ES = experimental setting; <emph>i</emph> = number of incidences.</item> <item>16 *<emph>p</emph> <.05. **<emph>p</emph> <.01.</item> </ulist> <p>Graph</p> <p>Table 14. Results of Sidak Test–based Multiple Pairwise Comparisons of Vignette Dimensions' Average Effect Sizes in Different Experimental Settings.</p> <p> <ephtml> <table><thead><tr><th align="center" colspan="3">Consequent Dimension Reduction (Person-related Scope)</th><th colspan="3">Consequent Dimension Reduction (Intensity-related Scope)</th><th colspan="3">Partial Dimension Reduction (Gender)</th></tr></thead><tbody><tr><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td></tr><tr><td>2</td><td>−.021</td><td /><td>2</td><td>−.175</td><td /><td>2</td><td>60.49</td><td /></tr><tr><td>3</td><td>−.047</td><td>−.026</td><td>3</td><td>−.194</td><td>−.020</td><td>3</td><td>58.57</td><td>−1.92</td></tr><tr><td align="center" colspan="3">Partial Dimension Reduction (Parenthood)</td><td align="center" colspan="3">Partial Dimension Reduction (Job Experience)</td><td align="center" colspan="3">Partial Dimension Reduction (Effort at Work)</td></tr><tr><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td></tr><tr><td>2</td><td>−43.81</td><td /><td>2</td><td>17.13</td><td /><td>2</td><td>−76.50</td><td /></tr><tr><td>3</td><td>58.31</td><td>102.11</td><td>3</td><td>47.20</td><td>30.07</td><td>3</td><td>−122.08</td><td>−45.58</td></tr><tr><td align="center" colspan="3">Partial Dimension Reduction (Pay Inequality)</td><td align="center" colspan="3">Partial Dimension Reduction (Low Average Salary)</td><td align="center" colspan="3">Partial Dimension Reduction (High Average Salary)</td></tr><tr><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td><td>Versus</td><td>1</td><td>2</td></tr><tr><td>2</td><td>−122.90**</td><td>57.34</td><td>2</td><td>27.79</td><td>5.33</td><td>2</td><td>−17.55</td><td>−13.94</td></tr><tr><td>3</td><td>−65.56</td><td /><td>3</td><td>33.12</td><td /><td>3</td><td>−3.61</td><td /></tr><tr><td /><td align="center" colspan="3">Response Inconsistency</td><td /><td align="center" colspan="3">Processing Time for Judging All 16 Vignettes</td><td /></tr><tr><td /><td>Versus</td><td>1</td><td>2</td><td /><td>Versus</td><td>1</td><td>2</td><td /></tr><tr><td /><td>2</td><td>−69,658</td><td /><td /><td>2</td><td>5.99</td><td /><td /></tr><tr><td /><td>3</td><td>35,637</td><td>105,295</td><td /><td>3</td><td>−42.61</td><td>−48.60*</td><td /></tr></tbody></table> </ephtml> </p> <ulist> <item>17 <emph>Note</emph>: Hypotheses tested one tailed, expect for partial dimension reduction where no directed hypotheses were formulated; a negative sign indicates that respondents of the setting in a row assigned on average a smaller amount to the fictitious person than respondents of the setting in a column.</item> <item>18 *<emph>p</emph> <.05. **<emph>p</emph> <.01.</item> </ulist> <hd id="AN0154953735-24">Partial dimension reduction</hd> <p>To account for respondents whose subjective reference frame was not constant across all 16 vignettes, we compared the average coefficients of the vignette dimensions across the settings. Table 13 shows for each vignette dimension its AAP in the respective setting. These AAP reflect the setting-specific adjusted isolated effects of a dimension on respondents' judgments. For instance, in setting 1, respondents assigned in accordance with existing stereotypes on average a lower salary (by €16.99) to fictitious vignette persons when these were females rather than males, while respondents in setting 3 contrary to existing stereotypes assigned to fictitious female vignette persons on average a €41.57 higher salary than to fictitious male vignette persons (this value is very close to €43.50, i.e., the average of setting 2).[<reflink idref="bib26" id="ref106">26</reflink>] However, in both settings, the vignette dimension gender was not significantly (α =.05) different from zero, meaning that male and female fictitious vignette persons were assigned the same salary.</p> <p>According to Hypotheses 1a and 1b, we would expect that for each vignette dimension, the AAP would be largest, in absolute terms, in setting 3 and smallest in setting 1. However, the rank orders of the AAP (in absolute terms) presented in Table 13 do not systematically match our expectation, as is the case, for example, with the vignette dimension effort at work (cf. Table 13, vignette dimension effort at work). Even more, the results of the Sidak test (cf. Table 14) show that only one comparison among the 21 comparisons for partial dimension reduction yielded a significant difference between the effect sizes. Respondents in setting 2 assigned on average €122.90 less to a fictitious vignette person than respondents in setting 1, if the pay inequality in the fictitious region was described as being high. This single significant difference at least partially undermines our expectation (Hypothesis 1a).</p> <hd id="AN0154953735-25">Response inconsistency</hd> <p>In order to retrieve the squared residuals of this estimation model, we estimated a random intercept multilevel model with respondents' answers to the 16 vignettes as dependent variable and the vignette dimensions as independent variables of the model. The squared residuals serve as indicator for response inconsistency (higher positive values mean higher inconsistency). They were used as dependent variable in a separate random intercept multilevel model in which we examined the impact of the presentation format on response inconsistency, while controlling for respondents' education, age, processing time, and the vignette's position. The AAP presented in Table 13 (cf. response inconsistency) were calculated on the basis of this model. Against our expectation (Hypothesis 1b), setting 3 (tabular vignette) performs worse than the settings in which text-based vignettes were used. In line with Hypothesis 1a, supportive text vignettes had a lower response inconsistency than nonsupportive text vignettes.[<reflink idref="bib27" id="ref107">27</reflink>] However, we do not find any significant differences across the three experimental settings using the Sidak test that accounts for multiple comparisons (cf. Table 14).</p> <hd id="AN0154953735-26">Processing time</hd> <p>The last column of Table 13 gives an overview of the average adjusted processing time. In accordance with our <emph>tabular versus text format–time</emph> hypothesis (Hypothesis 2b), setting 3 performs best: Respondents needed 334.5 seconds on average to judge all 16 vignettes, 48.6 seconds less than in setting 2, which performed worst. Against our <emph>nonsupportive text format–time</emph> hypothesis (Hypothesis 2a), the supportive text vignette format (setting 2) performs with an adjusted processing time of 383.1 seconds worse than the nonsupportive text format (setting 1). However, the difference between both text formats is negligible (six seconds).[<reflink idref="bib28" id="ref108">28</reflink>]</p> <p>According to Hypothesis 2a, the processing time for a supportive presentation format should be shorter than the processing time for a nonsupportive presentation format. Empirically, this expectation does not hold for the comparison between setting 1 (nonsupportive text vignettes) and setting 2 (supportive text vignettes, AAP difference = 5.99, <emph>p</emph> > 0.05, cf. Table 14). Hypothesis 2b, however, is at least partially confirmed empirically; the processing time for tabular vignette format (setting 3) is significantly lower than the processing time for the supportive text format (setting 2) but not significantly lower than the nonsupportive text format (setting 3).</p> <hd id="AN0154953735-27">Discussion and Conclusions</hd> <p>Former methodological studies strengthened the confidence in the factorial survey by examining issues of design complexity in terms of vignette dimension number, vignette number, and order effects. In this study, we addressed another design-related issue by examining the impact of the presentation format of vignettes (tabular vs. text) on the response behavior in factorial surveys. For our Internet-based study, we recruited 498 individuals living in Germany according to a quota plan with crossed quota on age, sex, and education. Our respondents were randomly assigned to one of the three experimental settings. In each of the experimental settings, respondents were presented with 16 vignettes. The presentation format varied across the three experimental settings (nonsupportive and supportive text vignette format, tabular vignette format). Respondents' answers to the vignette judgment task with open answer format were analyzed regarding different types of strong satisficing strategies (refusal, break-off, vignette nonresponse) as well as different types of weak satisficing strategies (consequent dimension reduction, partial dimension reduction, response inconsistency).</p> <p>For the respondent level, refusal rates varied in our factorial survey between 13.3 percent in setting 3 (tabular vignette format) and 21.7 percent in setting 1 (nonsupportive text vignette format). We used an open answer format and did not force our respondents to answer each judgment task by explicitly offering a "don't know" option. The open answer format has been shown to result in higher missing value rates than closed answer formats ([<reflink idref="bib44" id="ref109">44</reflink>], [<reflink idref="bib45" id="ref110">45</reflink>]; [<reflink idref="bib53" id="ref111">53</reflink>]) and offering a "don't know" option, which make it easier for respondents to strongly satisfice by simply answering "don't know" ([<reflink idref="bib26" id="ref112">26</reflink>]; [<reflink idref="bib28" id="ref113">28</reflink>]; [<reflink idref="bib27" id="ref114">27</reflink>]). Even though refusal rates might seem to be high in their absolute level, they still allow examining them in relative terms by comparing the presentation formats regarding the tendency to refuse participation in the factorial survey. Although the refusal rate was 8.3 percent points lower for tabular vignettes than for nonsupportive text vignettes, none of the presentation formats significantly increased the probability of refusing to participate in the factorial survey. Breaking off the factorial survey occurred systematically more often with nonsupportive text format vignettes (setting 1) than with supportive text format vignettes (setting 2), whereas break-off rates in setting 3 (tabular format) were not significantly different from break-off rates in the text format vignettes (settings 1 and 2). For future studies, it might be good to have a higher sample size on the respondent level, which would also allow to detect comparably small effects. Apart from that, although different nonresponse patterns shed light on different sources of nonresponse, using the total vignette nonresponse as a summary measure for all vignettes that are not available for substantial analyses is more important for the researcher. By using total vignette nonresponse as the central criterion for selecting the best performing presentation format, our results clearly show that setting 3 (tabular presentation format) performed best regarding strong satisficing.</p> <p>Beside these findings, which on average hold for all respondents, we examined whether our outcomes differ with regard to a respondent's education and age, respectively. Concerning total vignette nonresponse, we found that individuals with the highest level of education perform significantly better than persons with the lowest level of education. This applies to all presentation formats. The smallest difference between these two groups was found in setting 3, which indicates that the lowest educated could profit most from using a tabular format. Regarding age, we found that respondents aged 60 and older did show lower instead of higher total vignette nonresponse rates than respondents aged between 20 and 29 years. The difference, however, was only significant for the supportive text format. The former result might be explained by higher perseverance of older respondents, which compensates for their slower information intake. Taken together, systematic information loss and processing time were minimized by setting 3 (table presentation format).</p> <p>While table formats performed best regarding strong satisficing, our study reveals that response behavior grounded in weak satisficing was not triggered by any of the presentation formats. That is, we found no statistical evidence for differences across the three different presentation formats concerning consequent dimension reduction, partial dimension reduction, and response inconsistency. In contrast to answer behaviors rooted in strong satisficing, answer behaviors rooted in weak satisficing are less easy to detect for researchers. Respecting the processing time, we found that setting 3 (tabular presentation format) performed significantly better than supportive text format vignettes (settings 2), while there was no systematic difference between settings 3 and 1 (nonsupportive text format). Furthermore, analysis of consequent dimension reduction showed that in the process of judgment formation, the vast majority of respondents, who rated all 16 vignettes, did not fade out in their subjective reference frame any of the vignette dimensions of the objective reference frame that we designed for this study. This result might help to explain why respondents' answers in factorial surveys can be expected to be ascertained with higher reliability and higher validity than with single survey items in typical survey research. Finally, in line with earlier studies, our results point in the direction that for sensitive topics where perceived public norms of "political correctness" exist, supportive formats are more prone to social desirability bias than nonsupportive formats. In order to reduce such potential biases, it might be recommended to present only one vignette per respondent in a nonsupportive format.</p> <hd id="AN0154953735-28">Acknowledgments</hd> <p>The authors thank the Research Training Group "Social Order and Life Chances in Cross-National Comparison" at the University of Cologne, which is funded by the German Science Foundation, for funding this study. The authors also thank the anonymous reviewers for their constructive comments and suggestions.</p> <hd id="AN0154953735-29">Notes</hd> <ref id="AN0154953735-30"> <title> Notes </title> <blist> <bibl id="bib1" idref="ref2" type="bt">1</bibl> <bibtext> The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref3" type="bt">2</bibl> <bibtext> The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This work is funded by Research Training Group "Social Order and Life Chances in Cross-National Comparison" at the University of Cologne (grant no. GRK 1461).</bibtext> </blist> <blist> <bibl id="bib3" idref="ref4" type="bt">3</bibl> <bibtext> Hawal Shamon https://orcid.org/0000-0001-6304-9301</bibtext> </blist> <blist> <bibl id="bib4" idref="ref5" type="bt">4</bibl> <bibtext> For example, respondents asked to assess the prospects for the local economy based on their responses on the local unemployment rate and prospects for employment growth ([33]).</bibtext> </blist> <blist> <bibl id="bib5" idref="ref14" type="bt">5</bibl> <bibtext> A basic idea of factorial surveys is to combine the high internal validity of experimental designs with the high external validity of survey research (cf.[4]:128; [40]:15-16; [49]:377-78). The experimental setting of a factorial survey increases the internal validity (the observed variation in the outcome variable is caused by the experimental stimuli). The high external validity of survey research is based on using a random sample from the population, which allows the results to be generalized to the population as a whole. If a nonrandom sample is used, the factorial survey still permits general conclusions about causal mechanisms (high internal validity; cf. also [5]:11-12, 60-64).</bibtext> </blist> <blist> <bibl id="bib6" idref="ref25" type="bt">6</bibl> <bibtext> For instance, [43] recommend for factorial surveys administered to general population samples to use 20 or less vignettes with less than 12 dimensions.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref39" type="bt">7</bibl> <bibtext> An advantage of using a tabular format is, according to [5]; cf. also [8]), that it allows to randomize the vignette dimensions in order to avoid possible order effects. For text vignettes, this might conflict with a smooth text flow. The substantive logic of the order of the vignette dimension, however, does probably in most cases, also for table vignettes, not allow a randomization of the dimensions. It, furthermore, might also confuse the respondents.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref26" type="bt">8</bibl> <bibtext> Tabular formats may not be useful if researchers need to specify vignette dimensions by using one or more sentence(s) to ensure the pragmatic understanding by respondents. For instance, in a country where sudden electricity import interruptions have never lead to unplanned interruptions in the power supply (a situation that occurred, e.g., in Ukraine), it might be insufficient to describe on the vignettes that a hypothetical country imports 10 percent (vs. 70 percent) of the consumed electricity from another country. Instead, it is advisable to operationalize this energy security issue by a sentence such as: "Country A is dependent on other countries because it imports 10 percent (vs. 70 percent) of its electricity consumption." One might also argue that the tabular format cannot be applied for face-to-face interviews, since only the text format allows the interviewee to understand the vignette text read aloud by the interviewer. However, for face-to-face interviews, it is best practice to support information intake by using show cards (cf., for instance, the [18]). Without using show cards for the vignettes, one might furthermore assume that neither the table format nor the supportive text format would be applicable any longer. Although abandoning the use of vignette show cards is surely not ideal, highlighting words of text vignettes might also be done by emphasizing the pronunciation of the varying text information or for table vignettes by informing the respondents that the interviewer will read aloud different categories such as income and thereafter specify the respective amount for different situations or persons. How far table formats work for developing countries with a different cultural background (for research on factorial surveys, for instance, conducted in different African countries; cf.[31]) remains a question that should be addressed by future research.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref18" type="bt">9</bibl> <bibtext> Choosing the first best solution is a <emph>weak form of satisficing</emph>, since all cognitive steps have been completed, although not with due diligence. Choosing a "don't know" category in order to avoid the step of judgment formation based on retrieved information is seen as a <emph>strong form of satisficing</emph> ([26]). Constant answer behavior across different items of an item battery is, at least sometimes, another possible form of <emph>strong satisficing</emph>.</bibtext> </blist> <blist> <bibtext> A vignette question is a question that follows and refers to a vignette and makes explicit, from which aspect the vignette is supposed to be judged by a respondent.</bibtext> </blist> <blist> <bibtext> Semantic understanding means that it must be clear to a respondent what the meaning of a question, a formulation, or a term is; pragmatic understanding implies that it should be clear to a respondent what the researcher or interviewer intends to find out with the question ([37]:18-23).</bibtext> </blist> <blist> <bibtext> Consequent dimension reduction does only reflect weak satisficing if respondents aim at reducing cognitive effort in answering the vignette questions.</bibtext> </blist> <blist> <bibtext> It should be mentioned that one can expect a trade-off between consequent dimension reduction and response inconsistency (cf. also [6]:89) as well as between partial dimension reduction and response inconsistency. While consequent dimension reduction can be expected to decrease response inconsistency, partial dimension reduction can be expected to increase response inconsistency.</bibtext> </blist> <blist> <bibtext> According to the empirical results of [32], a strong relationship exists between reading competencies and formal education. Education instead of reading competence is used for this study because for an online survey, it is more difficult to capture reading competencies. Furthermore, good items for measuring such skills developed by the [38] are not published.</bibtext> </blist> <blist> <bibtext> In order to avoid potential confusion about the meaning of the dimensions and levels, which results in a potential bias of the estimated effects, dimensions and levels should ideally be formulated very precisely, which allows all respondents to interpret them unambiguously (cf. also [5]:21). This is easy to realize in cases where natural units (for instance, number of children) or established predefined units (for instance, income in Dollar) exist. In situations, however, where natural units do not exist, even textbooks prefer, for instance, to use relatively vague levels as "only few" and "a lot of" for job experience or "short" and "long" for job tenure (cf.[5]:21). Since in our factorial survey, there exists no natural or predefined unit for the level of work, we used instead the terms "low" and "high" for the reason that the participants are familiar with these labels from daily life conversations.</bibtext> </blist> <blist> <bibtext> This recommendation should not be misunderstood as a golden rule that applies to all factorial surveys. Rather, researchers are encouraged to reflect on the cognitive abilities of their target population, the survey mode, on the cognitive demands of their substantive question and on practical reasons (e.g., the necessity to conduct respondent-specific estimations) when deciding on a reasonable set size for their research endeavor.</bibtext> </blist> <blist> <bibtext> The open text field allowed respondents to enter numeric values between €0 and €15,000. This restriction was implemented in order to prevent typing errors. Values outside this range produced an automatic warning.</bibtext> </blist> <blist> <bibtext> American Association for Public Opinion Research ([3]:37) recommends to classify respondents of Internet surveys who visit the survey's URL and log in with an ID and/or password but leave all survey items blank without providing any explanation as implicit refusals. In the same vein, AAPOR (2015:29) recommends to classify respondents of mail surveys who return entirely blank questionnaires without any remarks as to why the questionnaire was returned as implicit refusals.</bibtext> </blist> <blist> <bibtext> In some factorial surveys, constant answer behavior might be a valid answer pattern: In a factorial survey about discrimination, for instance, constant answer behavior could indicate that a respondent treats all people equally (no discrimination). In our case, constant answer behavior may indicate a respondent's preference for the principle that everybody should be paid equally irrespective of personal or contextual factors. However, we could logically rule out such an interpretation because none of the respondents with constant answer behavior stated a clear preference for the equality principle in the conventional survey item following the factorial survey.</bibtext> </blist> <blist> <bibtext> While a "don't know" option conceptually allows respondents to give an adequate answer to a question when information retrieval does not result in sufficient information for judgment formation (cf. also [36]), this response option also constitutes an opportunity for respondents to implement strong satisficing by saying "don't know" (cf.[27]). This means that "don't know" in response to a survey question is often "another way of saying 'I don't want to get involved'" ([11]:344). In our self-administered factorial survey, vignettes provide respondents with (sufficient) information to respond to a relatively unambiguous judgment task. Therefore, it can be assumed that respondents choose the "don't know" option because they are not motivated to form a judgment.</bibtext> </blist> <blist> <bibtext> According to this operationalization, respondents who rated fewer than 16 vignettes in a constant manner are also counted as refusals. Following this procedure, we categorized the answer behavior of two respondents in setting 1 and of one respondent in setting 3 as invalid constant answer behavior. The results of our study presented in following section were robust toward these changes.</bibtext> </blist> <blist> <bibtext> We also estimated a survival model in the event of a break-off to assess whether the presentation format affects the timing of the decision to break off the factorial survey. However, the break-offs in our sample were too few to estimate a survival model that accounts for all necessary interaction effects.</bibtext> </blist> <blist> <bibtext> For the estimation model, we used the coding scheme presented in Table 1.</bibtext> </blist> <blist> <bibtext> Respondents of an Internet survey can interrupt the survey whenever they want. Therefore, the total time needed to finish the interview can become quite long. In order to correct for such cases, it was decided to exclude respondents who needed more than three times of the interquartile range to complete the interview. This criterion was fulfilled by nine participants.</bibtext> </blist> <blist> <bibtext> None of our respondents belonged to International Standard Classification of Education 4.</bibtext> </blist> <blist> <bibtext> Conscious or unconscious stereotypes in our societies, ceteris paribus, privilege males over females regarding their income, a result that unconsciously also applies to highly educated females ([35]; cf. also [39]: XV). Even though factorials surveys are less prone to <emph>social desirability bias</emph> than directly asked single-item questions ([7]:143-44; cf. also [5]:4, 11), [10] could show that factorial surveys are also susceptible to social desirability bias in case of sensitive topics where perceived public norms of "political correctness" exist (e.g., discrimination of minorities, income discrimination of females).[5] expect that supported presentation formats are more affected by social desirability bias than unsupported formats, since for table formats the respondents' attention is "more clearly bound to the manipulation of the researcher" (p. 71), whereas for supported text formats, "respondents can more easily identify the highlighted dimensions" (p. 72). In our study, the observed differences for the dimension "gender" tend to support the direction of a social desirability bias among the supportive formats as assumed by [5]; cf. Table 13). If tested one tailed, however, only the difference between the two text formats (setting 2 vs. setting 1) would become significant (43.50 − (−16.99) = 60.49, <emph>p</emph> =.041).</bibtext> </blist> <blist> <bibtext> The response inconsistency in setting 3 is not significantly lower than in setting 1 (α =.05).</bibtext> </blist> <blist> <bibtext> We replicated the analysis regarding the processing time on the basis of 364 respondents who answered eight or more vignettes (whereas 14 outliers were disregarded from the analysis) and on the basis of all 392 respondents who answered at least one vignette (whereas 14 outliers were disregarded from the analysis). These changes did not affect the pattern in the processing times reported in Table 13.</bibtext> </blist> </ref> <ref id="AN0154953735-31"> <title> References </title> <blist> <bibtext> Abdi Hervé. 2007. "The Bonferonni and Šidák Corrections for Multiple Comparisons." Pp. 103–7 in Encyclopedia of Measurement and Statistics, edited by Salkind Neil J.. Thousand Oaks, CA: Sage.</bibtext> </blist> <blist> <bibtext> Alexander Cheryl S., Becker Henry Jay. 1978. " The Use of Vignettes in Survey Research." Public Opinion Quarterly42(1):93–104.</bibtext> </blist> <blist> <bibtext> American Association for Public Opinion Research. 2015. Standard Definitions—Final Dispositions of Case Codes and Outcome Rates for Surveys. Retrieved September 25, 2018 (https://<ulink href="http://www.aapor.org/AAPOR%5fMain/media/MainSiteFiles/Standard-Definitions2015%5f8thEd.pdf">www.aapor.org/AAPOR%5fMain/media/MainSiteFiles/Standard-Definitions2015%5f8thEd.pdf</ulink>).</bibtext> </blist> <blist> <bibtext> Atzmüller Christiane, Steiner Peter M.. 2010. " Experimental Vignette Studies in Survey Research." Methodology: European Journal of Research Methods for the Behavioral and Social Sciences6(3):128–38.</bibtext> </blist> <blist> <bibtext> Auspurg Katrin, Hinz Thomas. 2015. Factorial Survey Experiments: Quantitative Applications in the Social Sciences 175. Thousand Oaks, CA: Sage.</bibtext> </blist> <blist> <bibtext> Auspurg Katrin, Hinz Thomas, Liebig Stefan. 2009. " Komplexität von Vignetten, Lerneffekte und Plausibilität im Faktoriellen Survey." Methoden, Daten, Analysen3(1):59–96.</bibtext> </blist> <blist> <bibtext> Auspurg Katrin, Hinz Thomas, Liebig Stefan, Sauer Carsten. 2015. "The Factorial Survey as a Method for Measuring Sensitive Issues." Pp. 137–49 in Improving Survey Methods. Lessons from Recent Research, edited by Engels Uwe, Jann Ben, Lynn Peter, Scherpenzeel Annette, Sturgis Patrick. New York: Routledge.</bibtext> </blist> <blist> <bibtext> Auspurg Katrin, Jäckle Annette. 2017. " First Equals Most Important? Order Effects in Vignette-based Measurement." Sociological Methods & Research46(3):490–539.</bibtext> </blist> <blist> <bibtext> Beck Michael, Opp Karl-Dieter. 2001. " Der faktorielle Survey und die Messung von Normen." Kölner Zeitschrift für Soziologie und Sozialpsychologie53(2):283–306.</bibtext> </blist> <blist> <bibtext> Beyer Heiko, Liebe Ulf. 2015. " Three Experimental Approaches to Measure the Social Context Dependence of Prejudice Communication and Discriminatroy Behavior." Social Science Research49:343–55.</bibtext> </blist> <blist> <bibtext> Bogart Leo. 1967. " No Opinion, Don't Know, and Maybe No Answer." The Public Opinion Quarterly31(3):331–345.</bibtext> </blist> <blist> <bibtext> Bose Christine E., Rossi Peter. H.. 1983. " Gender and Jobs: Prestige Standings of Occupations as Affected by Gender." American Sociological Review48(3):316–30.</bibtext> </blist> <blist> <bibtext> Converse Philip. 1964. "The Nature of Belief Systems in Mass Publics." Pp. 206–61 in Ideology and Discontent, edited by Apter David. New York: Free Press.</bibtext> </blist> <blist> <bibtext> Dillman Don A.1978. Mail and Telephone Surveys: The Total Design Method. New York: Wiley Interscience.</bibtext> </blist> <blist> <bibtext> Dülmer Hermann. 2007. " Experimental Plans in Factorial Surveys." Sociological Methods & Research35(3):382–409.</bibtext> </blist> <blist> <bibtext> Dülmer Hermann. 2016. " The Factorial Survey: Design Selection and Its Impact on Reliability and Internal Validity." Sociological Methods & Research45(2):304–47.</bibtext> </blist> <blist> <bibtext> Dülmer Hermann, Jagodzinski Wolfgang, Siegers Pascal. 2008. Classification Scheme for the Modified Education Variable of the EVS 1999/2000. Cologne: Mimeo.</bibtext> </blist> <blist> <bibtext> European Social Survey. 2016. United Kingdom. Data and Documents. Retrieved February 25, 2019 (https://<ulink href="http://www.europeansocialsurvey.org/data/country.html?c=united%5fkingdom">www.europeansocialsurvey.org/data/country.html?c=united%5fkingdom</ulink>).</bibtext> </blist> <blist> <bibtext> Flesch Rudolph. 1948. " A New Readability Yardstick." Journal of Applied Psychology32(3):221–33.</bibtext> </blist> <blist> <bibtext> Gunst Richard F., Mason Robert L.. 1993. How to Construct Fractional Factorial Experiments: The Basic References in Quality Control: Statistical Techniques 14. Milwaukee, WI: ASQC Quality Press.</bibtext> </blist> <blist> <bibtext> Hainmueller Jens, Hangartner Dominik, Yamamoto Teppei. 2015. " Validating Vignette and Conjoint Survey Experiments against Real-world Behavior." Proceedings of the National Academy of Sciences of the United States of America112(8):2395–2400.</bibtext> </blist> <blist> <bibtext> Hartshorne Joshua K., Germine Laura T.. 2015. " When Does Cognitive Functioning Peak? The Asynchronous Rise and Fall of Different Cognitive Abilities across the Life Span." Psychological Science26(4):433–43.</bibtext> </blist> <blist> <bibtext> Hermkens Piet L. J., Boerman Frank A.. 1989. " Consensus with Respect to the Fairness of Incomes: Differences between Social Groups." Social Justice Research3(3):201–15.</bibtext> </blist> <blist> <bibtext> Jasso Guillermina. 2006. " Factorial Survey Methods for Studying Beliefs and Judgments." Sociological Methods & Research34(3):334–423.</bibtext> </blist> <blist> <bibtext> Knäuper Baerbel, Belli Robert F., Hill Daniel H., Regula Herzog A.. 1997. " Question Difficulty and Respondents' Cognitive Ability: The Effect on Data Quality." Journal of Official Statistics13(2):181–99.</bibtext> </blist> <blist> <bibtext> Krosnick John A.1991. " Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys." Applied Cognitive Psychology5(3):213–236.</bibtext> </blist> <blist> <bibtext> Krosnick John A., Holbrook Allyson L., Berent Matthew K., Carson Richard T., Michael Hanemann W., Kopp Raymond J., Mitchell Robert Cameron. 2002. " The Impact of 'No Opinion' Response Options on Data Quality." Public Opinion Quarterly66(3):371–403.</bibtext> </blist> <blist> <bibtext> Krosnick John A., Fabrigar Leandre R.. 1997. "Designing Rating Scales for Effective Measurement in Surveys." Pp. 141–64 in Survey Measurement and Process Quality, edited by Lyberg Lars E., Biemer Paul P., Collins Martin, Leeuw Edith De, Dippo Cathryn., Schwarz Norbert, Trewin Dennis. New York: John Wiley & Sons Inc.</bibtext> </blist> <blist> <bibtext> Kuhfeld Warren F.2010. "Experimental Design: Efficiency, Coding, and Choice Design." Pp. 53–241 in Marketing Research Methods in SAS. Experimental Design, Choice, Conjoint, and Graphical Techniques, edited by Kuhfeld Warren F.. Retrieved October 5, 2013 (<ulink href="http://support.sas.com/resources/papers/tnote/tnote%5fmarketresearch.html">http://support.sas.com/resources/papers/tnote/tnote%5fmarketresearch.html</ulink>).</bibtext> </blist> <blist> <bibtext> Kuhfeld Warren F., Tobias Randall D., Garratt Mark. 1994. " Efficient Experimental Design with Marketing Research Applications." Journal of Marketing Research31:545–57. A revised version appeared in SAS Technical Papers—Marketing Research, 243-264. Retrieved August 28, 2014 (<ulink href="http://support.sas.com/techsup/technote/mr2010d.pdf">http://support.sas.com/techsup/technote/mr2010d.pdf</ulink>).</bibtext> </blist> <blist> <bibtext> Liebe Ulf, Moumouni Ismaïl M., Bigler Christine, Ingabire Chantal, Bieri Sabin. 2017. " Using Factorial Survey Experiments to Measure Attitudes, Social Norms, and Fairness Concerns in Developing Countries." Sociological Methods & Research1-32. doi: 10.1177/0049124117729707.</bibtext> </blist> <blist> <bibtext> Maehler Déborah B., Massing Natascha, Helmschrott Susanne, Rammstedt Beate, Staudinger Ursula M., Wolf Chistof. 2013. "Grundlegende Kompetenzen in verschiedenen Bevölkerungsgruppen." Pp. 77–126 in Grundlegende Kompetenzen Erwachsener im internationalen Vergleich: Ergebnisse von PIAAC 2012, edited by Rammstedt Beate. Münster: Waxmann.</bibtext> </blist> <blist> <bibtext> Mason Robert, Carlson John E.. 1994. " Contrast Effects and Subtraction in Part-whole Questions." Public Opinion Quarterly58(4):569–78.</bibtext> </blist> <blist> <bibtext> Miller George A.1994. " The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information." Psychological Review101(2):343–52.</bibtext> </blist> <blist> <bibtext> Moss-Racusin Corinne A., Dovidio John F., Brescoll Victoria L., Graham Mark J., Handelsman Jo. 2012. " Science Faculty's Subtle Gender Biases Favor Male Students." Proceedings of the National Academy of Sciences of the United States of America109(41):16474–479.</bibtext> </blist> <blist> <bibtext> Norman Donald A.1972. Memory, Knowledge, and the Answering of Questions. Paper Presented at the Loyola Symposium on Cognitive Psychology, Chicago, IL.</bibtext> </blist> <blist> <bibtext> Porst Rolf. 2008. Fragebogen: Ein Arbeitsbuch (Vol. 1). Wiesbaden: VS-Verlag.</bibtext> </blist> <blist> <bibtext> Programme for the International Assessment of Adult Competencies. 2016. Programme for the International Assessment of Adult Competencies. Retrieved April 9, 2017 (<ulink href="http://www.oecd.org/skills/piaac">http://www.oecd.org/skills/piaac</ulink>).</bibtext> </blist> <blist> <bibtext> Ross Howard J.2014. Everyday Bias: Identifying and Navigating Unconscious Judgments in Our Daily Lives. Lanham, MD: Rowman & Littlefield.</bibtext> </blist> <blist> <bibtext> Rossi Peter H., Anderson Andy B.. 1982. "The Factorial Survey Approach: An Introduction." Pp. 15–67 in Measuring Social Judgments: The Factorial Survey Approach, edited by Rossi Peter H., Nock Steven L.. Thousand Oaks, CA: Sage.</bibtext> </blist> <blist> <bibtext> Rubin Donald B.1976. " Inference and Missing Data." Biometrika63(3):581–92.</bibtext> </blist> <blist> <bibtext> Saris Willem. E., Gallhofer Irmtraud N.. 2007. Design, Evaluation, and Analysis of Questionnaires for Survey Research. Hoboken, NJ: Wiley Interscience.</bibtext> </blist> <blist> <bibtext> Sauer Carsten, Auspurg Katrin, Hinz Thomas, Liebig Stefan. 2011. " The Application of Factorial Surveys in General Population Samples: The Effects of Respondent Age and Education on Response Times and Response Consistency." Survey Research Methods5(3):89–102.</bibtext> </blist> <blist> <bibtext> Schuman Howard, Presser Stanley. 1979. " The Open and Closed Question." American Sociological Review44(5):692–712.</bibtext> </blist> <blist> <bibtext> Schuman Howard, Presser Stanley. 1996. Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. Thousand Oaks, CA: Sage.</bibtext> </blist> <blist> <bibtext> Shamon Hawal. 2014. Der Einfluss des sozialen Kontexts auf Gerechtigkeitsurteile über das Einkommen—eine Untersuchung des kausalen Effekts auf Grundlage von internetbasierten experimentellen Umfragedesigns. Druckschrift, Cologne: University of Cologne.</bibtext> </blist> <blist> <bibtext> Shamon Hawal, Dülmer Hermann. 2014. " Raising the Question on 'Who Should Get What?' Again: On the Importance of Ideal and Existential Standards." Social Justice Research27(3):340–68.</bibtext> </blist> <blist> <bibtext> Shoemaker Pamela J., Eichholz Martin, Skewes Elizabeth A.. 2002. " Item Nonresponse: Distinguishing between Don't Know and Refuse." International Journal of Public Opinion Research14(2):193–201.</bibtext> </blist> <blist> <bibtext> Sniderman Paul M., Grob Douglas B.. 1996. " Innovations in Experimental Design in Attitudes Surveys." Annual Review of Sociology22:377–99.</bibtext> </blist> <blist> <bibtext> Sudman Seymour, Bradburn Norman M., Schwarz Norbert. 1996. Thinking about Answers. San Francisco, CA: Jossey-Bass.</bibtext> </blist> <blist> <bibtext> Teti Andrea, Gross Christiane, Knoll Nina, Blüher Stefan. 2016. " Feasibility of the Factorial Survey Method in Aging Research: Consistency Effects among Older Respondents." Research on Aging38(7):715–41.</bibtext> </blist> <blist> <bibtext> Tourangeau Roger, Rips Lance J., Rasinski Kenneth. 2000. The Psychology of Survey Response. Cambridge, England: Cambridge University Press.</bibtext> </blist> <blist> <bibtext> Urša Reja, Manfreda Katja Lozar, Hlebec Valentina, Vehovar Vasja. 2003. " Open-ended vs. Close-ended Questions in Web Questionnaires." Retrieved September 12, 2018 (<ulink href="http://mrvar.fdv.uni-lj.si/pub/mz/mz19/reja.pdf">http://mrvar.fdv.uni-lj.si/pub/mz/mz19/reja.pdf</ulink>).</bibtext> </blist> <blist> <bibtext> Van der Veld Williams M., Saris Willem E.. 2003. "Separation of Error, Method Effects, Instability and Attitude Strength." Pp. 37–63 in The Issue of Belief: Essays in the Intersection of Nonattitudes and Attitude Change, edited by Saris Willem E., Sniderman Paul M.. Princeton, NJ: University Press.</bibtext> </blist> <blist> <bibtext> Williams Richard. 2012. " Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects." The Stata Journal12(2):308–31.</bibtext> </blist> </ref> <aug> <p>By Hawal Shamon; Hermann Dülmer and Adam Giza</p> <p>Reported by Author; Author; Author</p> <p></p> <p>Hawal Shamon is a postdoctoral researcher at the Institute of Energy and Climate Research – Systems Analysis and Technology Assessment (IEK-STE) at the Research Center Jülich. His substantive research interest covers the transformation of energy systems. His methodological research interest focuses on the conception of (experimental) surveys and the analysis of cross-sectional as well as longitudinal surveys. He wrote his doctoral thesis on the justice of earnings at the Chair of Empirical Social and Economic Research at the University of Cologne.</p> <p>Hermann Dülmer is an assistant professor of Sociology at the University of Cologne, Germany. His research interests focus on applications of factorial surveys and on multilevel analysis including multilevel structural equation modelling. Applications include values, morality, and culture as well as electoral research with a particular emphasis on right-wing extremism.</p> <p>Adam Giza is sociologist with the main focus on survey research and statistical methods. He worked in different social and medical research institutes. He is an expert in applying multivariate statistical methods on different kinds of data, calculating survey weights, and in accounting for statistical disclosure control in large datasets, like the German healthcare data.</p> </aug> <nolink nlid="nl1" bibid="bib52" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib37" firstref="ref6"></nolink> <nolink nlid="nl3" bibid="bib42" firstref="ref9"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref12"></nolink> <nolink nlid="nl5" bibid="bib54" firstref="ref13"></nolink> <nolink nlid="nl6" bibid="bib24" firstref="ref15"></nolink> <nolink nlid="nl7" bibid="bib40" firstref="ref16"></nolink> <nolink nlid="nl8" bibid="bib46" firstref="ref21"></nolink> <nolink nlid="nl9" bibid="bib43" firstref="ref27"></nolink> <nolink nlid="nl10" bibid="bib51" firstref="ref28"></nolink> <nolink nlid="nl11" bibid="bib34" firstref="ref32"></nolink> <nolink nlid="nl12" bibid="bib19" firstref="ref36"></nolink> <nolink nlid="nl13" bibid="bib25" firstref="ref37"></nolink> <nolink nlid="nl14" bibid="bib21" firstref="ref38"></nolink> <nolink nlid="nl15" bibid="bib41" firstref="ref42"></nolink> <nolink nlid="nl16" bibid="bib48" firstref="ref43"></nolink> <nolink nlid="nl17" bibid="bib47" firstref="ref44"></nolink> <nolink nlid="nl18" bibid="bib14" firstref="ref46"></nolink> <nolink nlid="nl19" bibid="bib26" firstref="ref47"></nolink> <nolink nlid="nl20" bibid="bib27" firstref="ref50"></nolink> <nolink nlid="nl21" bibid="bib10" firstref="ref53"></nolink> <nolink nlid="nl22" bibid="bib11" firstref="ref54"></nolink> <nolink nlid="nl23" bibid="bib50" firstref="ref55"></nolink> <nolink nlid="nl24" bibid="bib12" firstref="ref57"></nolink> <nolink nlid="nl25" bibid="bib22" firstref="ref63"></nolink> <nolink nlid="nl26" bibid="bib15" firstref="ref65"></nolink> <nolink nlid="nl27" bibid="bib16" firstref="ref70"></nolink> <nolink nlid="nl28" bibid="bib20" firstref="ref72"></nolink> <nolink nlid="nl29" bibid="bib30" firstref="ref73"></nolink> <nolink nlid="nl30" bibid="bib29" firstref="ref75"></nolink> <nolink nlid="nl31" bibid="bib17" firstref="ref80"></nolink> <nolink nlid="nl32" bibid="bib44" firstref="ref81"></nolink> <nolink nlid="nl33" bibid="bib45" firstref="ref82"></nolink> <nolink nlid="nl34" bibid="bib53" firstref="ref83"></nolink> <nolink nlid="nl35" bibid="bib23" firstref="ref84"></nolink> <nolink nlid="nl36" bibid="bib18" firstref="ref88"></nolink> <nolink nlid="nl37" bibid="bib55" firstref="ref101"></nolink> <nolink nlid="nl38" bibid="bib28" firstref="ref108"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1325868
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Factorial Survey: The Impact of the Presentation Format of Vignettes on Answer Behavior and Processing Time
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Shamon%2C+Hawal%22">Shamon, Hawal</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-6304-9301">0000-0001-6304-9301</externalLink>)<br /><searchLink fieldCode="AR" term="%22Dülmer%2C+Hermann%22">Dülmer, Hermann</searchLink><br /><searchLink fieldCode="AR" term="%22Giza%2C+Adam%22">Giza, Adam</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Sociological+Methods+%26+Research%22"><i>Sociological Methods & Research</i></searchLink>. Feb 2022 51(1):396-438.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 43
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Vignettes%22">Vignettes</searchLink><br /><searchLink fieldCode="DE" term="%22Surveys%22">Surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Style+%28Tests%29%22">Response Style (Tests)</searchLink><br /><searchLink fieldCode="DE" term="%22Reaction+Time%22">Reaction Time</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Adults%22">Adults</searchLink><br /><searchLink fieldCode="DE" term="%22Age+Differences%22">Age Differences</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Attainment%22">Educational Attainment</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Germany%22">Germany</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/0049124119852382
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0049-1241
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The factorial survey is an experimental design in which the researcher constructs varying descriptions of situations or individual persons (vignettes), which will be judged by respondents with regard to a particular aspect. Some researchers present vignettes in text format as short stories, others present the central information of vignettes in a tabular format. To date, only a few sentences have been published, by Auspurg and Hinz, on the impact of the presentation format (text vs. table) on the answer behavior of students. Empirically, no differences were found between either format. Based on an Internet experiment conducted with a quota sample, we find evidence that ordinary tabular formats outperform text vignettes in terms of total vignette nonresponse but not when it comes to processing time. The former result especially applies in the case of less well-educated people. We further find that tabular format does not perform worse than text format regarding response inconsistency.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1325868
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1325868
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/0049124119852382
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 43
        StartPage: 396
    Subjects:
      – SubjectFull: Vignettes
        Type: general
      – SubjectFull: Surveys
        Type: general
      – SubjectFull: Response Style (Tests)
        Type: general
      – SubjectFull: Reaction Time
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Adults
        Type: general
      – SubjectFull: Age Differences
        Type: general
      – SubjectFull: Educational Attainment
        Type: general
      – SubjectFull: Germany
        Type: general
    Titles:
      – TitleFull: The Factorial Survey: The Impact of the Presentation Format of Vignettes on Answer Behavior and Processing Time
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Shamon, Hawal
      – PersonEntity:
          Name:
            NameFull: Dülmer, Hermann
      – PersonEntity:
          Name:
            NameFull: Giza, Adam
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 0049-1241
          Numbering:
            – Type: volume
              Value: 51
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Sociological Methods & Research
              Type: main
ResultId 1