View in EDS HTML Full Text PDF Full Text

Computerized and Traditional Administration of Questionnaires: Psychometric Quality and Completion Time for Measures of Self-Concept

Saved in:
Bibliographic Details
Title: Computerized and Traditional Administration of Questionnaires: Psychometric Quality and Completion Time for Measures of Self-Concept
Language: English
Authors: Vispoel, Walter Peter, Morris, Carrie Ann, Sun, Linan
Source: Journal of Experimental Education. 2019 87(3):384-399.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 16
Publication Date: 2019
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Self Concept Measures, Computer Assisted Testing, Questionnaires, Psychometrics, Scores, Answer Sheets, Test Length, Test Format, College Students
Assessment and Survey Identifiers: Self Description Questionnaire
DOI: 10.1080/00220973.2018.1448748
ISSN: 0022-0973
Abstract: In two independent studies of questionnaire administration, respondents completed multidimensional self-concept inventories within four randomized research conditions that mirrored the most common administration formats used in practice: paper booklets with and without answer sheets and computer questionnaires with single versus multiple items per screen. Strong differences among conditions emerged for completion time, but not for psychometric properties of scores (means, variances, reliability coefficients, concurrent validity coefficients). Answer sheets increased completion time by 24% to 34%, single-item displays by 20% to 25%, and computerization by 13% to 17%. Completion time was longer for multiple-alternative than for rating-scale items, but relative effects of answer sheets, single-item displays, and computerization remained consistent. We discuss implications for questionnaire construction and administration.
Abstractor: As Provided
Entry Date: 2019
Accession Number: EJ1218243
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGCday1eE6AakFuLFnaU6tfAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDNOSszhVQs2XeSVwWAIBEICBm3Cg1a_2QLy3Fk2KYBKAVQSdrfU-WyRgGYb2ip5OuMeGjDoUHTr-X3XmS13M5O5PalXh-OsYjChDdDyMaJ4B7ygwFXvqlXXnLFMuskXlllmq7xcxCWj38jVUUnf9aiOE5PwGvdehkspMwGpDzaPeHT7e1KSDInMwHKtBeD7rq3S3sW-GXZNscrTFcm9h0sldAz40DyswnjXS9_js
Text:
  Availability: 1
  Value: <anid>AN0136822484;jxe01jul.19;2019Jun07.02:27;v2.2.500</anid> <title id="AN0136822484-1">Computerized and Traditional Administration of Questionnaires: Psychometric Quality and Completion Time for Measures of Self-Concept </title> <p>In two independent studies of questionnaire administration, respondents completed multidimensional self-concept inventories within four randomized research conditions that mirrored the most common administration formats used in practice: paper booklets with and without answer sheets and computer questionnaires with single versus multiple items per screen. Strong differences among conditions emerged for completion time, but not for psychometric properties of scores (means, variances, reliability coefficients, concurrent validity coefficients). Answer sheets increased completion time by 24% to 34%, single-item displays by 20% to 25%, and computerization by 13% to 17%. Completion time was longer for multiple-alternative than for rating-scale items, but relative effects of answer sheets, single-item displays, and computerization remained consistent. We discuss implications for questionnaire construction and administration.</p> <p>COMMONLY STATED ADVANTAGES of computerized measures are that they can retain the psychometric quality of paper measures and reduce administration time (Bunderson, Inouye, & Olsen, [<reflink idref="bib5" id="ref1">5</reflink>]; Butcher, [<reflink idref="bib6" id="ref2">6</reflink>]; Drasgow, Luecht, & Bennett, [<reflink idref="bib16" id="ref3">16</reflink>]; Drasgow & Olsen-Buchanan, [<reflink idref="bib17" id="ref4">17</reflink>]; Mills, Potenza, Fremer, & Ward, [<reflink idref="bib28" id="ref5">28</reflink>]; Parshall, Spray, Kalohn, & Davey, [<reflink idref="bib34" id="ref6">34</reflink>]). However, the research literature for questionnaire administration has frequently not supported these claims. Score comparability, for example, can be affected by the degree of maneuverability in recording answers (ability to skip, backtrack, and change answers), setting of administration, and purpose of the assessment (see, e.g., Moon, [<reflink idref="bib29" id="ref7">29</reflink>]; Richman, Kiesler, Weisband, & Drasgow, [<reflink idref="bib39" id="ref8">39</reflink>]; Rosenfeld, Booth-Kewley, Edwards, & Thomas, [<reflink idref="bib43" id="ref9">43</reflink>]; Wilkerson, Nagao, & Martin, [<reflink idref="bib67" id="ref10">67</reflink>]). Computerized measures also can take more or less time to complete than paper measures, depending on how items are presented and responses made (Clough, [<reflink idref="bib10" id="ref11">10</reflink>]; Simms & Clark, [<reflink idref="bib45" id="ref12">45</reflink>]; Vispoel, [<reflink idref="bib53" id="ref13">53</reflink>]; Vispoel, Boo, & Bleiler, [<reflink idref="bib54" id="ref14">54</reflink>]; Vispoel, Morris, & Clough, [<reflink idref="bib58" id="ref15">58</reflink>]; Weigold, Weigold, & Russell, [<reflink idref="bib65" id="ref16">65</reflink>]). The purpose of the present research was to clarify factors that affect responses and completion time for self-report questionnaires by examining four common ways such measures are constructed and answered in practice: paper booklets with and without answer sheets and computer questionnaires with single versus multiple items per screen.</p> <hd id="AN0136822484-2">Background</hd> <p></p> <hd id="AN0136822484-3">Computerized administration of questionnaires</hd> <p>Over recent decades, the use of computerized questionnaires has increased dramatically. Computerized measures can produce immediate score reports, reduce test production costs, enhance uniformity in test administration conditions, provide flexibility in changing test items and scoring algorithms, monitor completion time, display technology-enhanced items, and allow for adaptive administration of items (Booth-Kewley, Larson, & Miyoshi, [<reflink idref="bib4" id="ref17">4</reflink>]; Bunderson et al., [<reflink idref="bib5" id="ref18">5</reflink>]; Butcher, [<reflink idref="bib6" id="ref19">6</reflink>]; Byrne, [<reflink idref="bib7" id="ref20">7</reflink>]; Mazzeo & Harvey, [<reflink idref="bib25" id="ref21">25</reflink>]; Simms, & Clark, [<reflink idref="bib45" id="ref22">45</reflink>]). However, to use computerized and paper measures interchangeably, scores must be comparable across modes, and most research on computerized questionnaires has focused on this issue. Meta-analyses, for example, indicate that mean scores are usually similar across modes when measures are administered in private, ensure anonymity, and permit answer review and revision (Dodou, & de Winter, [<reflink idref="bib15" id="ref23">15</reflink>]; Dwight & Feigelson, [<reflink idref="bib18" id="ref24">18</reflink>]; Richman et al., [<reflink idref="bib39" id="ref25">39</reflink>]). However, these meta-analytic studies did not include indices of variability, reliability, and validity that are also crucial in establishing equivalence of scores across modes of administration (American Psychological Association, [<reflink idref="bib1" id="ref26">1</reflink>]; AERA, APA, & NCME, [<reflink idref="bib2" id="ref27">2</reflink>]; International Test Commission, [<reflink idref="bib23" id="ref28">23</reflink>]).</p> <p>In addition, researchers routinely ignore questionnaire completion time when comparing modes of administration despite its obvious importance in practical settings. Respondents, administrators, and users of results would seemingly share a desire to minimize completion time without sacrificing the credibility of information obtained. Excessively long completion times might elicit fatigue and boredom effects, whereas suspiciously short times may signal insincere responding, either of which can undermine validity and appropriate uses of scores.</p> <hd id="AN0136822484-4">Effects of computerization on questionnaire completion time</hd> <p>Despite the practical importance of monitoring completion time and its potential value when interpreting scores, we found only six studies of questionnaires in controlled laboratory settings in which completion time was included as a dependent variable when comparing computer and paper modes of administration (Clough, [<reflink idref="bib10" id="ref29">10</reflink>]; Simms & Clark, [<reflink idref="bib45" id="ref30">45</reflink>]; Vispoel, [<reflink idref="bib53" id="ref31">53</reflink>]; Vispoel et al., [<reflink idref="bib54" id="ref32">54</reflink>]; Vispoel, Morris, & Clough, [<reflink idref="bib58" id="ref33">58</reflink>]; Weigold et al., [<reflink idref="bib65" id="ref34">65</reflink>]). In Clough ([<reflink idref="bib10" id="ref35">10</reflink>]), Vispoel ([<reflink idref="bib53" id="ref36">53</reflink>]), and Vispoel et al., [<reflink idref="bib54" id="ref37">54</reflink>], computer and paper questionnaires had the same size type and font, but the computer versions had fewer items per screen and more displays to read due to the landscape orientation of the computer screen versus portrait orientation of the printed page. In addition, responses to paper measures were made by marking choices directly in questionnaire booklets and those for the computer measures by clicking on choices using a mouse. The researchers also used a common experimental design in which randomly equivalent groups completed Likert-style rating-scale questionnaires in both modes but in counterbalanced orders with a one-week interval between administrations. Approximately half of the respondents took the computer version at the first session and paper version at the second session, and the other half did the reverse.</p> <p>The Clough ([<reflink idref="bib10" id="ref38">10</reflink>]), Vispoel ([<reflink idref="bib53" id="ref39">53</reflink>]), and Vispoel et al., [<reflink idref="bib54" id="ref40">54</reflink>] studies differed primarily with regard to the target questionnaire used and the number of individuals sampled. The 136-item Self-Description Questionnaire-III (SDQ-III; Marsh, [<reflink idref="bib24" id="ref41">24</reflink>]) was administered to 212 respondents in Vispoel ([<reflink idref="bib53" id="ref42">53</reflink>]), the 10-item Self-Esteem Scale (SES; Rosenberg, [<reflink idref="bib41" id="ref43">41</reflink>], [<reflink idref="bib42" id="ref44">42</reflink>]) to 225 respondents in Vispoel et al. ([<reflink idref="bib54" id="ref45">54</reflink>]), and the 40-item Balanced Inventory of Desirable Responding (BIDR; Paulhus, [<reflink idref="bib35" id="ref46">35</reflink>]) to 1,210 respondents in Clough ([<reflink idref="bib10" id="ref47">10</reflink>]). Results across studies revealed few differences between modes in the psychometric properties of scores (central tendency, variability, reliability, construct validity) but consistent differences in completion time. Computer measures took from 8% to 40% longer to complete than did paper measures, with differences most pronounced for the questionnaire with fewest items (i.e., SES 40% versus SDQ-III and BIDR 8% to 11%). Standardized mean differences (<emph>d</emph> values) from these studies equaled 1.15 for the SES, 0.44 for the SDQ-III, and 0.33 to 0.52 for the BIDR. Vispoel el al. ([<reflink idref="bib54" id="ref48">54</reflink>]) attributed the large relative difference in completion time for the SES to the use of three displays for the computerized questionnaires (one screen for directions and a practice item and two screens for the actual items) compared to a single page for the paper questionnaire that included directions and all items. For instruments in the other studies, multiple screens or pages were used for both computer and paper administration.</p> <p>In a more recent investigation by Weigold et al. ([<reflink idref="bib65" id="ref49">65</reflink>]), respondents completed either web-based or paper versions of the 50-item International Personality Item Pool questionnaire (IPIP; Goldberg et al., [<reflink idref="bib21" id="ref50">21</reflink>]), a 13-item short form of the Marlowe-Crown Social Desirability Scale (MCSDS-13; Reynolds, [<reflink idref="bib38" id="ref51">38</reflink>]), and the 32-item Computer Self-Efficacy Scale (CSE; Murphy, Coover, & Owen, [<reflink idref="bib30" id="ref52">30</reflink>]) on a single occasion. Paper and internet questionnaires were completed in both lab conditions and within homes of respondents. In the paper conditions, respondents either circled or wrote their answers directly on the questionnaires with the number of items per page versus per computer screen held constant. Consistent with the other studies cited here, completion time on average within the lab conditions was longer (by about 5%) for computer than for paper administration (<emph>d</emph> = 0.15). However, paper questionnaire completion time within homes was significantly longer (<emph>d</emph> values ranged from 0.70 to 0.89) and more variable (<emph>SD</emph> in minutes = 7.15 versus 2.63 to 3.99) than completion time within the other three conditions (internet at home, paper in the lab, internet in the lab). Weigold et al. ([<reflink idref="bib65" id="ref53">65</reflink>]) speculated that the differences might be attributable to reduced pressure at home to finish paper questionnaires in a timely manner and a greater likelihood of becoming distracted and setting them aside.</p> <hd id="AN0136822484-5">Effects of answer sheets</hd> <p>The studies described thus far all included paper measures with responses made directly in the questionnaire itself. In investigations conducted by Simms and Clark ([<reflink idref="bib45" id="ref54">45</reflink>]) and Vispoel, Morris, and Clough ([<reflink idref="bib58" id="ref55">58</reflink>]), responses to reusable paper questionnaire booklets were made on separate answer sheets. Simms and Clark administered computerized and paper versions of a 297-item form of the Schedule for Adaptive and Nonadaptive Personality (SNAP; Clark, [<reflink idref="bib9" id="ref56">9</reflink>]) to 413 college students. Participants were assigned at random to four research conditions involving repeated administrations of the SNAP with a one-week interval between administrations: (a) Paper Session 1, Paper Session 2; (b) Paper Session 1, Computer Session 2; (c) Computer Session 1, Paper Session 2; and (d) Computer Session 1, Computer Session 2. The paper version of the SNAP consisted of a booklet in which responses were made on a separate answer sheet, whereas the computerized version was administered using the MicroCAT software package (Assessment Systems Corporation, [<reflink idref="bib3" id="ref57">3</reflink>]), with only one item appearing on each screen. Respondents used the space bar or arrow keys to toggle to a desired answer, and answer options were identical for all items. The results revealed few noteworthy differences in psychometric characteristics across modes (central tendency, rank ordering of scores, reliability, and concurrent validity) but strong differences in completion time. However, completion time for paper questionnaires was 49% (<emph>d</emph> = 2.35) and 55% (<emph>d</emph> = 2.45) longer than for computerized questionnaires at Sessions 1 and 2, respectively.</p> <p>Vispoel, Morris, and Clough ([<reflink idref="bib58" id="ref58">58</reflink>]) administered the 40-item Balanced Inventory of Desirable Responding (BIDR) to 601 college students randomly assigned to the same four research conditions examined by Simms and Clark ([<reflink idref="bib45" id="ref59">45</reflink>]). Like Simms and Clark, separate answer sheets were used with all paper questionnaires. However, in contrast to Simms and Clark, Vispoel, Morris, and Clough used computer and paper questionnaires with the same font and text size, the same number of items per page/screen, and the same number of total pages/screens. Rather than typing in or toggling to response choices, respondents clicked on the desired choices using a mouse. With multiple items per screen, mouse-based responses, and a shorter questionnaire, Vispoel, Morris, and Clough found smaller relative differences in completion time across modes (29% to 33%; <emph>d</emph> values from 0.90 to 1.07) than did Simms and Clark ([<reflink idref="bib45" id="ref60">45</reflink>]). Taken together, findings from these two studies reveal that use of separate answer sheets with paper questionnaires can lead to substantial increases in administration time when computer displays have either single or multiple items. In contrast, for other studies in controlled settings in which answers are marked directly in booklets, paper questionnaires have consistently taken less time to complete than computer questionnaires when paper questionnaires have equal or more items per page than computer questionnaires have per screen.</p> <hd id="AN0136822484-6">Screen density effects</hd> <p>Effects of number of items per screen on completion time within computerized questionnaires were examined further in three additional studies. Toepoel, Das, and Van Soest ([<reflink idref="bib49" id="ref61">49</reflink>]) compared completion time for four versions of a 40-item scale developed by Mehrabian and Russell ([<reflink idref="bib27" id="ref62">27</reflink>]) to measure arousal. The questionnaires differed only in number of items per screen (<reflink idref="bib1" id="ref63">1</reflink>, 4, 10, 40). The researchers found a statistically significant monotonically decreasing relationship between number of items and completion time with median completion time values of 384, 316, 303, and 302 seconds for one, four, 10, and 40 items per screen, respectively. Similarly, Couper, Traugott, and Lamias ([<reflink idref="bib11" id="ref64">11</reflink>]) and Tourangeau, Couper, and Conrad ([<reflink idref="bib50" id="ref65">50</reflink>]) found that completion time for single-item screens was significantly longer than for multi-item screens containing from three to eight items using measures of attitudes toward affirmative action and diet behaviors, respectively.</p> <p>Although multiple items per sheet seemingly have an advantage in reducing completion time, they have potential drawbacks in yielding more highly correlated responses and correspondingly higher internal consistency coefficients resulting from context and carryover effects caused by grouping items measuring the same construct together. These potential effects also were examined in the three studies just cited (Couper et al., [<reflink idref="bib11" id="ref66">11</reflink>]; Toepoel et al., [<reflink idref="bib49" id="ref67">49</reflink>]; Tourangeau et al., [<reflink idref="bib50" id="ref68">50</reflink>]). In each study, the investigators compared alpha coefficient values for sets of items presented together on the same screen compared to those same items presented on individual screens. Significant effects were found in the [<reflink idref="bib50" id="ref69">50</reflink>] study of diet behaviors in which a linear trend was observed between alpha coefficient values and numbers of items per screen (<reflink idref="bib1" id="ref70">1</reflink>, 4, and 8). The highest score consistency was found for eight items per screen and the lowest for one item per screen.</p> <p>In a related study of online questionnaires, Weijters, De Beuckelaer, and Baumgarter ([<reflink idref="bib66" id="ref71">66</reflink>]) presented eight items intended to measure the same construct (frugality) in two separate four-item blocks separated by a set of 32 filler items that measured other constructs. Two experimental conditions were included consisting of different items from the frugality scale administered in the two blocks. In Condition 1, the first block had items 1 through 4 and the second block had items 5 through 8; in Condition 2, the first block had items 1, 2, 7, and 8 and the second block had items 3, 4, 5, and 6. For both conditions, the researchers found that items within each block were more highly correlated with each other than were items across blocks.</p> <p>Given the item-grouping effects reported by Tourangeau et al. ([<reflink idref="bib50" id="ref72">50</reflink>]) and Weijters, De Beuckelaer, and Baumgartner ([<reflink idref="bib66" id="ref73">66</reflink>]), we deemed it important to perform analyses for possible item-screen-density effects on score consistency in the current studies. We speculated that such effects might be controlled by spiraling items from common subscales throughout a questionnaire rather than by grouping the items together.</p> <hd id="AN0136822484-7">Item types</hd> <p>A final important point to note about investigations of mode effects on completion time is that they have been limited exclusively to items with a common rating scale such as those routinely included in Likert-style questionnaires. We are aware of no related research study to date that has focused on questionnaires for which each question has a unique set of answer options, akin to multiple-choice items on cognitive tests. Questionnaire items with multiple alternatives can reduce the influence of socially desirable responding by making the preferred response choice less obvious (Edwards, [<reflink idref="bib19" id="ref74">19</reflink>]; Urbina, [<reflink idref="bib51" id="ref75">51</reflink>]). However, because such items require special attention to both the stem and choices for each individual item, we would expect average-item-completion time to be longer than with items that use a common rating scale. The relative effects on completion time for such items resulting from use of answer sheets, use of single-item displays, and converting identical questionnaires from paper to computer form remains an empirical question.</p> <hd id="AN0136822484-8">This investigation</hd> <p>The purpose of the present study was to clarify how specific characteristics of paper and computerized questionnaires might affect responses and completion time by combining four research conditions included in prior studies and controlling factors often confounded in those studies. We report results based on two independent samples of respondents to mirror and expand upon types of questionnaires considered in previous research. One sample completed Likert-style items similar to those used in prior studies that included a common rating scale for all items within a given questionnaire. The other sample completed items measuring similar constructs that each had a unique set of response alternatives.</p> <p>Based on the research literature, we did not expect to find noteworthy differences in the psychometric properties of subscale scores across research conditions nor differences in reliability coefficients between single versus multiple items per computer displays due to the systematic spiraling of items from different subscales throughout each questionnaire. For completion time, we expected:</p> <p></p> <ulist> <item> (<reflink idref="bib1" id="ref76">1</reflink>) paper questionnaires with answer sheets to take longer than paper and computerized questionnaires without answer sheets</item> <p></p> <item> (<reflink idref="bib2" id="ref77">2</reflink>) computerized questionnaires with single item displays to take longer than computerized and paper questionnaires with multiple items per screen/page when answer sheets are not used</item> <p></p> <item> (<reflink idref="bib3" id="ref78">3</reflink>) computerized multi-item questionnaires to take longer than parallel paper multi-item questionnaires when answer sheets are not used</item> <p></p> <item> (<reflink idref="bib4" id="ref79">4</reflink>) multiple-alternative items to take longer than rating-scale items across all questionnaire formats</item> </ulist> <hd id="AN0136822484-9">Methods</hd> <p></p> <hd id="AN0136822484-10">Participants and measures</hd> <p></p> <hd id="AN0136822484-11">Sample 1</hd> <p>Participants included 224 students from a large Midwestern university (71% female; 92% Caucasian, 2% African American, 2% Hispanic, 4% Asian; median age = 20 (<emph>M</emph> = 20.83, <emph>SD</emph> = 3.72); and 65% having at least one parent with a bachelor's degree or higher). Students reported that they had spent 17.83 hours per week on average working on computers over the last 12 months (<emph>SD</emph> = 11.72). They completed the Self-Description Questionnaire-III (SDQ-III; Marsh, [<reflink idref="bib24" id="ref80">24</reflink>]), which has 136 items scored on an 8-point rating scale (1 = <emph>definitely false</emph>, 8 = <emph>definitely true</emph>). The SDQ-III has 13 subscales with 10 to 12 items balanced for positive and negative phrasing that measure overall self-esteem (general self) and 12 domain-specific aspects of self-concept: Emotional stability, peer relations—opposite sex, peer relations—same sex, parent relations, honesty/trustworthiness, spiritual values/religion, physical appearance, physical ability, general academic ability, verbal skills, math skills, and problem solving ability. Evidence supporting the reliability and construct validity of SDQ-III scores reported in its manual (Byrne, [<reflink idref="bib7" id="ref81">7</reflink>]; Marsh, [<reflink idref="bib24" id="ref82">24</reflink>]), in Bryne ([<reflink idref="bib7" id="ref83">7</reflink>]), and in Vispoel ([<reflink idref="bib52" id="ref84">52</reflink>]) includes internal consistency estimates ranging from 0.76 to 0.96, factor analyses verifying that each subscale measures a distinguishable construct, and logically consistent relationships of subscale scores with each other and external criterion measures.</p> <hd id="AN0136822484-12">Sample 2</hd> <p>Two-hundred-fifty-seven students from the same Midwestern university completed the Self-Perception Profile for College Students (SPPCS; Neemann & Harter, [<reflink idref="bib32" id="ref85">32</reflink>], [<reflink idref="bib33" id="ref86">33</reflink>]; 72% female, 82% Caucasian, 2% African American, 1% Native American, 8% Hispanic, 7% Asian; median age = 20 (<emph>M</emph> = 22.56, <emph>SD</emph> = 6.02); 67% having at least one parent with a bachelor's degree or higher). These students reported that they had spent 16.28 hours per week on average working on computers over the last 12 months (<emph>SD</emph> = 10.76). The SPPCS is a 54-item inventory with 13 four-to-six item subscales that measure overall self-regard (global self-worth) and 12 domain-specific aspects of self-concept: social acceptance, romantic relationships, close friendships, parent relationships, physical appearance, athletic competence, intellectual ability, scholastic competence, job competence, creativity, and sense of humor. Items are answered using a four-alternative extended forced-choice format. For each item, a pair of conflicting descriptions is presented. For example, a hypothetical item might be: <emph>Some students prefer having lots of friends</emph>BUT<emph>Other students prefer having just a few close ones</emph>. Respondents would first decide which statement in the pair better describes them and then determine whether that statement is "really true" or "sort of true" for them. The item shown here might be scored as 4 (really true) or 3 (sort of true) for choosing the first statement and as 2 (sort of true) or 1 (really true) for choosing the second statement. Items are balanced so that choices indicative of high self-regard appear first and, second, an equal number of times for each subscale. Evidence supporting the reliability and validity of subscale scores from the SPPCS for college students is provided in its manuals (Neemann & Harter, [<reflink idref="bib32" id="ref87">32</reflink>], [<reflink idref="bib33" id="ref88">33</reflink>]), in Byrne, ([<reflink idref="bib7" id="ref89">7</reflink>]), and in Vispoel and He ([<reflink idref="bib56" id="ref90">56</reflink>]). This evidence includes internal consistency estimates ranging from.76 to.96, factor analyses verifying that the subscales measure distinguishable constructs, and logically consistent relationships of subscale scores with external criteria.</p> <p>We chose the SDQ-III and SPPCS because they are widely used and highly regarded self-report inventories that measure personal attributes relevant in both educational and psychological settings (see Byrne, [<reflink idref="bib7" id="ref91">7</reflink>]). The inventories also measure similar constructs, spiral item content across subscales, and provide a good contrast between rating-scale and multiple-alternative type items. We considered college students appropriate for this research not only because the self-concept inventories were well targeted to them but also because meta-analytic studies have shown that relative patterns of mode differences in scores do not vary with the type (student, substance abuse, psychiatric, other adult), percentage of women, or mean age of samples (see, e.g., Dodou & de Winter, [<reflink idref="bib15" id="ref92">15</reflink>]; Richman et al., [<reflink idref="bib39" id="ref93">39</reflink>]).</p> <p>Respondents in both studies also completed the Balanced Inventory of Desirable Responding (BIDR-Version 6; Paulhus, [<reflink idref="bib35" id="ref94">35</reflink>], [<reflink idref="bib36" id="ref95">36</reflink>]) and a demographics questionnaire following administration of their targeted self-concept inventory. The demographics questionnaire included items about gender, race, age, parental educational levels, and computer experience. The BIDR has two 20-item subscales that measure self-deceptive enhancement (SDE) and impression management (IM). SDE items tap rigid overconfidence as reflected in perceptions of skills exceeding actual abilities. IM items measure endorsement of desirable but uncommon behaviors. BIDR items are stated in first person and rated along a 7-point metric (1 = <emph>not true</emph>, 4 = somewhat true, 7 = <emph>very true</emph>). Half the items are positively keyed and half, negatively keyed, with negatively keyed items reverse scored. The BIDR can be scored either polytomously or dichotomously to reflect only exaggerated endorsement of targeted behaviors. We scored responses polytomously here because recent studies have consistently shown that polytomous scoring provides superior evidence of internal consistency, stability, and construct validity when measuring individual differences in SDE and IM (Cervellione, Lee, & Bonanno, [<reflink idref="bib8" id="ref96">8</reflink>]; Stöber, Dette, & Musch, [<reflink idref="bib47" id="ref97">47</reflink>]; Vispoel & Kim, [<reflink idref="bib57" id="ref98">57</reflink>]; Vispoel & Tao, [<reflink idref="bib63" id="ref99">63</reflink>]; Vispoel, Morris, & Clough, [<reflink idref="bib58" id="ref100">58</reflink>]; Vispoel, Morris, & Kilinc, [<reflink idref="bib60" id="ref101">60</reflink>], [<reflink idref="bib60" id="ref102">60</reflink>], [<reflink idref="bib62" id="ref103">62</reflink>], [<reflink idref="bib61" id="ref104">61</reflink>]). Scores from the BIDR served as external criterion measures in our analysis of concurrent validity of scores from the SDQ-III and SPPCS. Within the present low-stakes settings, SDE scores are expected to correlate more strongly with measures of adjustment (self-esteem, emotional stability, social relations) and IM scores more strongly with measures of perceived virtue (honesty, spiritual values, morality; de Vries, Zettler, & Hilbig; [<reflink idref="bib12" id="ref105">12</reflink>]; Huang, [<reflink idref="bib22" id="ref106">22</reflink>]; Paulhus, [<reflink idref="bib35" id="ref107">35</reflink>], [<reflink idref="bib36" id="ref108">36</reflink>]; Paulhus & Trapnell, [<reflink idref="bib37" id="ref109">37</reflink>]; Stöber et al., [<reflink idref="bib47" id="ref110">47</reflink>]; Vispoel & Forte Fast, [<reflink idref="bib55" id="ref111">55</reflink>]; Vispoel & Kim, [<reflink idref="bib57" id="ref112">57</reflink>]; Vispoel, Morris, & Clough, [<reflink idref="bib58" id="ref113">58</reflink>]; Vispoel et al., [<reflink idref="bib60" id="ref114">60</reflink>], [<reflink idref="bib60" id="ref115">60</reflink>], [<reflink idref="bib62" id="ref116">62</reflink>], [<reflink idref="bib61" id="ref117">61</reflink>]).</p> <hd id="AN0136822484-13">Procedure and research design</hd> <p>Students from both samples completed the study across adjacent semesters to receive research credit points counting toward their final grades in several sections of the same educational psychology and statistics classes. Within both samples, research conditions reflected the most common ways in which paper and computerized questionnaires are administered in practice. Paper questionnaires typically include many items per page, with responses made either on the questionnaire itself or on a separate answer sheet. Computerized questionnaires rarely involve the use of separate answer sheets, but frequently employ single-item-per-screen administration. For example, single-item screens are typically used when items are administered adaptively based on responses to previous items (McClarty, [<reflink idref="bib26" id="ref118">26</reflink>]; Roper, Ben-Porath, & Butcher, [<reflink idref="bib40" id="ref119">40</reflink>]; Scullard, [<reflink idref="bib44" id="ref120">44</reflink>]; Simms & Clark, [<reflink idref="bib45" id="ref121">45</reflink>]).</p> <p>In each sample, respondents were assigned at random to four conditions: (a) paper administration with responses made on a separate answer sheet (PA), (b) paper administration with responses made directly in the questionnaire booklet (PB), (c) computerized administration with multiple items per screen (CM), and (d) computerized administration with single items per screen (CS). All questionnaires except the computerized single-item-per-screen version contained the same number and groupings of items per page/screen. Character shape, font (i.e., Palatino), intercharacter space, size of characters (14 point), characters per line, and interline spacing were the same across all conditions. Within the multiple-items per-page/screen conditions, there were equal lines and words per page. In all conditions, black text was presented against a white background with the same general directions and practice items. Directions varied only in relation to how to input responses (i.e., "circle" for PB, "bubble in" for PA, and "click-on" for CS and CM). To ensure good readability and consistency across the multi-item conditions, six items were presented per display (page or screen) for both the rating-scale and multiple-option questionnaires except on the last displays in which four items appeared for the SDQ-III and two for the SPPCS.</p> <p>Respondents completed measures in private enclosed cubicles within a research lab on campus using Macintosh Pro computers with answer review, answer change, and backtracking permitted in all conditions. Completion time was recorded automatically in the computer conditions and manually using electronic stopwatches in the paper conditions.</p> <hd id="AN0136822484-14">Analysis</hd> <p></p> <hd id="AN0136822484-15">Subscale scores</hd> <p>We focused initial analyses for each questionnaire on the psychometric properties of subscale scores (means, variances, reliability estimates, and concurrent validity coefficients). We adopted a significance level of 0.05 for all relevant statistical tests. Equality of means across research conditions was evaluated globally with one-way MANOVAs using the PROC GLM procedure in SAS 9.4. MANOVAs rather than subscale-specific ANOVAs were used to examine all subscales simultaneously and improve power (Tabachnick, & Fidell, [<reflink idref="bib48" id="ref122">48</reflink>]). Corresponding global chi-square tests for equality of subscale variances and equality of concurrent validity coefficients were derived using the "covtest" option in PROC GLIMMIX in SAS 9.4. Because we are aware of no global chi-square test for equality of alpha reliability estimates across multiple subscales within an instrument, we performed tests developed by Feldt, Woodruff, and Salih ([<reflink idref="bib20" id="ref123">20</reflink>]) at the individual subscale level using the R package "cocron" (Version 1.0-1; Diedenhofen, [<reflink idref="bib13" id="ref124">13</reflink>]).</p> <hd id="AN0136822484-16">Within-screen consistency</hd> <p>As noted earlier, a concern raised regarding presentation of multiple items per computer screen is that items presented together could yield more-highly-correlated responses and correspondingly higher internal consistency coefficients than those for the same items presented as single items per screen (Tourangeau et al., [<reflink idref="bib50" id="ref125">50</reflink>]; Weijters et al., [<reflink idref="bib66" id="ref126">66</reflink>]). To address this issue here, we compared alpha coefficients for sets of items presented on the same screen to those same items presented on individual screens, again using tests recommended by Feldt et al. ([<reflink idref="bib20" id="ref127">20</reflink>]). We conducted these analyses in two ways: (<reflink idref="bib1" id="ref128">1</reflink>) comparing alpha coefficients for subscales between the computer multi- and single-item conditions and (<reflink idref="bib2" id="ref129">2</reflink>) comparing alpha coefficients for items within the 23 multi-item screens used to administer the SDQ-III and nine multi-item screens used to administer the SPPCS to corresponding alpha coefficients for the same items administered on individual screens. Given the sheer number of tests for alpha coefficients conducted here at individual subscale and screen levels (84 in all), we expected some to be statistically significant by chance alone, but not so many as to noticeably exceed the significance level of 0.05 we established for all statistical tests.</p> <hd id="AN0136822484-17">Completion time</hd> <p>To parallel hypotheses stated earlier, we included three orthogonal, a priori contrasts within a one-way ANOVA design for each sample. These tests contrasted the following questionnaire conditions: (<reflink idref="bib1" id="ref130">1</reflink>) answer sheets (PA) versus no answer sheets (the average of PB, CM, and CS), (<reflink idref="bib2" id="ref131">2</reflink>) the single-item, no answer sheet condition (CS) versus the multi-item, no answer sheet conditions (the average of CM and PB), and (<reflink idref="bib3" id="ref132">3</reflink>) the multi-item, no answer sheet conditions compared to each other (CM versus PB). Based on studies by Simms and Clark ([<reflink idref="bib45" id="ref133">45</reflink>]) and Vispoel, Morris, and Clough, ([<reflink idref="bib58" id="ref134">58</reflink>]), we expected questionnaires with answer sheets (PA) to take longer than questionnaires without answer sheets (CS, CM, & PB). In keeping with these studies and with Couper et al. ([<reflink idref="bib11" id="ref135">11</reflink>]), Toepoel et al. ([<reflink idref="bib49" id="ref136">49</reflink>]) and Tourangeau et al. ([<reflink idref="bib50" id="ref137">50</reflink>]), we expected computerized questionnaires with single-item displays (CS) to take longer than multi-item computerized questionnaires and paper questionnaires with no answer sheets (CM and PB). Finally, based on findings from Clough ([<reflink idref="bib10" id="ref138">10</reflink>]), Vispoel ([<reflink idref="bib53" id="ref139">53</reflink>]), Vispoel et al. ([<reflink idref="bib54" id="ref140">54</reflink>]), and Weigold et al. ([<reflink idref="bib65" id="ref141">65</reflink>]), we expected the computerized questionnaire with multi-items (CM) to take longer than the paper booklet questionnaire with multiple items (PB). To express results across studies on a common scale, we represented completion time for all analyses on the item metric by dividing total completion time minus direction time by the number of items included in each measure (136 for the SDQ-III and 54 for the SPPCS).</p> <p>To compare average-item-administration time between rating-scale and multiple-alternative items, we combined results across samples. Although respondents from both samples came from the same classes over adjacent semesters, they still may have differed from each other in systematic ways. We therefore tested for possible differences between samples in gender (male, female), race (Caucasian, non-Caucasian), age, computer experience, and highest level of parental education. Because statistically significant results were found between samples for age [<emph>t</emph>(<reflink idref="bib479" id="ref142">479</reflink>) = 3.724, <emph>p</emph> =.0002, ] and race [(<emph>χ<sups>2</sups></emph>(<reflink idref="bib1" id="ref143">1</reflink>) = 9.081, <emph>p</emph> =.0026, <emph>w</emph> = .14)], we included those variables as covariates in an ANCOVA design with item type and questionnaire condition as between-subjects factors and average-item-completion time as the dependent variable. We represented questionnaire conditions using the same three orthogonal contrasts as in the individual analyses for each sample. The item type by questionnaire condition interactions within the design allowed for evaluation of whether effects of answer sheets, number of items per display, and computerization varied with item type. We expected average-item-completion time to be longer for multiple-alternative than for rating scale items, and explored whether differences in time between item types varied across questionnaire conditions via the item type by questionnaire condition interaction terms.</p> <hd id="AN0136822484-18">Results</hd> <p></p> <hd id="AN0136822484-19">Subscale scores</hd> <p>Means, standard deviations, and reliability coefficients for questionnaire subscales appear in Table 1 for both samples. MANOVAs for subscale means yielded no statistically significant differences among questionnaire conditions [SDQ-III: <emph>F</emph>(<reflink idref="bib39" id="ref144">39</reflink>, 617) = 1.24, <emph>p</emph> = 0.16; SPPCS: <emph>F</emph>(<reflink idref="bib39" id="ref145">39</reflink>, 714) = 0.74, <emph>p</emph> = 0.88] nor did global chi-square tests for equality of variances [SDQ-III: <emph>χ<sups>2</sups></emph> (<reflink idref="bib39" id="ref146">39</reflink>) = 49.25, <emph>p</emph> =.13; SPPCS: <emph>χ<sups>2</sups></emph> (<reflink idref="bib39" id="ref147">39</reflink>) = 38.19, <emph>p</emph> =.51] or equality of concurrent validity coefficients [SDQ-III: <emph>χ<sups>2</sups></emph> (<reflink idref="bib78" id="ref148">78</reflink>) = 93.68, <emph>p</emph> =.11; SPPCS: <emph>χ<sups>2</sups></emph> (<reflink idref="bib78" id="ref149">78</reflink>) = 78.13, <emph>p</emph> =.47].</p> <p>Table 1. Means, standard deviations, and alpha coefficients for SDQ-III and SPPCS scores.</p> <p> <ephtml> <table><thead><tr><td /><td>PA (<italic>n</italic> = 57)</td><td>PB (<italic>n</italic> = 57)</td><td>CM (<italic>n</italic> = 57)</td><td>CS (<italic>n</italic> = 53)</td></tr><tr><td>SDQ-III Subscale</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td></tr></thead><tbody><tr><td>General Self</td><td char=".">77.81</td><td char=".">13.78</td><td char=".">.94</td><td char=".">79.02</td><td char=".">11.97</td><td char=".">.96</td><td char=".">78.74</td><td char=".">14.20</td><td char=".">.96</td><td char=".">77.26</td><td char=".">11.46</td><td char=".">.94</td></tr><tr><td>Emotional Stability</td><td char=".">54.56</td><td char=".">11.21</td><td char=".">.87</td><td char=".">55.60</td><td char=".">11.35</td><td char=".">.90</td><td char=".">54.58</td><td char=".">12.32</td><td char=".">.91</td><td char=".">52.49</td><td char=".">12.53</td><td char=".">.92</td></tr><tr><td>Peer Relations—Opposite Sex</td><td char=".">62.44</td><td char=".">9.67</td><td char=".">.86</td><td char=".">59.56</td><td char=".">10.13</td><td char=".">.87</td><td char=".">61.21</td><td char=".">10.66</td><td char=".">.90</td><td char=".">55.11</td><td char=".">13.85</td><td char=".">.93</td></tr><tr><td>Peer Relations—Same Sex</td><td char=".">63.40</td><td char=".">10.30</td><td char=".">.88</td><td char=".">63.79</td><td char=".">9.09</td><td char=".">.84</td><td char=".">61.46</td><td char=".">9.94</td><td char=".">.86</td><td char=".">61.60</td><td char=".">10.27</td><td char=".">.88</td></tr><tr><td>Parent Relations</td><td char=".">65.51</td><td char=".">10.05</td><td char=".">.88</td><td char=".">64.19</td><td char=".">11.87</td><td char=".">.91</td><td char=".">64.25</td><td char=".">13.31</td><td char=".">.94</td><td char=".">66.92</td><td char=".">9.51</td><td char=".">.88</td></tr><tr><td>Honesty/Trustworthiness</td><td char=".">75.05</td><td char=".">8.46</td><td char=".">.69</td><td char=".">74.30</td><td char=".">9.71</td><td char=".">.79</td><td char=".">73.58</td><td char=".">9.47</td><td char=".">.77</td><td char=".">77.00</td><td char=".">8.44</td><td char=".">.74</td></tr><tr><td>Spiritual Values/Religion</td><td char=".">64.47</td><td char=".">20.66</td><td char=".">.96</td><td char=".">63.95</td><td char=".">22.69</td><td char=".">.97</td><td char=".">61.58</td><td char=".">19.15</td><td char=".">.95</td><td char=".">67.19</td><td char=".">18.46</td><td char=".">.97</td></tr><tr><td>Physical Appearance</td><td char=".">53.72</td><td char=".">13.83</td><td char=".">.94</td><td char=".">53.81</td><td char=".">12.30</td><td char=".">.92</td><td char=".">54.63</td><td char=".">11.45</td><td char=".">.91</td><td char=".">53.55</td><td char=".">11.51</td><td char=".">.91</td></tr><tr><td>Physical Ability</td><td char=".">63.33</td><td char=".">13.20</td><td char=".">.94</td><td char=".">62.86</td><td char=".">15.36</td><td char=".">.97</td><td char=".">62.47</td><td char=".">14.11</td><td char=".">.97</td><td char=".">62.21</td><td char=".">13.86</td><td char=".">.96</td></tr><tr><td>General Academic Ability</td><td char=".">57.32</td><td char=".">12.01</td><td char=".">.91</td><td char=".">57.33</td><td char=".">11.67</td><td char=".">.93</td><td char=".">58.25</td><td char=".">9.97</td><td char=".">.90</td><td char=".">57.34</td><td char=".">9.81</td><td char=".">.90</td></tr><tr><td>Verbal Skills</td><td char=".">58.63</td><td char=".">11.10</td><td char=".">.87</td><td char=".">58.79</td><td char=".">9.67</td><td char=".">.85</td><td char=".">57.32</td><td char=".">9.46</td><td char=".">.80</td><td char=".">53.28</td><td char=".">12.03</td><td char=".">.88</td></tr><tr><td>Math Skills</td><td char=".">47.33</td><td char=".">18.55</td><td char=".">.96</td><td char=".">47.95</td><td char=".">18.21</td><td char=".">.96</td><td char=".">44.07</td><td char=".">17.66</td><td char=".">.95</td><td char=".">50.25</td><td char=".">15.29</td><td char=".">.95</td></tr><tr><td>Problem Solving Ability</td><td char=".">53.74</td><td char=".">9.39</td><td char=".">.79</td><td char=".">55.91</td><td char=".">9.33</td><td char=".">.83</td><td char=".">54.14</td><td char=".">7.88</td><td char=".">.72</td><td char=".">55.09</td><td char=".">9.68</td><td char=".">.83</td></tr><tr><td /><td>PA (<italic>n</italic> = 63)</td><td>PB (<italic>n</italic> = 64)</td><td>CM (<italic>n</italic> = 65)</td><td>CS (<italic>n</italic> = 65)</td></tr><tr><td>SPPCS Subscale</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td><td><italic>M</italic></td><td><italic>SD</italic></td><td>α</td></tr><tr><td>Global Self-worth</td><td char=".">18.08</td><td char=".">3.65</td><td char=".">.88</td><td char=".">18.70</td><td char=".">3.65</td><td char=".">.87</td><td char=".">19.03</td><td char=".">3.59</td><td char=".">.87</td><td char=".">18.57</td><td char=".">3.58</td><td char=".">.89</td></tr><tr><td>Social Acceptance</td><td char=".">11.78</td><td char=".">3.02</td><td char=".">.84</td><td char=".">12.22</td><td char=".">3.04</td><td char=".">.85</td><td char=".">13.23</td><td char=".">2.61</td><td char=".">.86</td><td char=".">12.43</td><td char=".">3.05</td><td char=".">.89</td></tr><tr><td>Romantic Relationships</td><td char=".">10.54</td><td char=".">3.21</td><td char=".">.88</td><td char=".">10.55</td><td char=".">3.48</td><td char=".">.89</td><td char=".">10.94</td><td char=".">3.11</td><td char=".">.88</td><td char=".">10.46</td><td char=".">3.76</td><td char=".">.93</td></tr><tr><td>Close Friendships</td><td char=".">13.13</td><td char=".">2.76</td><td char=".">.78</td><td char=".">13.38</td><td char=".">2.85</td><td char=".">.86</td><td char=".">13.69</td><td char=".">2.45</td><td char=".">.78</td><td char=".">13.02</td><td char=".">3.18</td><td char=".">.87</td></tr><tr><td>Parental Relations</td><td char=".">14.37</td><td char=".">2.38</td><td char=".">.86</td><td char=".">14.44</td><td char=".">2.50</td><td char=".">.89</td><td char=".">14.72</td><td char=".">2.00</td><td char=".">.83</td><td char=".">14.88</td><td char=".">1.79</td><td char=".">.70</td></tr><tr><td>Morality</td><td char=".">13.19</td><td char=".">2.21</td><td char=".">.75</td><td char=".">12.80</td><td char=".">2.55</td><td char=".">.83</td><td char=".">13.02</td><td char=".">2.50</td><td char=".">.81</td><td char=".">13.05</td><td char=".">2.74</td><td char=".">.88</td></tr><tr><td>Physical Appearance</td><td char=".">10.16</td><td char=".">3.06</td><td char=".">.86</td><td char=".">9.84</td><td char=".">2.82</td><td char=".">.87</td><td char=".">10.82</td><td char=".">2.98</td><td char=".">.84</td><td char=".">9.78</td><td char=".">3.35</td><td char=".">.93</td></tr><tr><td>Athletic Competence</td><td char=".">10.25</td><td char=".">3.95</td><td char=".">.94</td><td char=".">10.55</td><td char=".">3.70</td><td char=".">.94</td><td char=".">10.97</td><td char=".">3.41</td><td char=".">.92</td><td char=".">11.18</td><td char=".">3.22</td><td char=".">.91</td></tr><tr><td>Intellectual Ability</td><td char=".">12.33</td><td char=".">2.74</td><td char=".">.87</td><td char=".">12.42</td><td char=".">2.56</td><td char=".">.83</td><td char=".">12.15</td><td char=".">2.95</td><td char=".">.87</td><td char=".">12.00</td><td char=".">3.21</td><td char=".">.93</td></tr><tr><td>Scholastic Competence</td><td char=".">11.24</td><td char=".">2.71</td><td char=".">.81</td><td char=".">11.42</td><td char=".">2.65</td><td char=".">.80</td><td char=".">11.23</td><td char=".">2.70</td><td char=".">.86</td><td char=".">11.00</td><td char=".">2.97</td><td char=".">.88</td></tr><tr><td>Job Competence</td><td char=".">12.02</td><td char=".">2.30</td><td char=".">.72</td><td char=".">12.38</td><td char=".">2.36</td><td char=".">.73</td><td char=".">12.14</td><td char=".">2.56</td><td char=".">.71</td><td char=".">12.54</td><td char=".">2.37</td><td char=".">.72</td></tr><tr><td>Creativity</td><td char=".">10.81</td><td char=".">3.29</td><td char=".">.93</td><td char=".">11.20</td><td char=".">2.88</td><td char=".">.93</td><td char=".">11.52</td><td char=".">2.66</td><td char=".">.92</td><td char=".">11.05</td><td char=".">3.44</td><td char=".">.93</td></tr><tr><td>Sense of Humor</td><td char=".">13.29</td><td char=".">2.70</td><td char=".">.89</td><td char=".">13.44</td><td char=".">2.35</td><td char=".">.86</td><td char=".">13.83</td><td char=".">2.30</td><td char=".">.89</td><td char=".">13.97</td><td char=".">2.30</td><td char=".">.86</td></tr></tbody></table> </ephtml> </p> <p>5 <emph>Note</emph>. PA = paper with answer sheet, CS = computer with single item per screen, CM = computer with multiple items per screen, and PB = paper answered directly in the booklet.</p> <p>The highlighted validity coefficients (i.e., correlations between BIDR and targeted self-concept scores across the research conditions) in Table 2 confirm expectations that BIDR SDE scores would be correlated more strongly with measures of adjustment (self-esteem, emotional stability, social relations) and IM scores more strongly with measures of perceived virtue (honesty, spiritual values, morality). In nearly all cases for the remaining self-concept scales that focus on physical attributes, academic/job skills, and sense of humor, correlations were higher for SDE than for IM. These relationships make sense given that SDE, more so than IM, represents tendencies to overendorse transparent characteristics that most people would consider desirable. The overall pattern of differences between correlations for SDE and IM is also identical to that reported by Vispoel and Forte Fast ([<reflink idref="bib55" id="ref150">55</reflink>]) for SDQ-III subscale scores.</p> <p>Table 2. Correlations of SDQ-III and SPPCS scores with BIDR scores.</p> <p> <ephtml> <table><thead><tr><td>SDQ-III Subscale (<italic>n</italic> = 224)</td><td>SDE</td><td>IM</td></tr></thead><tbody><tr><td>General Self</td><td char="."><bold>.491</bold></td><td char="."><italic>.070</italic></td></tr><tr><td>Emotional Stability</td><td char="."><bold>.510</bold></td><td char="."><italic>.097</italic></td></tr><tr><td>Peer Relations—Opposite Sex</td><td char="."><bold>.384</bold></td><td char="."><italic>−.056</italic></td></tr><tr><td>Peer Relations—Same Sex</td><td char="."><bold>.255</bold></td><td char="."><italic>.058</italic></td></tr><tr><td>Parent Relations</td><td char="."><bold>.246</bold></td><td char="."><italic>.186</italic></td></tr><tr><td>Honesty/Trustworthiness</td><td char="."><italic>.303</italic></td><td char="."><bold>.577</bold></td></tr><tr><td>Spiritual Values/Religion</td><td char="."><italic>.017</italic></td><td char="."><bold>.229</bold></td></tr><tr><td>Physical Appearance</td><td char=".">.374</td><td char=".">.192</td></tr><tr><td>Physical Ability</td><td char=".">.223</td><td char=".">−.048</td></tr><tr><td>General Academic Ability</td><td char=".">.220</td><td char=".">.253</td></tr><tr><td>Verbal Skills</td><td char=".">.367</td><td char=".">.165</td></tr><tr><td>Math Skills</td><td char=".">.188</td><td char=".">.150</td></tr><tr><td>Problem Solving Ability</td><td char=".">.379</td><td char=".">.196</td></tr><tr><td>SPPCS Subscale (n = 257)</td><td /><td char="." /></tr><tr><td>Global Self-worth</td><td char="."><bold>.497</bold></td><td char="."><italic>.095</italic></td></tr><tr><td>Social Acceptance</td><td char="."><bold>.307</bold></td><td char="."><italic>.004</italic></td></tr><tr><td>Romantic Relationships</td><td char="."><bold>.373</bold></td><td char="."><italic>.012</italic></td></tr><tr><td>Close Friendships</td><td char="."><bold>.271</bold></td><td char="."><italic>.064</italic></td></tr><tr><td>Parent Relationships</td><td char="."><bold>.196</bold></td><td char="."><italic>.065</italic></td></tr><tr><td>Morality</td><td char="."><italic>.240</italic></td><td char=".">.<bold>403</bold></td></tr><tr><td>Physical Appearance</td><td char=".">.284</td><td char=".">.171</td></tr><tr><td>Athletic Competence</td><td char=".">.247</td><td char=".">−.040</td></tr><tr><td>Intellectual Ability</td><td char=".">.279</td><td char=".">−.025</td></tr><tr><td>Scholastic Competence</td><td char=".">.282</td><td char=".">.090</td></tr><tr><td>Job Competence</td><td char=".">.412</td><td char=".">.111</td></tr><tr><td>Creativity</td><td char=".">.241</td><td char=".">.059</td></tr><tr><td>Sense of Humor</td><td char=".">.271</td><td char=".">−.012</td></tr></tbody></table> </ephtml> </p> <p>6 <emph>Note</emph>. SDE = Self-deceptive enhancement; IM = Impression management. Boldface and italicized indices represent convergent and discriminant validity coefficients for BIDR scores with measures of adjustment and perceived virtue. Anticipated differences between these indices were tested using the R package "cocor" (Diedenhofen, & Musch, [<reflink idref="bib14" id="ref151">14</reflink>]) based on a procedure described by Steiger ([<reflink idref="bib46" id="ref152">46</reflink>]). All tests except the parent relations (<emph>p</emph> = 0.22) scale from the SDQ-III yielded statistically significant results (<emph>p</emph>s <.012), thereby confirming expectations that SDE would correlate better with adjustment (self-esteem, emotional stability, social relations) and IM, better with perceived virtue (honesty, spiritual values, morality). Nondirectional tests using the same procedure for differences between correlations for SDE and IM with the remaining self-concept scales yielded statistically significant results (<emph>p</emph>s <.017) for all but the general academic ability (<emph>p</emph> = 0.66) and math skills (<emph>p</emph> = 0.62) scales from the SDQ-III and the physical appearance (<emph>p</emph> = 0.13) scale from the SPPCS. Alpha reliability estimates for SDE and IM scores, respectively, were 0.72 and 0.82 for the SDQ-III sample and 0.71 and 0.81 for the SPPCS sample.</p> <p>No statistically significant differences were found across the four research conditions for alpha reliability estimates except for the parental relationship scales from both instruments [SDQ-III: <emph>χ<sups>2</sups></emph> (<reflink idref="bib3" id="ref153">3</reflink>) = 9.86, <emph>p</emph> = 0.02, <emph>w</emph> = 0.21: SPPCS: <emph>χ<sups>2</sups></emph> (<reflink idref="bib3" id="ref154">3</reflink>) = 9.73, <emph>p</emph> = 0.02, <emph>w</emph> = 0.19]. Follow-up pairwise contrasts with upward Bonferroni corrections for <emph>p</emph> values revealed that no contrasts were significant for the SDQ-III scale (<emph>p</emph>s ≥ 0.06) and that only one contrast was significant for the SPPCS scale (<emph>χ<sups>2</sups></emph>(<reflink idref="bib1" id="ref155">1</reflink>) = 9.42, <emph>p</emph> = 0.01, <emph>w</emph> = 0.19) in which the alpha coefficient for CS was lower than that for PB (see Table 1).</p> <hd id="AN0136822484-20">Within-screen consistency</hd> <p>We evaluated possible item-grouping effects for computerized administration further by directly comparing alpha coefficients for the CM and CS conditions at subscale and screen display levels. Among these 58 comparisons, only two (3.4%) yielded statistically significant results at the 0.05 level. None of the 32 comparisons for screens yielded significant results (<emph>p</emph> > 0.06) and only two did so for subscales. The significant results were for the SDQ-III Parent Relations scale [<emph>χ<sups>2</sups></emph>(<reflink idref="bib1" id="ref156">1</reflink>) = 6.69, <emph>p</emph> = 0.01, <emph>w</emph> = 0.17] in which the alpha coefficient for CM was higher, and for the SPPCS Physical Appearance scale [<emph>χ<sups>2</sups></emph>(<reflink idref="bib1" id="ref157">1</reflink>) = 5.61, <emph>p</emph> = 0.02, <emph>w</emph> = 0.15] in which the alpha coefficient for CS was higher (see Table 1).</p> <hd id="AN0136822484-21">Completion time</hd> <p>Unlike results for the psychometric properties of scores, those from the one-way ANOVA for completion time within each sample yielded statistically significant results for all a priori orthogonal contrasts (<emph>p</emph> < 0.02; see Table 3). Table 4 reveals that average-item-completion time was longest for PA (SDQ-III, <emph>M</emph> = 8.02 seconds; SPPCS, <emph>M</emph> = 10.41 seconds), followed respectively by CS (SDQ-III, <emph>M</emph> = 7.17 seconds; SPPCS, <emph>M</emph> = 9.44 seconds), CM (SDQ-III, <emph>M</emph> = 6.10 seconds; SPPCS, <emph>M</emph> = 8.49 seconds), and PB (SDQ-III, <emph>M</emph> = 5.38 seconds; SPPCS, <emph>M</emph> = 7.28 seconds). The strongest contrast effects were for answer sheets (<emph>d</emph>s = 1.15 for the SDQ-III and 0.96 for the SPPCS), followed by single-item displays (<emph>d</emph>s = 0.91 for the SDQ-III and 0.74 for the SPPCS), followed by computerization (<emph>d</emph>s = 0.46 for the SDQ-III and 0.58 for the SPPCS). Answer sheets increased completion times by 24% to 29%, single-item displays by 20% to 25%, and computerization by 13% to 17%.</p> <p>Table 3. Planned contrast t statistics, mean differences, effect sizes, and percent increases for item completion time.</p> <p> <ephtml> <table><thead><tr><td>Measure/Contrast</td><td><italic>t</italic><xref ref-type="fn" rid="tfn8" /></td><td><italic>p</italic> value</td><td><italic>M</italic><bold>difference</bold><sup>c</sup></td><td><italic>d</italic><sup>c</sup></td><td>% increase<xref ref-type="fn" rid="tfn9" /></td></tr></thead><tbody><tr><td>SDQ-III</td><td char="." /><td char="." /><td char="." /><td char="." /><td char="." /></tr><tr><td> PA versus Mean (CS, CM, PB)</td><td char=".">7.51</td><td char=".">< 0.0001</td><td char=".">1.80</td><td char=".">1.15</td><td char=".">29.00</td></tr><tr><td> CS versus Mean (CM, PB)</td><td char=".">5.49</td><td char=".">< 0.0001</td><td char=".">1.43</td><td char=".">0.91</td><td char=".">24.86</td></tr><tr><td> CM versus PB</td><td char=".">2.45</td><td char=".">0.0150</td><td char=".">0.72</td><td char=".">0.46</td><td char=".">13.35</td></tr><tr><td>SPPCS</td><td char="." /><td char="." /><td char="." /><td char="." /><td char="." /></tr><tr><td> PA versus Mean (CS, CM, PB)</td><td char=".">6.62</td><td char=".">< 0.0001</td><td char=".">2.01</td><td char=".">0.96</td><td char=".">23.93</td></tr><tr><td> CS versus Mean (CM, PB)</td><td char=".">4.87</td><td char=".">< 0.0001</td><td char=".">1.55</td><td char=".">0.74</td><td char=".">19.65</td></tr><tr><td> CM versus PB</td><td char=".">3.28</td><td char=".">0.0012</td><td char=".">1.21</td><td char=".">0.58</td><td char=".">16.61</td></tr><tr><td>SDQ-III & SPPCS</td><td char="." /><td char="." /><td char="." /><td char="." /><td char="." /></tr><tr><td> Item Type (SPPCS versus SDQ-III)</td><td char=".">12.17</td><td char=".">< 0.0001</td><td char=".">2.24 (2.03)</td><td char=".">1.20 (1.09)</td><td char=".">33.72</td></tr><tr><td> PA versus Mean (CS, CM, PB)</td><td char=".">10.47</td><td char=".">< 0.0001</td><td char=".">1.89 (1.98)</td><td char=".">1.01 (1.06)</td><td char=".">25.48</td></tr><tr><td> CS versus Mean (CM, PB)</td><td char=".">7.84</td><td char=".">< 0.0001</td><td char=".">1.54 (1.58)</td><td char=".">0.82 (0.84)</td><td char=".">22.34</td></tr><tr><td> CM versus PB</td><td char=".">3.70</td><td char=".">0.0002</td><td char=".">0.99 (0.85)</td><td char=".">0.53 (0.46)</td><td char=".">15.46</td></tr></tbody></table> </ephtml> </p> <ulist> <item>7 <emph>Note</emph>. PA = Paper with answer sheet; CS = Computer with single item per screen; CM = Computer with multiple items per screen; PB = Paper booklet; <emph>d</emph> = standardized mean difference. For the separate SDQ-III and SPPCS analyses, questionnaire condition, respectively, accounted for 29.9% and 23.5% of the variance in item completion time. For the combined analysis, covariates, item type, and questionnaire condition, respectively, accounted for 8.3%, 17.2%, and 21.0% of the variance. Preliminary tests within the combined analysis for the covariate by research condition interactions [<emph>F</emph>(<reflink idref="bib14" id="ref158">14</reflink>, 457) = 1.40, <emph>p</emph> = 0.15] and the item type by questionnaire condition interactions [<emph>F</emph>(<reflink idref="bib3" id="ref159">3</reflink>, 471) = 0.55, <emph>p</emph> = 0.65] did not yield statistically significant results.</item> <item>8 All <emph>t</emph> tests for a priori contrasts reported in the table are one-tailed. <emph>t</emph> statistics for completion time in the combined analysis for the SDQ-III and SPPCS represent unique effects for each contrast after controlling for the effects of the covariates and other contrasts within the design.</item> <item>9 Percentage increase values were computed as follows: 100 × (PA – Mean (CS, CM, PB))/Mean (CS, CM, PB); 100 × (CS – Mean (CM, PB))/Mean (CM, PB); 100 × (CM – PB)/PB; 100 × (SPPCS – SDQ-III)/SDQ-III.</item> </ulist> <p>Table 4. Administration time means and standard deviations (in seconds) across questionnaire conditions.</p> <p> <ephtml> <table><thead><tr><td /><td>Total Time<xref ref-type="fn" rid="tfn11" /></td><td>Total Time—Direction Time</td><td>Direction Time<xref ref-type="fn" rid="tfn12" /></td><td>Item Time</td></tr><tr><td /><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>M</italic></td><td><italic>SD</italic></td></tr></thead><tbody><tr><td>SDQ-III</td><td char="." /><td char="." /><td char="." /><td char="." /><td char="." /></tr><tr><td>Condition</td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> PA</td><td>1,138.88</td><td>241.40</td><td>1,090.23</td><td>230.49</td><td>48.65</td><td>19.87</td><td>8.02</td><td>1.69</td></tr><tr><td> CS</td><td>1,025.74</td><td>212.55</td><td>974.47</td><td>197.09</td><td>51.26</td><td>24.98</td><td>7.17</td><td>1.45</td></tr><tr><td> CM</td><td>879.26</td><td>253.25</td><td>829.26</td><td>242.71</td><td>50.00</td><td>29.30</td><td>6.10</td><td>1.78</td></tr><tr><td> PB</td><td>778.84</td><td>175.38</td><td>731.59</td><td>172.23</td><td>47.25</td><td>14.59</td><td>5.38</td><td>1.27</td></tr><tr><td> No answer sheet (CS, CM, PB)</td><td>891.47</td><td>237.33</td><td>842.01</td><td>227.80</td><td>49.46</td><td>23.66</td><td>6.19</td><td>1.67</td></tr><tr><td> No answer sheet, multi-item (CM, PB)</td><td>829.05</td><td>222.64</td><td>780.43</td><td>215.17</td><td>48.63</td><td>23.08</td><td>5.74</td><td>1.58</td></tr><tr><td>SPPCS</td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td>Condition</td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> PA</td><td>624.83</td><td>143.25</td><td>562.33</td><td>125.79</td><td>62.49</td><td>29.91</td><td>10.41</td><td>2.33</td></tr><tr><td> CS</td><td>571.94</td><td>123.61</td><td>509.55</td><td>109.51</td><td>62.38</td><td>22.77</td><td>9.44</td><td>2.03</td></tr><tr><td> CM</td><td>525.91</td><td>143.56</td><td>458.51</td><td>125.33</td><td>67.40</td><td>31.43</td><td>8.49</td><td>2.32</td></tr><tr><td> PB</td><td>455.03</td><td>105.94</td><td>393.20</td><td>87.27</td><td>61.83</td><td>42.29</td><td>7.28</td><td>1.62</td></tr><tr><td> No answer sheet (CS, CM, PB)</td><td>517.95</td><td>133.72</td><td>454.07</td><td>118.08</td><td>63.88</td><td>33.01</td><td>8.41</td><td>2.19</td></tr><tr><td> No answer sheet, multi-item (CM, PB)</td><td>490.74</td><td>130.74</td><td>426.11</td><td>112.59</td><td>64.63</td><td>37.18</td><td>7.89</td><td>2.09</td></tr><tr><td>SDQ-III & SPPCS</td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td>Condition</td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> PA</td><td>869.00</td><td>323.36</td><td>813.08</td><td>321.44</td><td>55.92</td><td>26.46</td><td>9.27</td><td>2.37</td></tr><tr><td> CS</td><td>775.76</td><td>282.53</td><td>718.37</td><td>278.85</td><td>57.39</td><td>24.33</td><td>8.42</td><td>2.11</td></tr><tr><td> CM</td><td>691.00</td><td>268.18</td><td>631.73</td><td>264.71</td><td>59.27</td><td>31.55</td><td>7.37</td><td>2.40</td></tr><tr><td> PB</td><td>607.57</td><td>215.84</td><td>552.61</td><td>215.89</td><td>54.96</td><td>33.04</td><td>6.39</td><td>1.74</td></tr><tr><td> No answer sheet (CS, CM, PB)</td><td>690.74</td><td>265.22</td><td>633.53</td><td>262.53</td><td>57.21</td><td>29.90</td><td>7.38</td><td>2.25</td></tr><tr><td> No answer sheet, multi-item (CM, PB)</td><td>649.46</td><td>246.60</td><td>592.33</td><td>244.38</td><td>57.12</td><td>32.31</td><td>6.88</td><td>2.15</td></tr></tbody></table> </ephtml> </p> <ulist> <item>10 <emph>Note</emph>. Values in parentheses are adjusted for the covariates. PA = Paper with answer sheet; CS = Computer with single-item screens; CM = Computer with multi-items screens; PB = Paper booklet.</item> <item>11 ANOVAs for total time with directions for each instrument yielded patterns of findings identical to those for item time reported in Table 3.</item> <item>12 ANOVAs for direction time failed to yield statistically significant condition effects for either instrument [SDQ-III: <emph>F</emph>(<reflink idref="bib3" id="ref160">3</reflink>,<reflink idref="bib220" id="ref161">220</reflink>) = 0.317, <emph>p</emph> = 0.813: SPPCS: <emph>F</emph>(<reflink idref="bib3" id="ref162">3</reflink>,<reflink idref="bib253" id="ref163">253</reflink>) = 0.409, <emph>p</emph> = 0.747].</item> </ulist> <p>ANCOVA results for average-item-administration time across samples revealed no statistically significant interactions between item type and questionnaire conditions [<emph>F</emph>(<reflink idref="bib3" id="ref164">3</reflink>, 471) = 0.55, <emph>p</emph> = 0.65] but statistically significant main effects for the covariates [<emph>F</emph>(<reflink idref="bib2" id="ref165">2</reflink>, 471) = 36.58, <emph>p</emph> < 0.0001], item types [<emph>F</emph>(<reflink idref="bib1" id="ref166">1</reflink>, 471) = 225.19, <emph>p</emph> < 0.0001], and questionnaire conditions [<emph>F</emph>(<reflink idref="bib3" id="ref167">3</reflink>, 471) = 136.80, <emph>p</emph> < 0.0001]. Covariates, item types, and questionnaire conditions respectively accounted for 8.3%, 17.2%, and 21.0% of the variance in average-item-completion time. Effects for the covariates indicated that average-item-completion time was longer for older than for younger individuals (<emph>r</emph> =.27) and for non-Caucasians (<emph>M</emph> = 8.94) than for Caucasians (<emph>M</emph> = 7.69; <emph>d</emph> = 0.70).</p> <p>Table 3 shows t statistics, effect sizes, and percentage of item time increases for all a priori contrasts in the combined sample analysis. As expected, multiple-alternative (<emph>M</emph> = 8.90) items took longer on average than did rating-scale items (<emph>M</emph> = 6.66, <emph>d</emph> = 1.20, % increase = 33.72). Consistent with results for the samples analyzed separately, item-completion time was longer with answer sheets (<emph>d</emph> = 1.01, % increase = 25.48), with single-item displays (<emph>d</emph> = 0.86, % increase = 22.34), and with computerized presentation (<emph>d</emph> = 0.53, % increase = 15.46).</p> <hd id="AN0136822484-22">Discussion</hd> <p></p> <hd id="AN0136822484-23">Psychometric properties of scores</hd> <p>Results from the present studies highlight at least three sets of noteworthy findings. Although not the primary focus of this study, the first key finding was to confirm a consistent trend over time in supporting congruence of psychometric results for computer and paper measures of affective constructs when computerized measures ensure privacy and allow for skipping, backtracking, and change of responses. However, the present research extends prior investigations by considering psychometric properties beyond means (i.e., variances, reliability coefficients, and concurrent validity coefficients), by including four different formats for administering questionnaires, and by replicating those findings in two independent studies using different types of items. The only statistically significant differences in psychometric properties of scores across questionnaire conditions were for alpha reliability estimates. Out of the 84 comparisons of alpha coefficients reported at subscale and screen display levels, only four (4.8%) yielded statistically significant results beyond the 5% level, which is comparable to what would be expected purely by chance.</p> <p>We also found no compelling evidence of differences in alpha coefficients when directly comparing single- and multi-item computer displays. Among those 58 comparisons, only two (3.4%) were statistically significant (<emph>p</emph> <.05), but favored CM in one instance and CS in the other. These findings are in line with Couper et al. ([<reflink idref="bib11" id="ref168">11</reflink>]) and Toepoel et al. ([<reflink idref="bib49" id="ref169">49</reflink>]) but differ from Tourangeau et al. ([<reflink idref="bib50" id="ref170">50</reflink>]) and Weijters et al. ([<reflink idref="bib66" id="ref171">66</reflink>]). We attribute our contrasting result to the spiraled presentation of items within each instrument that prevented multiple items from common subscales appearing on the same screen. However, we strongly encourage verification of our findings in future research by directly comparing grouped and spiraled presentation of the current items and those from other instruments.</p> <p>Finally, in addition to gauging effects of questionnaire formats on scores, the present analyses contribute to the broader research literature on relationships between facets of self-concept and socially desirable responding. Our findings confirmed that SDE is more highly correlated with measures of adjustment (self-esteem, emotional stability, social relations) and IM with perceived virtue (honesty, spiritual values, morality). However, the results further showed that SDE is more highly correlated than IM with other key facets of self-concept that included physical attributes, academic/job skills, and sense of humor.</p> <hd id="AN0136822484-24">Completion time</hd> <p>The second important set of findings and the primary focus of the present research centered on how modes and response formats for administering questionnaires affect completion time. To clarify and extend research in this area, we took great care in controlling factors often confounded in prior studies and used experimental designs that permitted direct comparisons of effects for answer sheets, single-item administration, and direct conversion of paper questionnaires to computerized formats. In contrast to psychometric characteristics of scores, average-item-completion time was markedly affected by methods of administration with use of answer sheets leading to the greatest increases, followed by single-item displays, and computerization of a paper questionnaire. The strong effects we found for answer sheets are consistent with findings from Sims and Clark (2005) but correspond more closely to those from Vispoel, Morris, and Clough ([<reflink idref="bib58" id="ref172">58</reflink>]), which, like the present study, included paper and computerized questionnaires containing the same numbers of items per page as per screen. In both studies, standardized mean differences reflecting greater completion time with answer sheets were large, ranging from 0.90 to 1.07 in Vispoel, Morris, and Clough ([<reflink idref="bib58" id="ref173">58</reflink>]) compared to 0.96 to 1.15 here.</p> <p>Our findings of increases in completion time from using single-item screens replicate results from Couper et al. ([<reflink idref="bib11" id="ref174">11</reflink>]), Toepoel et al. ([<reflink idref="bib49" id="ref175">49</reflink>]), and Tourangeau et al. ([<reflink idref="bib50" id="ref176">50</reflink>]). The differences we observed between single- and multi-item administration were substantial (<emph>d</emph> values ranging from 0.74 to 1.06) and nearly as strong as those for answer sheets. Such drawbacks to single-item administration would seem to undermine potential benefits in efficiency gained by adaptive administration of questionnaire items unless such adaptation could occur simultaneously with items from differing subscales appearing on the same multi-item screens.</p> <p>Lastly, and also in line with previous research (Clough, [<reflink idref="bib10" id="ref177">10</reflink>]; Vispoel, [<reflink idref="bib53" id="ref178">53</reflink>]; Vispoel et al., [<reflink idref="bib54" id="ref179">54</reflink>]; Weigold et al., [<reflink idref="bib65" id="ref180">65</reflink>]), we found that paper booklets without answer sheets required less time to complete than parallel computer questionnaires when multiple items were presented in both modes. The magnitude of standardized mean differences here (0.46 for SDQ-III and 0.58 for SPPCS) are similar to those for corresponding conditions in the Vispoel ([<reflink idref="bib53" id="ref181">53</reflink>]) study of the SDQ-III (0.44) and Clough ([<reflink idref="bib10" id="ref182">10</reflink>]) study of the BIDR (0.33 to 0.52) in which paper booklets on average contained more items per page than computers did per screen. However, because formatting and numbers of items per page/screen were controlled in the present study, display characteristics per se do not appear to be a compelling reason why paper questionnaires took less time.</p> <p>One reason why direct paper booklet completion time was shorter than parallel computer administration may be that respondents were more familiar with reading and completing paper questionnaires (Weigold, Weigold, Drakeford, Dykema, & Smith, [<reflink idref="bib64" id="ref183">64</reflink>]). This explanation is certainly credible for older individuals who may be less familiar with computers but seems unlikely for the computer savvy college students sampled here. Moreover, differences in average-item-completion times remained statistically significant even after controlling for age.</p> <p>An alternative, and perhaps better, explanation for greater response time on computer may be differences in physical aspects of the assessments. In addition to familiarity, Muter ([<reflink idref="bib31" id="ref184">31</reflink>]) lists 28 possible differences between book and computer reading that can lead to slower reading rates for computer. We controlled most of these in this study, including character shape, intercharacter space, size of characters, characters per line, interline spacing, black text and white background across all conditions, equal lines per page, equal words per page in the multiple items per page/screen conditions, and so forth. Resolution of the computer screens also was excellent here, and system response time to answer input was nearly instantaneous.</p> <p>However, among the differences between reading text from computers versus books listed by Muter ([<reflink idref="bib31" id="ref185">31</reflink>]) that we did not control or manipulate were the angle of reading material and posture of the reader. With direct paper booklet administration in the studies reported here, respondents looked downward and circled responses directly after the item stem. In contrast, with computer administration, respondents sat up and faced forward using a mouse to navigate around response alternatives. If such physical differences in orientation and inputting responses do indeed account for the observed differences in completion time across modes, we would expect them to disappear or lessen for computer administration using tablets with touch screens or response sensitive pens that share the same physical orientation as paper booklets. Furthermore, any added efficiency of direct paper booklet responding would need to compensate for disadvantages of booklets not being reusable, responses needing to be keypunched or hand scored, and respondents' generally preferring computer administration and perceiving paper administration as being more fatiguing even when taking less time (Clough, [<reflink idref="bib10" id="ref186">10</reflink>]; Vispoel, [<reflink idref="bib53" id="ref187">53</reflink>]; Vispoel et al., [<reflink idref="bib54" id="ref188">54</reflink>]).</p> <hd id="AN0136822484-25">Item types</hd> <p>Our last set of noteworthy findings was with regard to completion time differences for multiple-alternative versus Likert-style rating-scale items. Overall, the rank ordering of questionnaire conditions did not vary for item types, with paper booklets without answer sheets having the shortest completion time followed, respectively, by computers with multi-item screens, computers with single-item screens, and paper booklets with answer sheets. In addition, the relative pattern of effects of answer sheets, single-item displays, and computerization did not vary across item types. Instead, mean completion time per item was consistently longer for multiple-alternative items than for rating-scale items across all research conditions. Moreover, this pattern prevailed even though the multiple-alternative items had four rather than eight response options from which to choose. These differences likely resulted from the extra time needed to read and consider different answer options for each multiple-alternative item as compared to using the same reoccurring and easily remembered response choices for all rating scale items. Had number of response choices been equal across multiple-alternative and rating-scale items, we would expect even greater relative differences in completion time between the item types.</p> <hd id="AN0136822484-26">Implications and limitations</hd> <p>The primary implication of the findings reported here is that common formats for questionnaire administration can have large and consistent effects on completion time over different types of items without meaningfully affecting the psychometric properties of scores. Although paper questionnaires with answers marked directly in a booklet took the least time to complete on average, such questionnaires usually are perceived less favorably than are computer questionnaires and have practical shortcomings regarding scoring and booklet reuse (Clough, [<reflink idref="bib10" id="ref189">10</reflink>]; Simms & Clark, [<reflink idref="bib45" id="ref190">45</reflink>]; Vispoel, [<reflink idref="bib53" id="ref191">53</reflink>]; Vispoel et al., [<reflink idref="bib54" id="ref192">54</reflink>], Vispoel, Morris, & Clough, [<reflink idref="bib58" id="ref193">58</reflink>]). As a result, unless adaptive administration of single items per screen is necessary or item content is very homogeneous, we recommend using computer questionnaires with multiple items per screen as the most viable option in balancing efficiency, practicality, and respondent preferences in situations where computer administration is cost effective. We further recommend that content for questionnaires with multiple subscales be spiraled or randomly sequenced rather than grouped together for each subscale to guard against spurious correlations among item scores. Given the congruence of the present results for both psychometric properties of scores and completion time with those from studies of other constructs (e.g., personality, socially desirable responding, self-efficacy, attitudes), we would anticipate that the results would generalize to most self-report questionnaires with the same response formats administered in similar settings.</p> <p>As a final caveat, we remind readers that the present findings are limited to college students from a large Midwestern university who completed self-concept inventories within a controlled laboratory setting. We therefore encourage future extensions of this research to include questionnaires measuring other constructs; individuals sampled from younger, older, and clinical populations; and expansion of research conditions to include computer questionnaire tablets, personal computers owned and used regularly by the respondents themselves, and paper questionnaires with items printed directly on the answer sheets. We further encourage research into score comparability and completion time for the growing uses of cell phones in responding to questionnaires within both laboratory and naturalistic settings.</p> <ref id="AN0136822484-27"> <title> References </title> <blist> <bibl id="bib1" idref="ref26" type="bt">1</bibl> <bibtext> American Psychological Association. (1986). Guidelines for computer-based tests and interpretations. Washington, DC : Author.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref27" type="bt">2</bibl> <bibtext> AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, DC : Author.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref57" type="bt">3</bibl> <bibtext> Assessment Systems Corporation. (1996). User's manual for the MicroCAT testing system (3rd ed.). St. Paul, MN : Assessment Systems.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref17" type="bt">4</bibl> <bibtext> Booth-Kewley, S., Larson, G. E., & Miyoshi, D. K. (2007). Social desirability effects on computerized and paper-and-pencil questionnaires. Computers in Human Behavior, 23, 463 – 477. doi: 10.1016/j.chb.2004.10.020</bibtext> </blist> <blist> <bibl id="bib5" idref="ref1" type="bt">5</bibl> <bibtext> Bunderson, C. V., Inouye, D. K., & Olsen, J. B. (1989). The four generations of computerized educational measurement. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 367 – 408). New York, NY : Macmillan.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref2" type="bt">6</bibl> <bibtext> Butcher, J. N. (1987). The use of computers in psychological assessment: An overview of practices and issues. In J. N. Butcher (Ed.), Computerized psychological assessment: A practitioner's guide (pp. 292 – 324). New York, NY : Basic Books.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref20" type="bt">7</bibl> <bibtext> Byrne, B. M. (1996). Measuring self-concept across the life span: Methodological issues and selected instrumentation. Washington, DC : APA Books.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref96" type="bt">8</bibl> <bibtext> Cervellione, K. L., Lee, Y. S., & Bonanno, G. A. (2009). Rasch modeling of the Self-Deception Scale of the Balanced Inventory of Desirable Responding. Educational and Psychological Measurement, 69 (3), 438 – 458. doi: 10.1177/0013164408322020</bibtext> </blist> <blist> <bibl id="bib9" idref="ref56" type="bt">9</bibl> <bibtext> Clark, L. A. (1993). Schedule for Nonadaptive and Adaptive Personality (SNAP). Manual for administration, scoring, and interpretation. Minneapolis : University of Minnesota Press.</bibtext> </blist> <blist> <bibtext> Clough, S. J. (2008). Computerized versus paper-and-pencil assessment of socially desirable responding: Score congruence, completion time, and respondent preferences (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses (UMI No. 3340215).</bibtext> </blist> <blist> <bibtext> Couper, M. P., Traugott, M. W., & Lamias, M. J. (2001). Web survey design and administration. Public Opinion Quarterly, 65, 230 – 253. doi: 10.1086/322199</bibtext> </blist> <blist> <bibtext> De Vries, R. E., Zettler, I., & Hilbig, B. E. (2014). Rethinking trait conceptions of social desirability scales: Impression management as an expression of honesty-humility. Assessment, 21, 286 – 299. doi: 10.1177/1073191113504619</bibtext> </blist> <blist> <bibtext> Diedenhofen, B. (2016). Cocron: Statistical comparisons of two or more alpha coefficients (Version 1.0-1). Available from <ulink href="http://comparingcronbachalphas.org">http://comparingcronbachalphas.org</ulink></bibtext> </blist> <blist> <bibtext> Diedenhofen, B., & Musch, J. (2015). Cocor: A comprehensive solution for the statistical comparison of correlations. PLoS ONE, 10 (4), e0121945. doi: 10.1371/journal.pone.0121945</bibtext> </blist> <blist> <bibtext> Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online, and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487 – 495. doi: 10.1016/j.chb.2014.04.005</bibtext> </blist> <blist> <bibtext> Drasgow, F., Leucht, R. M., & Bennett, R. E. (2006). Technology and testing. In R. L. Brennan (Ed.) Educational measurement (4th ed., pp. 471 – 516). Westport, CT : Praeger.</bibtext> </blist> <blist> <bibtext> Drasgow, F., & Olson-Buchanan, J. (Eds.). (1999). Innovations in computerized assessment. Mahwah, NJ : Erlbaum.</bibtext> </blist> <blist> <bibtext> Dwight, S. A., & Feigelson, M. E. (2000). A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and Psychological Measurement, 60, 340 – 360. doi: 10.1177/00131640021970583</bibtext> </blist> <blist> <bibtext> Edwards, A. L. (1957). The social desirability variable in personality assessment and research. New York, NY : Dryden Press.</bibtext> </blist> <blist> <bibtext> Feldt, L. S., Woodruff, D. J., & Salih, F. A. (1987). Statistical inference for coefficient alpha. Applied Psychological Measurement, 11, 93 – 103. doi: 10.1177/014662168701100107</bibtext> </blist> <blist> <bibtext> Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84 – 96. doi: 10.1016/j.jrp.2005.08.007</bibtext> </blist> <blist> <bibtext> Huang, C. (2013). Relation between self-esteem and socially desirable responding and the role of socially desirable responding in the relation between self-esteem and performance. European Journal of Psychology of Education, 28, 663 – 683. doi: 10.1007/s10212-012-0134-5</bibtext> </blist> <blist> <bibtext> International Test Commission. (2006). International guidelines on computer-based and internet-delivered testing. International Journal of Testing, 6, 143 – 171. doi: 10.1207/s15327574ijt0602_4.</bibtext> </blist> <blist> <bibtext> Marsh, H. W. (1992). Self-Description Questionnaire (SDQ) III: A theoretical and empirical basis for the measurement of multiple dimensions of late adolescent self-concept. An interim test manual and research monograph. Macarthur, Australia : University of Western Sydney.</bibtext> </blist> <blist> <bibtext> Mazzeo, J., & Harvey, A. L. (1988). The equivalence of scores from automated and conventional educational and psychological tests: A review of the literature (College Board Report No. 88-8). Princeton, NJ : Educational Testing Service.</bibtext> </blist> <blist> <bibtext> McClarty, K. L. (2006). A feasibility study of a computerized adaptive test of the international personality item pool NEO (Doctoral dissertation). University of Texas at Austin, Austin, Texas.</bibtext> </blist> <blist> <bibtext> Mehrabian, A., & Russell, J. (1974). An approach to environmental psychology. Cambridge, MA : MIT Press.</bibtext> </blist> <blist> <bibtext> Mills, C. N., Potenza, M. T., Fremer, J. J., & Ward, W. C. (2002). Computer-based testing: Building the foundation for future assessments. Mahwah, NJ : Erlbaum.</bibtext> </blist> <blist> <bibtext> Moon, Y. (1998). Impression management in computer-based interviews: The effects of input modality, output modality, and distance. Public Opinion Quarterly, 62, 610 – 622. doi: 10.1086/297862</bibtext> </blist> <blist> <bibtext> Murphy, C. A., Coover, D., & Owen, S. V. (1989). Development and validation of the Computer Self-Efficacy Scale. Educational and Psychological Measurement, 49, 893 – 899. doi: 10.1177/001316448904900412</bibtext> </blist> <blist> <bibtext> Muter, P. (1996). Interface design and optimization of reading of continuous text. In H. van Oostendorp & S. de Mul (Eds.), Cognitive aspects of electronic text processing (pp. 161 – 180). Norwood, NJ : Ablex.</bibtext> </blist> <blist> <bibtext> Neemann, J., & Harter, S. (1986). Manual for the Self-Perception Profile for College Students. Denver, CO : University of Denver.</bibtext> </blist> <blist> <bibtext> Neemann, J., & Harter, S. (2012). Self-Perception Profile for College Students: Manual and questionnaires (Revised ed.). Denver, CO : University of Denver.</bibtext> </blist> <blist> <bibtext> Parshall, C. G., Spray, J. A., Kalohn, J. C., & Davey, T. (2002). Practical considerations in computer-based testing. New York, NY : Springer-Verlag.</bibtext> </blist> <blist> <bibtext> Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17 – 59). San Diego, CA : Academic.</bibtext> </blist> <blist> <bibtext> Paulhus, D. L. (1999). Paulhus Deception Scales (PDS): The Balanced Inventory of Desirable Responding-7, User's manual. Toronto, Canada : Multi-Health Systems.</bibtext> </blist> <blist> <bibtext> Paulhus, D. L., & Trapnell, P. D. (2008). Self-presentation: An agency-communion framework. In O. P. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality psychology: Theory and research (3rd ed., pp. 492 – 517). New York, NY : Guilford Press.</bibtext> </blist> <blist> <bibtext> Reynolds, W. M. (1982). Development of reliable and valid short forms of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology, 38, 119 – 125. doi: 10.1002/1097-4679(198201)38:1<119::AID-JCLP2270380118>3.0.CO;2-I</bibtext> </blist> <blist> <bibtext> Richman, W. L., Kiesler, S., Weisband, S., & Drasgow, F. (1999). A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology, 84, 754 – 775. doi: 10.1037/0021-9010.84.5.754</bibtext> </blist> <blist> <bibtext> Roper, B. L., Ben-Porath, Y. S., & Butcher, J. N. (1995). Comparability and validity of computerized adaptive testing with the MMPI-2. Journal of Personality Assessment, 65, 358 – 371. doi: 10.1207/s15327752jpa6502_10</bibtext> </blist> <blist> <bibtext> Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ : Princeton University Press.</bibtext> </blist> <blist> <bibtext> Rosenberg, M. (1979). Conceiving the self. New York, NY : Basic Books.</bibtext> </blist> <blist> <bibtext> Rosenfeld, P., Booth-Kewley, S., Edwards, J. E., & Thomas, M. D. (1996). Responses on computer surveys: Impression management, social desirability, and the Big Brother Syndrome. Computers in Human Behavior, 12, 263 – 274. doi: 10.1016/0747-5632(96)00006-4</bibtext> </blist> <blist> <bibtext> Scullard, M. G. (2007). Application of item response theory based computerized adaptive testing to the Strong Interest Inventory. (Unpublished doctoral dissertation). University of Minnesota, Minneapolis, Minnesota.</bibtext> </blist> <blist> <bibtext> Simms, L. J., & Clark, L. A. (2005). Validation of a computerized adaptive version of the Schedule for Nonadaptive and Adaptive Personality (SNAP). Psychological Assessment, 17, 28 – 43. doi: 10.1037/1040-3590.17.1.28</bibtext> </blist> <blist> <bibtext> Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245 – 251. doi: 10.1037//0033-2909.87.2.245</bibtext> </blist> <blist> <bibtext> Stöber, J., Dette, D. E., & Musch, J. (2002). Comparing continuous and dichotomous scoring of the Balanced Inventory of Desirable Responding. Journal of Personality Assessment, 78, 370 – 389. doi: 10.1207/S15327752JPA7802_10</bibtext> </blist> <blist> <bibtext> Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights, MA : Allyn and Bacon.</bibtext> </blist> <blist> <bibtext> Toepoel, V., Das, M., & Van Soest, A. (2009). Design of web questionnaires: The effects of the number of items per screen. Field Methods, 21, 200 – 213. doi: 10.1177/1525822X08330261</bibtext> </blist> <blist> <bibtext> Tourangeau, R., Couper, M. P., & Conrad, F. (2004). Spacing, position, and order interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368 – 393. doi: 10.1093/poq/nfh035</bibtext> </blist> <blist> <bibtext> Urbina, S. (2014). Essentials of psychological testing (2nd ed.). Hoboken, NJ : John Wiley.</bibtext> </blist> <blist> <bibtext> Vispoel, W. P. (1995). Self-concept in artistic domains: An extension of the Shavelson, Hubner, and Stanton (1976) model. Journal of Educational Psychology, 85, 134 – 153. doi.org/ 10.1037/0022-0663.87.1.134</bibtext> </blist> <blist> <bibtext> Vispoel, W. P. (2000). Computerized versus paper-and-pencil assessment of self-concept: Score comparability and respondent preferences. Measurement and Evaluation in Counseling and Development, 33, 130 – 143. Retrieved from EBSCOhost database.</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., Boo, J., & Bleiler, T. (2001). Computerized and paper-and-pencil versions of the Rosenberg Self-Esteem Scale: A comparison of psychometric features and respondent preferences. Educational and Psychological Measurement, 61, 461 – 474. doi: 10.1177/00131640121971329</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., & Forte Fast, E. E. (2000). Response biases and their relation to sex differences in multiple domains of self-concept. Applied Measurement in Education, 13, 79 – 97. doi: 10.1207/s15324818ame1301_4</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., & He, Y. (2010, April). Evaluating the multifaceted, hierarchical structure of self-concept using the Self-Description Questionnaire-III (SDQ-III), Self-Perception Profile for College Students (SPPCS) and Arts Self-Perception Inventory (ASPI). Paper presented at the annual meeting of the American Educational Research Association, Denver, Colorado.</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., & Kim, H. Y. (2014). Psychometric properties for the Balanced Inventory of Desirable Responding: Dichotomous versus polytomous conventional and IRT scoring. Psychological Assessment, 26, 878 – 891. doi: 10.1037/a0036430</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., Morris, C. A., & Clough, S. J. (2018). Interchangeability of results from computerized and traditional administration of the BIDR: Convenience can match reality. Journal of Personality Assessment, 1 – 16. doi: 10.1080/00223891.2017.1406361 [ Epub ahead of print ].</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018a). Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychological Methods, 23, 1 – 26. doi: 10.1037/met0000107.</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018b). Practical applications of generalizability theory for designing, evaluating, and improving psychological assessments. Journal of Personality Assessment, 100, 53 – 67. doi: 10.1080/00223891.2017.1296455</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., Morris, C. A., & Kilinc, M. (in press). Using generalizability theory to disattenuate correlation coefficients for multiple sources of measurement error. Multivariate Behavioral Research.</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018c). Using G-theory to enhance evidence of reliability and validity for common uses of the Paulhus Deception Scales. Assessment, 25, 69 – 83. doi: 10.1177/1073191116641182</bibtext> </blist> <blist> <bibtext> Vispoel, W. P., & Tao, S. (2013). A generalizability analysis of score consistency for the Balanced Inventory of Desirable Responding. Psychological Assessment, 25, 94 – 104. doi: 10.1037/a0029061</bibtext> </blist> <blist> <bibtext> Weigold, A., Weigold, I. K., Drakeford, N. M., Dykema, S. A., & Smith, C. A. (2016). Equivalence of paper-and-pencil and computerized self-report surveys in older adults. Computers in Human Behavior, 54, 407 – 413. doi: 10.1016/j.chb.2015.08.033</bibtext> </blist> <blist> <bibtext> Weigold, A., Weigold, I. K., & Russell, E. J. (2013). Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection methods. Psychological Methods, 18, 53 – 70. doi: 10.1037/a0031607</bibtext> </blist> <blist> <bibtext> Weijters, B., De Beuckelaer, A., & Baumgartner, H. (2014). Discriminant validity where there should be none: Positioning same-scale items in separated blocks of a questionnaire. Applied Psychological Measurement, 38, 450 – 463. doi: 10.1177/0146621614531850</bibtext> </blist> <blist> <bibtext> Wilkerson, J. M., Nagao, D. H., & Martin, C. L. (2002). Socially desirable responding in computerized questionnaires: When questionnaire purpose matters more than the mode. Journal of Applied Social Psychology, 32, 544 – 559. doi: 10.1111/j.1559-1816.2002.tb00229.x</bibtext> </blist> </ref> <aug> <p>By Walter Peter Vispoel; Carrie Ann Morris and Linan Sun</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib16" firstref="ref3"></nolink> <nolink nlid="nl2" bibid="bib17" firstref="ref4"></nolink> <nolink nlid="nl3" bibid="bib28" firstref="ref5"></nolink> <nolink nlid="nl4" bibid="bib34" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib29" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib39" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib43" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib67" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib10" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib45" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib53" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib54" firstref="ref14"></nolink> <nolink nlid="nl13" bibid="bib58" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib65" firstref="ref16"></nolink> <nolink nlid="nl15" bibid="bib25" firstref="ref21"></nolink> <nolink nlid="nl16" bibid="bib15" firstref="ref23"></nolink> <nolink nlid="nl17" bibid="bib18" firstref="ref24"></nolink> <nolink nlid="nl18" bibid="bib23" firstref="ref28"></nolink> <nolink nlid="nl19" bibid="bib24" firstref="ref41"></nolink> <nolink nlid="nl20" bibid="bib41" firstref="ref43"></nolink> <nolink nlid="nl21" bibid="bib42" firstref="ref44"></nolink> <nolink nlid="nl22" bibid="bib35" firstref="ref46"></nolink> <nolink nlid="nl23" bibid="bib21" firstref="ref50"></nolink> <nolink nlid="nl24" bibid="bib38" firstref="ref51"></nolink> <nolink nlid="nl25" bibid="bib30" firstref="ref52"></nolink> <nolink nlid="nl26" bibid="bib49" firstref="ref61"></nolink> <nolink nlid="nl27" bibid="bib27" firstref="ref62"></nolink> <nolink nlid="nl28" bibid="bib11" firstref="ref64"></nolink> <nolink nlid="nl29" bibid="bib50" firstref="ref65"></nolink> <nolink nlid="nl30" bibid="bib66" firstref="ref71"></nolink> <nolink nlid="nl31" bibid="bib19" firstref="ref74"></nolink> <nolink nlid="nl32" bibid="bib51" firstref="ref75"></nolink> <nolink nlid="nl33" bibid="bib52" firstref="ref84"></nolink> <nolink nlid="nl34" bibid="bib32" firstref="ref85"></nolink> <nolink nlid="nl35" bibid="bib33" firstref="ref86"></nolink> <nolink nlid="nl36" bibid="bib56" firstref="ref90"></nolink> <nolink nlid="nl37" bibid="bib36" firstref="ref95"></nolink> <nolink nlid="nl38" bibid="bib47" firstref="ref97"></nolink> <nolink nlid="nl39" bibid="bib57" firstref="ref98"></nolink> <nolink nlid="nl40" bibid="bib63" firstref="ref99"></nolink> <nolink nlid="nl41" bibid="bib60" firstref="ref101"></nolink> <nolink nlid="nl42" bibid="bib62" firstref="ref103"></nolink> <nolink nlid="nl43" bibid="bib61" firstref="ref104"></nolink> <nolink nlid="nl44" bibid="bib12" firstref="ref105"></nolink> <nolink nlid="nl45" bibid="bib22" firstref="ref106"></nolink> <nolink nlid="nl46" bibid="bib37" firstref="ref109"></nolink> <nolink nlid="nl47" bibid="bib55" firstref="ref111"></nolink> <nolink nlid="nl48" bibid="bib26" firstref="ref118"></nolink> <nolink nlid="nl49" bibid="bib40" firstref="ref119"></nolink> <nolink nlid="nl50" bibid="bib44" firstref="ref120"></nolink> <nolink nlid="nl51" bibid="bib48" firstref="ref122"></nolink> <nolink nlid="nl52" bibid="bib20" firstref="ref123"></nolink> <nolink nlid="nl53" bibid="bib13" firstref="ref124"></nolink> <nolink nlid="nl54" bibid="bib479" firstref="ref142"></nolink> <nolink nlid="nl55" bibid="bib78" firstref="ref148"></nolink> <nolink nlid="nl56" bibid="bib14" firstref="ref151"></nolink> <nolink nlid="nl57" bibid="bib46" firstref="ref152"></nolink> <nolink nlid="nl58" bibid="bib220" firstref="ref161"></nolink> <nolink nlid="nl59" bibid="bib253" firstref="ref163"></nolink> <nolink nlid="nl60" bibid="bib64" firstref="ref183"></nolink> <nolink nlid="nl61" bibid="bib31" firstref="ref184"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1218243
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Computerized and Traditional Administration of Questionnaires: Psychometric Quality and Completion Time for Measures of Self-Concept
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Vispoel%2C+Walter+Peter%22">Vispoel, Walter Peter</searchLink><br /><searchLink fieldCode="AR" term="%22Morris%2C+Carrie+Ann%22">Morris, Carrie Ann</searchLink><br /><searchLink fieldCode="AR" term="%22Sun%2C+Linan%22">Sun, Linan</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Experimental+Education%22"><i>Journal of Experimental Education</i></searchLink>. 2019 87(3):384-399.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Self+Concept+Measures%22">Self Concept Measures</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Questionnaires%22">Questionnaires</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Answer+Sheets%22">Answer Sheets</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Length%22">Test Length</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22Self+Description+Questionnaire%22">Self Description Questionnaire</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/00220973.2018.1448748
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0022-0973
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In two independent studies of questionnaire administration, respondents completed multidimensional self-concept inventories within four randomized research conditions that mirrored the most common administration formats used in practice: paper booklets with and without answer sheets and computer questionnaires with single versus multiple items per screen. Strong differences among conditions emerged for completion time, but not for psychometric properties of scores (means, variances, reliability coefficients, concurrent validity coefficients). Answer sheets increased completion time by 24% to 34%, single-item displays by 20% to 25%, and computerization by 13% to 17%. Completion time was longer for multiple-alternative than for rating-scale items, but relative effects of answer sheets, single-item displays, and computerization remained consistent. We discuss implications for questionnaire construction and administration.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2019
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1218243
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1218243
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/00220973.2018.1448748
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 384
    Subjects:
      – SubjectFull: Self Concept Measures
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Questionnaires
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Answer Sheets
        Type: general
      – SubjectFull: Test Length
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Self Description Questionnaire
        Type: general
    Titles:
      – TitleFull: Computerized and Traditional Administration of Questionnaires: Psychometric Quality and Completion Time for Measures of Self-Concept
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Vispoel, Walter Peter
      – PersonEntity:
          Name:
            NameFull: Morris, Carrie Ann
      – PersonEntity:
          Name:
            NameFull: Sun, Linan
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2019
          Identifiers:
            – Type: issn-print
              Value: 0022-0973
          Numbering:
            – Type: volume
              Value: 87
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Journal of Experimental Education
              Type: main
ResultId 1