View in EDS HTML Full Text PDF Full Text

Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5)

Saved in:
Bibliographic Details
Title: Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5)
Language: English
Authors: Eirini M. Mitropoulou (ORCID 0000-0003-4786-9128), Leonidas A. Zampetakis, Ioannis Tsaousis
Source: Evaluation Review. 2024 48(6):1146-1159.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 14
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Foreign Countries, Adults, Item Response Theory, Personality Traits, Evaluation Methods, Personality Measures, Personality Assessment, Response Style (Tests), Test Interpretation, Test Reliability, Measurement Techniques, Test Validity, Test Selection
Geographic Terms: Greece
DOI: 10.1177/0193841X231223374
ISSN: 0193-841X
1552-3926
Abstract: Unfolding item response theory (IRT) models are important alternatives to dominance IRT models in describing the response processes on self-report tests. Their usage is common in personality measures, since they indicate potential differentiations in test score interpretation. This paper aims to gain a better insight into the structure of trait personality, by investigating whether the dominance or alternatively the unfolding IRT model are better descriptors of the response processes on a personality measure constructed under the dominance response theorem. For the assessment of the dominant model, the Graded Response Model (GRM) is used; while for the unfolding model, the Generalized Graded Unfolding Model (GGUM) was examined. All analyses are conducted with the freely available R. A sample of 1340 Greek adults, employed in private and public organizations, fulfilled the Trait Personality Questionnaire 5 short-form (TPQue5). Findings contradict previous research on trait personality. In accordance to the construction method employed, the TPQue5 items are best understood by monotonically increasing item response functions (IRFs). Individuals responding to the TPQue5 increase their probability of endorsing its items as their trait level increases; this stands for all personality dimensions, although Openness to Experience exhibited mixed type of item response patterns. Further research directions, implications and limitations are also discussed.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1441315
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHdqIua8XKSkZbgTKDV4CdKAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBncKqKBnqMscSgBTQIBEICBms_NhUAoAQTh2kvo7oX4Wk71vAgA4omRlh_16lXIfNUdmuSzhT0bXG-Jrb408eQ0QQ2snE7Jyr4WjE8gPS6j_FfmqQOw8B2jbEaZNcTV7UJNhCcyMIglE1edWLfu6klDD3mSSkjehbycHqRvnjf833vL0wbICshU0GuXxdYLqA2k0Fc5DIHq_ykR5eDCfUrshn52PKKhwMV7zL0=
Text:
  Availability: 1
  Value: <anid>AN0179974325;evr01dec.24;2024Oct01.06:01;v2.2.500</anid> <title id="AN0179974325-1">Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5) </title> <p>Unfolding item response theory (IRT) models are important alternatives to dominance IRT models in describing the response processes on self-report tests. Their usage is common in personality measures, since they indicate potential differentiations in test score interpretation. This paper aims to gain a better insight into the structure of trait personality, by investigating whether the dominance or alternatively the unfolding IRT model are better descriptors of the response processes on a personality measure constructed under the dominance response theorem. For the assessment of the dominant model, the Graded Response Model (GRM) is used; while for the unfolding model, the Generalized Graded Unfolding Model (GGUM) was examined. All analyses are conducted with the freely available R. A sample of 1340 Greek adults, employed in private and public organizations, fulfilled the Trait Personality Questionnaire 5 short-form (TPQue5). Findings contradict previous research on trait personality. In accordance to the construction method employed, the TPQue5 items are best understood by monotonically increasing item response functions (IRFs). Individuals responding to the TPQue5 increase their probability of endorsing its items as their trait level increases; this stands for all personality dimensions, although Openness to Experience exhibited mixed type of item response patterns. Further research directions, implications and limitations are also discussed.</p> <p>Keywords: methodological development content area; measurement methodology (if appropriate); behavioral health care and policy content area; survey methodology methodology (if appropriate)</p> <hd id="AN0179974325-2">Introduction</hd> <p>Measurement is often conducted using self-report methods, with item analysis being fundamental for advancing such tests. Although self-report test construction relies mainly upon factor-analysis and internal consistency reliability maximization (by selecting items with high item-test correlations), Item Response Theory (IRT) models have also been increasingly used to enhance self-report assessments (such as that concerning personality). In IRT, the probability of endorsing an item (<emph>i</emph>) to a person's latent trait level (<emph>θ</emph>) is determined by the item's response function (IRF) ([<reflink idref="bib5" id="ref1">5</reflink>]). Most studies using an IRT approach in the personality domain (e.g., [<reflink idref="bib9" id="ref2">9</reflink>]) use models that assume a dominance process underlies an individual's response to an item. This dominance approach pertains to the fact that the higher the individual's score on the latent trait, the higher the probability of endorsing that item.</p> <p>Despite focusing on the dominance process in responding to personality traits, several studies have found that the probability of endorsing an item may decrease at the higher end of a trait continuum (e.g., [<reflink idref="bib3" id="ref3">3</reflink>]). To elucidate upon the controversies regarding response processes, [<reflink idref="bib16" id="ref4">16</reflink>] proposed an unfolding IRT model. The basic assumption of the unfolding model is that the individuals and items are both located on a trait continuum. However, the individual endorses an item if, and only if, they are located near the item on the latent continuum. This model is best described by a non-monotonic, single-peaked IRF which represents the ideal response point of individuals and which is considered to have important consequences in the selection context (e.g., [<reflink idref="bib16" id="ref5">16</reflink>]).</p> <p>Recent research has explored the fit of unfolding models in personality trait measures given that misfits have been identified in the response process of certain personality traits. These responses were not adequately described under a dominance IRT model despite being constructed as such (e.g., [<reflink idref="bib8" id="ref6">8</reflink>]; [<reflink idref="bib18" id="ref7">18</reflink>]).These response misfits led to misspecifications in the ordering of the respondents and seriously affected measurement precision (e.g., [<reflink idref="bib10" id="ref8">10</reflink>]). Exploration of the fit of unfolding models to personality traits is considered important to explicate response misspecifications and to further enhance decision-making. Therefore, this study aimed to (<reflink idref="bib1" id="ref9">1</reflink>) evaluate the model fit of a Greek self-report personality measure under the dominance and unfolding IRT models and (<reflink idref="bib2" id="ref10">2</reflink>) to scrutinize the measure's items to identify potential unfolding patterns of responses. Such knowledge will contribute to how personality assessment is conducted and further validate the usability of the measure in clinical and organizational settings.</p> <hd id="AN0179974325-3">Dominance IRT Models</hd> <p>Dominance IRT models pertain to an increase in the probability of endorsing an item as an individual's trait level increases ([<reflink idref="bib5" id="ref11">5</reflink>]). Correspondingly, the dominant IRFs exhibit monotonically increasing functions (Figure 1-(a)). Therefore, the amount of information an item provides with respect to the individuals' <emph>θ</emph> depends on the model used. Some dominance IRT models determine item properties by estimating discrimination and difficulty parameters (i.e., 2-polytomous models). The discrimination parameter (<emph>α</emph>) expresses the slope of the items' characteristic curve (ICC), which has an S-shape curve and reflects the degree to which the individual's response choices fluctuate among different items as the level of the individual's <emph>θ</emph> changes. The difficulty parameter (<emph>β</emph><subs><emph>i</emph></subs>) denotes the location of the ICC on the θ-scale axis and determines if an individual's <emph>θ</emph> is above or below an item. One of the most widely used dominance IRT models is the Graded Response Model (GRM; [<reflink idref="bib15" id="ref12">15</reflink>]). The GRM is one of the first developed models for graded polytomous items, is appropriate for items with different <emph>α</emph> parameters, and is considered a natural assessment model for rating scales compared to other IRT models ([<reflink idref="bib6" id="ref13">6</reflink>]).</p> <p>Graph: Figure 1.The typically expected item scores for Likert-type items under the dominance (a) and the unfolding (b) model.</p> <hd id="AN0179974325-4">Unfolding IRT Models</hd> <p>In unfolding IRT, individuals endorse items only if the items' location on the trait continuum (always denoted as the ideal point) is close to each other ([<reflink idref="bib4" id="ref14">4</reflink>]; [<reflink idref="bib14" id="ref15">14</reflink>]). Correspondingly, the unfolding IRFs increase non-monotonically and exhibit a bell-shaped curve (Figure1-(b)). According to an unfolding IRF, if the ideal points between an individual and an item are not close, the probability of an individual disagreeing with the corresponding item is high. As in dominance IRT models, unfolding models estimate item properties with the discrimination and difficulty parameters (also denoted as <emph>α</emph> and <emph>β,</emph> respectively).</p> <p>One of the most widely used unfolding models is the generalized graded unfolding model (GGUM; [<reflink idref="bib14" id="ref16">14</reflink>]). This model estimates the probability of selecting an objective response option as the sum of the probabilities of the subjective responses to the same option. Therefore, the GGUM estimates three different parameters: the item discrimination parameter (<emph>α</emph>), the item difficulty parameter (<emph>β</emph>), and the response option thresholds (denoted as <emph>δ</emph>). The <emph>δ</emph> parameter represents the location of the individual's thresholds or boundaries between the response options relative to item location parameters. Accordingly, the probability of selecting a response option will increase as the distance between an individual's <emph>θ</emph> and the response option thresholds <emph>δ</emph> decreases ([<reflink idref="bib13" id="ref17">13</reflink>]).</p> <hd id="AN0179974325-5">Personality Assessment Using Unfolding IRT Models</hd> <p>Research on personality assessment has stressed the necessity of examining alternative response patterns to amplify the information received by a single personality test. For example, [<reflink idref="bib16" id="ref18">16</reflink>] were among the first to identify IRF misfits for specific items on the 16 Personality Factor Questionnaire (16PFQ); these item responses were not described adequately under the dominance (monotonic) IRT model. According to the authors, the misfits revealed an ideal response point, where personality information was maximized without reaching the maximum item score, thus realizing that personality assessment may result in false conclusions and, consequently, lead to erroneous occupational decisions.</p> <p>Since then, several studies have applied unfolding IRT models to different personality measures to scrutinize their fit and usefulness in personality assessments. For example, [<reflink idref="bib10" id="ref19">10</reflink>] found that the depression subscale of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) is best described from an unfolding IRT perspective. In addition, the International Personality Item Pool (IPIP) measure —which estimates the five-factor model of personality— was found to have a significantly better fit to the unfolding model than to the dominance IRT model ([<reflink idref="bib1" id="ref20">1</reflink>]; [<reflink idref="bib18" id="ref21">18</reflink>]). Moreover, [<reflink idref="bib7" id="ref22">7</reflink>] examined the fit of the mini-marker scale of the Big Five Personality Traits and confirmed that individuals' responses were more consistent under the unfolding process when compared to the dominance model; they found that a substantial proportion of individuals exhibited an ideal point of response to the specific personality measure.</p> <p>As a result, attention should be paid to unfolding IRT models, and evidence of ideal response points should be provided for personality measurement. Research in the fields of scale analysis and scale construction should identify which domains (if any) of personality measurement unfolding models are particularly useful ([<reflink idref="bib18" id="ref23">18</reflink>]). Misspecifications in the item response process may cause serious consequences when ordering individuals according to their latent trait scores, thus seriously affecting organizational decision-making for diagnostic, classification, and/or selection purposes (e.g., [<reflink idref="bib16" id="ref24">16</reflink>]). Therefore, it is imperative to examine the ideal-point response process in personality tests.</p> <hd id="AN0179974325-6">Aims of the Present Study</hd> <p>In the present study, under the unfolding IRT paradigm, we aimed to scrutinize response patterns for an alternative personality measure —namely, the Trait Personality Questionnaire 5 (TPQue5). Personality assessments under the unfolding paradigm are considered important. However, the research has not clarified whether an ideal response point can be generalized to other personality inventories by measuring similar or different personality constructs. To our knowledge, only [<reflink idref="bib18" id="ref25">18</reflink>] have compared the fit of the unfolding IRT model to different personality tests, some of which were developed under the dominant response process and others under the unfolding process. Their mixed results confirm the necessity of further assessing unfolding IRT models for personality testing to improve our knowledge of each construct's assessment capabilities.</p> <p>This research focuses on the TPQue5, a test constructed under the dominance IRT paradigm, which corresponds to a monotonically increasing response process. To the best of our knowledge, TPQue5 has not been tested under the unfolding IRT paradigm. Hence, such an analysis may extend our knowledge of personality assessments and improve the test's estimation accuracy. Therefore, our <emph>first objective</emph> was to provide a test-level analysis and further validate the measure's response estimation method by comparing two IRT models: an unfolding model and a dominance model. As the measure was constructed under the dominant paradigm, we propose that a dominant IRT model will exhibit a better fit than an unfolding model.</p> <p>Our <emph>second objective</emph> was to provide an item-level analysis to identify potential IRF misspecifications using TPQue5 as a venue for our methodological assessment. Item analysis will enhance our understanding of the responses of individuals to the TPQue5. Although the TPQue5 items have not yet been examined under the unfolding IRT paradigm, their utility in decision-making in Greek occupational and educational settings is indisputable. Hence, identifying potential misspecifications in individuals' response processes may improve their personality assessment accuracy and prevent misguided occupational decisions.</p> <hd id="AN0179974325-7">Method</hd> <p></p> <hd id="AN0179974325-8">Instrument</hd> <p>The TPQue5 consists of 75 positively and negatively formulated statements that measure five major dimensions of personality ("Extraversion," "Neuroticism," "Openness to Experience," "Agreeableness," and "Conscientiousness"). The TPQue5 was developed in Greek and is based on Costa and McCrae's theoretical framework (1992). The items were developed based on both rational and factor-analysis techniques. Each of the five scales comprises 15 items, and participants are asked to respond using a Likert-type scale ranging from 1 (completely disagree) to 5 (completely agree). The TPQue5 has shown acceptable reliability and validity (e.g., [<reflink idref="bib17" id="ref26">17</reflink>]). Alpha and split-half reliabilities showed that all TPQue5 scales demonstrate acceptable internal consistency indices, ranging from 0.73 (for "Openness") to 0.87 (for "Neuroticism and Conscientiousness"). The test-retest reliability also supported the temporal stability of the TPQue5 scales (ranging from 0.74 to 0.87). Finally, different validity studies supported the concurrent, convergent, and discriminant validity of the TPQue5 scales by examining their conceptual relevance to alternative personality models, including the Eysenck Personality Questionnaire, the 16 Personality Factor Questionnaire, and the Big Five Inventory.</p> <hd id="AN0179974325-9">Participants and Procedure</hd> <p>The present study employed a sample of 1340 Greek adults with a mean age of 28.3 (<emph>SD</emph> = 9.26, age-range 18–78 years). The majority of the participants were female (<emph>n</emph> = 718; 53.6%) compared to males (<emph>n</emph> = 622; 46,4%). Data were obtained from the TPQue5 normative sample, which was used to develop norms for the working population. Participants in the normative sample came from public and private organizations (e.g., manufacturing, trading, and public services).</p> <hd id="AN0179974325-10">Analyses</hd> <p>Analyses were performed using the free R statistical package (version 4.1.2; [<reflink idref="bib11" id="ref27">11</reflink>]). To assess model fit for the generalized graded unfolding model (GGUM), we used the R package <emph>mirt</emph> ([<reflink idref="bib2" id="ref28">2</reflink>]). Mirt displays item location and discrimination parameters and allows for the differential use of response categories across the subscale's items. To assess the model fit for the dominance graded response model (GRM), we used the R package <emph>ltm</emph> ([<reflink idref="bib12" id="ref29">12</reflink>]), which provides different discrimination and category threshold parameters for all items. To determine which model provides the best fit, namely, the GGUM or GRM, we used each model's log-likelihood statistics and the commonly used comparative fit indices: the Bayesian information criterion (BIC) and Akaike's information criterion (AIC). Value differences greater than 10 on the AIC and BIC indicate model fit differences, with the lowest values indicating a better fit. The syntax for all R analyses is provided in Supplementary Material 1.</p> <hd id="AN0179974325-11">Results</hd> <p>The scale means, standard deviations, skewness and kurtosis, omega reliabilities, and inter-item correlation ranges are provided in Table 1. Inter-item correlations retained similar patterns and omega reliability was acceptable for all five subscales. The unidimensionality of each subscale was also assessed; all TPQue5 items exhibited communalities > 0.30, which is considered acceptable for the conformability of the items to the designated factor.</p> <p>Table 1. Descriptive Statistics for TPQue5 Subscales.</p> <p>Graph</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Scales</th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>ω</italic></th><th align="center">Skewness</th><th align="center">Kurtosis</th><th align="center">Inter-item correlation average<sup>a</sup></th></tr></thead><tbody valign="top"><tr><td align="left">Neuroticism</td><td align="left">43.8</td><td align="left">10.7</td><td align="left">.87</td><td align="center">.192</td><td align="center">−.247</td><td align="char" char="ndash">.05–.64</td></tr><tr><td align="left">Extraversion</td><td align="left">50.9</td><td align="left">9.02</td><td align="left">.83</td><td align="center">−.306</td><td align="center">.177</td><td align="char" char="ndash">.07–.55</td></tr><tr><td align="left">Agreeableness</td><td align="left">52.2</td><td align="left">7.95</td><td align="left">.81</td><td align="center">−.279</td><td align="center">.455</td><td align="char" char="ndash">.05–.52</td></tr><tr><td align="left">Conscientiousness</td><td align="left">50.3</td><td align="left">9.06</td><td align="left">.83</td><td align="center">−.136</td><td align="center">−.057</td><td align="char" char="ndash">.09–.58</td></tr><tr><td align="left">Openness to experience</td><td align="left">48.5</td><td align="left">8.36</td><td align="left">.73</td><td align="center">.061</td><td align="center">−.253</td><td align="char" char="ndash">.01–.51</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note. N</emph> = 1340; <emph>a</emph> = most inter-item correlations were statistically significant at <emph>ρ</emph> <.001 and <emph>ρ</emph> <.01.</p> <p>To explore our first objective, the GGUM and GRM model fit indices were estimated for all the TPQue5 subscales (Table 2). In line with the first research hypothesis, the GGUM did not fit our data better than that of the conventional dominant model for all the subscales. On the other hand, the GRM showed a significantly better fit in four out of five personality traits, as presented by the differences in both criteria used (indicated as <emph>ΔΑΙC</emph> and <emph>ΔBIC</emph> in Table 2). Discrepancies regarding the most appropriate model fit are found only for the "Openness to Experience" subscale; the estimated fit indices seem to disagree on which model fits our data best, because the AIC is smaller for the unfolding model and BIC for the dominant model.</p> <p>Table 2. GGUM and GRM Results for the TPQue5 Subscales.</p> <p>Graph</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left" rowspan="2">Scale</th><th align="center" colspan="3">Generalized graded unfolding model</th><th align="center" colspan="3">Graded response model</th><th align="center" rowspan="2"><italic>ΔΑΙC</italic></th><th align="center" rowspan="2"><italic>ΔBIC</italic></th></tr><tr><th align="left">LL</th><th align="center">AIC</th><th align="center">BIC</th><th align="center">LL</th><th align="center">AIC</th><th align="center">BIC</th></tr></thead><tbody valign="top"><tr><td align="left">Neuroticism</td><td align="char" char=".">−27,642.97</td><td align="char" char=".">55,465.94</td><td align="char" char=".">55,933.98</td><td align="char" char=".">−27,378.33</td><td align="char" char=".">54,906.65</td><td align="char" char=".">55,296.68</td><td align="char" char=".">559.29</td><td align="char" char=".">637.3</td></tr><tr><td align="left">Extraversion</td><td align="char" char=".">−27,071.50</td><td align="char" char=".">54,323.00</td><td align="char" char=".">54,791.04</td><td align="char" char=".">−26,898.68</td><td align="char" char=".">53,947.36</td><td align="char" char=".">54,337.39</td><td align="char" char=".">375.64</td><td align="char" char=".">453.65</td></tr><tr><td align="left">Agreeableness</td><td align="char" char=".">−26,460.18</td><td align="char" char=".">53,100.37</td><td align="char" char=".">53,568.41</td><td align="char" char=".">−26,271.32</td><td align="char" char=".">52,692.64</td><td align="char" char=".">53,082.67</td><td align="char" char=".">407.73</td><td align="char" char=".">485.74</td></tr><tr><td align="left">Conscientiousness</td><td align="char" char=".">−26,687.14</td><td align="char" char=".">53,554.27</td><td align="char" char=".">54,022.31</td><td align="char" char=".">−26,558.35</td><td align="char" char=".">53,266.77</td><td align="char" char=".">53,656.74</td><td align="char" char=".">287.5</td><td align="char" char=".">365.57</td></tr><tr><td align="left">Openness to experience</td><td align="char" char=".">−28,775.42</td><td align="char" char=".">57,730.84</td><td align="char" char=".">58,198.88</td><td align="char" char=".">−28,804.78</td><td align="char" char=".">57,759.57</td><td align="char" char=".">58,149.6</td><td align="char" char=".">−28.73</td><td align="char" char=".">49.28</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note.</emph> LL = Log-likelihood; AIC = Akaike's information criterion; BIC = Bayesian information criterion; Bold cells indicate better fitting model; ΔAIC = Estimated AIC difference between the two models; ΔBIC = Estimated BIC difference between the two models.</p> <p>Plots of the expected item and total scoring functions were also estimated to further validate the superior fit of the GRM. Most of the expected item plots (including the expected total score) were dissimilar to the GGUM bell-shaped curve ([<reflink idref="bib14" id="ref30">14</reflink>]). However, some items exhibit a single-peaked curve or have a trend to single-point at the higher end of the continuum and seem to be best endorsed by "intermediate personality types" of respondents (e.g., Neuroticism subscale, cf. Supplementary Material 2).</p> <p>To explore our second objective and further evaluate the TPQue5's potential item misspecifications, the item location parameters <emph>(δi)</emph> for both subscales were also estimated (Table 3). The discrimination and item difficulty parameters are reported separately in Supplementary Material 2. Observation of items' <emph>δ</emph> index indicates that the item location parameters present no evidence of ordered (or unfolding) clustering of the item location for all personality subscales. Αll TPQue5 items are located on the positive side of the continuum, with values ranging 0.175 <<emph>δ</emph> < 3.131.</p> <p>Table 3. TPQue5 Item Location Parameters (δ) for all Subscales.</p> <p>Graph</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Neuroticism</th><th align="center"><italic>δ</italic></th><th align="center">Extraversion</th><th align="center"><italic>δ</italic></th><th align="center">Agreeableness</th><th align="center"><italic>δ</italic></th><th align="center">Conscientiousness</th><th align="center"><italic>δ</italic></th><th align="center">Openness to experience</th><th align="center"><italic>δ</italic></th></tr></thead><tbody valign="top"><tr><td align="left">Item55</td><td align="char" char=".">.778</td><td align="left">Item11</td><td align="char" char=".">.425</td><td align="left">Item93</td><td align="char" char=".">.541</td><td align="left">Item25</td><td align="char" char=".">.394</td><td align="left">Item38</td><td align="char" char=".">.175</td></tr><tr><td align="left">Item33</td><td align="char" char=".">.823</td><td align="left">Item53</td><td align="char" char=".">.427</td><td align="left">Item8</td><td align="char" char=".">.603</td><td align="left">Item63</td><td align="char" char=".">.401</td><td align="left">Item89</td><td align="char" char=".">.536</td></tr><tr><td align="left">Item85</td><td align="char" char=".">.843</td><td align="left">Item31</td><td align="char" char=".">.508</td><td align="left">Item73</td><td align="char" char=".">.627</td><td align="left">Item99</td><td align="char" char=".">.409</td><td align="left">Item71</td><td align="char" char=".">.551</td></tr><tr><td align="left">Item34</td><td align="char" char=".">.860</td><td align="left">Item82</td><td align="char" char=".">.543</td><td align="left">Item74</td><td align="char" char=".">.636</td><td align="left">Item10</td><td align="char" char=".">.429</td><td align="left">Item70</td><td align="char" char=".">.585</td></tr><tr><td align="left">Item69</td><td align="char" char=".">.936</td><td align="left">Item65</td><td align="char" char=".">.584</td><td align="left">Item95</td><td align="char" char=".">.789</td><td align="left">Item46</td><td align="char" char=".">.430</td><td align="left">Item5</td><td align="char" char=".">.626</td></tr><tr><td align="left">Item35</td><td align="char" char=".">1.083</td><td align="left">Item29</td><td align="char" char=".">.620</td><td align="left">Item23</td><td align="char" char=".">.864</td><td align="left">Item27</td><td align="char" char=".">.487</td><td align="left">Item19</td><td align="char" char=".">.647</td></tr><tr><td align="left">Item17</td><td align="char" char=".">1.173</td><td align="left">Item83</td><td align="char" char=".">.624</td><td align="left">Item39</td><td align="char" char=".">1.02</td><td align="left">Item78</td><td align="char" char=".">.725</td><td align="left">Item22</td><td align="char" char=".">.686</td></tr><tr><td align="left">Item57</td><td align="char" char=".">1.369</td><td align="left">Item1</td><td align="char" char=".">.667</td><td align="left">Item91</td><td align="char" char=".">1.141</td><td align="left">Item62</td><td align="char" char=".">.757</td><td align="left">Item90</td><td align="char" char=".">.712</td></tr><tr><td align="left">Item2</td><td align="char" char=".">1.537</td><td align="left">Item81</td><td align="char" char=".">.749</td><td align="left">Item42</td><td align="char" char=".">1.173</td><td align="left">Item47</td><td align="char" char=".">.768</td><td align="left">Item86</td><td align="char" char=".">.724</td></tr><tr><td align="left">Item18</td><td align="char" char=".">1.556</td><td align="left">Item51</td><td align="char" char=".">.841</td><td align="left">Item75</td><td align="char" char=".">1.247</td><td align="left">Item79</td><td align="char" char=".">.923</td><td align="left">Item59</td><td align="char" char=".">.760</td></tr><tr><td align="left">Item66</td><td align="char" char=".">1.706</td><td align="left">Item30</td><td align="char" char=".">.843</td><td align="left">Item43</td><td align="char" char=".">1.287</td><td align="left">Item26</td><td align="char" char=".">1.001</td><td align="left">Item21</td><td align="char" char=".">.872</td></tr><tr><td align="left">Item67</td><td align="char" char=".">1.995</td><td align="left">Item50</td><td align="char" char=".">.886</td><td align="left">Item9</td><td align="char" char=".">1.625</td><td align="left">Item77</td><td align="char" char=".">1.100</td><td align="left">Item6</td><td align="char" char=".">1.442</td></tr><tr><td align="left">Item54</td><td align="char" char=".">2.081</td><td align="left">Item14</td><td align="char" char=".">.945</td><td align="left">Item61</td><td align="char" char=".">1.652</td><td align="left">Item45</td><td align="char" char=".">1.202</td><td align="left">Item58</td><td align="char" char=".">1.566</td></tr><tr><td align="left">Item3</td><td align="char" char=".">2.503</td><td align="left">Iem49</td><td align="char" char=".">1.114</td><td align="left">Item94</td><td align="char" char=".">1.874</td><td align="left">Item98</td><td align="char" char=".">1.291</td><td align="left">Item37</td><td align="char" char=".">2.145</td></tr><tr><td align="left">Item15</td><td align="char" char=".">3.131</td><td align="left">Item13</td><td align="char" char=".">1.622</td><td align="left">Item41</td><td align="char" char=".">2.350</td><td align="left">Item97</td><td align="char" char=".">1.449</td><td align="left">Item87</td><td align="char" char=".">2.912</td></tr></tbody></table> </ephtml> </p> <hd id="AN0179974325-12">Discussion</hd> <p>This study focused on calibrating trait personality items through the unfolding IRT paradigm using two different models: the dominance and unfolding models. Regarding the first aim, we used TPQue5 as a venue to test this methodology. The findings supported our first hypothesis: TPQue5 is best described under the dominance IRT model. Our results contradict previous findings (e.g., [<reflink idref="bib16" id="ref31">16</reflink>]; [<reflink idref="bib18" id="ref32">18</reflink>]), which indicate that an unfolding model would be preferable for explaining response processes in personality tests and conform to the dominance construction method used for evaluating and scoring TPQue5 items, which pertains to the classical test and common factor analytic processes ([<reflink idref="bib17" id="ref33">17</reflink>]). Trait personality, as measured by the TPQue5, is best understood by monotonically increasing IRFs; consequently, individuals responding to the TPQue5 increase their probability of endorsing its items as their trait levels increase.</p> <p>Regarding the second aim, we further analyzed the IRFs for the TPQue5, and the item location parameters <emph>(δi)</emph> for both subscales were estimated. The "Agreeableness", "Openness to Experience", and "Neuroticism" subscales contained mixed item response patterns, with "Openness to Experience" indicating the most problematic or mis-specified model adjustment. Our findings partially confirm the findings of [<reflink idref="bib4" id="ref34">4</reflink>], [<reflink idref="bib1" id="ref35">1</reflink>], and [<reflink idref="bib10" id="ref36">10</reflink>]. Despite the dominance response strategy followed in the construction of the TPQue5, the test contains items with monotonically increasing, decreasing, and single-peaked IRFs. One example is item 15 from the Neuroticism scale—"<emph>I usually remain calm, even when time frame narrows</emph>"—which presents an ideal-point response function (cf. Supplementary material 2). This item may be endorsed by individuals who exhibit an intermediate personality type because they have an increased probability of agreeing with the item. In contrast, individuals with low or high levels of neuroticism may disagree with this item. In the case of low neuroticism, individuals may consider their inability to retain calmness due to the timeframe provided. At the same time, individuals with high neuroticism levels may also disagree because they are less able to retain their calmness, regardless of the timeframe.</p> <p>The absence of folding in our data could be attributed to the presence of highly positive item location parameters. One example is item 54, which is highly discriminant (<emph>α</emph> = 2.06) and difficult to respond to (<emph>β</emph> = 2.25). Folding typically occurs at elevated levels of a trait that was sparsely represented in our sample (see [<reflink idref="bib19" id="ref37">19</reflink>], for a relevant discussion on the trait emotional intelligence).Thus, TPQue5 items may be characterized as moderately extreme because they exhibit large location parameters and are more informative at extreme trait levels (e.g., [<reflink idref="bib16" id="ref38">16</reflink>]). According to [<reflink idref="bib4" id="ref39">4</reflink>], moderately extreme items exhibit similar response characteristics for both the dominance and ideal-point models, with dominance models considered more suitable for psychological assessment.</p> <p>Moreover, item-total correlations were very large across all subscales, which may explain the differences in IRFs within certain subscales ([<reflink idref="bib16" id="ref40">16</reflink>]). Overall, the TPQue5 is considered an efficient application of the dominance model for personality measurement because of the congruence between the scoring method and item analytic processes. Therefore, its use in the occupational and clinical settings is encouraged.</p> <hd id="AN0179974325-13">Limitations</hd> <p>This study has some limitations. Although the sample size was considered adequate for examining item parameters in the dominance models, this prerequisite may not have been fulfilled in this case. According to [<reflink idref="bib13" id="ref41">13</reflink>], accurate person parameter estimates are typically obtained by using approximately 20 items distributed among 750 respondents. Each TPQue5 subscale comprises 15 items. This decrease in subscale length may have affected the accuracy of the ideal-point person parameterization. It would be interesting to identify unfolding response patterns in longer and more informative versions of the TPQue. Second, TPQue5 items were considered extreme. However, intermediate items (or neutrally worded items) may illuminate "hidden divisions" of personality that may not be assessed by extreme items (e.g., [<reflink idref="bib18" id="ref42">18</reflink>]). Thus, subscales pertaining to neutrally worded items may add to the measurement precision of theTPQue.</p> <hd id="AN0179974325-14">Conclusion</hd> <p>In conclusion, TPQue5 is considered to conform to the dominance response model, which best describes responses as a monotonic function. Respondents with high personality traits tend to have similar levels of agreement with statements. We also found that all the subscales included intermediate personality items that conformed to the ideal-point paradigm. Items with neutrally worded statements may be helpful in scrutinizing personality traits.</p> <hd id="AN0179974325-15">Supplemental Material</hd> <p>Graph: Supplemental Material for Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5) by Eirini M. Mitropoulou, Leonidas A. Zampetakis and Ioannis Tsaousis in Evaluation Review</p> <hd id="AN0179974325-16">Supplemental Material</hd> <p>Graph: Supplemental Material for Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5) by Eirini M. Mitropoulou, Leonidas A. Zampetakis and Ioannis Tsaousis in Evaluation Review</p> <hd id="AN0179974325-17">ORCID iD</hd> <p>Eirini M. Mitropoulou https://orcid.org/0000-0003-4786-9128</p> <ref id="AN0179974325-18"> <title> References </title> <blist> <bibl id="bib1" idref="ref9" type="bt">1</bibl> <bibtext> Cao M., Drasgow F., Cho S. (2015). Developing ideal intermediate personality items for the ideal point model. Organizational Research Methods, 18(2), 252–275. https://doi.org/10.1177/1094428114555993</bibtext> </blist> <blist> <bibl id="bib2" idref="ref10" type="bt">2</bibl> <bibtext> Chalmers R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://<ulink href="http://www.jstatsoft.org/index.php/jss/article/view/v048i06/598">www.jstatsoft.org/index.php/jss/article/view/v048i06/598</ulink></bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Chernyshenko O. S., Stark S., Chan K., Drasgow F., Williams B. (2001). Fitting item-response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523–562.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref14" type="bt">4</bibl> <bibtext> Chernyshenko O. S., Stark S., Dradgow F., Roberts B. W. (2007). Conducting personality scales under the assumptions of an ideal point response process: Towards increasing the flexibility of personality measures. Psychological Assessment, 19, 88–106. https://doi.org/10.1037/1040-3590.19.1.88</bibtext> </blist> <blist> <bibl id="bib5" idref="ref1" type="bt">5</bibl> <bibtext> Embretson S. E., Reise S. P. (2000). Item response theory for psychologists: Erlbaum.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref13" type="bt">6</bibl> <bibtext> Jansen P. G. W., Roskam E. E. (1986). Latent trait models and dichotomization of graded responses. Psychometrika, 51, 69–91.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref22" type="bt">7</bibl> <bibtext> LaPalme M., Tay L., Wang W. (2017). A within-person examination of the ideal-point response process. Psychological Assessment, 30, 567–581. https://doi.org/10.1037/pas0000499</bibtext> </blist> <blist> <bibl id="bib8" idref="ref6" type="bt">8</bibl> <bibtext> Mammadov S. (2022). Big five personality traits and academic performance: A meta-analysis. Journal of Personality, 90(2), 222–255. https://doi.org/10.1111/jopy.12663</bibtext> </blist> <blist> <bibl id="bib9" idref="ref2" type="bt">9</bibl> <bibtext> Maples-Keller J. L., Williamson R. L., Sleep C. E., Carter N. T., Campbell W. K., Miller J. D. (2019). Using Item Response Theory to develop a 60-item representation of the NEO PI-R using the international personality item pool: Development of the IPIP-NEO-60. Journal of Personality Assessment, 101(1), 4–15. https://doi.org/10.1080/00223891.2017.1381968</bibtext> </blist> <blist> <bibtext> Meijer R. R., Baneke J. J. (2004). Analyzing psychopathology items: A case for nonparametric item response theory. Psychological Methods, 9, 354–368.</bibtext> </blist> <blist> <bibtext> R Core Team. (2021). R: A language and environment for statistical computing: R Foundation for Statistical Computing. https://<ulink href="http://www.R-project.org/">www.R-project.org/</ulink></bibtext> </blist> <blist> <bibtext> Rizopoulos D. (2007). ltm: An R package for latent variable modeling and item response analysis. Journal of Statistical Software, 17(5), 1–25. https://<ulink href="http://www.jstatsoft.org/index.php/jss/article/view/v017i05/117">www.jstatsoft.org/index.php/jss/article/view/v017i05/117</ulink></bibtext> </blist> <blist> <bibtext> Roberts J. S., Donoghue J. R., Laughlin J. E. (1998). The generalized graded unfolding model: A general parametric item response model for unfolding graded responses. ETS Research Report Series, 2, 1–53.</bibtext> </blist> <blist> <bibtext> Roberts J. S., Donoghue J. R., Laughlin J. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24(1), 3–32. https://doi.org/10.1177/01466216000241001</bibtext> </blist> <blist> <bibtext> Samejima F. (1997). Graded response model. In van der Linden W. J., Hambleton R. K. (eds), Handbook of modern item response theory (pp. 85–100): Springer.</bibtext> </blist> <blist> <bibtext> Stark S., Chernyshenko O. S., Drasgow F., Williams B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring? Journal of Applied Psychology, 91, 25–39. https://doi.org/10.1037/0021-9010.91.1.25</bibtext> </blist> <blist> <bibtext> Tsaousis I., Kerpelis P. (2004). The traits personality questionnaire 5 (TPQue5). European Journal of Psychological Assessment, 20(3), 180–191. https://doi.org/10.1027/1015-5759.20.3.180</bibtext> </blist> <blist> <bibtext> Weekers A. M., Meijer R. R. (2008). Scaling response processes on personality items using unfolding and dominance models: An illustration with a Dutch dominance and unfolding personality inventory. European Journal of Psychological Assessment, 24(1), 65–77. https://doi.org/10.1027/1015-5759.24.1.65</bibtext> </blist> <blist> <bibtext> Zampetakis L. A. (2011). The measurement of trait emotional intelligence with TEIQue-SF: An analysis based on unfolding item response theory models. In What have we learned? Ten years on, Emerald Group Publishing Limited.</bibtext> </blist> </ref> <ref id="AN0179974325-19"> <title> Footnotes </title> <blist> <bibtext> The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibtext> The author(s) received no financial support for the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibtext> Supplemental material for this article is available online.</bibtext> </blist> </ref> <aug> <p>By Eirini M. Mitropoulou; Leonidas A. Zampetakis and Ioannis Tsaousis</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib16" firstref="ref4"></nolink> <nolink nlid="nl2" bibid="bib18" firstref="ref7"></nolink> <nolink nlid="nl3" bibid="bib10" firstref="ref8"></nolink> <nolink nlid="nl4" bibid="bib15" firstref="ref12"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref15"></nolink> <nolink nlid="nl6" bibid="bib13" firstref="ref17"></nolink> <nolink nlid="nl7" bibid="bib17" firstref="ref26"></nolink> <nolink nlid="nl8" bibid="bib11" firstref="ref27"></nolink> <nolink nlid="nl9" bibid="bib12" firstref="ref29"></nolink> <nolink nlid="nl10" bibid="bib19" firstref="ref37"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1441315
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5)
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Eirini+M%2E+Mitropoulou%22">Eirini M. Mitropoulou</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4786-9128">0000-0003-4786-9128</externalLink>)<br /><searchLink fieldCode="AR" term="%22Leonidas+A%2E+Zampetakis%22">Leonidas A. Zampetakis</searchLink><br /><searchLink fieldCode="AR" term="%22Ioannis+Tsaousis%22">Ioannis Tsaousis</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Evaluation+Review%22"><i>Evaluation Review</i></searchLink>. 2024 48(6):1146-1159.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Adults%22">Adults</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Personality+Traits%22">Personality Traits</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Personality+Measures%22">Personality Measures</searchLink><br /><searchLink fieldCode="DE" term="%22Personality+Assessment%22">Personality Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Style+%28Tests%29%22">Response Style (Tests)</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Interpretation%22">Test Interpretation</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement+Techniques%22">Measurement Techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Selection%22">Test Selection</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Greece%22">Greece</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/0193841X231223374
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0193-841X<br />1552-3926
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Unfolding item response theory (IRT) models are important alternatives to dominance IRT models in describing the response processes on self-report tests. Their usage is common in personality measures, since they indicate potential differentiations in test score interpretation. This paper aims to gain a better insight into the structure of trait personality, by investigating whether the dominance or alternatively the unfolding IRT model are better descriptors of the response processes on a personality measure constructed under the dominance response theorem. For the assessment of the dominant model, the Graded Response Model (GRM) is used; while for the unfolding model, the Generalized Graded Unfolding Model (GGUM) was examined. All analyses are conducted with the freely available R. A sample of 1340 Greek adults, employed in private and public organizations, fulfilled the Trait Personality Questionnaire 5 short-form (TPQue5). Findings contradict previous research on trait personality. In accordance to the construction method employed, the TPQue5 items are best understood by monotonically increasing item response functions (IRFs). Individuals responding to the TPQue5 increase their probability of endorsing its items as their trait level increases; this stands for all personality dimensions, although Openness to Experience exhibited mixed type of item response patterns. Further research directions, implications and limitations are also discussed.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1441315
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1441315
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/0193841X231223374
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 1146
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Adults
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Personality Traits
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Personality Measures
        Type: general
      – SubjectFull: Personality Assessment
        Type: general
      – SubjectFull: Response Style (Tests)
        Type: general
      – SubjectFull: Test Interpretation
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Measurement Techniques
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Test Selection
        Type: general
      – SubjectFull: Greece
        Type: general
    Titles:
      – TitleFull: Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5)
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Eirini M. Mitropoulou
      – PersonEntity:
          Name:
            NameFull: Leonidas A. Zampetakis
      – PersonEntity:
          Name:
            NameFull: Ioannis Tsaousis
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0193-841X
            – Type: issn-electronic
              Value: 1552-3926
          Numbering:
            – Type: volume
              Value: 48
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Evaluation Review
              Type: main
ResultId 1