View in EDS HTML Full Text PDF Full Text

The Video Engagement Scale (VES): Measurement Properties of the Full and Shortened VES across Studies

Saved in:
Bibliographic Details
Title: The Video Engagement Scale (VES): Measurement Properties of the Full and Shortened VES across Studies
Language: English
Authors: Lehmann, Vicky (ORCID 0000-0003-4898-3176), Hillen, Marij A. (ORCID 0000-0001-5724-7934), Verdam, Mathilde G. E. (ORCID 0000-0002-2479-8817), Pieterse, Arwen H. (ORCID 0000-0001-6395-0052), Labrie, Nanon H. M. (ORCID 0000-0002-5483-0152), Fruijtier, Agnetha D. (ORCID 0000-0003-0434-0101), Oreel, Tom H., Smets, Ellen M. A. (ORCID 0000-0002-8145-8595), Visser, Leonie N. C. (ORCID 0000-0003-3487-7938)
Source: International Journal of Social Research Methodology. 2023 26(3):305-318.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 14
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Likert Scales, Video Technology, Vignettes, Construct Validity, Test Reliability, Error of Measurement, Factor Analysis, Factor Structure, Gender Differences, Educational Attainment, Psychometrics, Scores, Psychological Patterns, Diseases, Health, College Students, Foreign Countries
Geographic Terms: Netherlands
DOI: 10.1080/13645579.2022.2052697
ISSN: 1364-5579
1464-5300
Abstract: The Video Engagement Scale (VES) is a quality indicator to assess engagement in experimental video-vignette studies, but its measurement properties warrant improvement. Data from previous studies were combined (N = 2676) and split into three subsamples for a stepped analytical approach. We tested construct validity, criterion validity, reliability, and measurement invariance. Confirmatory factor analysis (CFA) reiterated poor fit of the previously suggested subscales. An alternative 2-factor structure is presented, which was cross-validated and further shortened (VES-sf). Reliability of both subscales ("Immersion," "Emotional Impact") was very high ([alpha] > 0.86). Multi-group CFAs demonstrated partial and complete measurement invariance among male/female and differently educated participants respectively. Invariance between healthy versus ill participants was not established, but CFAs supported the VES-sf structure in both groups separately. To conclude, we provide the VES-sf with sound measurement properties and demonstrated the appropriateness of comparing certain subgroups. Overall, we recommend the VES-sf to assess engagement/ecological validity in video-vignette research.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1387392
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwH0leun8NGwcT6YL6DmKtJxAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDMURLQSw5X1vH_706gIBEICBm0yy4wy2UvlZaMCVq01Wsk71SHuKMeZzEz4iB2WBc2D-5izM3VeMSR7pM8bmWZuicLBTKGj9Onbg-9cI6xB0cAaF1_hbgIWVjhU5laJgI86eJl_2BacQThn8kaf-xXhoa9lpO8k7CRoUmZsPWxBRpD3l-ZuGp8qg9vlfeTTKFLxpJbp2HYu36h-E6TF56aVaDbIvpQl0-6p2472h
Text:
  Availability: 1
  Value: <anid>AN0163282891;9eb01may.23;2023Apr26.05:52;v2.2.500</anid> <title id="AN0163282891-1">The Video Engagement Scale (VES): measurement properties of the full and shortened VES across studies </title> <p>The Video Engagement Scale (VES) is a quality indicator to assess engagement in experimental video-vignette studies, but its measurement properties warrant improvement. Data from previous studies were combined (N = 2676) and split into three subsamples for a stepped analytical approach. We tested construct validity, criterion validity, reliability, and measurement invariance. Confirmatory factor analysis (CFA) reiterated poor fit of the previously suggested subscales. An alternative 2-factor structure is presented, which was cross-validated and further shortened (VES-sf). Reliability of both subscales (Immersion, Emotional Impact) was very high (α >.86). Multi-group CFAs demonstrated partial and complete measurement invariance among male/female and differently educated participants respectively. Invariance between healthy versus ill participants was not established, but CFAs supported the VES-sf structure in both groups separately. To conclude, we provide the VES-sf with sound measurement properties and demonstrated the appropriateness of comparing certain subgroups. Overall, we recommend the VES-sf to assess engagement/ecological validity in video-vignette research.</p> <p>Keywords: Measurement properties; validity; video engagement; experimental study design; video-vignette</p> <hd id="AN0163282891-2">1. Introduction</hd> <p>Experimental video vignettes have been used in research about communication in the healthcare setting to systematically manipulate and test effects of various communication styles and behaviors of healthcare providers. Such video vignettes are scripted scenes of healthcare consultations that are typically played by actors but closely resemble real-life situations (Blanch-Hartigan et al., [<reflink idref="bib1" id="ref1">1</reflink>]). Different versions of the same consultation are being created in which specific aspects are systematically varied, such as withholding vs. providing additional information, showing increased vs. little empathy, or displaying extensive vs. minimal non-verbal behaviors (e.g. eye contact). The resulting vignettes are then presented to participants, who are asked to imagine themselves being the patient in the video (Hillen et al., [<reflink idref="bib10" id="ref2">10</reflink>]; Van Vliet et al., [<reflink idref="bib22" id="ref3">22</reflink>]). These participants are called 'analogue patients' (Blanch-Hartigan et al., [<reflink idref="bib1" id="ref4">1</reflink>]), who function as a representative of a potential real-life patient. They can be disease-naïve individuals (i.e. healthy or free from the disease/unfamiliar with the healthcare setting of the vignette) or patients/survivors themselves. Various outcomes can then be assessed among these analogue patients, such as satisfaction with the observed conversation or information recall.</p> <p>Participation in such experimental study design requires a certain amount of engagement from analogue patients who may vary in their ability to do so. For example, some participants may be less able to engage due to lacking empathy, disagreeing with the attitude and/or behaviors of the video patient, having different own experiences with healthcare professionals, experiencing the video/actors to be of poor quality, or being distracted while watching (e.g. noise, phone call). There has been an ongoing debate about the validity of video-vignette designs, doubting that analogue patients can not represent real-life patient experiences, but several studies demonstrated the suitability and ability of analogue patients to serve as proxy for 'real' patients; underlining its ecological validity (Blanch-Hartigan et al., [<reflink idref="bib1" id="ref5">1</reflink>]; Gehenne et al., [<reflink idref="bib8" id="ref6">8</reflink>]; Hillen et al., [<reflink idref="bib10" id="ref7">10</reflink>]; Van Vliet et al., [<reflink idref="bib22" id="ref8">22</reflink>]; Visser, Tollenaar et al., [<reflink idref="bib24" id="ref9">24</reflink>]). Yet, researchers must assure that participants show an acceptable level of engagement with the video vignette, to bolster the validity of experimental video-vignette study designs, to produce meaningful study results, and to enable comparisons across manipulations and studies.</p> <p>The Video Engagement Scale (VES; Visser, Hillen et al., [<reflink idref="bib23" id="ref10">23</reflink>]) has been developed to serve as an indicator of participants' engagement with experimental video vignettes, reflecting the extent to which participants are able to view, immerse, and imagine themselves being in the video, and potentially being emotionally touched by the video. The VES contains 15 items and was based on the multidimensional construct of narrative engagement, which reflects the extent to which a reader becomes immersed in a story (De Graaf et al., [<reflink idref="bib5" id="ref11">5</reflink>], [<reflink idref="bib6" id="ref12">6</reflink>]). Based on this work, items were developed and grouped in five subscales: <emph>attention, entering the narrative world, identification, empathy</emph>, and <emph>emotions</emph>. Initial psychometric testing by means of confirmatory factor analysis did not support this 5-factor structure, and a 4-factor structure had been suggested instead (Visser, Hillen et al., [<reflink idref="bib23" id="ref13">23</reflink>]). Yet, this 4-factor structure simply combined the two subscales <emph>empathy</emph> and <emph>emotions</emph>, which suggests that the VES could be shortened. Besides, the VES serves as a quality indicator, and not primary outcome, which further substantiates the need to be as economic and short as possible. Other studies that used the VES combined scores of all items into one total score, serving as a generic measure of engagement with video vignettes (Hillen et al., [<reflink idref="bib9" id="ref14">9</reflink>]; Lehmann et al., [<reflink idref="bib14" id="ref15">14</reflink>]; Medendorp et al., [<reflink idref="bib16" id="ref16">16</reflink>]; Visser et al., [<reflink idref="bib25" id="ref17">25</reflink>]). Thus, the dimensional structure of the VES remains to be tested, while it could potentially be shortened and requires further validation.</p> <p>The current study aims to test the measurement properties of the full VES and a potentially alternative and shortened version. We will test the construct validity (i.e. original and alternative factor structures by means of confirmatory and exploratory factor analyses), reliability (i.e. Cronbach's alpha), and criterion validity (i.e. correlations with a related construct: perceived video realism). Finally, measurement invariance will be tested across various subgroups to ascertain validity in comparing study participants, based on their (a) level of education, (b) sex, and (c) disease status. These three factors were chosen as they are factors typically utilized in selecting and comparing analogue patients in experimental video-vignette studies.</p> <hd id="AN0163282891-3">2. Methods</hd> <p></p> <hd id="AN0163282891-4">2.1 Sample</hd> <p>To overcome common issues in human subjects research and psychometric testing (e.g. small and/or homogeneous samples), we maximized our sample size by combining data from previous experimental video-vignette studies that used the VES. We searched the literature and identified seven Dutch studies, that all contributed data to this project: Two studies exclusively recruited healthy/disease-naïve university students (Medendorp et al., [<reflink idref="bib16" id="ref18">16</reflink>]; Visser et al., [<reflink idref="bib25" id="ref19">25</reflink>]) and one project included healthy/disease-naïve older adults above age 50 (Fruijtier et al., [<reflink idref="bib7" id="ref20">7</reflink>]). Four other projects combined healthy/disease-naïve adults with either cancer patients/survivors (Hillen et al., [<reflink idref="bib9" id="ref21">9</reflink>]; Lehmann et al., [<reflink idref="bib15" id="ref22">15</reflink>]; Visser, Tollenaar et al., [<reflink idref="bib24" id="ref23">24</reflink>]) or participants who had one of various health conditions (cancer, rheumatoid arthritis, or kidney disease (Pieterse et al., [<reflink idref="bib18" id="ref24">18</reflink>])). An overview of all included studies is presented in Table 1. Combining data from these seven studies yielded a grand total of <emph>N</emph> = 2676 participants in the current project (see background characteristics in Table 2). In each included study, participants had completed standard written informed consent procedures, which included the consent to use their data for secondary analyses. Therefore, and by means of data share agreements, data used in this project included the VES items, ratings of perceived video realism, and sociodemographic characteristics: sex, age, level of education (low, middle high; (Nuffic, [<reflink idref="bib17" id="ref25">17</reflink>])), and disease status (i.e. previously/acutely ill participants vs. healthy/disease-naïve participants without a history of the disease/setting of the vignette; Table 2). Given that each study had been approved by their local institutional review board, no additional approval was needed for this secondary data analysis.</p> <p>Table 1. Overview of included projects and their settings.</p> <p> <ephtml> <table><thead><tr><td>sample</td><td>sex</td><td>age</td><td>Video setting</td><td>reported in:</td></tr><tr><td /><td><italic>% female</italic></td><td><italic>M</italic><italic>(range)</italic></td><td /></tr></thead><tbody><tr><td><italic>N</italic> = 433: <bold><italic>n</italic> = 388</bold> mixed cancer survivors <bold><italic>n</italic> = 45</bold> healthy/disease-naïve adults*</td><td>53%</td><td>62 (20–90)</td><td>Oncology: treatment plan (immunotherapy)</td><td>Lehmann et al., <xref ref-type="bibr" rid="bibr15">2019</xref>, Lehmann et al., <xref ref-type="bibr" rid="bibr14">2020</xref></td></tr><tr><td><italic>N</italic> = 46: <bold><italic>n</italic> = 22</bold> mixed cancer survivors <bold><italic>n</italic> = 24</bold> healthy/disease-naïve adults*</td><td>65%</td><td>23–81</td><td>Oncology: diagnosis/ bad news consultation</td><td>Visser, Hillen, <xref ref-type="bibr" rid="bibr23">2016</xref></td></tr><tr><td><italic>N</italic> = 97: <bold><italic>n</italic> = 97</bold> university students*</td><td>86%</td><td>22 (18–35)</td><td>Oncology: treatment plan (surgery)</td><td>Medendorp et al., <xref ref-type="bibr" rid="bibr16">2017</xref></td></tr><tr><td><italic>N</italic> = 217:<sup>a</sup><bold><italic>n</italic> = 215</bold> university students*</td><td>86%</td><td>21 (18–33)</td><td>Oncology: diagnosis/ bad news consultation</td><td>Visser et al., <xref ref-type="bibr" rid="bibr25">2019</xref></td></tr><tr><td><italic>N</italic> = 214: <bold><italic>n</italic> = 147</bold> breast cancer survivors <bold><italic>n</italic> = 67</bold> healthy/disease-naïve adults*</td><td>100%</td><td>55 (31–91) 51 (31–73)</td><td>Oncology: treatment plan (mastectomy, adjuvant chemotherapy)</td><td>Hillen et al., <xref ref-type="bibr" rid="bibr9">2015</xref></td></tr><tr><td><italic>N</italic> = 652:<sup>b</sup><bold><italic>n</italic> = 137</bold> mixed cancer survivors <bold><italic>n</italic> = 336</bold> healthy/disease-naïve adults* <bold><italic>n</italic> = 89</bold> rheumatoid arthritis <bold><italic>n</italic> = 82</bold> kidney disease</td><td>54%</td><td>53 (18–87)</td><td>Oncology: treatment plan (chemotherapy at home vs. at home and hospital) Kidney disease: treatment plan (hemodialysis vs. peritoneal dialysis) rheumatic disease: Treatment plan (at home vs. hospital)</td><td>Pieterse et al., <xref ref-type="bibr" rid="bibr18">2021</xref></td></tr><tr><td><italic>N</italic> = 1128:<sup>c</sup><bold><italic>n</italic> = 1027</bold> healthy/disease-naïve adults*</td><td>82%</td><td>63 (37–94)</td><td>Dementia/Memory clinic</td><td>Fruijtier et al., <xref ref-type="bibr" rid="bibr7">2022</xref></td></tr></tbody></table> </ephtml> </p> <p>1 * considered as healthy, i.e. disease-naïve/free from the disease of the video setting;</p> <ulist> <item>2 Excluded participants from current analysis: <sups>a</sups><emph>n</emph> = 2; <sups>b</sups><emph>n</emph> = 8; <sups>c</sups><emph>n</emph> = 101 (due to incomplete data);</item> <item>3 Included grand total <bold><emph>N</emph> = 2676</bold></item> </ulist> <p>Table 2. Background characteristics of the full sample and three subsamples.</p> <p> <ephtml> <table><thead><tr><td /><td>Full sample</td><td>Sample I</td><td>Sample II</td><td>Sample III</td><td>Difference tests sample I, II & III</td></tr><tr><td /><td><italic>N</italic> = 2676</td><td><italic>n</italic> = 867</td><td><italic>n</italic> = 878</td><td><italic>n</italic> = 931</td><td /></tr></thead><tbody><tr><td><bold>Age</bold></td><td>54.4 (16.6), 18–94</td><td>54.2 (16.7), 18–93</td><td>54.4 (17.1), 18–94</td><td>54.6 (16.1), 18–87</td><td><italic>F</italic>(2, 2673) = 0.1, <italic>p</italic> =.93</td></tr><tr><td><bold>Sex</bold></td><td /><td /><td /><td /><td><italic>χ<sup>2</sup></italic>(2) = 9.2, <italic>p</italic> =.01 <sup>c</sup></td></tr><tr><td>Female</td><td>1923 (71.9%)</td><td>590 (68.1%)</td><td>648 (73.8%)</td><td>685 (73.6%)</td><td /></tr><tr><td>Male</td><td>753 (28.1%)</td><td>277 (31.9%)</td><td>230 (26.2%)</td><td>246 (26.4%)</td><td /></tr><tr><td><bold>Level of education</bold><sup>a</sup></td><td /><td /><td /><td /><td><italic>χ<sup>2</sup></italic>(4) = 2.8, <italic>p</italic> =.58</td></tr><tr><td>Low</td><td>218 (8.2%)</td><td>66 (7.6%)</td><td>77 (8.8%)</td><td>75 (8.1%)</td><td /></tr><tr><td>Middle</td><td>996 (37.2%)</td><td>310 (35.8%)</td><td>325 (37.0%)</td><td>360 (38.7%)</td><td /></tr><tr><td>High</td><td>1460 (54.6%)</td><td>491 (56.6%)</td><td>475 (54.1%)</td><td>494 (53.1%)</td><td /></tr><tr><td><bold>Disease status</bold></td><td /><td /><td /><td /><td><italic>χ<sup>2</sup></italic>(2) = 0.7, <italic>p</italic> =.69</td></tr><tr><td>Healthy/disease-naïve</td><td>1811 (67.7%)</td><td>577 (66.6%)</td><td>599 (68.2%)</td><td>635 (68.2%)</td><td /></tr><tr><td>(chronically) ill</td><td>865 (32.3%)</td><td>290 (33.4%)</td><td>279 (31.8%)</td><td>296 (31.8%)</td><td /></tr><tr><td><bold>Years since diagnosis</bold><sup>b</sup></td><td>8.5 (9.5), 0–58</td><td>8.6 (9.0), 0–58</td><td>8.3 (9.5), 0–46</td><td>8.5 (9.9), 0–49</td><td><italic>F</italic>(2, 706) = 0.1, <italic>p</italic> =.94</td></tr></tbody></table> </ephtml> </p> <ulist> <item>4 <sups>a</sups>Education has <emph>n</emph> = 1 (0.1%) missing in Sample II, and <emph>n</emph> = 2 (0.2%) missings in Sample III;</item> <item>5 low = vocational training or less; middle = continued education; high = college/university</item> <item>6 <sups>b</sups>given the inclusion of healthy participants and missing values, years since diagnosis is reported for <emph>n</emph> = 709 in the full sample, <emph>n</emph> = 247 Sample I, <emph>n</emph> = 224 Sample II, and <emph>n</emph> = 238 Sample III</item> <item>7 <sups>c</sups>significant due to large sample sizes, but the sex distribution is rather similar across groups 70:30% and therefore negligible</item> </ulist> <hd id="AN0163282891-5">2.2 Measures</hd> <p></p> <hd id="AN0163282891-6">Video engagement</hd> <p>The VES includes 15 items (Table 3; Appendix 1), which are answered on a 7-point Likert scale (<emph>disagree – agree</emph>) with higher scores indicating greater engagement. Cronbach's alpha in previous studies was high (<emph>α </emph>≥.93) for the full scale (Lehmann et al., [<reflink idref="bib15" id="ref26">15</reflink>]; Visser, Tollenaar et al., [<reflink idref="bib24" id="ref27">24</reflink>]), while reported alphas for the original five subscales varied, <emph>α = </emph>.54-.92 (Visser, Hillen et al., [<reflink idref="bib23" id="ref28">23</reflink>]). Low scores were observed for the <emph>attention</emph> subscale (<emph>α = </emph>.54–66), whereas all other subscale <emph>α</emph>'s exceeded.82).</p> <p>Table 3. All 15 VES items and factor loadings of the exploratory factor analysis (EFA)* in sample I.</p> <p> <ephtml> <table><thead><tr><td /><td /><td /><td>Factor</td></tr><tr><td>#</td><td>Item</td><td>Initial subscale</td><td>1 <sup>a</sup></td><td>2 <sup>b</sup></td><td>3 <sup>c</sup></td></tr></thead><tbody><tr><td>4</td><td>After the video ended, I felt as if I was coming back into the 'real' world</td><td>Narrative world</td><td>.564</td><td /><td /></tr><tr><td>5</td><td>When I was watching for a while, it seemed as if I had become the patient in my mind</td><td>Identity</td><td>.821</td><td /><td /></tr><tr><td>8</td><td>As I watched the video, in my imagination I was in the world of the video</td><td>Narrative world</td><td>.568 *</td><td /><td /></tr><tr><td>12</td><td>While watching, I was hardly aware of my surroundings</td><td>Attention</td><td>.572 *</td><td /><td /></tr><tr><td>13</td><td>I felt as if I was going through what the patient was going through</td><td>Identity</td><td>.954 *</td><td /><td /></tr><tr><td>14</td><td>In my imagination it was as if I were the patient</td><td>Identity</td><td>.993 *</td><td /><td /></tr><tr><td>2</td><td>While watching, I felt like I was present at the events in the video</td><td>Narrative world</td><td>.417</td><td /><td>(.326)</td></tr><tr><td>6</td><td>I empathized with the patient</td><td>Empathy</td><td /><td>.746 *</td><td /></tr><tr><td>7</td><td>The video touched me</td><td>Emotions</td><td /><td>.926</td><td /></tr><tr><td>9</td><td>While watching, I felt sad when the patient felt sad</td><td>Empathy</td><td /><td>.602 *</td><td /></tr><tr><td>10</td><td>I thought the video was moving</td><td>Emotions</td><td /><td>.891 *</td><td /></tr><tr><td>11</td><td>I sympathized with the patient</td><td>Empathy</td><td /><td>.791</td><td /></tr><tr><td>15</td><td>The video made me emotional</td><td>Emotions</td><td>(.378)</td><td>.472 *</td><td /></tr><tr><td>1</td><td>While watching, I was completely focused on the video</td><td>Attention</td><td /><td /><td>.808</td></tr><tr><td>3</td><td>While watching the video, I was fully concentrated on it</td><td>Attention</td><td /><td /><td>.815</td></tr></tbody></table> </ephtml> </p> <p>8 Note. Double loading are shown for items with two loadings <.2 of each other; * KMO-Measure of Sampling adequacy:.911 and Bartlett's test of Sphericity: <emph>χ<sups>2</sups></emph>= 9351, <emph>p</emph> <.001 both assuring the suitability of used data; <sups>a</sups> factor 1 = retained as initial <emph>Immersion</emph> subscale (6 items); <sups>b</sups> factor 2 = retained as initial <emph>Emotional Impact</emph> subscale (6 items); <sups>c</sups> dropped from further analyses; * retained in the final VES-sf subscales and item #1 is retained as screener</p> <hd id="AN0163282891-7">Perceived video realism</hd> <p>Five of the seven included projects (Hillen et al., [<reflink idref="bib9" id="ref29">9</reflink>]; Lehmann et al., [<reflink idref="bib14" id="ref30">14</reflink>], [<reflink idref="bib15" id="ref31">15</reflink>]; Medendorp et al., [<reflink idref="bib16" id="ref32">16</reflink>]; Visser, Tollenaar et al., [<reflink idref="bib24" id="ref33">24</reflink>]; Visser et al., [<reflink idref="bib25" id="ref34">25</reflink>]) used an additional three items that measured whether participants perceived the video vignette as realistic, which was used to assess criterion validity. These three items were face-valid and measured whether participants thought the video was <emph>realistic, credible</emph>, and if it could have <emph>happened in real-life</emph>. Answers on a 7-point Likert scale (<emph>totally disagree – totally agree</emph>) were averaged with higher scores indicating greater perceived realism (<emph>α = </emph>.88; <emph>n</emph> = 1004).</p> <hd id="AN0163282891-8">2.3 Statistical analyses</hd> <p>Before testing measurement properties, a random number generator was used to randomly split the whole sample (<emph>N</emph> = 2676) into three equally sized subsamples (i.e. ~33% each) to enable cross-validation of our results. The resulting three subsamples were compared to test whether they were similar with regards to background factors, using <emph>χ<sups>2</sups></emph>-tests or <emph>F</emph>-tests as appropriate. After establishing similarity of the subsamples (Table 2), a stepwise analytical approach was conducted to test the measurement properties of the VES. The <emph>main</emph> steps included testing the original VES structure and suggesting an alternative structure by means of confirmatory and exploratory factor analyses (in Sample I), then further shortening the VES based on model fit indices and theoretical reasoning (Sample II), before testing its measurement invariance between subgroups (Sample III; see more details below).</p> <p>Using <bold>Sample I</bold> (<emph>n</emph> = 867), descriptive statistics, including skewness and kurtosis were calculated to examine all items. The previously suggested 5- and 4-factor structure of the VES were then tested using confirmatory factor analyses (CFAs). Analyses were performed in the Lavaan package (Rosseel, [<reflink idref="bib19" id="ref35">19</reflink>]) of the statistical software R (<ulink href="http://www.r-project.org">http://www.r-project.org</ulink>) by means of Robust Maximum Likelihood (RML). The RML procedure uses robust Huber-White sandwich standard errors and scaled chi-square (<emph>χ<sups>2</sups></emph>) tests that take non-normality into account (Yuan & Bentler, [<reflink idref="bib27" id="ref36">27</reflink>]). Model fit was examined based on the <emph>χ<sups>2</sups></emph>-test, Comparative Fit Index (CFI; Hu & Bentler, [<reflink idref="bib12" id="ref37">12</reflink>]), Root Mean Square Error of Approximation (RMSEA; Browne & Cudeck, [<reflink idref="bib2" id="ref38">2</reflink>]), and Standardized Root Mean Square Residual (SRMR; Hooper et al., [<reflink idref="bib11" id="ref39">11</reflink>]). Non-significant <emph>χ<sups>2</sups></emph>–tests indicate good model fit, but the <emph>χ<sups>2</sups></emph>–value is sensitive to sample size (Jöreskog & Sörbom, [<reflink idref="bib13" id="ref40">13</reflink>]; Schermelleh-Engel et al., [<reflink idref="bib21" id="ref41">21</reflink>]) and exact fit between model and data is rare in practice. Therefore, good fit is further indicated by CFI>.97, RMSEA<.06, and SRMR<.06, whereas values of CFI>.95, RMSEA<.08 and SRMR<.08 indicate acceptable fit (Browne & Cudeck, [<reflink idref="bib2" id="ref42">2</reflink>]; Hooper et al., [<reflink idref="bib11" id="ref43">11</reflink>]; Hu & Bentler, [<reflink idref="bib12" id="ref44">12</reflink>]; Schermelleh-Engel et al., [<reflink idref="bib21" id="ref45">21</reflink>]). Whenever appropriate, sample size-adjusted BIC (Bayesian Information Criterion) and AIC (Akaike Information Criterion) were used to compare different models, with lower scores indicating more favorable fit. Next, and to examine alternative factorial dimensions, an exploratory factor analysis (EFA) was carried out in SPSS (version 25), using the maximum likelihood method and Oblimin rotation with Kaiser Normalization. Finally, the research team met to discuss all results and to decide about possible alternative factor structures of the VES, based on theoretical reasoning and CFA and EFA results.</p> <p>In <bold>Sample II</bold> (<emph>n</emph> = 878), the newly suggested factor structure was cross-validated using CFAs. Subsequently, factor loadings (i.e. substantial loadings are >.4 (Watkins, [<reflink idref="bib26" id="ref46">26</reflink>])), potential multicollinearity of items (<emph>r</emph> >.8), internal consistency as indicated by Cronbach's α, and theoretical reasoning were used to further shorten the VES (VES-sf). Criterion validity was tested by correlating scores of the VES-sf with perceived video realism, where correlations >.30 are considered moderate (Cohen, [<reflink idref="bib4" id="ref47">4</reflink>]).</p> <p>Using <bold>Sample III</bold> (<emph>n</emph> = 931), we cross-validated the VES-sf using a CFA. Measurement invariance of the VES-sf across subgroups was tested to determine if the same construct (i.e. engagement) is measured in different groups of participants and thus to ensure the comparability of VES-scores across groups. We used multi-group CFAs to test <emph>metric</emph> invariance (i.e. equal factor loadings) and <emph>scalar</emph> invariance (i.e. equal factor loadings and intercepts) across three different groups based on participants' (a) level of education, (b) sex, and (c) disease status (i.e. healthy/disease-naïve vs. previously/acutely ill). Tenability of measurement invariance restrictions was evaluated using the difference in <emph>χ<sups>2</sups></emph>-values between models with scaled <emph>χ<sups>2</sups>Δ</emph>-tests (Satorra & Bentler, [<reflink idref="bib20" id="ref48">20</reflink>]), where a significant result indicates a violation of measurement invariance. In addition, differences in CFI values between models (ΔCFI) were evaluated, where ΔCFI>.10 indicates a violation of measurement invariance (Cheung & Rensvold, [<reflink idref="bib3" id="ref49">3</reflink>]).</p> <hd id="AN0163282891-9">3. Results</hd> <p></p> <hd id="AN0163282891-10">3.1 Sample I</hd> <p>Visual inspection of descriptive statistics of all 15 items indicated little variance and thus limited discriminatory value for items #1, #3, and #6, as shown by a combination of mean scores ≥5 and modes ≥6 on a scale from 1–7, as well as skewness and kurtosis ≥1 (Table 4). CFA results indicated poor model fit for both the 5- and 4-factor structure based on all fit indices (<emph>p</emph>s<.001, CFI<.87, RMSEA>.11, SRMR>.12), while a direct comparison between both showed a better fit of the 5-factor model (<emph>p</emph> <.001, lower AIC/ BIC; Table 5). The results of the EFA showed that all items were grouped into 3 factors, but the scree plot and eigenvalues favored a 1- or 2-factor solution (Table 3). Inspection of all items and factor loadings confirmed the limited added value of a 3<sups>rd</sups> factor, which included items #1, #2, and #3.</p> <p>Table 4. Descriptive statistics of the initial 15 VES items (Sample I).</p> <p> <ephtml> <table><thead><tr><td>Item</td><td>Mean</td><td>Median</td><td>Mode</td><td>range</td><td>skewness</td><td>kurtosis</td></tr></thead><tbody><tr><td>1</td><td>6.07</td><td>6</td><td>7</td><td>1–7</td><td>−1.43</td><td>2.13</td></tr><tr><td>2</td><td>4.65</td><td>5</td><td>6</td><td>1–7</td><td>−0.54</td><td>−0.64</td></tr><tr><td>3</td><td>5.58</td><td>6</td><td>7</td><td>1–7</td><td>−0.95</td><td>0.46</td></tr><tr><td>4</td><td>3.79</td><td>4</td><td>4</td><td>1–7</td><td>−0.02</td><td>−1.07</td></tr><tr><td>5</td><td>3.41</td><td>3</td><td>1</td><td>1–7</td><td>0.17</td><td>−1.13</td></tr><tr><td>6</td><td>5.29</td><td>6</td><td>6</td><td>1–7</td><td>−1.04</td><td>0.83</td></tr><tr><td>7</td><td>4.75</td><td>5</td><td>6</td><td>1–7</td><td>−0.50</td><td>−0.65</td></tr><tr><td>8</td><td>4.00</td><td>4</td><td>4</td><td>1–7</td><td>−0.21</td><td>−1.10</td></tr><tr><td>9</td><td>3.79</td><td>4</td><td>4</td><td>1–7</td><td>0.02</td><td>−1.08</td></tr><tr><td>10</td><td>4.35</td><td>4</td><td>4</td><td>1–7</td><td>−0.28</td><td>−0.79</td></tr><tr><td>11</td><td>5.15</td><td>5</td><td>6</td><td>1–7</td><td>−0.92</td><td>0.43</td></tr><tr><td>12</td><td>4.00</td><td>4</td><td>4</td><td>1–7</td><td>−0.08</td><td>−1.19</td></tr><tr><td>13</td><td>3.34</td><td>3</td><td>1</td><td>1–7</td><td>0.29</td><td>−1.11</td></tr><tr><td>14</td><td>3.16</td><td>3</td><td>1</td><td>1–7</td><td>0.40</td><td>−1.02</td></tr><tr><td>15</td><td>4.17</td><td>4</td><td>5</td><td>1–7</td><td>−0.21</td><td>−1.08</td></tr></tbody></table> </ephtml> </p> <p>Table 5. Results of all Confirmatory Factor Analyses; and tests of measurement invariance in subgroups by (a) level of education, (b), sex, and (c) disease status.</p> <p> <ephtml> <table><thead><tr><td /><td /><td>χ<sup>2</sup> (df), <italic>p</italic></td><td>RMSEA [95%CI]</td><td>CFI</td><td>SRMR</td><td>AIC</td><td>BIC</td></tr></thead><tbody><tr><td /><td><bold>Sample I</bold></td><td /><td /><td /><td /><td /><td /></tr><tr><td>#1</td><td>5-factor model <sup>a</sup></td><td>986.55 (80), <italic>p</italic> <.001</td><td>.113 [.108 –.119]</td><td>.867</td><td>.127</td><td>42,509</td><td>42,597</td></tr><tr><td>#2</td><td>4-factor model <sup>a</sup></td><td>984.26 (84), <italic>p</italic> <.001</td><td>.111 [.106 –.117]</td><td>.865</td><td>.126</td><td>42,554</td><td>42,635</td></tr><tr><td /><td><bold>Sample II</bold></td><td /><td /><td /><td /><td /><td /></tr><tr><td>#3</td><td>2-factor model</td><td>796.06 (53), <italic>p</italic> <.001</td><td>.127 [.121 –.134]</td><td>.875</td><td>.071</td><td>33,872</td><td>33,930</td></tr><tr><td>#4</td><td>short 2-factor model</td><td>283.46 (19), <italic>p</italic> <.001</td><td>.127 [.116 –.138]</td><td>.918</td><td>.052</td><td>23,460</td><td>23,500</td></tr><tr><td>#5</td><td>short 2-factor model <sup>b</sup></td><td>149.24 (18), <italic>p</italic> <.001</td><td>.092 [.080 –.104]</td><td>.960</td><td>.035</td><td>23,281</td><td>23,322</td></tr><tr><td /><td><bold>Sample III</bold></td><td /><td /><td /><td /><td /><td /></tr><tr><td>#5</td><td>short 2-factor model <sup>b</sup></td><td>97.01 (18), <italic>p</italic> <.001</td><td>.069 [.058 –.081]</td><td>.976</td><td>.031</td><td>24,854</td><td>24,897</td></tr><tr><td>#5</td><td>short 2-factor model <sup>b</sup> by <bold>education:</bold></td></tr><tr><td /><td>Configural invariance</td><td>128.29 (54), <italic>p</italic> <.001</td><td>.067 [.054 –.080]</td><td>.976</td><td>.033</td><td>24,795</td><td>24,923</td></tr><tr><td /><td>Metric invariance</td><td><italic>Δ </italic>= 14.73 (12), <italic>p</italic> =.257</td><td /><td><italic>Δ </italic>=.002</td><td /><td /><td /></tr><tr><td /><td>Scalar invariance</td><td><italic>Δ </italic>= 14.17 (12), <italic>p</italic> =.290</td><td /><td><italic>Δ </italic>=.001</td><td /><td /><td /></tr><tr><td>#5</td><td>short 2-factor model <sup>b</sup> by <bold>sex</bold>:</td></tr><tr><td /><td>Configural invariance</td><td>121.39 (36), <italic>p</italic> <.001</td><td>.072 [.060 –.084]</td><td>.975</td><td>.032</td><td>24,848</td><td>24,934</td></tr><tr><td /><td>Metric invariance</td><td><italic>Δ </italic>= 15.56 (6), <italic>p</italic> =.016</td><td /><td><italic>Δ </italic>=.003</td><td /><td /><td /></tr><tr><td /><td /><td><italic>Δ </italic>= 8.19 (5), <italic>p</italic> =.146 <sup>c</sup></td><td /><td><italic>Δ </italic>=.002</td><td /><td /><td /></tr><tr><td /><td>Scalar invariance</td><td><italic>Δ </italic>= 16.94 (6), <italic>p</italic> =.010 <sup>c</sup></td><td /><td><italic>Δ </italic>=.003</td><td /><td /><td /></tr><tr><td /><td /><td><italic>Δ </italic>= 8.78 (5), <italic>p</italic> =.118 <sup>c,d</sup></td><td /><td><italic>Δ </italic>=.002</td><td /><td /><td /></tr><tr><td>#5</td><td>short 2-factor model <sup>b</sup> by <bold>disease status</bold>: <sup>e</sup></td></tr><tr><td /><td>Configural invariance</td><td>125.422 (36), <italic>p</italic> <.001</td><td>.074 [.062 –.086]</td><td>.974</td><td>.030</td><td>24,636</td><td>24,721</td></tr><tr><td /><td>Metric invariance</td><td><italic>Δ </italic>= 28.84 (6), <italic>p</italic> <.001</td><td /><td><italic>Δ </italic>=.006</td><td /><td /><td /></tr><tr><td /><td /><td><italic>Δ </italic>= 8.54 (4), <italic>p</italic> =.074 <sup>f</sup></td><td /><td><italic>Δ </italic>=.002</td><td /><td /><td /></tr><tr><td /><td>Scalar invariance</td><td><italic>Δ </italic>= 139.28 (6), <italic>p</italic> <.001 <sup>f</sup></td><td /><td><italic>Δ </italic>=.030</td><td /><td /><td /></tr><tr><td /><td /><td><italic>Δ </italic>= 1.18 (2), <italic>p</italic> =.555 <sup>f, g</sup></td><td /><td><italic>Δ </italic>=.000</td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <ulist> <item>9 <sups>a</sups>direct comparison of model #1 vs. #2: <emph>χ<sups>2</sups>Δ</emph>(<reflink idref="bib4" id="ref50">4</reflink>) = 26.827, <emph>p</emph> <.001, favoring model #1;</item> <item>10 <sups>b</sups>covariances between item 13 and 14 were added, as suggested by modification fit indices of model #4</item> <item>11 <sups>c</sups>allowing unequal factor loadings for item 9</item> <item>12 <sups>d</sups>allowing unrestricted intercept for item 8</item> <item>13 <sups>e</sups>fit indices for CFAs in both groups separately: healthy participants: <emph>χ<sups>2</sups></emph>(<reflink idref="bib18" id="ref51">18</reflink>) = 66.62, RMSEA =.065 [.051-.080], CFI =.979, SRMR =.032 versus ill participants: <emph>χ<sups>2</sups></emph>(<reflink idref="bib18" id="ref52">18</reflink>) = 59.06, RMSEA =.089 [.068-.111], CFI =.965, SRMR =.027</item> <item>14 <sups>f</sups>allowing unequal factor loadings of item 10 and 14</item> <item>15 <sups>g</sups>allowing unrestricted intercepts for items 8, 10, 12, and 15</item> </ulist> <p>These results were discussed within the research team and it was concluded that a 2-factor solution represents the VES items and dimensions of video engagement best, including (<reflink idref="bib1" id="ref53">1</reflink>) <emph>Immersion</emph> in the video and (<reflink idref="bib2" id="ref54">2</reflink>) <emph>Emotional Impact</emph> of the video. Both subscales included six items and items #2 and #3 were dropped from further analyses (see, Table 3 for an overview of all retained items per scale). Cronbach's alpha was excellent at.906 (Immersion) and.907 (Emotional Impact) in Sample I. Moreover, item #1 was suggested to be used as screener item, because it assesses whether participants pay attention and it is considered a prerequisite to engage in video-vignette research. If participants endorse item #1 '<emph>I was fully concentrated on the video while watching</emph>' with a score of 2 or lower on the 7-point scale (i.e. <emph>(totally) disagree</emph>), we recommend the exclusion of these participants from further analyses.</p> <hd id="AN0163282891-11">3.2 Sample II</hd> <p>Based on screener item #1 (score≤2), a total of 1.6% of participants (<emph>n</emph> = 14/878) were excluded from further analyses, resulting in a sample of <emph>N</emph> = 864. The newly suggested 2-factor structure was tested in this sample, using a CFA. Relative to the previous CFAs, a slightly better (SRMR =.071) but still poor model fit was found (<emph>p</emph> <.001, CFI =.875, RMSEA =.127; Table 5). Nevertheless, Cronbach's alpha for both subscales was excellent at.922 (Immersion) and.917 (Emotional Impact).</p> <p>Given such suboptimal findings and as intended, we discussed all factor loadings, multicollinearity between items, and alpha-if-item-deleted, together with theoretical considerations within the research team to further shorten and improve the VES. This resulted in omitting 2 items from each subscale:</p> <p>Scale (<reflink idref="bib1" id="ref55">1</reflink>) <bold>Immersion</bold>: Model fit indices specified improved fit if adding a covariance between item #4 and #14, but item #4 was deemed to be too abstract/difficult and its exclusion did not influence Cronbach's alpha. Moreover, multicollinearity between items #5 and #14 was detected (<emph>r</emph> >.8) and their content was also deemed similar (i.e. feeling like the video patient). The wording of item #14 was considered more face valid, while item #5 also had a lower factor loading, all substantiating the exclusion of item #5. Thus, the subscale <emph>Immersion</emph> was shortened to a 4-item subscale, and with still very good Cronbach's alpha of.894, all included items showing substantial factor loadings (>.7), and thus assessing the extent to which participants become engaged with the video and identify with the video character.</p> <p>Scale (<reflink idref="bib2" id="ref56">2</reflink>) <bold>Emotional Impact</bold>: The content of items #6 and #11 (i.e. empathizing with video patient) as well as items #7 and #10 (i.e. being moved by the video) were evaluated as being similar. Accordingly, multicollinearity was detected within both item pairs (<emph>r ≥ </emph>.8) and model fit could be improved by including covariances. However, for the sake of brevity, and taking both difficulty of item wording and factor loadings into account, one item of each pair was excluded: items #11 and #7. This resulted in a 4-item subscale, still retaining a very good Cronbach's alpha of.864, all items showing substantial factor loadings (>.6), and assessing the extent to which participants are emotionally affected by the video.</p> <p>Subsequently, this shortened 2-factor structure was tested using a CFA, indicating still not optimal fit (CFI =.918, RMSEA =.127). However, this was improved by allowing two items of the Immersion subscale to covary (i.e. #13 and #14; CFI =.960, RMSEA =.092, SRMR =.035; Table 5), supporting confidence in the newly suggested shortened VES (VES-sf). All retained items are indicated in Table 3 and see Appendix 1 for the English and Dutch version of the full and VES-sf.</p> <p>Descriptive statistics of both subscales and total score are presented in Table 6. The VES-sf scores were moderately correlated with perceived video realism (<emph>r</emph> =.373-.414, <emph>p</emph> <.001; Table 7), suggesting adequate criterion validity.</p> <p>Table 6. Descriptive statistics and group comparisons based on background factors of the VES-sf.</p> <p> <ephtml> <table><thead><tr><td /><td /><td>sex</td><td /><td>education</td><td /><td>disease status</td><td /></tr><tr><td /><td /><td>female</td><td>male</td><td /><td>low</td><td>middle</td><td>high</td><td /><td>healthy</td><td>ill</td><td /></tr></thead><tbody><tr><td>SAMPLE II</td><td><italic>N</italic> = 864/ 878 <sup>a</sup></td><td><italic>n = </italic>637</td><td><italic>n = </italic>227</td><td /><td><italic>n = </italic>74</td><td><italic>n = </italic>320</td><td><italic>n = </italic>469</td><td /><td><italic>n = </italic>591</td><td><italic>n = </italic>273</td><td /></tr><tr><td /><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>t; p</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>F; p</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>t, p</italic></td></tr><tr><td>VES Total score</td><td>4.1 (1.5)</td><td>4.1 (1.5)</td><td>4.1 (1.4)</td><td>0.05;.963</td><td>4.6 (1.5)</td><td>4.0 (1.5)</td><td>4.1 (1.5)</td><td>4.69;.009 <sup>b</sup></td><td>4.1 (1.4)</td><td>4.2 (1.6)</td><td>1.52;.130</td></tr><tr><td>VES Immersion</td><td>3.7 (1.7)</td><td>3.7 (1.7)</td><td>3.9 (1.6)</td><td>1.38;.170</td><td>4.2 (1.8)</td><td>3.7 (1.7)</td><td>3.7 (1.6)</td><td>3.98;.019 <sup>b</sup></td><td>3.6 (1.6)</td><td>4.0 (1.8)</td><td>3.71; <.001 <sup>c</sup></td></tr><tr><td>VES Emotional Impact</td><td>4.5 (1.5)</td><td>4.5 (1.5)</td><td>4.4 (1.5)</td><td>1.43;.154</td><td>5.0 (1.5)</td><td>4.4 (1.5)</td><td>4.5 (1.5)</td><td>4.31;.014 <sup>b</sup></td><td>4.5 (1.4)</td><td>4.4 (1.6)</td><td>1.08;.280</td></tr><tr><td>SAMPLE III</td><td><italic>N</italic> = 917/ 931 <sup>d</sup></td><td><italic>n = 678</italic></td><td><italic>n = 239</italic></td><td /><td><italic>n = 74</italic></td><td><italic>n = 354</italic></td><td><italic>n = 487</italic></td><td /><td><italic>n = 631</italic></td><td><italic>n = 286</italic></td><td /></tr><tr><td /><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>t; p</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>F; p</italic></td><td><italic>M(SD)</italic></td><td><italic>M(SD)</italic></td><td><italic>t, p</italic></td></tr><tr><td>VES Total score</td><td>4.1 (1.4)</td><td>4.1 (1.4)</td><td>4.0 (1.4)</td><td>0.83;.407</td><td>4.7 (1.6)</td><td>4.1 (1.5)</td><td>4.0 (1.4)</td><td>7.70; <.001 <sup>f</sup></td><td>4.0 (1.4)</td><td>4.2 (1.6)</td><td>1.13;.260</td></tr><tr><td>VES Immersion</td><td>3.7 (1.7)</td><td>3.6 (1.7)</td><td>3.7 (1.6)</td><td>0.89;.377</td><td>4.4 (1.7)</td><td>3.9 (1.7)</td><td>3.5 (1.6)</td><td>10.60; <.001 <sup>f</sup></td><td>3.5 (1.6)</td><td>3.9 (1.7)</td><td>3.49; <.001 <sup>g</sup></td></tr><tr><td>VES Emotional Impact</td><td>4.5 (1.5)</td><td>4.6 (1.5)</td><td>4.3 (1.5)</td><td>2.65;.008 <sup>e</sup></td><td>4.9 (1.5)</td><td>4.4 (1.5)</td><td>4.5 (1.4)</td><td>3.72;.024 <sup>f</sup></td><td>4.6 (1.4)</td><td>4.4. (1.6)</td><td>1.71;.087</td></tr></tbody></table> </ephtml> </p> <p>16 <sups>a</sups><emph>n</emph> = 14 (1.6%) were excluded based on the screener item score ≤2; <sups>b</sups> post hoc analyses: lower educated participants always reported significantly higher scores (<emph>p</emph>s<.031; <emph>d</emph> = 0.29–0.40), middle and highly educated participants had comparable scores; <sups>c</sups><emph>d</emph> = 0.24; <sups>d</sups><emph>n</emph> = 14 (1.5%) were screened out; <sups>e</sups><emph>d</emph> = 0.20; <sups>f</sups> post hoc: lowest educated always significantly highest scores (<emph>p</emph>s<.022; <emph>d</emph> = 0.29–0.56), middle and highly educated participants reported comparable scores; <sups>g</sups><emph>d</emph> = 0.25</p> <p>Table 7. Pearson's correlations (<emph>r</emph>) of the VES-sf total score and subscales with age and video realism.</p> <p> <ephtml> <table><thead><tr><td /><td>Age</td><td>Perceived video realism</td></tr></thead><tbody><tr><td>SAMPLE II</td><td><italic>N</italic> = 864</td><td><italic>n</italic> = 336</td></tr><tr><td>VES Total score</td><td>−.110 ***</td><td>.413 ***</td></tr><tr><td>Immersion</td><td>−.127 ***</td><td>.414 ***</td></tr><tr><td>Emotional Impact</td><td>−.077 *</td><td>.373 ***</td></tr><tr><td>SAMPLE III</td><td><italic>N</italic> = 931</td><td><italic>n</italic> = 332</td></tr><tr><td>VES Total score</td><td>−.048</td><td>.368 ***</td></tr><tr><td>Immersion</td><td>−.089 **</td><td>.348 ***</td></tr><tr><td>Emotional Impact</td><td>.006</td><td>.352 ***</td></tr></tbody></table> </ephtml> </p> <ulist> <item>17 * <emph>p</emph> <.050; **<emph>p</emph> <.010; ***<emph>p</emph> <.001;</item> <item>18 Note that correlations with years since diagnosis were not significant</item> </ulist> <p>Differences in VES scores based on participants' background characteristics were tested and showed that lower educated participants reported significantly higher VES total and subscale scores (<emph>d</emph> = 0.29–0.40), whereas middle and highly educated participants reported similar scores (Table 6). Participants who were disease-naïve reported significantly lower scores on Immersion than (previously) ill participants, which constituted a small difference (<emph>d</emph> = 0.24, Table 6). No sex differences were identified, and associations with age were significant yet negligible (<emph>r</emph><-.13; Table 7). However, to substantiate whether such group comparisons yield valid results, the measurement invariance of the VES-sf was tested between groups in the third sample.</p> <hd id="AN0163282891-12">3.3 Sample III</hd> <p>First, the VES-sf was cross-validated, after 1.5% of participants (<emph>n</emph> = 14/931) were excluded based on the screener item. Although the model was overall still significant based on the <emph>χ<sups>2</sups></emph>-test, all model fit indices suggested acceptable (RMSEA =.069) to good model fit (CFI =.976, SRMR =.031; Table 5).</p> <p>To further substantiate group comparisons as reported above, multi-group confirmatory factor analyses (mCFA) were conducted to test measurement invariance of the VES-sf between subgroups, based on (a) level of education, (b) sex, and (c) disease status.</p> <p> <bold>(a) Level of education</bold>: Both metric (<emph>χ<sups>2</sups></emph>(<reflink idref="bib12" id="ref57">12</reflink>) = 14.73, <emph>p</emph> =.257) and scalar measurement invariance (<emph>χ<sups>2</sups></emph>(<reflink idref="bib12" id="ref58">12</reflink>) = 14.17, <emph>p</emph> =.290; Table 5) were demonstrated between participants with low, moderate, or high education by showing no significant differences in model fit.</p> <p> <bold>(b) Sex</bold>: Metric invariance was not fully demonstrated between men and women (<emph>Δχ<sups>2</sups></emph>(<reflink idref="bib6" id="ref59">6</reflink>) = 15.56, <emph>p</emph> =.016), because the factor loading of item #9 ('feeling sad/emotional') was unequal for both groups. The factor loading was higher among women, demonstrating that feelings of sadness contributed more strongly to the underlying subscale <emph>Emotional Impact</emph> than for men. However, relaxing the restriction on item #9 demonstrated partial metric invariance (<emph>Δχ<sups>2</sups></emph>(<reflink idref="bib5" id="ref60">5</reflink>) = 8.19, <emph>p</emph> =.146; <emph>Δ</emph>CFI =.003). Similarly, scalar invariance was only partially demonstrated (<emph>Δχ<sups>2</sups></emph>(<reflink idref="bib6" id="ref61">6</reflink>) = 16.94, <emph>p</emph> =.010), given unequal intercepts for item #8 ('submerge into video') for men and women. Men tended to score somewhat lower on this item, irrespective of their <emph>Immersion</emph> subscale score. We estimated scores for men and women before and after accounting for such difference, which showed that sex differences may be underestimated (i.e. smaller if not taken into account). Nevertheless, estimated sex differences without (<emph>d</emph> = 0.04) and after (<emph>d</emph> = 0.11) taking different intercepts into account indicated only a very small difference (<emph>dΔ </emph>= 0.07). Overall, relaxing two restrictions (i.e. factor loading item #9, intercept item #8) demonstrated partial measurement invariance (<emph>Δχ<sups>2</sups></emph>(<reflink idref="bib5" id="ref62">5</reflink>) = 8.78, <emph>p</emph> =.118, <emph>Δ</emph>CFI =.002; Table 5).</p> <p> <bold>(c) Disease status</bold>: Measurement invariance was not entirely supported between participants who were disease-naïve vs. (previously) ill. Several items showed unequal factor loadings (i.e. no metric invariance; <emph>Δχ<sups>2</sups></emph>(<reflink idref="bib6" id="ref63">6</reflink>) = 28.84, <emph>p</emph> <.001) or unequal intercepts (i.e. no scalar invariance; <emph>Δχ<sups>2</sups></emph>(<reflink idref="bib6" id="ref64">6</reflink>) = 139.28, <emph>p</emph> <.001) between both groups. Specifically, factor loadings for items #10 ('video was moving') and #14 ('becoming the patient') were higher in ill than in disease-naïve participants. Model fit improved somewhat if allowing both factor loadings to differ between groups (<emph>Δχ<sups>2</sups></emph>(<reflink idref="bib4" id="ref65">4</reflink>) = 8.54, <emph>p</emph> =.074, <emph>Δ</emph>CFI =.002; Table 5). Unequal intercepts were indicated for four out of 8 items (i.e. two from each subscale; #8, #10, #12, #15) demonstrating no scalar invariance. It appeared that (previously) ill participants tended to score lower on <emph>Immersion</emph> and higher on <emph>Emotional Impact</emph>. Estimating group differences without and after taking these differences in intercepts into account, indicated relatively small effects (<emph>Immersion: dΔ </emph>= 0.22; <emph>Emotional Impact: dΔ </emph>= 0.06). Thus, <emph>not</emph> accounting for such differences would typically result in somewhat overestimating differences between groups. Nevertheless, CFAs in both groups separately demonstrated acceptable to good fit, which supports the use of the VES-sf in both groups separately (CFIs>.96, RMSEAs<.09, SRMR<.03; Table 5).</p> <hd id="AN0163282891-13">4. Discussion</hd> <p>We thoroughly tested the measurement properties of the Video Engagement Scale (VES; Visser, Hillen et al., [<reflink idref="bib23" id="ref66">23</reflink>]) and suggest an improved shorter version (VES-sf), which measures two dimensions of engagement: Immersion (i.e. engagement with video, identification with video character) and Emotional Impact (i.e. emotional effects of watching a video vignette). Factorial and criterion validity along with very good reliability were demonstrated supporting the use of the VES-sf as a sound quality indicator in future experimental video-vignette research. The suitability of directly comparing VES-scores between male and female participants and between participants with different levels of education was generally indicated. However, if directly comparing healthy/disease-naïve versus (previously) ill participants, caution is advised as ill participants tend to experience a stronger emotional response and less immersion.</p> <p>Efforts are often extensive to craft experimental video vignettes to appear realistic and technically sound, but there are still various factors that may influence whether participants can immerse themselves in a video and be moved by it. Sometimes, even minor disruptions such as a phone call, the staging/set of the video, or behaviors of the actors could interfere with a participant's ability to engage with the vignette and therefore lessen the effects of manipulations in experimental video vignettes. This underlines the importance of measuring engagement to being able to draw valid conclusions about the effects of manipulations for which the vignettes were intended.</p> <p>The CFAs confirmed the poor factorial validity of the original 4 and 5 dimensions (Visser, Hillen et al., [<reflink idref="bib23" id="ref67">23</reflink>]). By shortening the VES, we present a more efficient and less burdensome quality indicator, which still contains items from all initial five theory-based dimensions (De Graaf et al., [<reflink idref="bib5" id="ref68">5</reflink>], [<reflink idref="bib6" id="ref69">6</reflink>]; Visser, Hillen et al., [<reflink idref="bib23" id="ref70">23</reflink>]). Future research may use the two subscale of the VES-sf based on their research questions. For example, studies emphasizing participants' emotional responses to a video vignette may use the Emotional Impact subscale, whereas others may focus on the Immersion subscale if they are interested in evaluations of the video itself, communication, information recall, or other (more cognitive) outcomes.</p> <p>Researchers are also advised to carefully choose their participants, based on their research question and ethical considerations. We demonstrated that Immersion and Emotional Impact weigh differently between healthy/disease-naïve participants versus those who are/were ill (i.e. measurement invariance was not demonstrated; but the 2-factor structure was confirmed in both groups separately). This implies two things, namely that disease status differently affects people's engagement with video vignettes and that VES-scores should not be readily compared between both groups. It may be speculated that disease-naïve participants can better immerse themselves because they are unlikely to have comparable personal experiences with the healthcare system that may conflict with the scripted video scenes. In contrast, ill participants may be more emotionally affected due to their illness experiences, which could be particularly emotional if including patients faced with life threatening diseases (e.g. cancer). When designing a video-vignette study, researchers could actually utilize such differences and select either group, depending on their research questions. Alternatively, if studies are intended to include both healthy and (previously) ill participants, researchers should ensure an appropriate randomization of both groups across all conditions to maximize variability. Overall, our findings encourage the use of the VES to assess engagement <emph>and</emph> to ensure the inclusion of engaged participants in analyses, but it remains to be tested whether different levels of engagement translate into differences in study outcomes in experimental video-vignette studies. Future research may also test whether different levels of engagement may mediate the effects of certain communication behaviors/vignette characteristics on study outcomes.</p> <p>Moreover, using the VES-sf to assess engagement can be particularly helpful in the phase of developing and validating realistic and believable video vignettes to ensure their high quality (e.g. staging, setting, scripts, acting, camera angles). This will also be important when drawing conclusions from experiments, for example, when assessing manipulation checks or (mis)matches between analogue and video patients. The use of the screener item (i.e. exclude participants with scores ≤2) may be of particular use. As demonstrated in Sample II and III, less than 2% of participants were excluded from further analyses based on the screener item. We argue that participants who report they were not at all concentrated on the video (irrespective their reason) would not render useful data and should be removed from further analyses to prevent invalid conclusions (e.g. findings may not be ascribed to video vignettes and their manipulation, but to other personal or environmental factors).</p> <p>The two-factor structure of the VES-sf was further demonstrated between participants with different levels of education and thus allowing direct comparisons of VES-scores between groups. Measurement invariance among men and women was partially established (i.e. different factor loadings item #9; different intercept item #8). This indicated that comparing VES scores between men and women may be underestimating actual sex differences, but results from our estimated means analyses reassured that such underestimation is negligible (<emph>dΔ </emph>= 0.07). Yet, researchers should also pay attention to randomizing equal portions of men and women to all experimental conditions to account for potential sex differences. Please also note that the item #9 was worded as 'feeling sad/emotional' in the included projects, but the VES was designed in such way that researchers can assess any emotion, depending on their video vignette (see Appendix 1). Hence, factor loadings and intercepts may act somewhat different if other emotions are assessed.</p> <p>This study thoroughly tested the measurement properties of the VES and a definite strength is our large sample size and stepwise analytical approach in cross-validating alternative structures of the VES. The combined data also included video vignettes of different healthcare settings (i.e. oncology, dementia/memory clinics, urology, orthopedics) and diverse analogue patients further broadening the scope and variability of our data. Nevertheless, we would like to highlight that the VES is the only quality indicator assessing engagement with video vignettes. Therefore, we did not have other measures serving as 'gold standard' to test criterion validity. As a close substitute, we tested correlations with perceived video realism, but that may still not indicate whether participants truly engaged with the videos. In the future, it may be an option to interview participants regarding their perceived engagement as a means to assess criterion validity. Moreover, participants were predominantly female (~70%) and additional background or clinical factors may be tested in relation to the VES-sf, which was not feasible here due to using data from seven existing projects. Highly educated participants were also overrepresented in this study, which is partly due to two projects with university students. Thus, although our findings suggest measurement invariance across educational levels, more research among lower educated and lower literate individuals is needed to bolster these findings. The included projects were exclusively conducted in the Netherlands, as the VES has been developed in 2016 in the Dutch language. Therefore, cultural differences could not be tested, but we encourage future research elsewhere and we thoroughly translated all items of the full and short VES into English by independent native speakers (see Appendix 1).</p> <hd id="AN0163282891-14">Conclusion & implication</hd> <p>We recommend the use of the shortened VES (VES-sf) as a quality indicator of engagement of participants in experimental video-vignette studies of all kinds (see English and Dutch version in Appendix 1). Thereby, the use of the screener item and two subscales, Immersion and Emotional Impact, is suggested. The sound measurement properties of the VES-sf were demonstrated and the appropriateness of comparing subgroups based on education or sex were shown. Using the VES-sf in disease-naïve and ill participants is feasible, but direct comparisons are not recommended. Instead, future research may strategically utilize either or both groups, depending on their research questions. Future studies should further test the effects of different levels of engagement on various study outcomes.</p> <hd id="AN0163282891-15">Disclosure statement</hd> <p>No potential conflict of interest was reported by the author(s).</p> <hd id="AN0163282891-16">Supplementary material</hd> <p>Supplemental data for this article can be accessed https://doi.org/10.1080/13645579.2022.2052697</p> <ref id="AN0163282891-17"> <title> References </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Blanch-Hartigan, D., Hall, J. A., Krupat, E., & Irish, J. T. (2013). Can naive viewers put themselves in the patients' shoes?: Reliability and validity of the analogue patient methodology. Medical Care, 51 (3), e16 – e21. https://doi.org/10.1097/MLR.0b013e31822945cc</bibtext> </blist> <blist> <bibl id="bib2" idref="ref38" type="bt">2</bibl> <bibtext> Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21 (2), 230 – 258. https://doi.org/10.1177/0049124192021002005</bibtext> </blist> <blist> <bibl id="bib3" idref="ref49" type="bt">3</bibl> <bibtext> Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9 (2), 233 – 255. https://doi.org/10.1207/S15328007SEM0902_5</bibtext> </blist> <blist> <bibl id="bib4" idref="ref47" type="bt">4</bibl> <bibtext> Cohen, J. (1992). A power primer. Psychological Bulletin, 112 (1), 155. https://doi.org/10.1037/0033-2909.112.1.155</bibtext> </blist> <blist> <bibl id="bib5" idref="ref11" type="bt">5</bibl> <bibtext> de Graaf, A., Hoeken, H., Sanders, J., & Beentjes, H. (2009). The role of dimensions of narrative engagement in narrative persuasion. Communications-European Journal of Communication Research, 34 (4), 385 – 405. https://doi.org/10.1515/Comm.2009.024</bibtext> </blist> <blist> <bibl id="bib6" idref="ref12" type="bt">6</bibl> <bibtext> de Graaf, A., Hoeken, H., Sanders, J., & Beentjes, J. W. J. (2012). Identification as a mechanism of narrative persuasion. Communication Research, 39 (6), 802 – 823. https://doi.org/10.1177/0093650211408594</bibtext> </blist> <blist> <bibl id="bib7" idref="ref20" type="bt">7</bibl> <bibtext> Fruijtier, A. D., van der Schaar, J., van Maurik, I. S., Zwan, M. D., Scheltens, P., Bouwman, F., Pijnenburg, Y. A. L., van Berckel, B. N. M., Ebenau, J., van der Flier, W. M., Smets, E. M. A., & Visser, L. N. C. (2022). Identifying best-practices for amyloid imaging results disclosure: A Randomized Controlled Trial. Alzheimer's & Dementia. [in press] https://doi.org/10.1002/alz.12630</bibtext> </blist> <blist> <bibl id="bib8" idref="ref6" type="bt">8</bibl> <bibtext> Gehenne, L., Christophe, V., Eveno, C., Carnot, A., Turpin, A., Pannier, D., Piessen, G., & Lelorain, S. (2021). Creating scripted video-vignettes in an experimental study on two empathic processes in oncology: Reflections on our experience. Patient Education and Counseling, 104 (3), 654 – 662. https://doi.org/10.1016/j.pec.2020.08.041</bibtext> </blist> <blist> <bibl id="bib9" idref="ref14" type="bt">9</bibl> <bibtext> Hillen, M. A., de Haes, H. C. J. M., van Tienhoven, G., Bijker, N., van Laarhoven, H. W. M., Vermeulen, D. M., & Smets, E. M. A. (2015). All eyes on the patient: The influence of oncologists' nonverbal communication on breast cancer patients' trust. Breast Cancer Research and Treatment, 153 (1), 161 – 171. https://doi.org/10.1007/s10549-015-3486-0</bibtext> </blist> <blist> <bibtext> Hillen, M. A., van Vliet, L. M., de Haes, H. C. J. M., & Smets, E. M. A. (2013). Developing and administering scripted video vignettes for experimental research of patient–provider communication. Patient Education and Counseling, 91 (3), 295 – 309. https://doi.org/10.1016/j.pec.2013.01.020</bibtext> </blist> <blist> <bibtext> Hooper, D., Coughlan, J., & Mullen, M. (2008). Evaluating model fit: A synthesis of the structural equation modelling literature. 7th European Conference on research methodology for business and management studies London, UK (Academic Publishing Limited).</bibtext> </blist> <blist> <bibtext> Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6 (1), 1 – 55. https://doi.org/10.1080/10705519909540118</bibtext> </blist> <blist> <bibtext> Jöreskog, K. G., & Sörbom, D. (1982). Recent developments in structural equation modeling. Journal of Marketing Research, 19 (4), 404 – 416. https://doi.org/10.1177/002224378201900402</bibtext> </blist> <blist> <bibtext> Lehmann, V., Labrie, N. H. M., van Weert, J. C. M., van Dulmen, S., de Haes, H., Kersten, M. J., Pieterse, A. H., & Smets, E. M. A. (2020). Provider caring and structuring treatment information to improve cancer patients' recall: Does it help? Patient Education and Counseling, 103 (1), 55 – 62. https://doi.org/10.1016/j.pec.2019.07.011</bibtext> </blist> <blist> <bibtext> Lehmann, V., Labrie, N. H. M., van Weert, J. C. M., van Dulmen, S., de Haes, H., Kersten, M. J., and Smets, E. M. A. (2019). Tailoring the amount of treatment information to cancer patients' and survivors' preferences: Effects on patient-reported outcomes. Patient Educ Couns 103 3 514 – 520. https://doi.org/10.1016/j.pec.2019.09.024</bibtext> </blist> <blist> <bibtext> Medendorp, N. M., Visser, L. N. C., Hillen, M. A., de Haes, J., & Smets, E. M. A. (2017). How oncologists' communication improves (analogue) patients' recall of information. A randomized video-vignettes study. Patient Education and Counseling, 100 (7), 1338 – 1344. https://doi.org/10.1016/j.pec.2017.02.012</bibtext> </blist> <blist> <bibtext> Nuffic: The Dutch Organisation for internationalisation in education. (2019). Education and diplomas in the Netherlands. https://<ulink href="http://www.nuffic.nl/en/subjects/education-and-diplomas-the-netherlands/">www.nuffic.nl/en/subjects/education-and-diplomas-the-netherlands/</ulink></bibtext> </blist> <blist> <bibtext> Pieterse, A. H., Brandes, K., de Graaf, J., de Boer, J. E., Labrie, N. H. M., Knops, A., ... Stiggelbout, A. M. (2021). Fostering patient choice awareness and presenting treatment options neutrally: A randomized trial to assess the effect on perceived room for involvement in decision making. Medical Decision Making, 0272989X211056334. https://doi.org/10.1177/0272989x211056334</bibtext> </blist> <blist> <bibtext> Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48 (2), 1 – 36. https://doi.org/10.18637/jss.v048.i02</bibtext> </blist> <blist> <bibtext> Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66 (4), 507 – 514. https://doi.org/10.1007/BF02296192</bibtext> </blist> <blist> <bibtext> Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8 (2), 23 – 74.</bibtext> </blist> <blist> <bibtext> Van Vliet, L. M., Van Der Wall, E., Albada, A., Spreeuwenberg, P. M., Verheul, W., & Bensing, J. M. (2012). The validity of using analogue patients in practitioner–patient communication research: Systematic review and meta-analysis. Journal of General Internal Medicine, 27 (11), 1528 – 1543. https://doi.org/10.1007/s11606-012-2111-8</bibtext> </blist> <blist> <bibtext> Visser, L. N., Hillen, M. A., Verdam, M. G., Bol, N., de Haes, H. C., & Smets, E. M. (2016). Assessing engagement while viewing video vignettes; validation of the Video Engagement Scale (VES). Patient Education and Counseling, 99 (2), 227 – 235. https://doi.org/10.1016/j.pec.2015.08.029</bibtext> </blist> <blist> <bibtext> Visser, L. N., Tollenaar, M. S., Bosch, J. A., van Doornen, L. J., de Haes, H. C., & Smets, E. M. (2016). Analogue patients' self-reported engagement and psychophysiological arousal in a video-vignettes design: Patients versus disease-naive individuals. Patient Education and Counseling, 99 (10), 1724 – 1732. https://doi.org/10.1016/j.pec.2016.04.012</bibtext> </blist> <blist> <bibtext> Visser, L. N., Tollenaar, M. S., van Doornen, L. J. P., de Haes, H., & Smets, E. M. A. (2019). Does silence speak louder than words? The impact of oncologists' emotion-oriented communication on analogue patients' information recall and emotional stress. Patient Education and Counseling, 102 (1), 43 – 52. https://doi.org/10.1016/j.pec.2018.08.032</bibtext> </blist> <blist> <bibtext> Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44 (3), 219 – 246. https://doi.org/10.1177/0095798418771807</bibtext> </blist> <blist> <bibtext> Yuan, K.-H., & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30 (1), 165 – 200. https://doi.org/10.1111/0081-1750.00078</bibtext> </blist> </ref> <aug> <p>By Vicky Lehmann; Marij A. Hillen; Mathilde G. E. Verdam; Arwen H. Pieterse; Nanon H.M. Labrie; Agnetha D. Fruijtier; Tom H. Oreel; Ellen M.A. Smets and Leonie N.C. Visser</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib22" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib24" firstref="ref9"></nolink> <nolink nlid="nl4" bibid="bib23" firstref="ref10"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref15"></nolink> <nolink nlid="nl6" bibid="bib16" firstref="ref16"></nolink> <nolink nlid="nl7" bibid="bib25" firstref="ref17"></nolink> <nolink nlid="nl8" bibid="bib15" firstref="ref22"></nolink> <nolink nlid="nl9" bibid="bib18" firstref="ref24"></nolink> <nolink nlid="nl10" bibid="bib17" firstref="ref25"></nolink> <nolink nlid="nl11" bibid="bib19" firstref="ref35"></nolink> <nolink nlid="nl12" bibid="bib27" firstref="ref36"></nolink> <nolink nlid="nl13" bibid="bib12" firstref="ref37"></nolink> <nolink nlid="nl14" bibid="bib11" firstref="ref39"></nolink> <nolink nlid="nl15" bibid="bib13" firstref="ref40"></nolink> <nolink nlid="nl16" bibid="bib21" firstref="ref41"></nolink> <nolink nlid="nl17" bibid="bib26" firstref="ref46"></nolink> <nolink nlid="nl18" bibid="bib20" firstref="ref48"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1387392
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Video Engagement Scale (VES): Measurement Properties of the Full and Shortened VES across Studies
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Lehmann%2C+Vicky%22">Lehmann, Vicky</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-4898-3176">0000-0003-4898-3176</externalLink>)<br /><searchLink fieldCode="AR" term="%22Hillen%2C+Marij+A%2E%22">Hillen, Marij A.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-5724-7934">0000-0001-5724-7934</externalLink>)<br /><searchLink fieldCode="AR" term="%22Verdam%2C+Mathilde+G%2E+E%2E%22">Verdam, Mathilde G. E.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-2479-8817">0000-0002-2479-8817</externalLink>)<br /><searchLink fieldCode="AR" term="%22Pieterse%2C+Arwen+H%2E%22">Pieterse, Arwen H.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-6395-0052">0000-0001-6395-0052</externalLink>)<br /><searchLink fieldCode="AR" term="%22Labrie%2C+Nanon+H%2E+M%2E%22">Labrie, Nanon H. M.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-5483-0152">0000-0002-5483-0152</externalLink>)<br /><searchLink fieldCode="AR" term="%22Fruijtier%2C+Agnetha+D%2E%22">Fruijtier, Agnetha D.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-0434-0101">0000-0003-0434-0101</externalLink>)<br /><searchLink fieldCode="AR" term="%22Oreel%2C+Tom+H%2E%22">Oreel, Tom H.</searchLink><br /><searchLink fieldCode="AR" term="%22Smets%2C+Ellen+M%2E+A%2E%22">Smets, Ellen M. A.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-8145-8595">0000-0002-8145-8595</externalLink>)<br /><searchLink fieldCode="AR" term="%22Visser%2C+Leonie+N%2E+C%2E%22">Visser, Leonie N. C.</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-3487-7938">0000-0003-3487-7938</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Social+Research+Methodology%22"><i>International Journal of Social Research Methodology</i></searchLink>. 2023 26(3):305-318.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Likert+Scales%22">Likert Scales</searchLink><br /><searchLink fieldCode="DE" term="%22Video+Technology%22">Video Technology</searchLink><br /><searchLink fieldCode="DE" term="%22Vignettes%22">Vignettes</searchLink><br /><searchLink fieldCode="DE" term="%22Construct+Validity%22">Construct Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Factor+Analysis%22">Factor Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Factor+Structure%22">Factor Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Gender+Differences%22">Gender Differences</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Attainment%22">Educational Attainment</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Psychological+Patterns%22">Psychological Patterns</searchLink><br /><searchLink fieldCode="DE" term="%22Diseases%22">Diseases</searchLink><br /><searchLink fieldCode="DE" term="%22Health%22">Health</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Netherlands%22">Netherlands</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/13645579.2022.2052697
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1364-5579<br />1464-5300
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The Video Engagement Scale (VES) is a quality indicator to assess engagement in experimental video-vignette studies, but its measurement properties warrant improvement. Data from previous studies were combined (N = 2676) and split into three subsamples for a stepped analytical approach. We tested construct validity, criterion validity, reliability, and measurement invariance. Confirmatory factor analysis (CFA) reiterated poor fit of the previously suggested subscales. An alternative 2-factor structure is presented, which was cross-validated and further shortened (VES-sf). Reliability of both subscales ("Immersion," "Emotional Impact") was very high ([alpha] > 0.86). Multi-group CFAs demonstrated partial and complete measurement invariance among male/female and differently educated participants respectively. Invariance between healthy versus ill participants was not established, but CFAs supported the VES-sf structure in both groups separately. To conclude, we provide the VES-sf with sound measurement properties and demonstrated the appropriateness of comparing certain subgroups. Overall, we recommend the VES-sf to assess engagement/ecological validity in video-vignette research.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1387392
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1387392
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/13645579.2022.2052697
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 305
    Subjects:
      – SubjectFull: Likert Scales
        Type: general
      – SubjectFull: Video Technology
        Type: general
      – SubjectFull: Vignettes
        Type: general
      – SubjectFull: Construct Validity
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
      – SubjectFull: Factor Analysis
        Type: general
      – SubjectFull: Factor Structure
        Type: general
      – SubjectFull: Gender Differences
        Type: general
      – SubjectFull: Educational Attainment
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Psychological Patterns
        Type: general
      – SubjectFull: Diseases
        Type: general
      – SubjectFull: Health
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Netherlands
        Type: general
    Titles:
      – TitleFull: The Video Engagement Scale (VES): Measurement Properties of the Full and Shortened VES across Studies
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Lehmann, Vicky
      – PersonEntity:
          Name:
            NameFull: Hillen, Marij A.
      – PersonEntity:
          Name:
            NameFull: Verdam, Mathilde G. E.
      – PersonEntity:
          Name:
            NameFull: Pieterse, Arwen H.
      – PersonEntity:
          Name:
            NameFull: Labrie, Nanon H. M.
      – PersonEntity:
          Name:
            NameFull: Fruijtier, Agnetha D.
      – PersonEntity:
          Name:
            NameFull: Oreel, Tom H.
      – PersonEntity:
          Name:
            NameFull: Smets, Ellen M. A.
      – PersonEntity:
          Name:
            NameFull: Visser, Leonie N. C.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 1364-5579
            – Type: issn-electronic
              Value: 1464-5300
          Numbering:
            – Type: volume
              Value: 26
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: International Journal of Social Research Methodology
              Type: main
ResultId 1