View in EDS HTML Full Text PDF Full Text

Decision-Making Efficiency with Aided Information: The Impact of Automation Reliability and Task Difficulty

Saved in:
Bibliographic Details
Title: Decision-Making Efficiency with Aided Information: The Impact of Automation Reliability and Task Difficulty
Language: English
Authors: Hanshu Zhang (ORCID 0000-0002-0273-3254), Ran Zhou, Cheng-You Cheng, Sheng-Hsu Huang, Ming-Hui Cheng, Cheng-Ta Yang
Source: Cognitive Research: Principles and Implications. 2025 10.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 18
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Task Analysis, Difficulty Level, Decision Making, Automation, Reliability, Cues, Classification, Validity, Accuracy, Cognitive Processes, Computer Software
DOI: 10.1186/s41235-025-00659-w
ISSN: 2365-7464
Abstract: Although it is commonly believed that automation aids human decision-making, conflicting evidence raises questions about whether individuals would gain greater advantages from automation in difficult tasks. Our study examines the combined influence of task difficulty and automation reliability on aided decision-making. We assessed decision efficiency by employing the single-target self-terminating (STST) capacity coefficient in Systems Factorial Technology, estimating the ratio of performance with aided information to that without it. Participants were instructed to perform a shape categorization task, wherein they assessed whether the presented stimulus belonged to one category or another. In Experiment 1, three automation reliability conditions (high reliability, low reliability, and unaided) were tested in separate blocks. Our results indicated that, in general, participants exhibited unlimited capacity when provided with valid automated cues, implying that the decision efficiency was unaltered by automated assistance. Despite the failure to gain extra efficiency, the benefits of automated aids in decision-making for difficult tasks were evident. In Experiment 2, various types of automation reliability were randomly intermixed. In this scenario, the impact of automation reliability on participants' performance diminished; however, the significance of information accuracy increased. Our study illustrates how the presentation of automation, its reliability, and task difficulty interactively influence participants' processing of automated information for decision-making. Our study may improve processing efficiency in automated systems, hence facilitating superior interface design and automation execution.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1478691
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFe8cXIsZSaMkLXudaemNTkAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDODdY3l_Y143NA2u1wIBEICBm4ePf2YxMVxyohQQgVy7L05UigfOaBZw94YJ-XWPAqTqIizTtuEdhJM7f14favt5sQcanHzrQxpS_03rtuAN4jewo_VnMATqlaeL8G3nd0ZQgOn5BGqggGasFLzRRHBJxii7ntCumDOhZfVOwVFFnil9VWkMEdjHhBD9g1l7XoAuDdg58qvGm_krfpPw0edVH4qo8F6v4mbyViQo
Text:
  Availability: 1
  Value: <anid>AN0187092980;[k1e6]30jul.25;2025Aug05.03:34;v2.2.500</anid> <title id="AN0187092980-1">Decision-making efficiency with aided information: the impact of automation reliability and task difficulty </title> <p>Although it is commonly believed that automation aids human decision-making, conflicting evidence raises questions about whether individuals would gain greater advantages from automation in difficult tasks. Our study examines the combined influence of task difficulty and automation reliability on aided decision-making. We assessed decision efficiency by employing the single-target self-terminating (STST) capacity coefficient in Systems Factorial Technology, estimating the ratio of performance with aided information to that without it. Participants were instructed to perform a shape categorization task, wherein they assessed whether the presented stimulus belonged to one category or another. In Experiment 1, three automation reliability conditions (high reliability, low reliability, and unaided) were tested in separate blocks. Our results indicated that, in general, participants exhibited unlimited capacity when provided with valid automated cues, implying that the decision efficiency was unaltered by automated assistance. Despite the failure to gain extra efficiency, the benefits of automated aids in decision-making for difficult tasks were evident. In Experiment 2, various types of automation reliability were randomly intermixed. In this scenario, the impact of automation reliability on participants' performance diminished; however, the significance of information accuracy increased. Our study illustrates how the presentation of automation, its reliability, and task difficulty interactively influence participants' processing of automated information for decision-making. Our study may improve processing efficiency in automated systems, hence facilitating superior interface design and automation execution.</p> <p>Keywords: Aided decision-making; Automation reliability; Decision efficiency; Systems factorial technology; Psychology and Cognitive Sciences Psychology</p> <hd id="AN0187092980-2">Significance statement</hd> <p>Decision makers engage with automation in intricate systems. Nevertheless, findings regarding the benefits of automated information in difficult tasks have been inconsistent. Our study explores whether these discrepancies arise from the ambiguous information on automation reliability. Using the single-target self-terminating (STST) capacity coefficient—a theory-driven tool for assessing decision efficiency—we evaluate the processing efficiency of unaided versus aided decision-making. The impact of task difficulty is evident when automation enables decision makers to gather reliable information effectively. However, this effect diminishes when the information regarding automation reliability is difficult to accumulate. Our findings indicate a potential synergy between task difficulty and automation reliability, which may enhance decision efficiency with automated assistance. This contributes to a deeper understanding of human-automation collaboration and improves interface designs in automated systems.</p> <hd id="AN0187092980-3">Introduction</hd> <p>Humans are required to interact with automation in complex and complicated systems (Sheridan & Parasuraman, [<reflink idref="bib45" id="ref1">45</reflink>]). Comprehending the factors that affect the effectiveness of interaction, therefore, is essential for enhancing human-automation collaboration. Nevertheless, research remains ambiguous about the extent to which individuals gain benefits from automation, especially in the context of difficult tasks. Our current study examines the interaction between task complexity and automation reliability on decision efficiency, employing Systems Factorial Technology (hereafter SFT; Little et al., [<reflink idref="bib28" id="ref2">28</reflink>]; Townsend & Nozawa, [<reflink idref="bib46" id="ref3">46</reflink>]) for a detailed analysis of automated aids' effect on human decision-making processes. We aim to elucidate how decision makers react to differing degrees of automation reliability across various task situations.</p> <hd id="AN0187092980-4">Task difficulty and aided performance</hd> <p>While automation is often seen as beneficial for decision makers in both easy and difficult tasks (e.g., Chen et al., [<reflink idref="bib6" id="ref4">6</reflink>]), there is little agreement on its actual benefits in challenging scenarios. Some studies suggest that automation enhances performance in difficult tasks (Xu et al., [<reflink idref="bib52" id="ref5">52</reflink>]), while others observe a decline in problem-solving capabilities when automation is introduced (Bowers et al., [<reflink idref="bib4" id="ref6">4</reflink>]). Using workload capacity analysis within the framework of SFT, Kneeland et al. ([<reflink idref="bib24" id="ref7">24</reflink>]) found no evidence that decision makers performed better with combined automated support under varying task difficulty conditions. This finding introduces a third perspective: Task difficulty may not influence the efficiency of aided decision-making.</p> <p>Regardless of whether automation consistently yields performance benefits, previous research has often observed an increased reliance on automation in difficult tasks. This trend is partly attributed to the cognitive workload being offloaded by the presence of automated support. For instance, participants demonstrated higher accuracy performance when aided by highly reliable automation, especially under conditions of high cognitive load (Gamble et al., [<reflink idref="bib14" id="ref8">14</reflink>]). Similarly, reliance on automation increased under high time pressure (Hoff & Bashir, [<reflink idref="bib18" id="ref9">18</reflink>]). However, as previously discussed, increased reliance does not necessarily translate to improved performance. Rieger and Manzey ([<reflink idref="bib40" id="ref10">40</reflink>]) found that while aided information helped mitigate some negative effects of time pressure, joint human-automation performance still remained inferior to automation-only performance.</p> <p>The decision to rely on automation is further shaped by numerous other factors. Decision makers who are confident in their abilities may mistakenly believe that they do not need automated support for difficult tasks (Patton, [<reflink idref="bib38" id="ref11">38</reflink>]). This confidence trade-off can also reveal their belief in the task's importance. Schwark et al. ([<reflink idref="bib44" id="ref12">44</reflink>]) found that in visual search tasks, participants were more likely to use automated aids when tasks were explicitly labeled as difficult, regardless of the actual difficulty level. However, participants tended to rely more on their abilities when tasks labeled as difficult were associated with higher rewards, underscoring the influence of perceived importance on the utilization of automation.</p> <hd id="AN0187092980-5">Trust toward the imperfect automation</hd> <p>Among the factors influencing the willingness to utilize automation, the most extensively discussed was automation reliability. Reliability is a critical sub-component of trust (Hoff & Bashir, [<reflink idref="bib18" id="ref13">18</reflink>]; Lee & Moray, [<reflink idref="bib27" id="ref14">27</reflink>]; Schaefer et al., [<reflink idref="bib43" id="ref15">43</reflink>]), and automated aids that demonstrate high reliability are more likely to be perceived as trustworthy. Decision makers who utilize highly reliable automation exhibited superior accuracy and response time performance compared to those using less reliable ones (Gamble et al., [<reflink idref="bib14" id="ref16">14</reflink>]). By contrast, poor automation can even worsen existing problems rather than alleviate them. Metzger and Parasuraman ([<reflink idref="bib33" id="ref17">33</reflink>]) found that decision makers' performance with aids was even worse than in the unaided condition. Despite these potential drawbacks, decision makers often choose to rely on flawed automation to preserve cognitive resources for other tasks. Indeed, when reliability falls below a critical threshold (in the range of 70–75%), automation may lead to increased errors and inefficiencies, perhaps yielding inferior performance compared to the absence of automation (Wickens & Dixon, [<reflink idref="bib48" id="ref18">48</reflink>]).</p> <p>In decision-making contexts, decision makers often underestimate the actual reliability of imperfectly reliable automation (Chen et al., [<reflink idref="bib6" id="ref19">6</reflink>]; Wiegmann et al., [<reflink idref="bib50" id="ref20">50</reflink>]). This tendency may stem from a "perfect automation" schema, wherein decision makers expect flawless performance. As a result, automation errors can disproportionately influence decision makers' estimates of aid reliability and reduce their willingness to rely on automation—despite evidence that aided diagnosis may outperform unaided diagnosis (Wiegmann et al., [<reflink idref="bib50" id="ref21">50</reflink>]). In a simulated warehouse management task, Barg-Walkow and Rogers ([<reflink idref="bib1" id="ref22">1</reflink>]) examined perceptions of automation reliability. Participants were told that the automation was either 60%, 75%, or 90% reliable, although all groups experienced the same actual reliability of 75%. Participants in the 60% and 75% conditions underestimated the automation's reliability, while those in the 90% condition provided more accurate estimations. Similarly, Pop et al. ([<reflink idref="bib39" id="ref23">39</reflink>]) investigated perception of automation reliability in a simulated X-ray task, manipulating initial and subsequent reliability levels (100–80%, 60–80%, or 80–80%). The results indicated that participants adjusted their perceived reliability in response to changes in the actual performance of automation, with decision makers who had high expectations of automation's reliability being more sensitive to changes. Interestingly, contrary to the assumption that a more accurate perception of automation reliability should lead to better performance and increased error detection, Merritt et al. ([<reflink idref="bib32" id="ref24">32</reflink>]) found no evidence that decision makers' adjustments in perceived reliability and trust in response to changes in the aid's actual reliability over time significantly predicted task performance or the ability to identify automation failures.</p> <p>In addition to the underestimation of automation's reliability, self-confidence, which often varies across tasks, is a particularly influential factor in shaping trust formation (Lee & See, [<reflink idref="bib26" id="ref25">26</reflink>]). Decision makers frequently hold an inflated perception of their competence, especially when tasks appear easy, leading them to overestimate their capabilities (Dunning et al., [<reflink idref="bib10" id="ref26">10</reflink>]). This overconfidence can bias individual toward favoring their capabilities over automated support. For example, Kantowitz et al. ([<reflink idref="bib23" id="ref27">23</reflink>]) found that when self-confidence exceeds trust in automation, decision makers were more likely to disregard automated advice—particularly in familiar contexts—and rely instead on their own decisions. Similarly, De Vries et al. ([<reflink idref="bib8" id="ref28">8</reflink>]) also found that the difference between measures of trust and self-confidence was highly predictive of control allocation. In addition, their results also suggested a fundamental bias to trust one's own abilities over those of the system. Notably, excessive reliance on automation also presents challenges. Highly reliable automation can lead to "complacency," where decision makers neglect to monitor the information provided (Parasuraman & Wickens, [<reflink idref="bib36" id="ref29">36</reflink>]). The risks associated with complacency become more pronounced as automation reliability increases, as failures in highly reliable systems can result in more severe consequences (e.g., Rovira et al., [<reflink idref="bib42" id="ref30">42</reflink>]), underscoring the delicate balance required in managing trust and reliance on automation.</p> <hd id="AN0187092980-6">The present study</hd> <p>The (mis)perception of task difficulty or the reliability of automation can lead to critical issues that maladaptively affect goal achievement during the decision-making process. Previous research has suggested that increasing task difficulty would amplify the influence of automation reliability (Wickens, [<reflink idref="bib49" id="ref31">49</reflink>]). Reliance on automation is more likely in situations where the automation is highly reliable and manual operation is effortful or cognitively demanding (Dzindolet et al., [<reflink idref="bib11" id="ref32">11</reflink>]; Wickens, [<reflink idref="bib47" id="ref33">47</reflink>]). Conversely, automation failures on easy tasks tend to have a more pronounced negative impact on trust and reliance, as such errors are more noticeable compared to difficulty tasks (Dzindolet et al., [<reflink idref="bib11" id="ref34">11</reflink>]; Madhavan et al., [<reflink idref="bib30" id="ref35">30</reflink>]).</p> <p>Our research investigates the interplay between the task difficulty and automation reliability and their impact on processing efficiency in aided decision-making. Specifically, the automation is presented as an aided decision support system that provides recommended answers to decision makers (e.g., Rieger & Manzey, [<reflink idref="bib40" id="ref36">40</reflink>]). In Experiment 1, participants were instructed to perform a shape categorization task where the automation in high and low reliability provided aided information in the easy and difficult tasks. We hypothesize that the judgments with greater task difficulty will encourage decision makers to rely more on automation, and thereby, the automation reliability has a larger impact on decision-making efficiency. In contrast, for easy tasks, automation reliability may have minimal influence on its utilization.</p> <p>To further examine the role of perceived automation reliability, Experiment 2 introduced three randomly presented automations with varying reliability levels, whereas the task difficulty was in a block design as in Experiment 1. We predict that when the automation reliability is difficult to assess, the impact of task difficulty on automation usage will become more pronounced. Following the methodology of Kneeland et al. ([<reflink idref="bib24" id="ref37">24</reflink>]), our current research employs capacity analysis in SFT to compare decision-making performance with automated assistance to a null model in which decision makers performed tasks without aided information.</p> <hd id="AN0187092980-7">Experiment 1</hd> <p></p> <hd id="AN0187092980-8">Methods</hd> <p></p> <hd id="AN0187092980-9">Participants</hd> <p>Thirty-six university students (Ages [<reflink idref="bib18" id="ref38">18</reflink>, 28], <emph>N</emph><subs>female</subs> = 19) from National Cheng Kung University participated in the experiment. Participants signed the informed consent before the start of the experiment. The study was approved by the Human Research Ethics Committee of National Cheng Kung University (Protocol code: NCKU HREC-E109-257-2). Participants received NT$528 (approximately USD 16.5) as reimbursement for their total 3-h participation in the study.</p> <hd id="AN0187092980-10">Experimental apparatus</hd> <p>The experiment was conducted using a computer equipped with an Intel Core i7-8700 CPU running at 3.20 GHz. The experiment was programmed using MATLAB (MathWorks Inc.) and Psychtoolbox (<ulink href="http://psychtoolbox.org/">http://psychtoolbox.org/</ulink>), running on a 19-inch LED monitor with a screen resolution of 1920 × 1080 pixels and a refresh rate of 75 Hz.</p> <hd id="AN0187092980-11">Design, stimulus, and procedure</hd> <p>Figure 1 depicts stimuli employed in the study that the stimulus transient from an ellipse-like shape to a circle (subtended visual angle from left to right: 0.53° × 0.72°, 0.51° × 0.75°, 0.48° × 0.79°, 0.46° × 0.81°), in which the left two stimuli were arbitrarily categorized as category A and the right two stimuli as category B. In addition, the middle of the two stimuli was considered more difficult to categorize.</p> <p>Graph: Fig. 1 The stimuli in the study were in combinations of different categories and difficulty levels. From left to right: Category B (easy), Category B (difficult), Category A (difficult), Category A (easy)</p> <p>Participants were instructed to decide as quickly as possible the category of the presented stimuli. In addition to the presented stimuli, participants were also provided with the automation suggestion (explained further below) when they were in the condition of the automated aid. In the <emph>high-reliability condition</emph>, the overall accuracy for the automation was 82%, and in the <emph>low-reliability condition</emph>, the overall accuracy for the automation was 50%, represented by different automation suggestion displays. That is, given a stimulus A, 82 out of 100 trials that the high-reliability automation provided valid suggestions, whereas 50 out of 100 trials that the low-reliability automation provided valid suggestions. A detailed explanation can be found in the Appendix. This resulted in a 2 (task difficulty: easy, difficult) × 3 (automation reliability: high, low, unaided) within-subject design.</p> <p>The experiment is structured into three sessions, each focusing on a different automation reliability condition, with each session lasting approximately one hour. The test order was counterbalanced across participants. Figure 2 depicts a trial for the categorization task. Each trial began with a blank screen, followed by a fixation cross displayed at the center of the screen, ranging from 500 to 1000 ms. Then the test stimulus was presented for participants to make a judgment for 2000 ms or until a response. Participants were instructed to press "J" if the presented stimulus was (more closely resembled) a circle (category A) or "F" if the stimulus was (closely resembled) an ellipse (category B) as quickly and accurately as possible. During the response period in sessions featuring high or low-reliability automated aids, an automated aid (displayed as an "avatar") appeared above the stimulus, with each type of reliability represented by a different avatar, offering classification suggestions to the participants. Participants received feedback after each trial.[<reflink idref="bib1" id="ref39">1</reflink>] Each session included 400 trials, randomized across difficult and easy tasks. In total, each participant accomplished 1200 trials for the whole experiment.</p> <p>Graph: Fig. 2 The procedure of a test trial. In the automated presented session, participants were also given the aids' suggestions</p> <hd id="AN0187092980-12">Capacity coefficient</hd> <p>In the current research, we applied workload capacity within the framework of systems factorial technology (SFT; Townsend & Nozawa, [<reflink idref="bib46" id="ref40">46</reflink>]) to address the shortcomings of the traditional efficiency indicators that are merely implied by response times or accuracy. Workload capacity refers to the changes in decision efficiency as a function of workload, which provides a suite of response-based distributional measures for assessing changes in information processing efficiency when the aided information is provided. The single-target self-terminating (<emph>C</emph><subs><emph>STST</emph></subs>, STST) stopping rule (Blaha et al., Under Review), a measurement for workload capacity, is calculated as a ratio:</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mrow><mi mathvariant="italic">STST</mi></mrow></msub><mfenced close=")" open="("><mi>t</mi></mfenced><mo>=</mo><mfrac><mrow><msub><mi>K</mi><mrow><mi mathvariant="italic">unaided</mi></mrow></msub><mfenced close=")" open="("><mi>t</mi></mfenced></mrow><mrow><msub><mi>K</mi><mrow><mi mathvariant="italic">aided</mi></mrow></msub><mfenced close=")" open="("><mi>t</mi></mfenced></mrow></mfrac></mrow></math> </ephtml> </p> <p>Graph</p> <p>where <emph>K</emph> represents the cumulative reverse hazard function at time <emph>t,</emph> from both the aided and unaided trials. The cumulative reverse hazard function is represented by the natural logarithm of the cumulative distribution function (CDF), denoted as In[<emph>F</emph>(t)] = <emph>K</emph>(t), where <emph>F</emph>(t) stands for the CDF of response times. In this way, the ratio presents participants' performance with the aided information compared to the unaided trials, during which participants have the option to either take into account the added information or disregard it during aided trials. The ratio was compared against a null model, which assumed the predicted performance based on an unlimited-capacity, parallel, and independent (UCIP) model (Townsend & Nozawa, [<reflink idref="bib46" id="ref41">46</reflink>]). The <emph>C</emph><subs><emph>STST</emph></subs> < 1 indicates participants of limited capacity, referring to that the decision efficiency declines when the automated aids are introduced. The super capacity,<emph> C</emph><subs><emph>STST</emph></subs> > 1, suggests that the presence of automated aids enhances decision efficiency. The <emph>C</emph><subs><emph>STST</emph></subs> = 1, on the other hand, suggests that there is no performance decrement or increment with the automated aids.</p> <hd id="AN0187092980-13">Results</hd> <p>The goal of our current study was to examine the performance of participants when presented with and without the automated aids' suggestions. However, a consistently low accuracy when the high-reliability aids produced incorrect responses (i.e., invalid cues) also indicated that participants did not actively engage in the task. In this way, four participants were excluded from further analysis because their accuracy was below 0.5 when high-reliability aids presented invalid cues for the difficult task. Additionally, trials with responses faster than 100 ms or slower than 1500 ms (approximately 0.88% of total trials) were excluded for further analysis. The behavior analysis in response times and mean accuracy was conducted in ANOVA using JASP (JASP Team, [<reflink idref="bib22" id="ref42">22</reflink>]). We use Greenhouse–Geisser for sphericity correction and Bonferroni correction for the post hoc tests. The SFT package (Houpt et al., [<reflink idref="bib21" id="ref43">21</reflink>]) in R was used for STST analysis.</p> <hd id="AN0187092980-14">RT</hd> <p>We first examined the influence of automation conditions (unaided, high reliability, low reliability) and task difficulty (difficult, easy) on the response times of correct responses. The repeated measures ANOVA (Fig. 3) indicated that there was a main effect of the automation condition (<emph>F</emph> (<reflink idref="bib2" id="ref44">2</reflink>,<reflink idref="bib62" id="ref45">62</reflink>) = 4.19, <emph>p</emph> =.02, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>12), indicating that the unaided condition led to significantly faster responses compared to the low-reliability-aided condition (<emph>t</emph> = 2.79, <emph>p</emph> =.02, Cohen's <emph>d</emph> = 0.29). There was no statistical significance between the responses in the high-reliability-aided condition and low-reliability-aided condition (<emph>t</emph> = 2.07, <emph>p</emph> =.13, <emph>d</emph> = 0.22), as well as the unaided condition (<emph>t</emph> = 0.72, <emph>p</emph> = 1, <emph>d</emph> = 0.08). In addition, there was a main effect of the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref46">1</reflink>,<reflink idref="bib31" id="ref47">31</reflink>) = 116.86, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>79), participants responded faster to the easy task compared to the difficult task (<emph>t</emph> = 10.81, <emph>p</emph> <.001, <emph>d</emph> = 0.95). The results also indicated that there was an interaction between the two factors (<emph>F</emph> (<reflink idref="bib2" id="ref48">2</reflink>,<reflink idref="bib62" id="ref49">62</reflink>) = 3.75, <emph>p</emph> =.03, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> =.11): The reaction time (RT) discrepancy between the high-reliability automated condition and the unaided condition in difficult tasks (<emph>M</emph><subs><emph>diff</emph></subs> = -1.00, 95%CI [ -31.51, 29.52])) was notably smaller compared to the difference observed in easy tasks (<emph>M</emph><subs><emph>diff</emph></subs> = 14.84, 95%CI [-15.67, 45.36]). This suggests that decision makers responded even more quickly on their own in easier tasks.</p> <p>Graph: Fig. 3 Mean response times across conditions of task difficulty (difficult, easy) and automation reliability (high, low, unaided) in Experiment 1. Error bars represent ± one standard-error</p> <p>Considering that automation reliability could lead to distinct attentional allocations, an additional categorization for the automated session was implemented based on whether the cue was valid to examine its impact on the participants' response times. The repeated measures ANOVA (Fig. 4) on automation reliability (high, low), task difficulty (difficult, easy), and cue validity (valid, invalid) suggested that there were main effects of task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref50">1</reflink>,<reflink idref="bib31" id="ref51">31</reflink>) = 100.53, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>76) and cue validity (<emph>F</emph> (<reflink idref="bib1" id="ref52">1</reflink>,<reflink idref="bib31" id="ref53">31</reflink>) = 37.24, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>55): The difficult task resulted in slower responses (<emph>t</emph> = 10.03, <emph>p</emph> <.001, <emph>d</emph> = 0.88) compared to the easy task, and the slower responses were also exacerbated by invalid cues compared to valid cues (<emph>t</emph> = 6.10, <emph>p</emph> <.001, <emph>d</emph> = 0.40).</p> <p>Graph: Fig. 4 Mean response times across conditions of cue validity (valid, invalid) and task difficulty (difficult, easy) for different automation reliability (high, low) types in Experiment 1. Error bars represent ± one standard-error</p> <p>The results indicated that there was also a three-way interaction (<emph>F</emph> (<reflink idref="bib1" id="ref54">1</reflink>,<reflink idref="bib31" id="ref55">31</reflink>) = 6.34, <emph>p</emph> =.017, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>17). To further investigate the three-way interaction, the two-way repeated measures ANOVA on task difficulty and cue validity was conducted for the high- and low-reliability conditions, respectively. When the automation reliability was high, there was an interaction between the cue validity and task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref56">1</reflink>,<reflink idref="bib31" id="ref57">31</reflink>) = 7.36, <emph>p</emph> =.01, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>19), resulting from a larger RT difference between the valid cues and invalid cues when the task was difficult (<emph>M</emph><subs><emph>diff</emph></subs> = 77.02, 95%CI [42.56, 111.48]) compared to when the task was easy (<emph>M</emph><subs><emph>diff</emph></subs> = 49.84, 95%CI [15.38, 84.30]). However, such an interaction effect was not significant when the automation reliability was low (<emph>F</emph> (<reflink idref="bib1" id="ref58">1</reflink>,<reflink idref="bib31" id="ref59">31</reflink>) = 0.001, <emph>p</emph> =.97, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> < <emph>.</emph>001).</p> <hd id="AN0187092980-15">Accuracy</hd> <p>Similar to RT analyses, the repeated measures ANOVA on the automation reliability and task difficulty were conducted on the response accuracy (Fig. 5). There was a main effect of task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref60">1</reflink>,<reflink idref="bib31" id="ref61">31</reflink>) = 62.42, <emph>p</emph> < <emph>.</emph>001, <emph>η</emph><subs><emph>p</emph></subs><sups><emph>2</emph></sups> = 0.67). Participants performed better in the easy task compared to the difficult task (<emph>t</emph> = 7.90, <emph>p</emph> <.001, <emph>d</emph> = 1.45). There was also an interaction between the two factors (<emph>F</emph> (1.57,48.58) = 4.84, <emph>p</emph> = <emph>.</emph>011, <emph>η</emph><subs><emph>p</emph></subs><sups><emph>2</emph></sups> = 0.14), likely resulting in the higher accuracy lead by the high-reliability automation compared to the low-reliability automation (<emph>M</emph><subs><emph>diff</emph></subs> = 0.02, 95%CI [5.30 × 10<sups>–4</sups>, 0.03]) and unaided condition (<emph>M</emph><subs><emph>diff</emph></subs> = 0.01, 95%CI [-0.004, 0.03]) when the task was difficult whereas it led to relatively lower accuracy in the easy condition (high reliability vs. low reliability: <emph>M</emph><subs><emph>diff</emph></subs> = - 0.003, 95%CI [ − 0.02, 0.14]; high reliability vs. unaided: <emph>M</emph><subs><emph>diff</emph></subs> = -0.007, 95%CI [ -0.02, 0.009]).</p> <p>Graph: Fig. 5 Mean accuracy across conditions of task difficulty (difficult, easy) and automation reliability (high, low, unaided) in Experiment 1. Error bars represent ± one standard-error</p> <p>The further analysis with cue validity included (Fig. 6) suggested that there were statistically significant main effects of the automation reliability (<emph>F</emph> (<reflink idref="bib1" id="ref62">1</reflink>,<reflink idref="bib31" id="ref63">31</reflink>) = 9.60, <emph>p</emph> =.004, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.24), task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref64">1</reflink>,<reflink idref="bib31" id="ref65">31</reflink>) = 44.39, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.59), and cue validity (<emph>F</emph> (<reflink idref="bib1" id="ref66">1</reflink>,<reflink idref="bib31" id="ref67">31</reflink>) = 27.13, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.47). The post hoc analysis indicated that low reliable automation (<emph>t</emph> = 3.10, <emph>p</emph> =.004, <emph>d</emph> = 0.32), easy task (<emph>t</emph> = 6.66, <emph>p</emph> <.001, <emph>d</emph> = 0.94), and valid cues (<emph>t</emph> = 5.21, <emph>p</emph> <.001, <emph>d</emph> = 0.79) were more likely to result in higher accuracy. Furthermore, there was an interaction between the automation reliability and cue validity (<emph>F</emph> (<reflink idref="bib1" id="ref68">1</reflink>,<reflink idref="bib31" id="ref69">31</reflink>) = 16.26, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.34). When the automation was highly reliable, the valid cues produced significantly higher accuracy compared to invalid cues (<emph>t</emph> = 6.58, <emph>p</emph> <.001, <emph>d</emph> = 1.31) where such effect was not observed in the low-reliability condition (<emph>t</emph> = 1.43, <emph>p</emph> =.95, <emph>d</emph> = 1.43). In addition, the cue validity also interacted with the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref70">1</reflink>,<reflink idref="bib31" id="ref71">31</reflink>) = 15.21, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.33), suggesting that the accuracy difference between the valid and invalid cues (i.e., a larger validity effect) was observed only in the difficult task (<emph>M</emph><subs><emph>diff</emph></subs> = 0.08, 95%CI [0.05, 0.12], <emph>t</emph> = 6.39, <emph>p</emph> <.001) rather than in the easy task (<emph>M</emph><subs><emph>diff</emph></subs> = 0.04, 95%CI [0.002, 0.07], <emph>t</emph> = 2.96, <emph>p</emph> =.03).</p> <p>Graph: Fig. 6 Mean accuracy across conditions of cue validity (valid, invalid) and task difficulty (difficult, easy) for different automation reliability (high, low) types in Experiment 1. Error bars represent ± one standard-error</p> <hd id="AN0187092980-16">Capacity coefficient</hd> <p>The STST capacity was calculated to indicate whether the aided information influenced processing efficiency compared to the predicted baseline based on the participant's performance in the unaided decisions. Figure 7 displays the dynamic change of the capacity coefficient across response times. The visual inspection suggests that participants performed at limited to unlimited capacity. Notably, the figure also indicated that at the visual inspection, participants processed super capacity during the 500 ms and 750 ms period (Fig. 7, the left top panel), suggesting the facilitated processing aided by the automation.</p> <p>Graph: Fig. 7 The temporal changes of capacity functions in Experiment 1. The thick line represents the average mean group level and the thin lines represent each individual's capacity function</p> <p>To statistically quantify the observed capacity performance, the raw capacity scores are transformed to the statistic <emph>Cz</emph> to provide a summarized measure of processing efficiency across the entire response times as depicted in Fig. 8 (Houpt & Townsend, [<reflink idref="bib20" id="ref72">20</reflink>]). The summarized <emph>Cz</emph> scores are tested against 0, in which the <emph>Cz</emph> < 0 signals the limited capacity and <emph>Cz</emph> > 0 signals the super capacity. The one-sample <emph>t-</emph>test suggested the performance was unlimited capacity at the group level (i.e., not statistically significantly different from <emph>Cz</emph> = 0) except that the limited capacity performance when the automation presented the invalid cues in the easy task (high reliability: <emph>M</emph> = -1.10, <emph>t</emph>(<reflink idref="bib31" id="ref73">31</reflink>) = 2.52, <emph>p</emph> =.017, <emph>d</emph> = 0.45; low reliability: <emph>M</emph> =-1.87, <emph>t</emph> = 2.88, <emph>p</emph> =.007, <emph>d</emph> = 0.51).</p> <p>Graph: Fig. 8 Mean Cz values for STST capacity coefficients for processing additional automated aids in Experiment 1. Cz at 0 implies unlimited capacity and below 0 implies limited capacity. The error bars represent ± one standard-error</p> <p>The repeated measures ANOVA indicated that there was a main effect of the cue validity (<emph>F</emph> (<reflink idref="bib1" id="ref74">1</reflink>,<reflink idref="bib31" id="ref75">31</reflink>) = 8.05, <emph>p</emph> =.008, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.21), the valid cue resulted in higher processing efficiency compared to the invalid cues (<emph>t</emph> = 2.84, <emph>p</emph> =.008, <emph>d</emph> = 0.18). In addition, there was a main effect of the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref76">1</reflink>,<reflink idref="bib31" id="ref77">31</reflink>) = 14.01, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.31) where the aided decisions in a difficult task were more efficient than in an easy task (<emph>t</emph> = 3.74, <emph>p</emph> <.001, <emph>d</emph> = 0.20). There was also a three-way interaction (<emph>F</emph> (<reflink idref="bib1" id="ref78">1</reflink>,<reflink idref="bib31" id="ref79">31</reflink>) = 4.34, <emph>p</emph> =.045, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.12). The separate ANOVA analyses were conducted on the high- and low-reliability aids to further elucidate the three-way interaction. There was an interaction between the task difficulty and cue validity when participants performed in the high-reliability condition: There was a larger disparity in processing efficiency between the valid and invalid cues when the task was difficult (<emph>M</emph><subs><emph>diff</emph></subs> = 1.00, 95%CI [ -0.10, 2.11]) compared in the easy task (<emph>M</emph><subs><emph>diff</emph></subs> = 0.36, 95%CI [-0.75, 1.46]). The interaction between the task difficulty and cue validity, nevertheless, failed to display when the automation reliability was low.</p> <hd id="AN0187092980-17">Discussion</hd> <p>In Experiment 1, participants were asked to perform a shape categorization task with aids (high reliability and low reliability) and without aids. The behavioral analysis indicated that the easy task resulted in faster response times and higher accuracy, validating our experimental design for difficulty manipulation. Additionally, consistent with previous research (Kneeland et al., [<reflink idref="bib24" id="ref80">24</reflink>]; Yamani & McCarley, [<reflink idref="bib53" id="ref81">53</reflink>]), our behavioral results also suggested that automated aids in the low-reliability condition resulted in slower responses compared to unaided condition, suggesting that the automated aids could potentially result in additional costs. One notable finding was the interaction between automation reliability and cue validity on accuracy. As we hypothesized, in situations where automation had high reliability, valid cues resulted in significantly higher accuracy compared to invalid cues, a distinction not observed in the low-reliability condition. This suggested that low reliability produced higher accuracy performance when the cue was invalid, demonstrating the complacency effect for the high-reliability automation, where overreliance led to a "neglect" of the "raw" stimuli information (Parasuraman & Wickens, [<reflink idref="bib36" id="ref82">36</reflink>]).</p> <p>Furthermore, we performed the STST capacity analysis to examine the impact of automated aids on decision efficiency. The results demonstrated that participants operated at unlimited capacity across most combinations of reliability and task difficulty conditions, except for a noted limitation in capacity when automated aids supplied invalid cues during easy tasks. Contrary to other studies suggesting that participants may rely on aided information to save time (Chen et al., [<reflink idref="bib6" id="ref83">6</reflink>]), the observed unlimited-capacity performance demonstrated that automation neither improved nor diminished decision-making efficiency. Consistent with our hypothesis that automation reliability has a larger impact on difficult tasks, our results also demonstrated improved decision efficiency with aided information during difficult tasks compared to easy ones. To assess the extent to which task difficulty interacts with automation reliability, participants in Experiment 2 engaged in the categorization task with varied automated aids randomly presented. In this circumstance, we anticipated that the impact of task difficulty would be further intensified.</p> <hd id="AN0187092980-18">Experiment 2</hd> <p></p> <hd id="AN0187092980-19">Methods</hd> <p></p> <hd id="AN0187092980-20">Participants</hd> <p>A group of 36 university students (Ages: [<reflink idref="bib19" id="ref84">19</reflink>, 28], <emph>N</emph><subs>female</subs> = 18) from the same participant pool were recruited for Experiment 2. Similar to Experiment 1, participants signed the consent form before the experiment and they received NT$528 (approximately USD 16.5) as reimbursement for their participation.</p> <hd id="AN0187092980-21">Experimental procedure</hd> <p>In Experiment 2, participants were instructed to perform the same categorization task as used in Experiment 1. The key difference was that in Experiment 1, participants encountered only one type of automation reliability per session, while in Experiment 2, they were presented with three different types of automation reliability, which were displayed in a random order within each session.</p> <hd id="AN0187092980-22">Results</hd> <p>The analysis conducted in Experiment 2 mirrored that of Experiment 1. Additionally, trials with response times below 100 ms or above 1500 ms were omitted from further analysis, accounting for about 1.7% of the total trials. This exclusion criterion was consistent with the approach taken in Experiment 1.</p> <hd id="AN0187092980-23">RT</hd> <p>Similar to Experiment 1, we first explored the influence of task difficulty and automation reliability on response times for corrected responses. The repeated measures ANOVA (Fig. 9) suggested that there was a main effect of the automation reliability (<emph>F</emph> (1.59, 55.88) = 21.83, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0<emph>.</emph>38). The high-reliability automation resulted in faster responses than the low-reliability automation (<emph>t</emph> = 4.03, <emph>p</emph> < <emph>.</emph>001, <emph>d</emph> = 0.19). Unaided trials also resulted in a faster response than the high- (<emph>t</emph> = 2.53, <emph>p</emph> =.04, <emph>d</emph> = 0.12) and low-reliability-aided trials (<emph>t</emph> = 6.55, <emph>p</emph> <.001, <emph>d</emph> = 0.31), respectively. There was also a main effect of the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref85">1</reflink>,<reflink idref="bib35" id="ref86">35</reflink>) = 280.26, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = <emph>.</emph>89); the difficult task resulted in slower responses compared to the easy task (<emph>t</emph> = 16.74, <emph>p</emph> <.001, <emph>d</emph> = 1.15). The results failed to indicate the interaction between the two factors (<emph>F</emph> (<reflink idref="bib2" id="ref87">2</reflink>,<reflink idref="bib70" id="ref88">70</reflink>) = 1.68, <emph>p</emph> =.19, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.05).</p> <p>Graph: Fig. 9 Mean response times across conditions of task difficulty (difficult, easy) and automation reliability (high, low, unaided) in Experiment 2. Error bars represent ± one standard-error</p> <p>Further analysis was conducted to investigate the influence of the cue validity (Fig. 10). The results indicated that there was a main effect of task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref89">1</reflink>,<reflink idref="bib35" id="ref90">35</reflink>) = 244.73, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.88); the difficult task resulted in significantly slower responses compared to the easy task (<emph>t</emph> = 15.64, <emph>p</emph> <.001, <emph>d</emph> = 1.06). Furthermore, there was a main effect of the cue validity; the invalid cues resulted in significantly slower responses compared to the valid cues (<emph>t</emph> = 6.90, <emph>d</emph> = 0.56, <emph>p</emph> <.001). The analysis also indicated an interaction between the cue validity and task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref91">1</reflink>,<reflink idref="bib35" id="ref92">35</reflink>) = 6.33, <emph>p</emph> =.017, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.15). The interaction was influenced by a more significant disparity in response times between the valid and invalid cues when the task was difficult (<emph>M</emph><subs><emph>diff</emph></subs> = 53.94, 95%CI [33.67, 74.22]) compared to when it was easy (<emph>M</emph><subs><emph>diff</emph></subs> = 37.86, 95%CI [17.55, 58.09]).</p> <p>Graph: Fig. 10 Mean response times across conditions of cue validity (valid, invalid) and task difficulty (difficult, easy) for different automation reliability (high, low) types in Experiment 2. Error bars represent ± one standard-error</p> <hd id="AN0187092980-24">Accuracy</hd> <p>As for response accuracy (Fig. 11), the repeated measures ANOVA analysis suggested that there was a main effect of the automation reliability (<emph>F</emph> (<reflink idref="bib2" id="ref93">2</reflink>, 70) = 17.00, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.33). The high-reliability automation resulted in more accurate responses compared to the low-reliability automation (<emph>t</emph> = 5.76, <emph>p</emph> <.001, <emph>d</emph> = 0.46) and unaided responses (<emph>t</emph> = 3.67, <emph>p</emph> =.001, <emph>d</emph> = 0.29). There was no accuracy difference between the low-reliability and unaided responses (<emph>t</emph> = 2.10, <emph>p</emph> =.119, <emph>d</emph> = 0.17). In addition, there was also a main effect of the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref94">1</reflink>,<reflink idref="bib35" id="ref95">35</reflink>) = 99.09, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.74); the easy task resulted in more accurate responses compared to the difficult task (<emph>t</emph> = 9.95, <emph>p</emph> <.001, <emph>d</emph> = 1.55). The interaction between the two factors was also significant (<emph>F</emph> (<reflink idref="bib2" id="ref96">2</reflink>,<reflink idref="bib70" id="ref97">70</reflink>) = 18.06, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.34): When the task was difficult, the high-reliability automation resulted in more accurate responses compared to low-reliability automation (<emph>t</emph> = 7.73, <emph>p</emph> < <emph>.</emph>001, <emph>d</emph> = 0.80) and unaided responses (<emph>t</emph> = 6.15, <emph>p</emph> <.001, <emph>d</emph> = 0.63) while there was no statistical significance between the low-reliability aids and unaided trials (<emph>t</emph> = 1.58, <emph>p</emph> = 1, <emph>d</emph> = 0.16); there was no statistical difference in accuracy among the three automated conditions when the task was easy (Fig. 12).</p> <p>Graph: Fig. 11 Mean accuracy across conditions of task difficulty (difficult, easy) and automation reliability (high, low, unaided) in Experiment 2. Error bars represent ± one standard-error</p> <p>Graph: Fig. 12 Mean accuracy across conditions of cue validity (valid, invalid) and task difficulty (difficult, easy) for different automation reliability (high, low) types in Experiment 2. Error bars represent ± one standard-error</p> <p>The automation information was further categorized into valid and invalid cues to evaluate the influence of cue validity on response accuracy. The analysis (Fig. 11) revealed that there was a main effect of the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref98">1</reflink>,<reflink idref="bib35" id="ref99">35</reflink>) = 86.33, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.71), in which the difficult task resulted in lower accuracy compared to the easy task (<emph>t</emph> = 9.29, <emph>p</emph> <.001, <emph>d</emph> = 1.24). The main effect of cue validity was also statistically significant (<emph>F</emph> (<reflink idref="bib1" id="ref100">1</reflink>,<reflink idref="bib35" id="ref101">35</reflink>) = 44.70, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.56), suggesting that the trials with valid cues resulted in better performance compared to the trials with invalid cues (<emph>t</emph> = 6.69, <emph>p</emph> <.001, <emph>d</emph> = 0.92). In addition, there was also an interaction between cue validity and task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref102">1</reflink>,<reflink idref="bib35" id="ref103">35</reflink>) = 30.86, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.47). The interaction was driven by the statistical significance between the valid and invalid cues when the task was difficult (<emph>t</emph> = 8.65, <emph>p</emph> <.001, <emph>d</emph> = 1.43) whereas such difference was not significant when the task was easy (<emph>t</emph> = 2.43, <emph>p</emph> =.11, <emph>d</emph> = 0.40).</p> <hd id="AN0187092980-25">Capacity coefficient</hd> <p>The visual inspection of the STST capacity for Experiment 2 (Fig. 13) suggested that group performance was unlimited or limited capacity (below the <emph>C</emph>(t) = 1 baseline). In addition, there was no visually distinct difference between the two levels of task difficulty. The one-sample <emph>t</emph>-test suggests that when the cue was valid, the group performance was unlimited capacity regardless of the automation reliability and task difficulty; the Cz scores were not significantly different from 0. Nevertheless, participants performed at a limited capacity (i.e., Cz < 0) when the cue was invalid. The repeated measures ANOVA suggested that there was a main effect of the cue validity (<emph>F</emph> (<reflink idref="bib1" id="ref104">1</reflink>,<reflink idref="bib35" id="ref105">35</reflink>) = 21.75, <emph>p</emph> <.001, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.38), suggesting that the valid cue resulted in statistically more efficient processing compared to the invalid cues (<emph>t</emph> = 4.66, <emph>p</emph> < <emph>.</emph>001, <emph>d</emph> = 0.75). There was also a main effect of the automation reliability (<emph>F</emph> (<reflink idref="bib1" id="ref106">1</reflink>,<reflink idref="bib35" id="ref107">35</reflink>) = 5.37, <emph>p</emph> =.026, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.13); the high-reliability aid resulted in more efficient performance (<emph>t</emph> = 2.31, <emph>p</emph> =.026, <emph>d</emph> = 0.21). The results failed to find evidence suggesting the main effect of the task difficulty (<emph>F</emph> (<reflink idref="bib1" id="ref108">1</reflink>,<reflink idref="bib35" id="ref109">35</reflink>) = 0.77, <emph>p</emph> =.39, <emph>η</emph><subs><emph>p</emph></subs><sups>2</sups> = 0.09 nor was there statistical significance for interactions among main effects (Fig. 14).</p> <p>Graph: Fig. 13 The temporal changes of capacity functions in Experiment 2. The thick line represents the average mean group level and the thin lines represent each individual's capacity function</p> <p>Graph: Fig. 14 Mean Cz values for STST capacity coefficients for processing additional automated aids in Experiment 2. Cz at 0 implies unlimited capacity and below 0 implies limited capacity. The error bars represent ± one standard-error</p> <hd id="AN0187092980-26">Discussion</hd> <p>In Experiment 2, participants were instructed to perform a shape categorization task under intermixed automation conditions. Unlike Experiment 1, in which high and low automation reliability resulted in different behavioral performances, Experiment 2 indicated that the random presentation of different types of automation reliability in an intermixed design did not yield a significant reliability effect on both response times and accuracy. The findings indicated that the impact of reliability was likely replaced by cue validity, as the introduction of valid cues led to increased accuracy and faster responses. Furthermore, an interaction was observed between cue validity and task difficulty. In difficult tasks, notable differences emerged between responses influenced by valid and invalid cues, both in terms of response times and accuracy. This indicates that invalid cues presented a greater challenge for participants in their responses.</p> <p>Unlike Experiment 1, the STST analysis indicated that participants' performance capacity was limited when the automated aids delivered invalid cues, regardless of the reliability type. This disparity is likely due to participants' difficulty in distinguishing between high- and low-reliability automated aids when limited by cognitive resources. This may also indicate that participants tend to disregard information when its reliability is uncertain. Consequently, instead of selectively assessing the dependability of automation, participants may focus on evaluating the validity of information in each trial.</p> <p>Furthermore, Experiment 2 did not show the beneficial use of automated aids in difficult tasks, suggesting that the repeated display of automation, as in Experiment 1, was essential for participants to allocate attention resources to automation that enhanced the task performance in difficult tasks. In addition, our evidence indicated an interaction between cue validity and automation reliability, where the difference in decision efficiency between valid and invalid cues was more pronounced in trials supported by high-reliability automation. This difference may be attributed to an uneven number of cues associated with various types of reliability automation, as high-reliability automation offers a greater number of valid cues. Considering that cue validity may influence participants' attention (e.g., Horowitz, [<reflink idref="bib19" id="ref110">19</reflink>]), we propose that future research should investigate this specific response tendency further.</p> <hd id="AN0187092980-27">General discussion</hd> <p>The impact of task difficulty and automation reliability played a crucial role in shaping participants' use of automated assistance. Participants were more likely to seek assistance from automation during difficult tasks and were more influenced by it; nevertheless, when the automation was unreliable, it could become a hindrance and potentially have a negative impact on performance (e.g., Wohleber et al., [<reflink idref="bib51" id="ref111">51</reflink>]). In our current study, participants were instructed to categorize the shape of the stimulus. Experiment 1 had participants perform the task under one specific type of automation condition (low reliability, high reliability, unaided) at a time, whereas Experiment 2 instructed participants to perform the task with an intermixed design in which the three types of automation were presented randomly. We analyzed the data using STST capacity analysis, comparing participants' performance against a baseline established by participants' performance in the unaided condition. This allows us to measure the decision efficiency of human information processing with automated aids. The findings indicated that when the automation provided valid cues, participants processed information more efficiently than when given invalid cues. Furthermore, the efficiency was enhanced when automation aided in difficult tasks, as demonstrated in Experiment 1. By contrast, this efficiency gain was not as apparent in Experiment 2, where participants encountered varying types of automated aids randomly. Furthermore, the temporal analysis of the capacity coefficient in Experiment 1 revealed that individuals had super capacity during the 500ms and 750 ms intervals, underscoring the dynamic fluctuations in decision efficiency throughout the response period.</p> <p>Our STST capacity analysis indicated that automated aids are beneficial in difficult tasks, especially when participants could distinctly discern between low- and high-reliability automation. Nevertheless, this discovery contrasts with the findings of Yamani and McCarley ([<reflink idref="bib53" id="ref112">53</reflink>]), who found that the easier task resulted in more efficient processing. This distinction depends on whether the responses of automated aids are seen as independent or an essential component of the decision-making process (Parasuraman et al., [<reflink idref="bib37" id="ref113">37</reflink>]). Our investigation revealed that when information was abundant for decision-making and participants could consider aided suggestions, they exhibited unlimited capacity when the aids provided correct recommendations. This aligns with the findings of Kneeland et al. ([<reflink idref="bib24" id="ref114">24</reflink>]), which demonstrated that participants displayed similar, if not inferior, efficiency with the automated aids compared to independent decision-making, despite no observed impact of difficulty.</p> <p>Kneeland et al. ([<reflink idref="bib25" id="ref115">25</reflink>]) observed that participants generally processed automated assistance and raw stimulus information sequentially, often neglecting to incorporate both into their decision-making considerations. Participants did not consistently rely solely on automated aids; rather, they seemed to switch between both sources across trials. This may partially explain our observation in Experiment 2, where the reliability of automation in an intermixed block design led participants to allocate attentional resources toward verifying the cues' validity rather than accumulating reliability information for each presented automated aid. An alternative explanation is that when automation is presented in an intermixed block design, participants may struggle to form consistent expectations regarding the automation's reliability. Following the expectancy violation theory (Burgoon, [<reflink idref="bib5" id="ref116">5</reflink>]), Fox et al. ([<reflink idref="bib13" id="ref117">13</reflink>]) found that differences in team efficiency between low- and high-purported reliability teammates were moderated by whether the teammates' actual performance aligned with expectations. Consistent with research on collaborative decision-making, this suggests the potential for induced coordination costs, as participants must synchronize their efforts (e.g., Fific & Gigerenzer, [<reflink idref="bib12" id="ref118">12</reflink>]; Neider et al., [<reflink idref="bib35" id="ref119">35</reflink>]; Yamani et al., [<reflink idref="bib54" id="ref120">54</reflink>]; Zhang et al., [<reflink idref="bib55" id="ref121">55</reflink>]). In the context of aided decision-making, decision makers were also less likely to respond to or assign weight to information from the automation when it was perceived to have low informative value (Maltz & Meyer, [<reflink idref="bib31" id="ref122">31</reflink>]; Meyer, [<reflink idref="bib34" id="ref123">34</reflink>]).</p> <p>Together, our results suggested that complacency regarding highly reliable automation may be partially ascribed to the validity of information, represented by a weighted trade-off between decision makers' belief in their performance and automation capability. Accordingly, we propose that future research in aided decision-making should ensure that decision makers are fully informed of the automation's limitations to mitigate the complacency effect. Furthermore, considering the difficulties encountered by participants in dynamic decision-making contexts (e.g., Cronin et al., [<reflink idref="bib7" id="ref124">7</reflink>]; Gonzalez, [<reflink idref="bib15" id="ref125">15</reflink>]), it is essential for future research to elucidate the manner in which information regarding automation capability is accumulated on a trial-by-trial basis (e.g., Gonzalez et al., [<reflink idref="bib17" id="ref126">17</reflink>]; Gonzalez & Dutt, [<reflink idref="bib16" id="ref127">16</reflink>]), in conjunction with various influential factors (Lyons & Stokes, [<reflink idref="bib29" id="ref128">29</reflink>]).</p> <p>In contrast to previous research where low-reliability automated aids resulted in performance costs (Wickens & Dixon, [<reflink idref="bib48" id="ref129">48</reflink>]), in our current study, participants had processing efficiency that was equally likely with low-reliability automated aids, as it provided accurate cues. We suspected that this could be accounted for by our limitation in the current research, which only adopted a restricted set of stimuli, and the range of task difficulty levels is limited. Additionally, the likelihood of valid cues was directly tied to the automation's reliability. To address such limitations, future research can benefit from employing a broader array of stimulus variations (e.g., Kneeland et al., [<reflink idref="bib24" id="ref130">24</reflink>]), various automated decision-making stages (Rovira et al., [<reflink idref="bib41" id="ref131">41</reflink>]), and automated aids that indicate the probability of a stimulus belonging to a certain category (e.g., Bergert & Nosofsky, [<reflink idref="bib2" id="ref132">2</reflink>]). Furthermore, Dixon et al. ([<reflink idref="bib9" id="ref133">9</reflink>]) proposed that automation prone to false alarms and misses might influence participants' interactions with automated aids differently. Specifically, they suggested that false alarm-prone automation impacts both decision makers' compliance (action following automation's signal diagnosis) and reliance (action following automation's noise diagnosis) whereas miss-prone automation seemed to affect only reliance. In our current research, we oversimplified by using overall accuracy as the measure of automation reliability and our design of the automated aids display differs from previous research in human-automation interaction (e.g., Kneeland et al., [<reflink idref="bib24" id="ref134">24</reflink>]). All can contribute to variations in participants' cognitive abilities in our current research. In this way, future research can explore whether participants adopt different strategies when faced with automated aids that exhibit various types of errors in a different display design, providing deeper insights into how decision makers interact with and adapt to automation in decision-making process.</p> <p>To conclude, our research explored the impact of task difficulty on processing efficiency by automation reliability. Our results using the STST capacity coefficient in SFT suggested that participants performed the categorization task comparably with or without the presence of automated aids, provided they were not distracted by invalid cues. In accordance with our hypothesis, participants exhibited a significant advantage when engaged in difficult categorization tasks with the assistance of automated aids. The advantageous condition observed in the difficult task became apparent when participants interacted with a singular sort of automation, thereby accumulating reliable information for that automation. Our research suggested the existence of an optimal scenario for employing automation to aid decision makers and can contribute to interface design to optimize processing efficiency inside the established automation system.</p> <hd id="AN0187092980-28">Acknowledgements</hd> <p>We are grateful to P.-F. Zhu for programming assistance and to H.-L. Fu for contributions to an earlier draft of the manuscript.</p> <hd id="AN0187092980-29">Author contributions</hd> <p>CY conceived the experimental design. CC, SH, and MC collected the experimental data. CCan HZ performed the data analysis. MC, CC, SH, HZ, and RZ prepared the initial draft. CY and HZ revised the manuscript. All authors read and approved the final manuscript.</p> <hd id="AN0187092980-30">Funding</hd> <p>This work is supported by the National Science and Technology Council, Taiwan (NSTC 112-2423-H-038-001, NSTC 109-2410-H-006 -049 -MY3, NSTC 107-2410-H-006 -055 -MY2, NSTC 105-2410-H-006 -020 -MY2, NSTC 113-2410-H-006 -129 -MY3, and NSTC 114-2627-M-A49 -003 -), to C.-T. Yang; the National Natural Science Foundation of China (32300910) and self-determined research funds of CCNU from the colleges' basic research and operation of MOE (CCNU24JCPT038, CCNU24JC004, CCNU25JC028, CCNU25ZZ155) to H. Zhang; the National Natural Science Foundation of China (32200887) to R. Zhou.</p> <hd id="AN0187092980-31">Declarations</hd> <p></p> <hd id="AN0187092980-32">Ethical approval</hd> <p>The experiment was approved by the Institutional Review Board at the Department of Psychology, National Cheng Kung University (protocol code: NCKU HREC-E109-257–2).</p> <hd id="AN0187092980-33">Competing interests</hd> <p>All authors declare that they have no conflict of interest.</p> <hd id="AN0187092980-34">Appendix</hd> <p>The overall reliability of the automation was 82% in the high-reliability condition and 50% in the low-reliability condition. Each block consisted of 4 sets of 100 trials, with each stimulus (see Fig. 1) presented 25 times, totaling 400 trials per block (i.e., 200 trials for category A and 200 for category B).</p> <p>For every 100 trials involving category A:</p> <p></p> <ulist> <item> In the high reliability condition, the automation provided 82 valid suggestions and 18 invalid ones. Of these, 1 out of the 82 valid suggestions and 9 out of the 18 invalid suggestions were accompanied by incorrect feedback (see Footnote 1 for details).</item> <p></p> <item> In the low-reliability condition, the automation provided 50 valid and 50 invalid suggestions, with 5 of each suggestion receiving incorrect feedback.</item> </ulist> <p>See Table 1.</p> <p>Table 1 Automation design example for each set of 100 trials with category A</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" colspan="5"><p>Advised</p></th></tr><tr><th align="left" rowspan="2"><p>Feedback</p></th><th align="left" colspan="2"><p>High reliability</p></th><th align="left" colspan="2"><p>Low reliability</p></th></tr><tr><th align="left"><p>A</p></th><th align="left"><p>B</p></th><th align="left"><p>A</p></th><th align="left"><p>B</p></th></tr></thead><tbody><tr><td align="left"><p>A</p></td><td align="left"><p><bold>81</bold></p></td><td align="left"><p><bold>1</bold></p></td><td align="left"><p><bold>45</bold></p></td><td align="left"><p><bold>5</bold></p></td></tr><tr><td align="left"><p>B</p></td><td align="left"><p>9</p></td><td align="left"><p>9</p></td><td align="left"><p>45</p></td><td align="left"><p>5</p></td></tr></tbody></table> </ephtml> </p> <p>The correct answers (i.e., valid information) were in bold</p> <hd id="AN0187092980-35">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0187092980-36"> <title> References </title> <blist> <bibl id="bib1" idref="ref22" type="bt">1</bibl> <bibtext> Barg-Walkow LH, Rogers WA. The effect of incorrect reliability information on expectations, perceptions, and use of automation. Human Factors. 2016; 58; 2: 242-260. 10.1177/001872081561027. 26519483</bibtext> </blist> <blist> <bibl id="bib2" idref="ref44" type="bt">2</bibl> <bibtext> Bergert FB, Nosofsky RM. A response-time approach to comparing generalized rational and take-the-best models of decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2007; 33; 1: 107. 17201556</bibtext> </blist> <blist> <bibl id="bib3" type="bt">3</bibl> <bibtext> Blaha, L. M, Townsend, J. T, Kneeland, C. M, & Houpt, J. W. (Under Review). Capacity coefficient analysis for single-target selfterminating processes. Journal of Mathematical Psychology.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref6" type="bt">4</bibl> <bibtext> Bowers C, Thornton C, Braun C, Morgan BB Jr, Salas E. Automation, task difficulty, and aircrew performance. Military Psychology. 1998; 10; 4: 259-274. 11541776</bibtext> </blist> <blist> <bibl id="bib5" idref="ref116" type="bt">5</bibl> <bibtext> Burgoon JKBerger CR, Roloff ME, Wilson SR, Dillard JP, Caughlin J, Solomon D. Expectancy violations theory. International encyclopedia of interpersonal communication. 2016; Wiley: 1-9</bibtext> </blist> <blist> <bibl id="bib6" idref="ref4" type="bt">6</bibl> <bibtext> Chen J, Mishler S, Hu B, Li N, Proctor RW. The description-experience gap in the effect of warning reliability on user trust and performance in a phishing-detection context. International Journal of Human-Computer Studies. 2018; 119: 35-47</bibtext> </blist> <blist> <bibl id="bib7" idref="ref124" type="bt">7</bibl> <bibtext> Cronin MA, Gonzalez C, Sterman JD. Why don't well-educated adults understand accumulation? A challenge to researchers, educators, and citizens. Organizational Behavior and Human Decision Processes. 2009; 108; 1: 116-130</bibtext> </blist> <blist> <bibl id="bib8" idref="ref28" type="bt">8</bibl> <bibtext> De Vries P, Midden C, Bouwhuis D. The effects of errors on system trust, self-confidence, and the allocation of control in route planning. International Journal of Human-Computer Studies. 2003; 58; 6: 719-735</bibtext> </blist> <blist> <bibl id="bib9" idref="ref133" type="bt">9</bibl> <bibtext> Dixon SR, Wickens CD, McCarley JS. On the independence of compliance and reliance: Are automation false alarms worse than misses?. Human Factors. 2007; 49; 4: 564-572. 17702209</bibtext> </blist> <blist> <bibtext> Dunning D, Johnson K, Ehrlinger J, Kruger J. Why people fail to recognize their own incompetence. Current Directions in Psychological Science. 2003; 12; 3: 83-87. 10.1111/1467-8721.01235</bibtext> </blist> <blist> <bibtext> Dzindolet MT, Peterson SA, Pomranky RA, Pierce LG, Beck HP. The role of trust in automation reliance. International Journal of Human-Computer Studies. 2003; 58; 6: 697-718. 10.1016/S1071-5819(03)00038-7</bibtext> </blist> <blist> <bibtext> Fifić M, Gigerenzer G. Are two interviewers better than one?. Journal of Business Research. 2014; 67; 8: 1771-1779. 10.1016/j.jbusres.2014.03.003</bibtext> </blist> <blist> <bibtext> Fox EL, Capiola A, Bowers G, Stephenson A. A metric of team multitasking throughput. Journal of Experimental Psychology: Applied. 2025; 31; 1: 58-70. 10.1037/xap0000519. 39680014</bibtext> </blist> <blist> <bibtext> Gamble KR, Cassenti DN, Buchler N. Effects of information accuracy and volume on decision making. Military Psychology. 2018; 30; 4: 311-320</bibtext> </blist> <blist> <bibtext> Gonzalez C. Learning to make decisions in dynamic environments: Effects of time constraints and cognitive abilities. Human Factors. 2004; 46; 3: 449-460. 15573545</bibtext> </blist> <blist> <bibtext> Gonzalez C, Dutt V. Instance-based learning: Integrating sampling and repeated decisions from experience. Psychological Review. 2011; 118; 4: 523-551. 10.1037/a0024558. 21806307</bibtext> </blist> <blist> <bibtext> Gonzalez C, Lerch JF, Lebiere C. Instance-based learning in dynamic decision making. Cognitive Science. 2003; 27; 4: 591-635</bibtext> </blist> <blist> <bibtext> Hoff KA, Bashir M. Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors. 2015; 57; 3: 407-434. 10.1177/0018720814547570. 25875432</bibtext> </blist> <blist> <bibtext> Horowitz TS. Prevalence in visual search: From the clinic to the lab and back again. Japanese Psychological Research. 2017; 59; 2: 65-108</bibtext> </blist> <blist> <bibtext> Houpt, J. W, & Townsend, J. T. (2012). Statistical measures for workload capacity analysis. Journal of Mathematical Psychology, 56(5), 341–355.</bibtext> </blist> <blist> <bibtext> Houpt JW, Blaha LM, McIntire JP, Havig PR, Townsend JT. Systems factorial technology with R. Behavior Research Methods. 2013; 46; 2: 307-330. 10.3758/s13428-013-0377-3</bibtext> </blist> <blist> <bibtext> JASP Team. (2024). JASP (Version 0.19.3) [Computer software].</bibtext> </blist> <blist> <bibtext> Kantowitz BH, Hanowski RJ, Kantowitz SC. Driver acceptance of unreliable traffic information in familiar and unfamiliar settings. Human Factors. 1997; 39; 2: 164-176. 10.1518/001872097778543831</bibtext> </blist> <blist> <bibtext> Kneeland CM, Houpt JW, Bennett KB. Exploring the performance consequences of target prevalence and ecological display designs when using an automated aid. Computational Brain & Behavior. 2021; 4: 335-354</bibtext> </blist> <blist> <bibtext> Kneeland CM, Houpt JW, Juvina I. How do people process information from automated decision aids: An application of systems factorial technology. Computational Brain & Behavior. 2024; 7; 1: 106-128. 10.1007/s42113-023-00188-z</bibtext> </blist> <blist> <bibtext> Lee JD, See KA. Trust in automation: Designing for appropriate reliance. Human Factors. 2004; 46; 1: 50-80. 10.1518/hfes.46.1.50_30392. 15151155</bibtext> </blist> <blist> <bibtext> Lee J, Moray N. Trust, control strategies and allocation of function in human-machine systems. Ergonomics. 1992; 35; 10: 1243-1270. 1516577</bibtext> </blist> <blist> <bibtext> Little D, Altieri N, Fific M, Yang CT. Systems factorial technology: A theory driven methodology for the identification of perceptual and cognitive mechanisms. 2017; Academic Press</bibtext> </blist> <blist> <bibtext> Lyons JB, Stokes CK. Human-human reliance in the context of automation. Human Factors. 2012; 54; 1: 112-121. 22409106</bibtext> </blist> <blist> <bibtext> Madhavan P, Wiegmann DA, Lacson FC. Automation failures on tasks easily performed by operators undermine trust in automated aids. Human Factors. 2006; 48; 2: 241-256. 10.1518/001872006777724408. 16884046</bibtext> </blist> <blist> <bibtext> Maltz M, Meyer J. Use of warnings in an attentionally demanding detection task. Human Factors: THe Journal of the Human Factors and Ergonomics Society. 2001; 43; 2: 217-226. 10.1518/001872001775900931</bibtext> </blist> <blist> <bibtext> Merritt SM, Lee D, Unnerstall JL, Huber K. Are well-calibrated users effective users? Associations between calibration of trust and performance on an automation-aided task. Human Factors. 2015; 57; 1: 34-47. 10.1177/0018720814561675. 25790569</bibtext> </blist> <blist> <bibtext> Metzger U, Parasuraman R. Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors. 2005; 47; 1: 35-49. 10.1518/0018720053653802. 15960085</bibtext> </blist> <blist> <bibtext> Meyer J. Effects of warning validity and proximity on responses to warnings. Human Factors. 2001; 43; 4: 563-572. 10.1518/001872001775870395. 12002005</bibtext> </blist> <blist> <bibtext> Neider MB, Chen X, Dickinson CA, Brennan SE, Zelinsky GJ. Coordinating spatial referencing using shared gaze. Psychonomic Bulletin & Review. 2010; 17: 718-724. 10.3758/PBR.17.5.718</bibtext> </blist> <blist> <bibtext> Parasuraman R, Wickens CD. Humans: Still vital after all these years of automation. Decision Making in Aviation. 2017; Routledge: 251-260</bibtext> </blist> <blist> <bibtext> Parasuraman R, Sheridan TB, Wickens CD. A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics-Part a: Systems and Humans. 2000; 30; 3: 286-297</bibtext> </blist> <blist> <bibtext> Patton, C. E. (2023). The influence of trust, self-confidence and task difficulty on automation use. Doctoral dissertation, Colorado State University.</bibtext> </blist> <blist> <bibtext> Pop VL, Shrewsbury A, Durso FT. Individual differences in the calibration of trust in automation. Human Factors. 2015; 57; 4: 545-556. 10.1177/0018720814564422. 25977317</bibtext> </blist> <blist> <bibtext> Rieger T, Manzey D. Human performance consequences of automated decision aids: The impact of time pressure. Human Factors. 2022; 64; 4: 617-634. 10.1177/0018720820965019. 33111557</bibtext> </blist> <blist> <bibtext> Rovira, E, McGarry, K, & Parasuraman, R. (2002). Effects of unreliable automation on decision making in command and control. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 46, No. 3, pp. 428–432). SAGE Publications.</bibtext> </blist> <blist> <bibtext> Rovira E, McGarry K, Parasuraman R. Effects of imperfect automation on decision making in a simulated command and control task. Human Factors. 2007; 49; 1: 76-87. 17315845</bibtext> </blist> <blist> <bibtext> Schaefer KE, Chen JYC, Szalma JL, Hancock PA. A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors. 2016; 58; 3: 377-400. 10.1177/0018720816634228. 27005902</bibtext> </blist> <blist> <bibtext> Schwark, J, Dolgov, I, Graves, W, & Hor, D. (2010). The influence of perceived task difficulty and importance on automation use. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 54, No. 19, pp. 1503–1507). SAGE Publications.</bibtext> </blist> <blist> <bibtext> Sheridan TB, Parasuraman R. Human-automation interaction. Reviews of Human Factors and Ergonomics. 2005; 1; 1: 89-129</bibtext> </blist> <blist> <bibtext> Townsend JT, Nozawa G. Spatio-temporal properties of elementary perception: An investigation of parallel, serial, and coactive theories. Journal of Mathematical Psychology. 1995; 39; 4: 321-359. 10.1006/jmps.1995.1033</bibtext> </blist> <blist> <bibtext> Wickens CD. Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science. 2002; 3; 2: 159-177. 10.1080/14639220210123806</bibtext> </blist> <blist> <bibtext> Wickens CD, Dixon SR. The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomics Science. 2007; 8; 3: 201-212. 10.1080/14639220110110306</bibtext> </blist> <blist> <bibtext> Wickens, C. D. (2000). Imperfect and unreliable automation and its implications for attention allocation, information access and situation awareness. Technical Report ARL00-l OMASA-00-2.</bibtext> </blist> <blist> <bibtext> Wiegmann DA, Rich A, Zhang H. Automated diagnostic aids: The effects of aid reliability on users' trust and reliance. Theoretical Issues in Ergonomics Science. 2001; 2; 4: 352-367</bibtext> </blist> <blist> <bibtext> Wohleber, R. W, Calhoun, G. L, Funke, G. J, Ruff, H, Chiu, C. Y. P, Lin, J, & Matthews, G. (2016). The impact of automation reliability and operator fatigue on performance and reliance. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 60, No. 1, pp. 211–215). SAGE Publications.</bibtext> </blist> <blist> <bibtext> Xu X, Wickens CD, Rantanen EM. Effects of conflict alerting system reliability and task difficulty on pilots' conflict detection with cockpit display of traffic information. Ergonomics. 2007; 50; 1: 112-130. 17178655</bibtext> </blist> <blist> <bibtext> Yamani Y, McCarley JS. Effects of task difficulty and display format on automation usage strategy: A workload capacity analysis. Human Factors. 2018; 60; 4: 527-537. 29470135</bibtext> </blist> <blist> <bibtext> Yamani Y, Neider MB, Kramer AF, McCarley JS. Characterizing the efficiency of collaborative visual search with systems factorial technology. Archives of Scientific Psychology. 2017; 5; 1: 1-9. 10.1037/arc0000030</bibtext> </blist> <blist> <bibtext> Zhang H, Zhu PF, Yang CT. Group efficiency based on the termination rule in the multiple-targets visual search task. Psychonomic Bulletin & Review. 2025; 32; 1: 463-471</bibtext> </blist> </ref> <ref id="AN0187092980-37"> <title> Footnotes </title> <blist> <bibtext> To test the aid reliance in the context of decision under uncertainty, participants received 90% "correct" feedback and 10% "incorrect" feedback when they made correct responses, the same applies to when participants made incorrect responses in the current design. We expected that this design would help participants adjust their reliance on automation and encourage participants to process the stimuli information. See the Appendix for more detailed explanations.</bibtext> </blist> </ref> <aug> <p>By Hanshu Zhang; Ran Zhou; Cheng-You Cheng; Sheng-Hsu Huang; Ming-Hui Cheng and Cheng-Ta Yang</p> <p>Reported by Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib45" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib28" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib46" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib52" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib24" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib14" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib18" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib40" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib38" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib44" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib27" firstref="ref14"></nolink> <nolink nlid="nl12" bibid="bib43" firstref="ref15"></nolink> <nolink nlid="nl13" bibid="bib33" firstref="ref17"></nolink> <nolink nlid="nl14" bibid="bib48" firstref="ref18"></nolink> <nolink nlid="nl15" bibid="bib50" firstref="ref20"></nolink> <nolink nlid="nl16" bibid="bib39" firstref="ref23"></nolink> <nolink nlid="nl17" bibid="bib32" firstref="ref24"></nolink> <nolink nlid="nl18" bibid="bib26" firstref="ref25"></nolink> <nolink nlid="nl19" bibid="bib10" firstref="ref26"></nolink> <nolink nlid="nl20" bibid="bib23" firstref="ref27"></nolink> <nolink nlid="nl21" bibid="bib36" firstref="ref29"></nolink> <nolink nlid="nl22" bibid="bib42" firstref="ref30"></nolink> <nolink nlid="nl23" bibid="bib49" firstref="ref31"></nolink> <nolink nlid="nl24" bibid="bib11" firstref="ref32"></nolink> <nolink nlid="nl25" bibid="bib47" firstref="ref33"></nolink> <nolink nlid="nl26" bibid="bib30" firstref="ref35"></nolink> <nolink nlid="nl27" bibid="bib22" firstref="ref42"></nolink> <nolink nlid="nl28" bibid="bib21" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib62" firstref="ref45"></nolink> <nolink nlid="nl30" bibid="bib31" firstref="ref47"></nolink> <nolink nlid="nl31" bibid="bib20" firstref="ref72"></nolink> <nolink nlid="nl32" bibid="bib53" firstref="ref81"></nolink> <nolink nlid="nl33" bibid="bib19" firstref="ref84"></nolink> <nolink nlid="nl34" bibid="bib35" firstref="ref86"></nolink> <nolink nlid="nl35" bibid="bib70" firstref="ref88"></nolink> <nolink nlid="nl36" bibid="bib51" firstref="ref111"></nolink> <nolink nlid="nl37" bibid="bib37" firstref="ref113"></nolink> <nolink nlid="nl38" bibid="bib25" firstref="ref115"></nolink> <nolink nlid="nl39" bibid="bib13" firstref="ref117"></nolink> <nolink nlid="nl40" bibid="bib12" firstref="ref118"></nolink> <nolink nlid="nl41" bibid="bib54" firstref="ref120"></nolink> <nolink nlid="nl42" bibid="bib55" firstref="ref121"></nolink> <nolink nlid="nl43" bibid="bib34" firstref="ref123"></nolink> <nolink nlid="nl44" bibid="bib15" firstref="ref125"></nolink> <nolink nlid="nl45" bibid="bib17" firstref="ref126"></nolink> <nolink nlid="nl46" bibid="bib16" firstref="ref127"></nolink> <nolink nlid="nl47" bibid="bib29" firstref="ref128"></nolink> <nolink nlid="nl48" bibid="bib41" firstref="ref131"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1478691
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Decision-Making Efficiency with Aided Information: The Impact of Automation Reliability and Task Difficulty
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hanshu+Zhang%22">Hanshu Zhang</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-0273-3254">0000-0002-0273-3254</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ran+Zhou%22">Ran Zhou</searchLink><br /><searchLink fieldCode="AR" term="%22Cheng-You+Cheng%22">Cheng-You Cheng</searchLink><br /><searchLink fieldCode="AR" term="%22Sheng-Hsu+Huang%22">Sheng-Hsu Huang</searchLink><br /><searchLink fieldCode="AR" term="%22Ming-Hui+Cheng%22">Ming-Hui Cheng</searchLink><br /><searchLink fieldCode="AR" term="%22Cheng-Ta+Yang%22">Cheng-Ta Yang</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Cognitive+Research%3A+Principles+and+Implications%22"><i>Cognitive Research: Principles and Implications</i></searchLink>. 2025 10.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 18
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Task+Analysis%22">Task Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+Making%22">Decision Making</searchLink><br /><searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Reliability%22">Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Cues%22">Cues</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Validity%22">Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Processes%22">Cognitive Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1186/s41235-025-00659-w
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2365-7464
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Although it is commonly believed that automation aids human decision-making, conflicting evidence raises questions about whether individuals would gain greater advantages from automation in difficult tasks. Our study examines the combined influence of task difficulty and automation reliability on aided decision-making. We assessed decision efficiency by employing the single-target self-terminating (STST) capacity coefficient in Systems Factorial Technology, estimating the ratio of performance with aided information to that without it. Participants were instructed to perform a shape categorization task, wherein they assessed whether the presented stimulus belonged to one category or another. In Experiment 1, three automation reliability conditions (high reliability, low reliability, and unaided) were tested in separate blocks. Our results indicated that, in general, participants exhibited unlimited capacity when provided with valid automated cues, implying that the decision efficiency was unaltered by automated assistance. Despite the failure to gain extra efficiency, the benefits of automated aids in decision-making for difficult tasks were evident. In Experiment 2, various types of automation reliability were randomly intermixed. In this scenario, the impact of automation reliability on participants' performance diminished; however, the significance of information accuracy increased. Our study illustrates how the presentation of automation, its reliability, and task difficulty interactively influence participants' processing of automated information for decision-making. Our study may improve processing efficiency in automated systems, hence facilitating superior interface design and automation execution.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1478691
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1478691
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1186/s41235-025-00659-w
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
    Subjects:
      – SubjectFull: Task Analysis
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Decision Making
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Reliability
        Type: general
      – SubjectFull: Cues
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Validity
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Cognitive Processes
        Type: general
      – SubjectFull: Computer Software
        Type: general
    Titles:
      – TitleFull: Decision-Making Efficiency with Aided Information: The Impact of Automation Reliability and Task Difficulty
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hanshu Zhang
      – PersonEntity:
          Name:
            NameFull: Ran Zhou
      – PersonEntity:
          Name:
            NameFull: Cheng-You Cheng
      – PersonEntity:
          Name:
            NameFull: Sheng-Hsu Huang
      – PersonEntity:
          Name:
            NameFull: Ming-Hui Cheng
      – PersonEntity:
          Name:
            NameFull: Cheng-Ta Yang
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-electronic
              Value: 2365-7464
          Numbering:
            – Type: volume
              Value: 10
          Titles:
            – TitleFull: Cognitive Research: Principles and Implications
              Type: main
ResultId 1