Explanation strategies in humans versus current explainable artificial intelligence: Insights from image classification.
Saved in:
| Title: | Explanation strategies in humans versus current explainable artificial intelligence: Insights from image classification. |
|---|---|
| Authors: | Qi, Ruoxi (AUTHOR), Zheng, Yueyuan (AUTHOR), Yang, Yi (AUTHOR), Cao, Caleb Chen (AUTHOR), Hsiao, Janet H. (AUTHOR) |
| Source: | British Journal of Psychology. May2026, Vol. 117 Issue 2, p479-502. 24p. |
| Subjects: | Task performance, Research funding, Corn, Winter sports, Portable computers, T-test (Statistics), Artificial intelligence, Tennis, Multiple regression analysis, Eye movement measurements, Lemon, Cell phones, Analysis of covariance, Attention, Large-scale brain networks, Analysis of variance, Concepts, Visual perception, Insects, Mollusks, Horses, Mushrooms, Transducers |
| Geographic Terms: | China |
| Abstract: | Explainable AI (XAI) methods provide explanations of AI models, but our understanding of how they compare with human explanations remains limited. Here, we examined human participants' attention strategies when classifying images and when explaining how they classified the images through eye‐tracking and compared their attention strategies with saliency‐based explanations from current XAI methods. We found that humans adopted more explorative attention strategies for the explanation task than the classification task itself. Two representative explanation strategies were identified through clustering: One involved focused visual scanning on foreground objects with more conceptual explanations, which contained more specific information for inferring class labels, whereas the other involved explorative scanning with more visual explanations, which were rated higher in effectiveness for early category learning. Interestingly, XAI saliency map explanations had the highest similarity to the explorative attention strategy in humans, and explanations highlighting discriminative features from invoking observable causality through perturbation had higher similarity to human strategies than those highlighting internal features associated with higher class score. Thus, humans use both visual and conceptual information during explanation, which serve different purposes, and XAI methods that highlight features informing observable causality match better with human explanations, potentially more accessible to users. [ABSTRACT FROM AUTHOR] |
| Copyright of British Journal of Psychology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Psychology and Behavioral Sciences Collection |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Explainable AI (XAI) methods provide explanations of AI models, but our understanding of how they compare with human explanations remains limited. Here, we examined human participants' attention strategies when classifying images and when explaining how they classified the images through eye‐tracking and compared their attention strategies with saliency‐based explanations from current XAI methods. We found that humans adopted more explorative attention strategies for the explanation task than the classification task itself. Two representative explanation strategies were identified through clustering: One involved focused visual scanning on foreground objects with more conceptual explanations, which contained more specific information for inferring class labels, whereas the other involved explorative scanning with more visual explanations, which were rated higher in effectiveness for early category learning. Interestingly, XAI saliency map explanations had the highest similarity to the explorative attention strategy in humans, and explanations highlighting discriminative features from invoking observable causality through perturbation had higher similarity to human strategies than those highlighting internal features associated with higher class score. Thus, humans use both visual and conceptual information during explanation, which serve different purposes, and XAI methods that highlight features informing observable causality match better with human explanations, potentially more accessible to users. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 00071269 |
| DOI: | 10.1111/bjop.12714 |
