View in EDS

Personality in 3D: multimodal deep learning framework for big five trait prediction.

Saved in:
Bibliographic Details
Title: Personality in 3D: multimodal deep learning framework for big five trait prediction.
Authors: Patel, Devraj1 (AUTHOR) devraj_pcse19@diat.ac.in, Dhavale, Sunita V.1 (AUTHOR) sunitadhavale@diat.ac.in, Mhetre, Bhushan B.2 (AUTHOR) bhushanbmhetre@gmail.com
Source: Neural Computing & Applications. Apr2026, Vol. 38 Issue 7, p1-23. 23p.
Abstract: Automatic Personality Prediction (APP) is a key area in affective computing, aiming to infer human personality traits from behavioural cues. This study proposes a multimodal deep learning framework for predicting Big Five personality traits using text, audio, and video data. We employ modality-specific and multimodal datasets annotated for the Big Five traits and explore a range of ardy proposes a multimodal deep learning framework for predicting Big Five personality traits using text, audio, and video data. We employ modality-specific and multimodal datasets annotated for the Big Five traits and explore a range of architectures, including transformers, CNNs, and recurrent models (LSTM, CRNN). Results show that audio features, especially MFCC-1 and MFCC-2, offer high predictive power, while sentiment-aware textual embeddings enhance linguistic modelling. Visual features capture non-verbal cues vital for comprehensive trait assessment. We implement both early and late fusion strategies to integrate affective, linguistic, and visual signals, improving robustness and generalisation. To ensure transparency, we incorporate Explainable AI (XAI) techniques, including SHAP and Grad-CAM, to identify influential features across modalities. This enables human-centred analysis and builds trust in model predictions. Our findings highlight the effectiveness of deep multimodal learning for personality modelling and demonstrate how combining behavioural signals with interpretability tools leads to more adaptive and transparent personality-aware AI systems. [ABSTRACT FROM AUTHOR]
Copyright of Neural Computing & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Automatic Personality Prediction (APP) is a key area in affective computing, aiming to infer human personality traits from behavioural cues. This study proposes a multimodal deep learning framework for predicting Big Five personality traits using text, audio, and video data. We employ modality-specific and multimodal datasets annotated for the Big Five traits and explore a range of ardy proposes a multimodal deep learning framework for predicting Big Five personality traits using text, audio, and video data. We employ modality-specific and multimodal datasets annotated for the Big Five traits and explore a range of architectures, including transformers, CNNs, and recurrent models (LSTM, CRNN). Results show that audio features, especially MFCC-1 and MFCC-2, offer high predictive power, while sentiment-aware textual embeddings enhance linguistic modelling. Visual features capture non-verbal cues vital for comprehensive trait assessment. We implement both early and late fusion strategies to integrate affective, linguistic, and visual signals, improving robustness and generalisation. To ensure transparency, we incorporate Explainable AI (XAI) techniques, including SHAP and Grad-CAM, to identify influential features across modalities. This enables human-centred analysis and builds trust in model predictions. Our findings highlight the effectiveness of deep multimodal learning for personality modelling and demonstrate how combining behavioural signals with interpretability tools leads to more adaptive and transparent personality-aware AI systems. [ABSTRACT FROM AUTHOR]
ISSN:09410643
DOI:10.1007/s00521-026-11979-3