View in EDS

Chinese ethnic minority book classification by large language models within CLC.

Saved in:
Bibliographic Details
Title: Chinese ethnic minority book classification by large language models within CLC.
Authors: Jia, Junzhi1 (AUTHOR) junzhij@163.com, Xu, Hui1 (AUTHOR) xuhui111@ruc.edu.cn, Guo, Yiqian1 (AUTHOR) guoyiqian@ruc.edu.cn, Gao, Wenjing2 (AUTHOR) 437189571@qq.com
Source: Electronic Library. 2026, Vol. 44 Issue 2, p251-270. 20p.
Subject Terms: *Library science, *Artificial intelligence, *Books, *Experimental design, *Information retrieval, *Comparative studies, Classification of books, Research funding, Natural language processing, Ethnology, Descriptive statistics, Minorities, Data analysis software
Geographic Terms: China
Abstract: Purpose: This study evaluates the capabilities and limitations of large language models (LLMs) in classifying Chinese ethic minority books under the scheme of Chinese Library Classification. Design/methodology/approach: A test collection of Chinese ethnic minority bibliographic records was constructed, and prompt engineering was used to compare the classification performance of DeepSeek-v3 and ChatGPT-4o under two input scenarios: "title + abstract" and "title only." By designing evaluation metrics that include accuracy, granularity and error-type analysis, this study systematically evaluates the performance differences between the models, diagnoses the causes of errors and proposes improvement strategies. Findings: Experimental results show both models performed well in the broad category classification of Chinese ethnic minority books, with DeepSeek-v3 exceeding 80% accuracy. Incorporating abstracts further improved accuracy and prompted longer, more detailed classification codes. However, accuracy declined for both as classification codes grew more specific. DeepSeek-v3 significantly outperformed ChatGPT-4o, achieving an overall accuracy of 40.78% and 33.50% with and without abstracts, respectively, while ChatGPT-4o remained below 6%. On the basis of classification error analysis, this study proposes improvements in classification system design, model capability enhancement and human–artificial intelligence (AI) collaboration to guide practical improvements in organizing ethnic minority resources. Originality/value: Combining librarianship, ethnography and artificial intelligence, this study is the first to compare the classification ability of different large-language models for Chinese ethnic minority books. It reveals cultural limitations in knowledge organization systems, identifies the "capability threshold" of LLMs in cultural context processing and establishes an empirical basis for developing culturally-aware AI governance frameworks. [ABSTRACT FROM AUTHOR]
Copyright of Electronic Library is the property of Emerald Publishing Limited and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Description
Abstract:Purpose: This study evaluates the capabilities and limitations of large language models (LLMs) in classifying Chinese ethic minority books under the scheme of Chinese Library Classification. Design/methodology/approach: A test collection of Chinese ethnic minority bibliographic records was constructed, and prompt engineering was used to compare the classification performance of DeepSeek-v3 and ChatGPT-4o under two input scenarios: "title + abstract" and "title only." By designing evaluation metrics that include accuracy, granularity and error-type analysis, this study systematically evaluates the performance differences between the models, diagnoses the causes of errors and proposes improvement strategies. Findings: Experimental results show both models performed well in the broad category classification of Chinese ethnic minority books, with DeepSeek-v3 exceeding 80% accuracy. Incorporating abstracts further improved accuracy and prompted longer, more detailed classification codes. However, accuracy declined for both as classification codes grew more specific. DeepSeek-v3 significantly outperformed ChatGPT-4o, achieving an overall accuracy of 40.78% and 33.50% with and without abstracts, respectively, while ChatGPT-4o remained below 6%. On the basis of classification error analysis, this study proposes improvements in classification system design, model capability enhancement and human–artificial intelligence (AI) collaboration to guide practical improvements in organizing ethnic minority resources. Originality/value: Combining librarianship, ethnography and artificial intelligence, this study is the first to compare the classification ability of different large-language models for Chinese ethnic minority books. It reveals cultural limitations in knowledge organization systems, identifies the "capability threshold" of LLMs in cultural context processing and establishes an empirical basis for developing culturally-aware AI governance frameworks. [ABSTRACT FROM AUTHOR]
ISSN:02640473