Development of a Virtual Assistant Based on LLMs for the Knowledge Domain in Biomedical Metrology.

Saved in:

Bibliographic Details
Title:	Development of a Virtual Assistant Based on LLMs for the Knowledge Domain in Biomedical Metrology.
Authors:	Carreon, Yamilet¹ u18203594@utp.edu.pe, Chicchon, Miguel¹ c11201@utp.edu.pe
Source:	International Journal of Online & Biomedical Engineering. 2025, Vol. 21 Issue 9, p125-137. 13p.
Subjects:	Language models, Source code, Metrology, Biomes
Abstract:	Accurate measurements are essential for effective diagnosis and treatment in the healthcare sector. However, there is limited training in biomedical metrology for Biomedical Engineers, which may hinder their performance. This study evaluated the knowledge of large language models (LLMs) in biomedical metrology to develop a specialized virtual assistant that supports these professionals. The effectiveness of the LLMs was assessed based on the accuracy and coherence of their responses using the CBET Certification exam and metrics such as Rouge-L, F1 score, and cosine similarity. The Llama 3.2-3B Mini model, optimized with retrieval-augmented generation (RAG), showed an increase in the F1 score from 0.402 to 0.526, a Rouge-L score of 0.497, and a cosine similarity of 0.657, demonstrating its ability to generate relevant and accurate responses. The developed virtual assistant represents a promising tool for improving the training and performance of biomedical engineers, ensuring access to precise and reliable information, thereby strengthening safety in the healthcare sector. Our source code is publicly available at https://github.com/yamilet2662/assistant_biome. [ABSTRACT FROM AUTHOR]
	Copyright of International Journal of Online & Biomedical Engineering is the property of International Journal of Online Engineering and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	Accurate measurements are essential for effective diagnosis and treatment in the healthcare sector. However, there is limited training in biomedical metrology for Biomedical Engineers, which may hinder their performance. This study evaluated the knowledge of large language models (LLMs) in biomedical metrology to develop a specialized virtual assistant that supports these professionals. The effectiveness of the LLMs was assessed based on the accuracy and coherence of their responses using the CBET Certification exam and metrics such as Rouge-L, F1 score, and cosine similarity. The Llama 3.2-3B Mini model, optimized with retrieval-augmented generation (RAG), showed an increase in the F1 score from 0.402 to 0.526, a Rouge-L score of 0.497, and a cosine similarity of 0.657, demonstrating its ability to generate relevant and accurate responses. The developed virtual assistant represents a promising tool for improving the training and performance of biomedical engineers, ensuring access to precise and reliable information, thereby strengthening safety in the healthcare sector. Our source code is publicly available at https://github.com/yamilet2662/assistant_biome. [ABSTRACT FROM AUTHOR]
ISSN:	26268493
DOI:	10.3991/ijoe.v21i09.55653