Prostate magnetic resonance imaging/transrectal ultrasound registration using vision transformer and convolutional neural network.

Saved in:

Bibliographic Details
Title:	Prostate magnetic resonance imaging/transrectal ultrasound registration using vision transformer and convolutional neural network.
Authors:	Mahmoudi, Hanae¹ hanae.mahmoudi@usmba.ac.ma, Ramadan, Hiba¹ hiba.ramadan@usmba.ac.ma, Riffi, Jamal¹ riffi.jamal@gmail.com, Tairi, Hamid¹ hamidtairi@gmail.com
Source:	International Journal of Electrical & Computer Engineering (2088-8708). Jun2026, Vol. 16 Issue 3, p1188-1198. 11p.
Subjects:	Image registration, Convolutional neural networks, Three-dimensional imaging, Diagnostic imaging, Transformer models
Abstract:	Multimodal registration of 3D medical images (3D-MReg) plays a key role in several medical applications and remains a very challenging task as it deals with multimodal images and volumetric objects at the same time. Recently, convolutional neural networks (CNNs) based approaches have been proposed to solve 3D-MReg. However, these techniques cannot preserve the global spatial context required for accurate affine registration since they rely on convolution and regional clustering operations. To solve these problems, we propose a supervised approach that combines both CNN and the vision transformer (ViT) to predict a dense displacement field (DDF). In a first step, our method investigates the power of ViT to capture global voxels dependencies for initial rigid alignment. Then we exploit the force of CNNs to focus on local details within pre-aligned concatenated input 3D moving and fixed images and estimate DDF, which is then applied to the moving labels. Our method has been validated in a prostate magnetic resonance imaging/transrectal ultrasound (MRI/TRUS) dataset and achieved promising results compared to previous work based on only CNNs. [ABSTRACT FROM AUTHOR]
	Copyright of International Journal of Electrical & Computer Engineering (2088-8708) is the property of Institute of Advanced Engineering & Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	Multimodal registration of 3D medical images (3D-MReg) plays a key role in several medical applications and remains a very challenging task as it deals with multimodal images and volumetric objects at the same time. Recently, convolutional neural networks (CNNs) based approaches have been proposed to solve 3D-MReg. However, these techniques cannot preserve the global spatial context required for accurate affine registration since they rely on convolution and regional clustering operations. To solve these problems, we propose a supervised approach that combines both CNN and the vision transformer (ViT) to predict a dense displacement field (DDF). In a first step, our method investigates the power of ViT to capture global voxels dependencies for initial rigid alignment. Then we exploit the force of CNNs to focus on local details within pre-aligned concatenated input 3D moving and fixed images and estimate DDF, which is then applied to the moving labels. Our method has been validated in a prostate magnetic resonance imaging/transrectal ultrasound (MRI/TRUS) dataset and achieved promising results compared to previous work based on only CNNs. [ABSTRACT FROM AUTHOR]
ISSN:	20888708
DOI:	10.11591/ijece.v16i3.pp1188-1198