View in EDS

Weakly-supervised convolutional neural networks for multimodal image registration.

Saved in:
Bibliographic Details
Title: Weakly-supervised convolutional neural networks for multimodal image registration.
Authors: Hu, Yipeng1,2 yipeng.hu@ucl.ac.uk, Modat, Marc1,3, Gibson, Eli1, Li, Wenqi1,3, Ghavami, Nooshin1, Bonmati, Ester1, Wang, Guotai1,3, Bandula, Steven4, Moore, Caroline M.5, Emberton, Mark5, Ourselin, Sébastien1,3, Noble, J. Alison2, Barratt, Dean C.1,3, Vercauteren, Tom1,3
Source: Medical Image Analysis. Oct2018, Vol. 49, p1-13. 13p.
Subjects: Neural circuitry, Diagnostic imaging, Prostate cancer, Ultrasonic imaging, Organs (Anatomy)
Abstract: Highlights • A method to infer voxel-level correspondence from higher-level anatomical labels. • Efficient and fully-automated registration for MR and ultrasound prostate images. • Validation experiments with 108 pairs of labelled interventional patient images. • Open-source implementation. Abstract One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels. Graphical abstract Image, graphical abstract [ABSTRACT FROM AUTHOR]
Copyright of Medical Image Analysis is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 131660848
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Weakly-supervised convolutional neural networks for multimodal image registration.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hu%2C+Yipeng%22">Hu, Yipeng</searchLink><relatesTo>1,2</relatesTo><i> yipeng.hu@ucl.ac.uk</i><br /><searchLink fieldCode="AR" term="%22Modat%2C+Marc%22">Modat, Marc</searchLink><relatesTo>1,3</relatesTo><br /><searchLink fieldCode="AR" term="%22Gibson%2C+Eli%22">Gibson, Eli</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Li%2C+Wenqi%22">Li, Wenqi</searchLink><relatesTo>1,3</relatesTo><br /><searchLink fieldCode="AR" term="%22Ghavami%2C+Nooshin%22">Ghavami, Nooshin</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Bonmati%2C+Ester%22">Bonmati, Ester</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Wang%2C+Guotai%22">Wang, Guotai</searchLink><relatesTo>1,3</relatesTo><br /><searchLink fieldCode="AR" term="%22Bandula%2C+Steven%22">Bandula, Steven</searchLink><relatesTo>4</relatesTo><br /><searchLink fieldCode="AR" term="%22Moore%2C+Caroline+M%2E%22">Moore, Caroline M.</searchLink><relatesTo>5</relatesTo><br /><searchLink fieldCode="AR" term="%22Emberton%2C+Mark%22">Emberton, Mark</searchLink><relatesTo>5</relatesTo><br /><searchLink fieldCode="AR" term="%22Ourselin%2C+Sébastien%22">Ourselin, Sébastien</searchLink><relatesTo>1,3</relatesTo><br /><searchLink fieldCode="AR" term="%22Noble%2C+J%2E+Alison%22">Noble, J. Alison</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Barratt%2C+Dean+C%2E%22">Barratt, Dean C.</searchLink><relatesTo>1,3</relatesTo><br /><searchLink fieldCode="AR" term="%22Vercauteren%2C+Tom%22">Vercauteren, Tom</searchLink><relatesTo>1,3</relatesTo>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Medical+Image+Analysis%22">Medical Image Analysis</searchLink>. Oct2018, Vol. 49, p1-13. 13p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Neural+circuitry%22">Neural circuitry</searchLink><br /><searchLink fieldCode="DE" term="%22Diagnostic+imaging%22">Diagnostic imaging</searchLink><br /><searchLink fieldCode="DE" term="%22Prostate+cancer%22">Prostate cancer</searchLink><br /><searchLink fieldCode="DE" term="%22Ultrasonic+imaging%22">Ultrasonic imaging</searchLink><br /><searchLink fieldCode="DE" term="%22Organs+%28Anatomy%29%22">Organs (Anatomy)</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Highlights • A method to infer voxel-level correspondence from higher-level anatomical labels. • Efficient and fully-automated registration for MR and ultrasound prostate images. • Validation experiments with 108 pairs of labelled interventional patient images. • Open-source implementation. Abstract One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels. Graphical abstract Image, graphical abstract [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Medical Image Analysis is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=131660848
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.media.2018.07.002
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 1
    Subjects:
      – SubjectFull: Neural circuitry
        Type: general
      – SubjectFull: Diagnostic imaging
        Type: general
      – SubjectFull: Prostate cancer
        Type: general
      – SubjectFull: Ultrasonic imaging
        Type: general
      – SubjectFull: Organs (Anatomy)
        Type: general
    Titles:
      – TitleFull: Weakly-supervised convolutional neural networks for multimodal image registration.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hu, Yipeng
      – PersonEntity:
          Name:
            NameFull: Modat, Marc
      – PersonEntity:
          Name:
            NameFull: Gibson, Eli
      – PersonEntity:
          Name:
            NameFull: Li, Wenqi
      – PersonEntity:
          Name:
            NameFull: Ghavami, Nooshin
      – PersonEntity:
          Name:
            NameFull: Bonmati, Ester
      – PersonEntity:
          Name:
            NameFull: Wang, Guotai
      – PersonEntity:
          Name:
            NameFull: Bandula, Steven
      – PersonEntity:
          Name:
            NameFull: Moore, Caroline M.
      – PersonEntity:
          Name:
            NameFull: Emberton, Mark
      – PersonEntity:
          Name:
            NameFull: Ourselin, Sébastien
      – PersonEntity:
          Name:
            NameFull: Noble, J. Alison
      – PersonEntity:
          Name:
            NameFull: Barratt, Dean C.
      – PersonEntity:
          Name:
            NameFull: Vercauteren, Tom
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: Oct2018
              Type: published
              Y: 2018
          Identifiers:
            – Type: issn-print
              Value: 13618415
          Numbering:
            – Type: volume
              Value: 49
          Titles:
            – TitleFull: Medical Image Analysis
              Type: main
ResultId 1