View in EDS

Towards neural foundation models for vision: Aligning EEG, MEG, and fMRI representations for decoding, encoding, and modality conversion.

Saved in:
Bibliographic Details
Title: Towards neural foundation models for vision: Aligning EEG, MEG, and fMRI representations for decoding, encoding, and modality conversion.
Authors: Ferrante, Matteo1 (AUTHOR) matteo.ferrante@uniroma2.it, Boccato, Tommaso1 (AUTHOR), Rashkov, Grigorii1 (AUTHOR), Toschi, Nicola1,2 (AUTHOR)
Source: Information Fusion. Feb2026:Part B, Vol. 126, pN.PAG-N.PAG. 1p.
Subjects: Electroencephalography, Functional magnetic resonance imaging, Magnetoencephalography, Artificial neural networks, Machine learning
Abstract: This paper presents a novel approach towards creating a foundational model for aligning neural data and visual stimuli across cross-modal representations of brain activity by leveraging contrastive learning. We leverage electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI) data. The framework is validated through three key experiments: decoding visual information from neural data, encoding images into neural representations, and converting between neural modalities. The results highlight the model's ability to accurately capture semantic information across different brain imaging techniques, underscoring its potential in decoding, encoding, and modality conversion tasks. Our results reveal that EEG, MEG, and fMRI signals – despite being collected from different subjects and datasets – can be aligned in a shared vision-grounded space that preserves semantic information. This finding suggests the existence of modality-invariant neural codes and offers a new framework for comparing and decoding brain activity across heterogeneous non-invasive recordings. • Unified Multimodal Representation : Aligns neural data with visual stimuli • Three Key Capabilities : Neural data decoding, encoding, and cross-modality conversion • High Decoding Accuracy : Achieves 93% CLIP 2-way accuracy with fMRI • Cross-Modality Consistency: Translates semantic content across EEG, MEG, and fMRI. • Towards a Neural Foundation Model : Advances a unified framework for neural data, [ABSTRACT FROM AUTHOR]
Copyright of Information Fusion is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 188708363
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Towards neural foundation models for vision: Aligning EEG, MEG, and fMRI representations for decoding, encoding, and modality conversion.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ferrante%2C+Matteo%22">Ferrante, Matteo</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> matteo.ferrante@uniroma2.it</i><br /><searchLink fieldCode="AR" term="%22Boccato%2C+Tommaso%22">Boccato, Tommaso</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Rashkov%2C+Grigorii%22">Rashkov, Grigorii</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Toschi%2C+Nicola%22">Toschi, Nicola</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Information+Fusion%22">Information Fusion</searchLink>. Feb2026:Part B, Vol. 126, pN.PAG-N.PAG. 1p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Electroencephalography%22">Electroencephalography</searchLink><br /><searchLink fieldCode="DE" term="%22Functional+magnetic+resonance+imaging%22">Functional magnetic resonance imaging</searchLink><br /><searchLink fieldCode="DE" term="%22Magnetoencephalography%22">Magnetoencephalography</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+neural+networks%22">Artificial neural networks</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This paper presents a novel approach towards creating a foundational model for aligning neural data and visual stimuli across cross-modal representations of brain activity by leveraging contrastive learning. We leverage electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI) data. The framework is validated through three key experiments: decoding visual information from neural data, encoding images into neural representations, and converting between neural modalities. The results highlight the model's ability to accurately capture semantic information across different brain imaging techniques, underscoring its potential in decoding, encoding, and modality conversion tasks. Our results reveal that EEG, MEG, and fMRI signals – despite being collected from different subjects and datasets – can be aligned in a shared vision-grounded space that preserves semantic information. This finding suggests the existence of modality-invariant neural codes and offers a new framework for comparing and decoding brain activity across heterogeneous non-invasive recordings. • Unified Multimodal Representation : Aligns neural data with visual stimuli • Three Key Capabilities : Neural data decoding, encoding, and cross-modality conversion • High Decoding Accuracy : Achieves 93% CLIP 2-way accuracy with fMRI • Cross-Modality Consistency: Translates semantic content across EEG, MEG, and fMRI. • Towards a Neural Foundation Model : Advances a unified framework for neural data, [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Information Fusion is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=188708363
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.inffus.2025.103650
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 1
        StartPage: N.PAG
    Subjects:
      – SubjectFull: Electroencephalography
        Type: general
      – SubjectFull: Functional magnetic resonance imaging
        Type: general
      – SubjectFull: Magnetoencephalography
        Type: general
      – SubjectFull: Artificial neural networks
        Type: general
      – SubjectFull: Machine learning
        Type: general
    Titles:
      – TitleFull: Towards neural foundation models for vision: Aligning EEG, MEG, and fMRI representations for decoding, encoding, and modality conversion.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ferrante, Matteo
      – PersonEntity:
          Name:
            NameFull: Boccato, Tommaso
      – PersonEntity:
          Name:
            NameFull: Rashkov, Grigorii
      – PersonEntity:
          Name:
            NameFull: Toschi, Nicola
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 05
              M: 02
              Text: Feb2026:Part B
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 15662535
          Numbering:
            – Type: volume
              Value: 126
          Titles:
            – TitleFull: Information Fusion
              Type: main
ResultId 1