View in EDS

Characterizing Datasets for Social Visual Question Answering, and the New TinySocial Dataset

Saved in:
Bibliographic Details
Title: Characterizing Datasets for Social Visual Question Answering, and the New TinySocial Dataset
Language: English
Authors: Chen, Zhanwen, Li, Shiyao, Rashedi, Roxanne, Zi, Xiaoman, Elrod-Erickson, Morgan, Hollis, Bryan, Maliakal, Angela, Shen, Xinyu, Zhao, Simeng, Kunda, Maithilee
Source: Grantee Submission. 2020.
Peer Reviewed: Y
Page Count: 6
Publication Date: 2020
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R324A180171
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Junior High Schools
Middle Schools
Secondary Education
Descriptors: Visual Stimuli, Questioning Techniques, Social Cognition, Video Technology, Scoring Rubrics, Artificial Intelligence, Theory of Mind, Data Collection, Heuristics, Films, Middle School Students, Autism Spectrum Disorders
Abstract: Modern social intelligence includes the ability to watch videos and answer questions about social and theory-of-mind-related content, e.g., for a scene in "Harry Potter," "Is the father really upset about the boys flying the car?" Social visual question answering (social VQA) is emerging as a valuable methodology for studying social reasoning in both humans (e.g., children with autism) and AI agents. However, this problem space spans enormous variations in both videos and questions. We discuss methods for creating and characterizing social VQA datasets, including: (1) crowdsourcing versus in-house authoring, including sample comparisons of two new datasets that we created (TinySocial-Crowd and TinySocial-InHouse) and the previously existing Social-IQ dataset; (2) a new rubric for characterizing the difficulty and content of a given video; and (3) a new rubric for characterizing question types. We close by describing how having well-characterized social VQA datasets will enhance the explainability of AI agents and can also inform assessments and educational interventions for people.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2022
Access URL: https://ieeexplore.ieee.org/abstract/document/9278057
Accession Number: ED622666
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED622666
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED622666
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Characterizing Datasets for Social Visual Question Answering, and the New TinySocial Dataset
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chen%2C+Zhanwen%22">Chen, Zhanwen</searchLink><br /><searchLink fieldCode="AR" term="%22Li%2C+Shiyao%22">Li, Shiyao</searchLink><br /><searchLink fieldCode="AR" term="%22Rashedi%2C+Roxanne%22">Rashedi, Roxanne</searchLink><br /><searchLink fieldCode="AR" term="%22Zi%2C+Xiaoman%22">Zi, Xiaoman</searchLink><br /><searchLink fieldCode="AR" term="%22Elrod-Erickson%2C+Morgan%22">Elrod-Erickson, Morgan</searchLink><br /><searchLink fieldCode="AR" term="%22Hollis%2C+Bryan%22">Hollis, Bryan</searchLink><br /><searchLink fieldCode="AR" term="%22Maliakal%2C+Angela%22">Maliakal, Angela</searchLink><br /><searchLink fieldCode="AR" term="%22Shen%2C+Xinyu%22">Shen, Xinyu</searchLink><br /><searchLink fieldCode="AR" term="%22Zhao%2C+Simeng%22">Zhao, Simeng</searchLink><br /><searchLink fieldCode="AR" term="%22Kunda%2C+Maithilee%22">Kunda, Maithilee</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2020.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 6
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2020
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R324A180171
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Visual+Stimuli%22">Visual Stimuli</searchLink><br /><searchLink fieldCode="DE" term="%22Questioning+Techniques%22">Questioning Techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Social+Cognition%22">Social Cognition</searchLink><br /><searchLink fieldCode="DE" term="%22Video+Technology%22">Video Technology</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring+Rubrics%22">Scoring Rubrics</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Theory+of+Mind%22">Theory of Mind</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Collection%22">Data Collection</searchLink><br /><searchLink fieldCode="DE" term="%22Heuristics%22">Heuristics</searchLink><br /><searchLink fieldCode="DE" term="%22Films%22">Films</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Autism+Spectrum+Disorders%22">Autism Spectrum Disorders</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Modern social intelligence includes the ability to watch videos and answer questions about social and theory-of-mind-related content, e.g., for a scene in "Harry Potter," "Is the father really upset about the boys flying the car?" Social visual question answering (social VQA) is emerging as a valuable methodology for studying social reasoning in both humans (e.g., children with autism) and AI agents. However, this problem space spans enormous variations in both videos and questions. We discuss methods for creating and characterizing social VQA datasets, including: (1) crowdsourcing versus in-house authoring, including sample comparisons of two new datasets that we created (TinySocial-Crowd and TinySocial-InHouse) and the previously existing Social-IQ dataset; (2) a new rubric for characterizing the difficulty and content of a given video; and (3) a new rubric for characterizing question types. We close by describing how having well-characterized social VQA datasets will enhance the explainability of AI agents and can also inform assessments and educational interventions for people.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://ieeexplore.ieee.org/abstract/document/9278057" linkWindow="_blank">https://ieeexplore.ieee.org/abstract/document/9278057</link>
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED622666
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED622666
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 6
    Subjects:
      – SubjectFull: Visual Stimuli
        Type: general
      – SubjectFull: Questioning Techniques
        Type: general
      – SubjectFull: Social Cognition
        Type: general
      – SubjectFull: Video Technology
        Type: general
      – SubjectFull: Scoring Rubrics
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Theory of Mind
        Type: general
      – SubjectFull: Data Collection
        Type: general
      – SubjectFull: Heuristics
        Type: general
      – SubjectFull: Films
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Autism Spectrum Disorders
        Type: general
    Titles:
      – TitleFull: Characterizing Datasets for Social Visual Question Answering, and the New TinySocial Dataset
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chen, Zhanwen
      – PersonEntity:
          Name:
            NameFull: Li, Shiyao
      – PersonEntity:
          Name:
            NameFull: Rashedi, Roxanne
      – PersonEntity:
          Name:
            NameFull: Zi, Xiaoman
      – PersonEntity:
          Name:
            NameFull: Elrod-Erickson, Morgan
      – PersonEntity:
          Name:
            NameFull: Hollis, Bryan
      – PersonEntity:
          Name:
            NameFull: Maliakal, Angela
      – PersonEntity:
          Name:
            NameFull: Shen, Xinyu
      – PersonEntity:
          Name:
            NameFull: Zhao, Simeng
      – PersonEntity:
          Name:
            NameFull: Kunda, Maithilee
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2020
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1