View in EDS

Hierarchically Organized Behavior and Its Neural Foundations: A Reinforcement Learning Perspective

Saved in:
Bibliographic Details
Title: Hierarchically Organized Behavior and Its Neural Foundations: A Reinforcement Learning Perspective
Language: English
Authors: Botvinick, Matthew M., Niv, Yael, Barto, Andrew C.
Source: Cognition. Dec 2009 113(3):262-280.
Availability: Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Peer Reviewed: Y
Physical Description: PDF
Page Count: 19
Publication Date: 2009
Document Type: Journal Articles
Reports - Research
Descriptors: Intelligent Tutoring Systems, Animal Behavior, Reinforcement, Models, Cognitive Processes, Learning Strategies, Brain, Problem Solving, Behavior, Correlation
DOI: 10.1016/j.cognition.2008.08.011
ISSN: 0010-0277
Abstract: Research on human and animal behavior has long emphasized its hierarchical structure--the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as "hierarchical reinforcement learning," which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior. (Contains 6 figures.)
Abstractor: As Provided
Entry Date: 2009
Accession Number: EJ860870
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ860870
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Hierarchically Organized Behavior and Its Neural Foundations: A Reinforcement Learning Perspective
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Botvinick%2C+Matthew+M%2E%22">Botvinick, Matthew M.</searchLink><br /><searchLink fieldCode="AR" term="%22Niv%2C+Yael%22">Niv, Yael</searchLink><br /><searchLink fieldCode="AR" term="%22Barto%2C+Andrew+C%2E%22">Barto, Andrew C.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Cognition%22"><i>Cognition</i></searchLink>. Dec 2009 113(3):262-280.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: PhysDesc
  Label: Physical Description
  Group: PhysDesc
  Data: PDF
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 19
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2009
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Intelligent+Tutoring+Systems%22">Intelligent Tutoring Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Animal+Behavior%22">Animal Behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Reinforcement%22">Reinforcement</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Processes%22">Cognitive Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Strategies%22">Learning Strategies</searchLink><br /><searchLink fieldCode="DE" term="%22Brain%22">Brain</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Behavior%22">Behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1016/j.cognition.2008.08.011
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0010-0277
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Research on human and animal behavior has long emphasized its hierarchical structure--the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as "hierarchical reinforcement learning," which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior. (Contains 6 figures.)
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2009
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ860870
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ860870
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.cognition.2008.08.011
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 19
        StartPage: 262
    Subjects:
      – SubjectFull: Intelligent Tutoring Systems
        Type: general
      – SubjectFull: Animal Behavior
        Type: general
      – SubjectFull: Reinforcement
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Cognitive Processes
        Type: general
      – SubjectFull: Learning Strategies
        Type: general
      – SubjectFull: Brain
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Behavior
        Type: general
      – SubjectFull: Correlation
        Type: general
    Titles:
      – TitleFull: Hierarchically Organized Behavior and Its Neural Foundations: A Reinforcement Learning Perspective
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Botvinick, Matthew M.
      – PersonEntity:
          Name:
            NameFull: Niv, Yael
      – PersonEntity:
          Name:
            NameFull: Barto, Andrew C.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2009
          Identifiers:
            – Type: issn-print
              Value: 0010-0277
          Numbering:
            – Type: volume
              Value: 113
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Cognition
              Type: main
ResultId 1