The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers
Saved in:
| Title: | The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers |
|---|---|
| Language: | English |
| Authors: | Botvinick, Matthew M., Cohen, Jonathan D. |
| Source: | Cognitive Science. Aug 2014 38(6):1249-1285. |
| Availability: | Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA |
| Peer Reviewed: | Y |
| Page Count: | 37 |
| Publication Date: | 2014 |
| Sponsoring Agency: | National Science Foundation National Institute of Mental Health (DHHS) |
| Contract Number: | CRCNS 1207833 R01MH098815-01 |
| Document Type: | Journal Articles Reports - Evaluative |
| Descriptors: | Cognitive Ability, Guidelines, Models, Cognitive Processes, Brain Hemisphere Functions, Computers, Cognitive Science |
| DOI: | 10.1111/cogs.12126 |
| ISSN: | 0364-0213 |
| Abstract: | Cognitive control has long been one of the most active areas of computational modeling work in cognitive science. The focus on computational models as a medium for specifying and developing theory predates the PDP books, and cognitive control was not one of the areas on which they focused. However, the framework they provided has injected work on cognitive control with new energy and new ideas. On the occasion of the books' anniversary, we review computational modeling in the study of cognitive control, with a focus on the influence that the PDP approach has brought to bear in this area. Rather than providing a comprehensive review, we offer a framework for thinking about past and future modeling efforts in this domain. We define control in terms of the optimal parameterization of task processing. From this vantage point, the development of control systems in the brain can be seen as responding to the structure of naturalistic tasks, through the filter of the brain systems with which control directly interfaces. This perspective lays open a set of fascinating but difficult research questions, which together define an important frontier for future computational research. |
| Abstractor: | As Provided |
| Entry Date: | 2014 |
| Accession Number: | EJ1035921 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEd8LaDQCe3RlpvVh_Y_8FxAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDLqiFNuztFsbMcy54wIBEICBmnIbvuMAocxk-dAknEGGDytRhCZXle4wrJbMVX1eSlN-gLC2hH8sXW15dzhbRzdeOUCNh8em2RaCBPCIxqad_iBE8MBA4rlqI-cVE9PL6I3s7zmMMg5lCvS7noDPjKJAXPiFSw-MiLggY66zOMohywgYydOITPMaA94YUyJqolZxVJTOLgPi2XxeVQKdAHNY5K9W-LUZdgrDxG4= Text: Availability: 1 Value: <anid>AN0097523350;cgn01aug.14;2018Jun28.13:57;v2.2.500</anid> <title id="AN0097523350-1">The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers. </title> <p>Cognitive control has long been one of the most active areas of computational modeling work in cognitive science. The focus on computational models as a medium for specifying and developing theory predates the PDP books, and cognitive control was not one of the areas on which they focused. However, the framework they provided has injected work on cognitive control with new energy and new ideas. On the occasion of the books' anniversary, we review computational modeling in the study of cognitive control, with a focus on the influence that the PDP approach has brought to bear in this area. Rather than providing a comprehensive review, we offer a framework for thinking about past and future modeling efforts in this domain. We define control in terms of the optimal parameterization of task processing. From this vantage point, the development of control systems in the brain can be seen as responding to the structure of naturalistic tasks, through the filter of the brain systems with which control directly interfaces. This perspective lays open a set of fascinating but difficult research questions, which together define an important frontier for future computational research.</p> <p>Cognitive control; Computational modeling</p> <p>As we reflect on the impact of the PDP volumes over the quarter century since their initial publication, it also seems a good time to assess developments, over the same period, in the study of cognitive control. There is, after all, a close historical alignment between the emergence of connectionism and the emergence of cognitive control as a well‐defined topic of research. Like connectionism, which drew ideas from early pioneers such as Hebb ([<reflink idref="bib71" id="ref1">71</reflink>] ), Selfridge ([<reflink idref="bib139" id="ref2">139</reflink>] ), and Rosenblatt ([<reflink idref="bib132" id="ref3">132</reflink>] ), the field of cognitive control also took root in trail‐blazing research in the middle of the 20th century, including that of Lashley ([<reflink idref="bib87" id="ref4">87</reflink>] ), Miller, Galanter, and Pribram ([<reflink idref="bib104" id="ref5">104</reflink>] ), and Atkinson and Shiffrin ([<reflink idref="bib7" id="ref6">7</reflink>] ). However, just as connectionism crystallized in the 1980s with the publication of the PDP books, it was not until that same time that various threads came together to establish cognitive control as a well‐defined and intensive area of research. In cognitive psychology, the work of Posner and Snyder ([<reflink idref="bib125" id="ref7">125</reflink>] ), Shiffrin and Schneider ([<reflink idref="bib144" id="ref8">144</reflink>] ), Norman and Shallice ([<reflink idref="bib108" id="ref9">108</reflink>] ), and Baddeley (Baddeley, [<reflink idref="bib8" id="ref10">8</reflink>] ; Baddeley &amp; Hitch, [<reflink idref="bib9" id="ref11">9</reflink>] ) combined to introduce the distinction between controlled and automatic processing, and the notion of a central executive or supervisory system, within the psychological canon. These ideas resonated with a contemporaneous burgeoning of research in neuropsychology and neurophysiology focused on the role of the prefrontal cortex in control functions such as task representation (Lhermitte, [<reflink idref="bib88" id="ref12">88</reflink>] ; Luria, [<reflink idref="bib92" id="ref13">92</reflink>] ), the temporal integration of behavior (Fuster [<reflink idref="bib62" id="ref14">62</reflink>] , [<reflink idref="bib63" id="ref15">63</reflink>] ), planning (Duncan, [<reflink idref="bib53" id="ref16">53</reflink>] ; Passingham, [<reflink idref="bib116" id="ref17">116</reflink>] ; Shallice, [<reflink idref="bib142" id="ref18">142</reflink>] ), performance monitoring (Petrides &amp; Milner, [<reflink idref="bib119" id="ref19">119</reflink>] ), and working memory maintenance (Fuster &amp; Alexander, [<reflink idref="bib64" id="ref20">64</reflink>] ; Goldman‐Rakic, [<reflink idref="bib66" id="ref21">66</reflink>] ).</p> <p>An initial point of contact between the PDP movement and research on control was in the domain of computational modeling. The PDP approach stimulated efforts to develop explicit, runnable computational models of control, capable of addressing both detailed behavioral data and relevant neuroscientific findings. Certainly, computational ideas had played an important role in cognitive control research from very early on. Indeed, the very notion of control arose in part through an analogy to computer architecture (Atkinson &amp; Shiffrin, [<reflink idref="bib7" id="ref22">7</reflink>] ) and was inspired by cybernetic control theory in engineering (e.g., Miller et al., [<reflink idref="bib104" id="ref23">104</reflink>] ). However, the advent of PDP, alongside the growing influence of production system modeling (Anderson, [<reflink idref="bib3" id="ref24">3</reflink>] ; Laird, Newell, &amp; Rosenbloom, [<reflink idref="bib85" id="ref25">85</reflink>] ), ushered in a new era in control research in which computational modeling assumed a central role in the expression and development of theory and the generation of experimental predictions. Furthermore, unlike other theoretical developments, the PDP framework tethered the development of models to a consideration of how the mechanisms they specified could be implemented by the brain.</p> <p>Despite the side‐by‐side emergence of connectionism and research on cognitive control, as well as their shared focus on computation, the theoretical perspectives adopted by these two appeared at least initially to be quite different. The PDP framework, in some of its most influential applications, focused on distributed representation, heterarchical processing, and emergence. In contrast, early control research seemed fundamentally tied to principles of symbolic representation, sequential hierarchical processing, and modularity. Some of the most important early developments in both PDP and cognitive control sprang from efforts to confront and perhaps reconcile these two perspectives (see Cooper, [<reflink idref="bib46" id="ref26">46</reflink>] ). Nevertheless, the PDP approach initially came under attack for its supposed failure to address complex cognitive processes that demand cognitive control, such as planning and problem solving. From this, it was often inferred that understanding such processes (and cognitive control itself) requires a more abstract, symbolic level of analysis than the PDP approach affords. However, subsequent work has shown this to be a misconception. Substantial progress has been made using the PDP approach to understand the mechanisms upon which cognitive control is built. Furthermore, as this work matures, we find that it is coming into increasing contact with issues that dominated the early, classic PDP work, especially those concerning the influence of learning mechanisms and the environment on the nature of representations.</p> <p>With an eye toward such issues, the present article pursues two goals. The first is to consider the still‐unfolding computational era in control research and attempt to digest what it has yielded so far. Our second, and perhaps more important, goal is to consider where the field needs to head next. In pursuing these goals, we will not attempt to provide a comprehensive survey of past work (for this, see Botvinick, [<reflink idref="bib19" id="ref27">19</reflink>] ; Miller &amp; Cohen, [<reflink idref="bib103" id="ref28">103</reflink>] ; O'Reilly, Herd, &amp; Pauli, [<reflink idref="bib111" id="ref29">111</reflink>] ). Furthermore, with apologies to esteemed colleagues, we will tend to concentrate on our own efforts in the field as illustrative of the contributions that the PDP approach has brought to bear. Rather than a literature review, what we seek to provide is a conceptual framework within which the accomplishments and limitations of past work can be clearly identified, and which throws into relief the most pressing current challenges.</p> <p>The framework we offer groups computational modeling studies into three categories, pursued in three sequential epochs, each defined by a particular focal question. The earliest PDP‐inspired models of cognitive control focused primarily on the entry‐level question of how control functions influence information processing. These models sought to specify the mechanisms by which control might exert its “top‐down,” regulative effects on information processing. This first wave of models sets the scene for a second phase, the scope of which expanded to embrace the question of how control is adapted to current demands, and the closely related question of how control might emerge from learning and experience. These models asked: How do control functions decide what regulative actions to take? What are the “bottom‐up” factors that govern the selection and modulation of “top‐down” control signals? And how is the response of the control system to such factors tuned through learning? As we shall detail, an important development in this phase of the modeling enterprise involved bringing to bear principles of optimization, with links being established to work on reinforcement learning and perceptual decision making.</p> <p>Our view, and the central assertion of the present paper, is that a computational understanding of cognitive control is presently on the threshold of a third, critically important phase. The pivotal question for this phase of research is harder to articulate, and indeed is only slowly coming into focus, but it centers on the issue of structure: How and why does the control system—its architecture and the representations and operations that inhere within it—come to assume its specific form or functional organization? And how does this reflect and adapt to the structure of the task environment? As we shall explain, we believe that confronting these and related questions concerning the structure of control represent a necessary step toward understanding how control operates in the setting of complex, naturalistic behavior.</p> <p>Cutting across each of the territories of research that we will examine are two critical features of control: Its remarkable flexibility (regarding the diversity of behavior it supports), and its equally striking constraints (regarding the number of control‐demanding behaviors that can be executed at once). The challenge of accounting for these two characteristics will provide an additional, overarching theme in our survey.</p> <p>Although the anniversary of the PDP books provides the occasion for the present work, our objective is not specifically to argue for a connectionist approach to control function, and indeed we will not limit ourselves strictly to a discussion of PDP‐inspired models. Nevertheless, two core tenets of the PDP approach are central to the work we discuss. The first is the basic assumption that behavior emerges out of an interaction among three elements: (a) the structure of the behavioral domain, defined by a specific set of environmental states or circumstances, available actions, and action‐effects; (b) the native structure of the information‐processing architecture itself, which is necessarily constrained by its implementation in the brain; and (c) a set of domain‐general learning and decision‐making mechanisms, which can be understood as optimizing behavior relative to a specific objective function. The second basic tenet of the PDP approach that we will advocate is the dictum that one should avoid stipulating what one seeks to explain (see McClelland et al., [<reflink idref="bib97" id="ref30">97</reflink>] ; Plaut &amp; McClelland, [<reflink idref="bib121" id="ref31">121</reflink>] ). The ability to explain seemingly irreducible aspects of behavior in terms of more basic underlying principles and mechanisms is a core strength of the PDP approach, and this will provide the gold standard against which we evaluate past and prospective future computational work on cognitive control.</p> <hd id="AN0097523350-2">Phase 1: The implementation of control mechanisms and their influence on processing</hd> <hd id="AN0097523350-3">Controlled versus automatic processing</hd> <p>By the early 1980s, when the PDP framework first began to emerge as a theoretical force, the distinction between controlled and automatic processes had assumed a central position in cognitive psychology. Controlled processes were characterized as slow, effortful, and dependent on a limited‐capacity central resource that rendered them subject to competition from other control‐demanding processes, and to interference from automatic processes that favored competing responses. In their landmark treatment of the topic, Posner and Snyder ([<reflink idref="bib125" id="ref32">125</reflink>] ) pointed to the Stroop task (Stroop, [<reflink idref="bib150" id="ref33">150</reflink>] ) as a canonical example of the distinction between a controlled process (color naming) and an automatic one (word reading). In the Stroop task, participants are presented with a written word displayed in color (e.g., GREEN), and asked to either read the word (say “green”) or name the color in which it is displayed (say “red”). Word reading is invariably faster and relatively impervious to influence from the color of the display. Conversely, color naming is slower and is influenced by the nature of the word—if it agrees (e.g., RED), then color naming is faster than if it does not (as in the example above). Posner and Snyder pointed to the characteristics of word reading as diagnostic of an automatic process (fast, immune to interference, but able to produce it), and of color naming as controlled (slower and subject to interference). Posner and Snyder's interpretation was compelling and played a dominant role in framing ensuing research on attention and control. However, several problems began to emerge with their account and other theories that invoked the notion of control (e.g., Baddeley, [<reflink idref="bib8" id="ref34">8</reflink>] ; Baddeley &amp; Hitch, [<reflink idref="bib9" id="ref35">9</reflink>] ).</p> <p>First, these accounts implied a qualitative and absolute distinction between controlled and automatic processing: Either a process was controlled or it was automatic. However, this idea quickly came under attack, as observations accumulated of putatively automatic processes that were subject to interference and, conversely, controlled processes that could be made to appear automatic (see Kahneman &amp; Treisman, [<reflink idref="bib79" id="ref36">79</reflink>] ; MacLeod, [<reflink idref="bib94" id="ref37">94</reflink>] and Cohen et al., [<reflink idref="bib36" id="ref38">36</reflink>] for reviews). The discrete distinction between controlled and automatic processing gradually yielded to the view that processes lie along a continuum of automaticity (e.g., Kahneman &amp; Treisman, [<reflink idref="bib79" id="ref39">79</reflink>] ; Cohen et al., [<reflink idref="bib36" id="ref40">36</reflink>] ), based on degree of practice (see Shiffrin &amp; Schneider, [<reflink idref="bib144" id="ref41">144</reflink>] ), and that the reliance of a process on control depends not only on its absolute position along the continuum but also on its position relative to other processes with which it finds itself in competition (e.g., MacLeod &amp; Dunbar, [<reflink idref="bib95" id="ref42">95</reflink>] ).</p> <p>Another, more important limitation of early theories of control was their focus on phenomenology rather than mechanism. Controlled processing was described as “effortful,” “capacity limited,” and sequential in nature. However, no account was given for how it was implemented in a physical system, no less the brain. The emergence of production system models provided one response to this concern. Production systems comprise collections of condition‐action rules that compete for expression based on the activity of propositions in declarative (working) memory. Norman and Shallice ([<reflink idref="bib108" id="ref43">108</reflink>] ) defined controlled processes in terms of a superordinate set of productions that controlled the state of working memory and thus the execution of behavior. Frameworks such as Anderson's ACT* (Anderson, [<reflink idref="bib3" id="ref44">3</reflink>] ) and Newell's SOAR (Laird et al., [<reflink idref="bib85" id="ref45">85</reflink>] ) offered the first “unified theories of cognition” in terms of production systems and were used to implement specific cognitive processes in terms of production system mechanisms, including the direction of attention and control of behavior. Others followed, such as EPIC (Meyer &amp; Kieras, [<reflink idref="bib100" id="ref46">100</reflink>] ) and ACT‐R (Anderson, [<reflink idref="bib4" id="ref47">4</reflink>] ), a revision of ACT* that incorporates normative constraints on its mechanisms and continues to play an important role in modeling cognitive function. However, an initial limitation of production system architectures was the lack of a clear mapping onto brain function (see Anderson et al., [<reflink idref="bib5" id="ref48">5</reflink>] for some more recent developments in ACT‐R). Cohen, Dunbar, and McClelland ([<reflink idref="bib36" id="ref49">36</reflink>] ) addressed this challenge by developing a PDP model of the Stroop effect. This model provided an account of automaticity and the role of control in processing in terms of the basic elements of typical PDP models, without recourse to any special or qualitatively distinct mechanisms needed for control. This model became a foundation for much of the subsequent work on cognitive control using the PDP framework. Their model is depicted in Fig. [NaN] .</p> <hd id="AN0097523350-4">The Stroop model</hd> <p>Task performance is simulated as the flow of activity from a set of processing units representing features of the stimulus through a set of associative units to ones representing potential responses. Connections along the word pathway are stronger than the color pathway, implementing the easier (faster, more accurate) performance of word reading relative to color naming arising from greater (and/or more consistent) experience with the former. As a result, when the model is presented with conflicting inputs (e.g., the color red and the word green), it will respond to the word—just as a person might if not instructed to name the color. However, given the task of naming the color, a person can exercise control by responding “red” to such a stimulus. In the model, this is achieved by activating the color‐naming unit in the task layer of the network. This unit sends additional biasing activity to the intermediate units in the color pathway, so that they are more responsive to their inputs. In this way, the model can selectively “attend” to the color dimension and respond accordingly. It is worth emphasizing that the increase in responsivity of the intermediate units is achieved simply by the additional top–down input provided by the task unit. This effect exploits the non‐linearity of the activation function of the processing units (see Cohen et al., [<reflink idref="bib36" id="ref50">36</reflink>] or Cohen, Aston‐Jones, &amp; Gilzenrat, [<reflink idref="bib37" id="ref51">37</reflink>] for fuller explanations), but it does not require any special or qualitatively distinct apparatus. The key observation is that attention, and corresponding control of behavior, emerges from the activation of a representation of the task to be performed and its influence on units that implement the set of possible mappings from stimuli to responses.</p> <p>The model provides a mechanistically explicit and neurobiologically plausible account of the phenomena associated with automaticity and controlled processing. It describes the continuum of automaticity in terms of the strengths of the connections in processing pathways that are accrued through learning. It explains why reliance on control is dependent not only on the absolute strength of a pathway (weaker ones need more top–down support) but also its strength relative to competing pathways: When a stimulus processed by a stronger pathway favors a competing response, greater top–down support is needed for responses that depend on processing in the weaker, task‐relevant pathway, so that it can compete with processing in the stronger but “distracting” pathway. The same process, performed in isolation or when placed in competition with a weaker pathway will rely less on control. Thus, the demands for control depend dually on practice and circumstance.</p> <hd id="AN0097523350-5">Guided activation theory</hd> <p>The idea that a pattern of activity over units in a neural network can function as an explicit representation of a task, and thereby implement a form of control, has proven useful for interpreting the contribution of prefrontal cortex to control. This has led to the development of the guided activation theory (GAT), which proposes that representations in prefrontal cortex (of tasks or goals) exert control over behavior by providing top–down biases that guide the flow of activity along processing pathways (“intermediate units”) in posterior structures responsible for task execution (Cohen &amp; Servan‐Schreiber, [<reflink idref="bib42" id="ref52">42</reflink>] ; Cohen, Braver, &amp; O'Reilly, [<reflink idref="bib39" id="ref53">39</reflink>] ; Miller &amp; Cohen, [<reflink idref="bib103" id="ref54">103</reflink>] ). GAT helps integrate the well‐documented associations of PFC with working memory (e.g., Cohen et al., [<reflink idref="bib41" id="ref55">41</reflink>] ; Goldman‐Rakic, [<reflink idref="bib66" id="ref56">66</reflink>] ), executive function (Duncan, [<reflink idref="bib53" id="ref57">53</reflink>] ; Shallice, [<reflink idref="bib142" id="ref58">142</reflink>] ), attention (Banich et al., [<reflink idref="bib11" id="ref59">11</reflink>] ; Knight, [<reflink idref="bib80" id="ref60">80</reflink>] ; Stuss &amp; Benson, [<reflink idref="bib151" id="ref61">151</reflink>] ), and behavioral inhibition (Fuster, [<reflink idref="bib61" id="ref62">61</reflink>] ; Lhermitte, [<reflink idref="bib88" id="ref63">88</reflink>] ; Luria, [<reflink idref="bib91" id="ref64">91</reflink>] ), by proposing that PFC is responsible for the active maintenance (representation in working memory) of task information (responsible for the execution of goal‐directed behavior) that is particularly critical when task‐relevant behavior demands that interference from distracting sources of information be ignored (attention) and/or competing response tendencies be overcome (inhibition). Models implementing this type of mechanism have successfully simulated detailed aspects of normal human performance in a variety of tasks known to engage PFC (e.g., Braver, Cohen, &amp; Servan‐Schreiber, [<reflink idref="bib32" id="ref65">32</reflink>] ; Cohen &amp; Servan‐Schreiber, [<reflink idref="bib42" id="ref66">42</reflink>] ; Dehaene &amp; Changeux, [<reflink idref="bib49" id="ref67">49</reflink>] , [<reflink idref="bib50" id="ref68">50</reflink>] ), as well as behavioral deficits in conditions associated with disturbances of PFC (e.g., Braver, Barch, &amp; Cohen, [<reflink idref="bib30" id="ref69">30</reflink>] ; Frank, Seeberger, &amp; O'Reilly, [<reflink idref="bib60" id="ref70">60</reflink>] ).</p> <p>Stepping back from the details of the Stroop model, or GAT more generally, we can characterize the view of control they offer as one in which a set of representations is used to parameterize the processes required to perform a given task. This parameterization is different from the one involved in learning: Whereas learning involves changes in system structure (e.g., synaptic weights) based on the absorption of new information from the environment, control involves the activation of a set of established representations, whose effect is to transiently adapt the parameters of information processing elsewhere in the system in the service of performing a particular task. The representations and corresponding processing parameters are not limited in kind to those involved in the Stroop model. They may pertain to a wide variety of processes and regulate everything from response thresholds in simple decision tasks to the criteria used for retrieval from semantic memory. What unites the relevant representations and parameters under the rubric of control is that they transiently adapt the target processes (perceptual, motor, attentional, or memory) to the specific demands of the current situation, as defined by experienced or inferred outcome and reward contingencies.</p> <p>This general view of control as process parameterization characterizes a number of recent formal theories, in addition to GAT (e.g., Cooper &amp; Shallice, [<reflink idref="bib47" id="ref71">47</reflink>] ; Dayan, [<reflink idref="bib48" id="ref72">48</reflink>] ; Dehaene &amp; Changeux, [<reflink idref="bib51" id="ref73">51</reflink>] ; Salinas, [<reflink idref="bib135" id="ref74">135</reflink>] ; Shenhav, Botvinick, &amp; Cohen, [<reflink idref="bib143" id="ref75">143</reflink>] ). Despite differences in the way the idea is applied across these accounts, a common assumption is that the system already possesses the appropriate representations required to perform (i.e., parameterize processing for) a given task, and that these representations are activated to an appropriate degree when needed. Accounts that begin with this assumption are subject to the same criticism as earlier models of control: They invoke a “homunculus”—a source of unexplained intelligence—to explain critical features of the phenomena or behaviors in question. Recent work has begun to address this concern. One line of work has focused on the question of how task representations arise and how they are organized. We will consider this below. First, however, we consider efforts that address how task representations are regulated based on the demands for control and updated in response to changes in the task environment. Elaborations of GAT have addressed these concerns.</p> <hd id="AN0097523350-6">Phase 2: Disarming the homunculus: The self‐regulation of control</hd> <hd id="AN0097523350-7">Recruitment of control</hd> <p>A long‐standing observation, consistent with the limited capacity for control, is that people adjust their allocation of control as circumstances demand. This is most clearly demonstrated by sequential adjustment effects, in which improvements in performance are observed on trials following a lapse in performance signaling a need for greater allocation of control. For example, an early observation was that people are less prone to an error on a trial following one in which they previously erred, an effect commonly known as the Rabbitt ([<reflink idref="bib127" id="ref76">127</reflink>] ) effect after its discoverer. This effect is frequently associated with slower responding, presumably reflecting more cautious and controlled performance on post‐error trials (Laming, [<reflink idref="bib86" id="ref77">86</reflink>] ). Numerous similar observations have been made, including an enhancement in selective attention following trials in which there was processing conflict (such as the Stroop task, described above), even in the absence of any errors (e.g., Carter et al., [<reflink idref="bib35" id="ref78">35</reflink>] ; Gratton, Coles, &amp; Donchin, [<reflink idref="bib68" id="ref79">68</reflink>] ; Logan, Zbrodoff, &amp; Fostey, [<reflink idref="bib90" id="ref80">90</reflink>] ; Tzelgov, Henik, &amp; Berger, [<reflink idref="bib155" id="ref81">155</reflink>] ). These observations suggest that people adaptively adjust control, allocating less to a task when it is needed less but increasing it when circumstances signal the need.</p> <p>Two classes of models have been proposed for how a system might adaptively adjust its allocation of control. One proposes that this involves an error‐monitoring system. This was motivated by the observation of a scalp‐recorded electrophysiological potential—the error‐related negativity (ERN)—that is selectively enhanced shortly after the commission of errors in speeded‐response tasks (Falkenstein, Hohnsbein, &amp; Hoorman, [<reflink idref="bib56" id="ref82">56</reflink>] ; Gehring, Goss, Coles, Meyer, &amp; Donchin, [<reflink idref="bib65" id="ref83">65</reflink>] ). The ERN has been proposed to reflect a negative reward prediction error mediated by the dopamine reinforcement learning system that signals the failure to receive an expected reward following commission of an error (Holroyd &amp; Coles, [<reflink idref="bib73" id="ref84">73</reflink>] ). This hypothesis has been implemented in the form of a PDP model (Holroyd, Yeung, Coles, &amp; Cohen, [<reflink idref="bib74" id="ref85">74</reflink>] ) and used to account for a variety of data concerning the ERN. One problem with error‐monitoring theories, however, is that they fail to account for adjustments of control when no error has been made. As noted above, such adjustments have been observed even in the absence of errors. Remarkably, so has the ERN (Yeung, Botvinick, &amp; Cohen, [<reflink idref="bib160" id="ref86">160</reflink>] ). Furthermore, these observations closely paralleled one of the most consistent findings in the neuroimaging literature: The association of activity in anterior cingulate cortex (ACC) with the difficulty of task performance (Paus, Koski, Caramanos, &amp; Westbury, [<reflink idref="bib117" id="ref87">117</reflink>] ). These observations inspired the development of a complementary proposal regarding the regulation of control: The conflict monitoring hypothesis (Botvinick, [<reflink idref="bib23" id="ref88">23</reflink>] ; Botvinick, Cohen, &amp; Carter, [<reflink idref="bib23" id="ref89">23</reflink>] ; Cole, Yeung, Freiwald, &amp; Botvinick, [<reflink idref="bib44" id="ref90">44</reflink>] ; see Shenhav et al., [<reflink idref="bib143" id="ref91">143</reflink>] for a review).</p> <p>This hypothesis built on the observation, articulated early on by Berlyne ([<reflink idref="bib13" id="ref92">13</reflink>] ), that those circumstances that demand control are typically characterized by the presence of processing conflict (e.g., the conflict between saying “red” vs. “green” in the example of the Stroop task above). Such conflict predisposes to errors. However, even in the absence of an error, conflict incurs performance costs in the form of slower responses. Both can be mitigated by the recruitment of control. A series of empirical and computational modeling studies have demonstrated that both ACC activity and the ERN are closely associated with conflicts in processing, and that their temporal profiles (at the resolution of tens of milliseconds in the case of the ERN) can be predicted by the level of processing conflict in the network (Botvinick, Nystrom, Fissell, Carter, &amp; Cohen, [<reflink idref="bib25" id="ref93">25</reflink>] ; Botvinick et al., [<reflink idref="bib21" id="ref94">21</reflink>] ; Carter et al., [<reflink idref="bib34" id="ref95">34</reflink>] ; Yeung et al., [<reflink idref="bib160" id="ref96">160</reflink>] ). In the models, conflict is quantified as the coactivation of competing representations (e.g., the red and green response units in Fig. [NaN] ; see Fig. [NaN] for relationship with conflict monitoring). More important, modeling work has shown that using the conflict signal to recruit control provides a quantitative account of sequential adjustments in performance that are observed across a variety of tasks and measures of performance (Botvinick, [<reflink idref="bib23" id="ref97">23</reflink>] ; Botvinick et al., [<reflink idref="bib21" id="ref98">21</reflink>] ). Recent work has extended this idea, suggesting that the ACC may be responsive more broadly to uncertainty, surprise, or both (Alexander &amp; Brown, [<reflink idref="bib1" id="ref99">1</reflink>] ; Ide, Shenoy, Angela, &amp; Chiang‐shan, [<reflink idref="bib76" id="ref100">76</reflink>] ; Rushworth &amp; Behrens, [<reflink idref="bib134" id="ref101">134</reflink>] ).</p> <p>Models of error, conflict, and uncertainty monitoring all provide examples of how adjustments in control can emerge in a self‐regulating manner, based on local computations, without recourse to a homunculus. Understanding the relevant processes—and resolving some attendant controversies (see Botvinick, [<reflink idref="bib17" id="ref102">17</reflink>] ; Cole et al., [<reflink idref="bib44" id="ref103">44</reflink>] ; Ullsperger et al., [<reflink idref="bib156" id="ref104">156</reflink>] )—remains an important challenge for contemporary research. However, there are also important reasons for expanding beyond performance monitoring as a paradigm for understanding the self‐regulation of control (see, e.g., Pezzulo &amp; Castelfranchi, [<reflink idref="bib120" id="ref105">120</reflink>] ; Shenhav et al., [<reflink idref="bib143" id="ref106">143</reflink>] ). While current theories of performance monitoring do take a first step toward “disarming the homunculus,” they do not address a looming, and perhaps more fundamental question: How are control representations themselves chosen, and updated as task circumstances change? The homunculus continues to lurk in explanatory recesses.</p> <hd id="AN0097523350-8">Updating of control representations</hd> <p>An important advance that addressed the question of how control representations are updated was the introduction of a gating mechanism. This is proposed to regulate access to the part of the system responsible for actively maintaining task representations (e.g., PFC). In the absence of a gating signal, inputs have a weak influence on PFC, allowing representations that are currently active to persist and guide performance. However, when a gating signal occurs, inputs are enhanced, allowing the updating of representations. The gating signal is presumed to occur only when there is an indication that the task to be performed has changed, and a new one should be pursued. The idea of a gating mechanism was inspired by computational work demonstrating that, in general, gating mechanisms are an effective way to regulate the updating of working memory (Hochreiter &amp; Schmidhuber, [<reflink idref="bib72" id="ref107">72</reflink>] ; Zipser, Kehoe, Littlewort, &amp; Fuster, [<reflink idref="bib162" id="ref108">162</reflink>] ) and have been shown to be of particular importance for the updating of task and/or goal representations (Todd, Niv, &amp; Cohen, [<reflink idref="bib154" id="ref109">154</reflink>] ). Several models have been proposed for how a gating mechanism might be implemented in the brain, most of which assign an important role to dopamine (e.g., Braver &amp; Cohen, [<reflink idref="bib31" id="ref110">31</reflink>] ; Frank, Loughry, &amp; O'Reilly, [<reflink idref="bib59" id="ref111">59</reflink>] ).</p> <p>The involvement of dopamine presents a potential solution to a critical challenge for the gating hypothesis: To explain how the system learns when to trigger a gating signal. Phasic dopamine signals have been proposed to implement a form of reinforcement learning (Montague, Dayan, &amp; Sejnowski, [<reflink idref="bib106" id="ref112">106</reflink>] ). According to this theory, the phasic release of dopamine acts as a learning signal that is used to predict when rewards will occur. Consistent with this theory, there is growing evidence that dopamine neurons fire in response to events associated with reward prediction errors—that is, unexpected events that are associated with a subsequent reward (e.g., Roesch, Calu, &amp; Schoenbaum, [<reflink idref="bib130" id="ref113">130</reflink>] ; Schultz, Dayan, &amp; Montague, [<reflink idref="bib137" id="ref114">137</reflink>] ). These are also precisely the conditions under which a gating signal should occur—when an unexpected event signals the opportunity for reward that can be obtained by redirecting behavior. Thus, the release of dopamine when a gating signal should occur can strengthen the likelihood that this signal will occur under similar circumstances in the future, providing an adaptive, self‐organizing mechanism for learning the timing of gating signals.</p> <p>The computational plausibility of the gating hypothesis has been established in at least two different types of models. One proposes that dopamine release simultaneously implements the gating signal in PFC (where task representations are presumed to reside) and the learning signal used to train dopaminergic nuclei when this should occur (Braver &amp; Cohen, [<reflink idref="bib31" id="ref115">31</reflink>] ; see Fig. [NaN] ). This model is simple and exploits the idea that both the gating and learning effects of dopamine may be implemented by the same physiological mechanism: gain control (Cohen, Braver, &amp; Brown, [<reflink idref="bib38" id="ref116">38</reflink>] ; Seamans &amp; Yang, [<reflink idref="bib138" id="ref117">138</reflink>] ; Servan‐Schreiber, Printz, &amp; Cohen, [<reflink idref="bib140" id="ref118">140</reflink>] ). A variant of this model proposes that dopamine is used to train the timing of the gating signal, but that the gating mechanism itself is implemented by the basal ganglia (Frank et al., [<reflink idref="bib59" id="ref119">59</reflink>] ; see Fig. [NaN] , top left). While these models differ with regard to the source of the gating signal, both make the same general prediction: That, at least while performing a new task, gating signals and updating of representations in PFC should be accompanied by the phasic release of dopamine.</p> <hd id="AN0097523350-9">Optimization and control</hd> <p>A growing trend in psychological and neuroscientific research is the development and testing of normative theory, which seeks to define the optimal mechanisms for performing a given function. This approach is particularly natural for the study of control, which can be defined as the optimization of performance in the service of a goal. GAT can be viewed as an instantiation of this approach, in asserting that the purpose of control representations is to parameterize (bias) processing to perform the desired task most effectively. This approach has been developed most explicitly in analyses of simple forms of decision making (e.g., using the drift diffusion model of two‐alternative forced choice tasks; Ratcliff, [<reflink idref="bib128" id="ref120">128</reflink>] ), which have revealed that there are unique parameters (e.g., the threshold for information accumulation that triggers a response, or biases in the starting point for the accumulation process) that optimize ecologically relevant metrics (such as the rate of rewards accrued). Empirical studies suggest that people can often approximate these optimal parameters (e.g., Balci et al., [<reflink idref="bib10" id="ref121">10</reflink>] ; Bogacz, Hu, Holmes, &amp; Cohen, [<reflink idref="bib16" id="ref122">16</reflink>] ; Bogacz et al., [<reflink idref="bib15" id="ref123">15</reflink>] ; Simen et al., [<reflink idref="bib146" id="ref124">146</reflink>] ), and similar analyses have been applied to adaptive adjustments in attentional control (e.g., Yeung &amp; Monsell, [<reflink idref="bib161" id="ref125">161</reflink>] ). The models of performance monitoring described above provide a mechanism by which these optimizations may be implemented. On this view, performance monitoring mechanisms serve to improve performance by optimizing the deployment of control itself. For example, they may be used to identify and implement the optimal threshold for a decision making process (e.g., Simen, Cohen, &amp; Holmes, [<reflink idref="bib145" id="ref126">145</reflink>] ), or the optimal trade‐off between different forms of control (e.g., anticipatory vs. reactive; see Braver, Gray, &amp; Burgess, [<reflink idref="bib33" id="ref127">33</reflink>] ; Botvinick, [<reflink idref="bib17" id="ref128">17</reflink>] ; Jimura, Locke, &amp; Braver, [<reflink idref="bib78" id="ref129">78</reflink>] ; Kool, McGuire, Rosen, &amp; Botvinick, [<reflink idref="bib82" id="ref130">82</reflink>] ; McGuire &amp; Botvinick, [<reflink idref="bib82" id="ref131">82</reflink>] ).</p> <p>Most work on optimization to date has focused either on how rewards are maximized, or on how costs of performance (i.e., errors, conflict, or uncertainty) can be minimized. However, there is strong reason to believe that the execution of control itself registers as a cost. Experiments on task choice have identified a robust pattern of demand avoidance (Kool &amp; Botvinick, [<reflink idref="bib81" id="ref132">81</reflink>] ; Kool et al., [<reflink idref="bib82" id="ref133">82</reflink>] ), with the degree of task avoidance tied to its tendency to engage neural regions implicated in control function, including dorsal cingulate and dorsolateral prefrontal cortices (Kool, Wang, McGuire, &amp; Botvinick, [<reflink idref="bib83" id="ref134">83</reflink>] ; McGuire &amp; Botvinick, [<reflink idref="bib99" id="ref135">99</reflink>] ). Integrating the cost of control with the broader optimization perspective, we recently proposed that people make decisions about whether and how to engage in a control‐demanding task by taking account of the expected value of control (EVC)—that is, an estimate of the benefits that would accrue from performance of the task, discounted both by the risks of failure as well as the costs of control scaled by the intensity of control required to perform the task (Shenhav et al., [<reflink idref="bib143" id="ref136">143</reflink>] ). Thus, the EVC theory offers a normative perspective on how decisions about the allocation of control may be made.</p> <p>Recent work has also begun to apply normative approaches to understanding the function of the gating mechanism. One fundamental observation that has emerged is that working memory coupled with a gating mechanism is a powerful, and potentially optimal, solution to the problem faced by reinforcement learning mechanisms operating in partially observable environments (i.e., ones in which the information needed to choose optimally is not all presently available). Working memory can be used to preserve such information until it is needed, in effect translating a partially observable Markovian decision process into a simple Markovian one, for which reinforcement learning algorithms are provably optimal (Bertsekas &amp; Tsitsiklis, [<reflink idref="bib14" id="ref137">14</reflink>] ; Sutton &amp; Barto, [<reflink idref="bib152" id="ref138">152</reflink>] ). The challenge then becomes how to determine what information should be gated into and stored in WM, and when this should be updated. Using reinforcement learning to train the gating mechanism addresses this problem (Peshkin, Meuleau, &amp; Kaelbling, [<reflink idref="bib118" id="ref139">118</reflink>] ), with performance characteristics that closely resemble those of human performance (Todd et al., [<reflink idref="bib154" id="ref140">154</reflink>] ).</p> <hd id="AN0097523350-10">Phase 3: Banishing the homunculus: The structure of control</hd> <p>The critical message to be drawn from the work reviewed above is that control can be viewed usefully as a tuning process, whereby parameters are dynamically adjusted to suit current task demands. However, there is an important limitation of this work, which is that it presupposes a set of representations (e.g., of tasks, thresholds, biases, etc.) that can implement the parameterization, and a set of processes upon which the parameter optimization operates. That is, most models of cognitive control assume from the outset a particular structure for the processing system, and a particular set of control signals (i.e., representations of tasks, thresholds, etc.) that can parameterize the processing system to perform the desired task, limiting consideration to the dynamics of the resulting interactions. Without further development, this approach runs the risk of stipulating at least part of what it purports to explain. A full computational account of control must also explain how control signals and their influence on task processes are themselves learned. If control can be thought of as optimizing over a set of parameters governing information‐processing operations, how are the parameterizations themselves learned? Rather than asking only how control processes regulate information processing, or how they are themselves regulated, we need to try to understand how the representations and processes that are intrinsic to control themselves emerge. Only then can we consider the homunculus to have been truly banished.</p> <p>In pursuing this effort, one attractive approach is to think about the computational problem that these representations and processes are meant to solve (Marr, [<reflink idref="bib96" id="ref141">96</reflink>] ). We suggest that the overall problem facing the control system is to find an efficient but flexible way of representing a wide range of task parameters, given a distribution of naturally occurring behavioral situations. The notion of optimization comes into play here once again, but in a new role. In considering the role of control above, the optimization problem was to find a set of parameters that optimizes performance of a particular task. Here, we are looking at a metaoptimization problem: Given a particular distribution of naturally occurring tasks, how can the control system itself be configured so as to optimize performance across tasks? The objective function here involves not only single‐task performance but also the generalizability of control—that is, the efficiency (“economy of scale”) gained by using similar representations to control multiple tasks. The latter relies on the opportunity to transfer across tasks, including scaling up from simple to more complex tasks (see Taylor &amp; Stone, [<reflink idref="bib153" id="ref142">153</reflink>] ).</p> <p>Our suspicion is that control exploits structure in the space of naturally occurring tasks just as coding in vision is shaped by the statistics of naturally occurring scenes (see, e.g., Bell &amp; Sejnowski, [<reflink idref="bib12" id="ref143">12</reflink>] ; Fiser, Berkes, Orban, &amp; Lengyel, [<reflink idref="bib58" id="ref144">58</reflink>] ) and motor codes are shaped by the structure of natural movements (Braun, Mehring, &amp; Wolpert, [<reflink idref="bib29" id="ref145">29</reflink>] ; Graziano &amp; Aflalo, [<reflink idref="bib69" id="ref146">69</reflink>] ). As in these domains and others (see, e.g., Griffiths, Tenenbaum, &amp; Kemp, [<reflink idref="bib70" id="ref147">70</reflink>] ), we assume that control leverages patterns of independency and opportunities for decomposition and abstraction, identifying the basic “building blocks” out of which naturalistic tasks are comprised. On this view, control develops a representational basis set (see Pouget &amp; Sejnowski, [<reflink idref="bib126" id="ref148">126</reflink>] ), suitable for parameterizing processing in an open‐ended set of naturalistic tasks. The nature of this basis set and the processes by which it is discovered and leveraged are not yet clear. However, recent work has begun the effort to puzzle them out.</p> <hd id="AN0097523350-11">Discovering control‐relevant structure in perception</hd> <p>One simple form of structure that characterizes many naturally occurring tasks concerns the perceptual feature dimensions of objects. Different dimensions of perception (shape, color, texture, size) are differentially relevant for different tasks. When sorting laundry, color may be most relevant; but when sorting silverware, shape is likely to be more relevant. Indeed, the ability to switch control of behavior from one stimulus dimension to another is a central feature of many neuropsychological tasks thought to index cognitive control and PFC function (such as the Stroop, Wisconsin Card Sort, and Intradimensional–Extradimensional Set‐shifting tasks—MacDonald, Cohen, Stenger, &amp; Carter, [<reflink idref="bib93" id="ref149">93</reflink>] ; Milner, [<reflink idref="bib105" id="ref150">105</reflink>] ; Owen, Roberts, Polkey, Sahakian, &amp; Robbins, [<reflink idref="bib113" id="ref151">113</reflink>] ; Weinberger, Berman, &amp; Daniel, [<reflink idref="bib157" id="ref152">157</reflink>] ). Accordingly, models of these tasks all include explicit, pre‐specified representations of the stimulus dimensions relevant for that task (e.g., the Stroop Model in Fig. [NaN] ; see also: Dehaene &amp; Changeux, [<reflink idref="bib50" id="ref153">50</reflink>] ; O'Reilly, Noelle, Braver, &amp; Cohen, [<reflink idref="bib112" id="ref154">112</reflink>] ). The question is how and under what conditions do such abstract, dimensional representations arise? Rougier, Noelle, Braver, Cohen, and O'Reilly ([<reflink idref="bib133" id="ref155">133</reflink>] ) addressed this by constructing a model that incorporated the learning and gating mechanisms outlined above, and exposing it to a task environment with an underlying dimensional structure that was not pre‐encoded in the model. The purpose of this effort was to determine whether the mechanisms, thought to be characteristic of PFC and associated control structures in the brain, were sufficient to develop abstract control representations that generalized within and between tasks. Here, we focus on the issue of generalization, and its relationship with the types of control representations that developed in the model. Below, we will return to a consideration of how this generalization relied on the learning and gating mechanisms implemented in the model.</p> <p>The training environment for the Rougier et al. ([<reflink idref="bib133" id="ref156">133</reflink>] ) model comprised four different tasks performed on a set of multidimensional stimuli (i.e., that varied in shape, color, size, etc.). Each task involved a different set of sensory‐motor mappings, but all tasks shared two critical characteristics in common: Only one stimulus dimension was relevant at a given time, and that dimension remained relevant (i.e., the task remained the same) for a sequence of trials, after which it switched to another. Such conditions were constructed to emulate naturalistic conditions, in which individual tasks are typically performed for some temporally extended period during which a particular subset of information in the environment (e.g., stimulus dimension) remains relevant, and then the task switches rendering other information relevant. Simulations were conducted to test how the breadth of training experience influenced the model's ability to generalize performance to novel stimuli—that is, the flexibility of control. Accordingly, during training the model was exposed to only a small subset of all potential stimuli for a given task, and then tested on its ability to generalize performance to a much broader range of novel ones. Furthermore, versions of the model were trained in each of two environments: One in which it was exposed to all four tasks; and another in which it was exposed to only two of the tasks but received twice as much training in each (to equate for overall training and number of stimuli seen). The purpose of this was to determine the extent to which the breadth of experience (i.e., the number of different tasks) influenced the types of control representations that were learned and their ability to generalize.</p> <p>The model learned to perform the tasks and stimuli on which it was trained equally well in both conditions. However, when the model was trained in the broader environment, it exhibited substantially better generalization: It was able to respond accurately to stimuli it had not previously seen in a given task substantially more often. This ability was directly associated with the development of discrete, componential representations of the relevant stimulus dimensions in the control layer of the network. Each unit in that layer came to represent a single dimension and all features in that dimension. Collectively, these representations formed a basis set of orthogonal vectors that spanned the space of task‐relevant processes, and that were aligned with the dimensions along which features had to be distinguished for task performance. Thus, the control system in the model was able to extract the dimensional structure that was relevant for performance of the tasks. Switching between tasks became a simple matter of identifying the appropriate dimensional representation to activate in the control layer, affording the network the ability to rapidly and flexibly adapt (i.e., reparameterize processing) as the task switched.</p> <p>Rougier et al. ([<reflink idref="bib133" id="ref157">133</reflink>] ) also tested other network architectures that were matched for overall size (e.g., number of units), but each of which lacked one or more critical elements of the control architecture outlined above (e.g., active memory, adaptive gating, as well as top–down connectivity from the control layer to processing layers of the network). All of these other models tended to memorize specific combinations of stimulus features and responses rather than develop abstract representations of feature dimensions. As a consequence, although they did as well during training, they fared substantially worse at generalization.</p> <p>Like some other work in computational cognitive control, the results of the Rougier et al. ([<reflink idref="bib133" id="ref158">133</reflink>] ) model may appear to strain against the PDP tradition, by having favored the development of “localist” (discrete) over “distributed” representations—that is, units that are each committed to representing a single dimension rather than participating in the overlapping representation of several dimensions. However, it is important to recognize that neural networks capable of supporting distributed representation can also learn componential representations when trained on tasks for which such representations are appropriate (see Plaut, McClelland, Seidenberg, &amp; Patterson, [<reflink idref="bib122" id="ref159">122</reflink>] ), and “localist” systems that represent stimuli, actions, or other entities (such as dimensions) as combinations of feature values can be interpreted as employing distributed representations. In this regard, the localist/distributed opposition may not be the most useful way to differentiate types of representations. Rather, it may be more useful to distinguish between orthogonal, componential representations and multicolinear or conjunctive representations. It is this opposition that is brought to the fore by the Rougier et al. ([<reflink idref="bib133" id="ref160">133</reflink>] ) model, where orthogonal, fully componential representations were shown to support generalization better than the conjunctive representations that emerged under other training conditions. Recent empirical evidence accords well with the idea of componential coding in cognitive control (see, e.g., Cole, Etzel, Zacks, Schneider, &amp; Braver, [<reflink idref="bib43" id="ref161">43</reflink>] ). However, other computational work has also highlighted the potential usefulness of multicolinear representation in supporting transfer to new tasks (e.g., Botvinick &amp; Plaut, [<reflink idref="bib26" id="ref162">26</reflink>] , [<reflink idref="bib27" id="ref163">27</reflink>] ). Furthermore, the neuroscientific literature provides ample evidence for conjunctive coding of action and action context (see Botvinick &amp; Plaut, [<reflink idref="bib28" id="ref164">28</reflink>] ), and recent computational work has considered how such conjunctive representation may support credit assignment during learning (Botvinick, Niv, &amp; Barto, [<reflink idref="bib24" id="ref165">24</reflink>] ). It thus appears likely that control may involve both orthogonal, componential and multicolinear and conjunctive representations. Understanding where and when control deploys these two forms of representation, and the computational trade‐offs involved in choosing between them, is an important challenge for future research, one that falls squarely within the best PDP tradition (McClelland, McNaughton, &amp; O'Reilly, [<reflink idref="bib98" id="ref166">98</reflink>] ).</p> <hd id="AN0097523350-12">Conditional structures</hd> <p>The Rougier et al. model focused on one aspect of task structure that might shape the structure of control, namely the fact that different perceptual features can become persistently task‐relevant at different times. A related structural motif was addressed in recent work by Collins and Frank ([<reflink idref="bib45" id="ref167">45</reflink>] ). In many tasks, the appropriate response to one perceptual feature dimension depends on the value of a different perceptual feature. The most ubiquitous example of such conditional structure in everyday life arises in social contexts: It is acceptable to open the refrigerator at home but usually not at someone else's house. A simpler version of this kind of conditional structure is often used in experimental studies of control, in which one set of stimulus (task cue) features determine how to respond to other (“target”) features. Collins and Frank ([<reflink idref="bib45" id="ref168">45</reflink>] ) studied the processes by which learners discover the conditional relationship between cue and target features. They began by presenting participants with stimuli defined by two features: color and shape. The correct response to each stimulus depended on a simple conjunction of these two features, as shown in Fig. [NaN] (left). Although inherently symmetric, behavioral analyses indicated that participants treated the feature dimensions asymmetrically, arbitrarily encoding one (e.g., color) as a task cue and the other (e.g., shape) as the target feature. This was evident in reaction times, which displayed switch‐cost effects when the feature value on the “task‐cue” dimension alternated between trials. It also manifested in generalization. When shapes in a new color were introduced into the stimulus set, the ease with which participants learned to respond to these new stimuli was influenced by the way in which they had initially assigned feature dimensions to cue and target roles (see Fig. [NaN] ).</p> <p>An interesting aspect of the Collins and Frank ([<reflink idref="bib45" id="ref169">45</reflink>] ) paradigm is that participants' separation of stimulus features into cue and target roles was not, in fact, computationally necessary. The task could, in principle, have been performed accurately by simply encoding shape and color conjunctively, without ordering the stimulus features in any way. Collins and Frank ([<reflink idref="bib45" id="ref170">45</reflink>] ) offered two interrelated explanations for why participants nonetheless arrived at an ordered, cue‐target encoding. Simulating the observed behavior using a Bayesian model, they suggested that learning is shaped by an inductive prior implementing the assumption that stimulus features often cue “task sets,” latent states specifying particular stimulus‐response mappings, dissociable from the specific cues that signal their relevance. Clearly, a prior of this kind would only be adaptive if it captures a genuine regularity in the task environment, namely one whereby disparate stimulus cues can map to a single underlying set of behavioral contingencies. It does seem plausible that this is a commonly encountered phenomenon in everyday activity.</p> <p>Alongside their Bayesian model, Collins and Frank ([<reflink idref="bib45" id="ref171">45</reflink>] ) presented a neural network model, centering on the gating mechanism discussed earlier. They showed that introducing a layered or hierarchical arrangement into the gating model (in much the same spirit as Reynolds &amp; O'Reilly, [<reflink idref="bib129" id="ref172">129</reflink>] ) can spontaneously yield the kind of ordered, cue‐target encoding observed experimentally. Collins and Frank ([<reflink idref="bib45" id="ref173">45</reflink>] ) proposed that the architecture and learning mechanisms implemented in this network model could be viewed as implementing the inductive prior involved in their Bayesian model.</p> <hd id="AN0097523350-13">Hierarchical structure</hd> <p>Collins and Frank ([<reflink idref="bib45" id="ref174">45</reflink>] ) conceptualized the “higher level” feature dimensions in their task as cueing task sets or response policies. However, there is another (not incompatible) way of interpreting the kind of conditional structure revealed by their work that provides another perspective on its relevance to control. Specifically, one could regard one set of stimulus features as setting preconditions for particular action outcomes: In the Collins and Frank task (see Fig. [NaN] ), an A1 response to a triangle stimulus will result in positive feedback only if the stimulus color is red. Color here sets a precondition for a particular shape‐response outcome. Preconditions of this kind are ubiquitous in naturalistic task settings—To enter one's home, one must have the door open. To open the door, it must be unlocked. To unlock the door, one must have the key…—a fact that has led preconditions to play a central role in artificial intelligence models of planning (a core function of cognitive control).</p> <p>An interesting structural property of preconditions, as they arise in real‐world tasks, is that they are characteristically asymmetric: To unlock a door, I must have the key, but whether the door is locked has no bearing on my ability to access the key. This kind of asymmetry has important ramifications for task structure, and consequently for the structure of control. As illustrated in Fig. [NaN] (right), based on a classic cognitive control task (the Tower of Hanoi task), networks of asymmetric preconditions naturally induce hierarchical structure in behavioral state space. When the state space is visualized as a graph, this hierarchical structure manifests as a clustering of nodes into densely interconnected groups separated by bottleneck‐like edges, a pattern that is referred to in the complex networks literature as “community structure” (Botvinick, [<reflink idref="bib19" id="ref175">19</reflink>] , [<reflink idref="bib20" id="ref176">20</reflink>] ).</p> <p>This kind of structure presents an opportunity for control processes. Computational studies have firmly established that when an agent inhabits a state space with community structure, the agent's capacity to plan can be dramatically enhanced if it learns to treat the bottlenecks between clusters as subgoals (Simsek, [<reflink idref="bib147" id="ref177">147</reflink>] ; Simsek, Wolfe, &amp; Barto, [<reflink idref="bib148" id="ref178">148</reflink>] ). This kind of strategic subgoal selection is a perfect example of the kind of metaoptimization we have hypothesized for human cognitive control, according to which the representations involved in control are themselves adapted to the structure of target tasks.</p> <p>Recent behavioral and fMRI work suggest that human planners do indeed identify bottlenecks in task space, using these as subgoals for planning. In a study by Solway et al. ([<reflink idref="bib149" id="ref179">149</reflink>] ) (see Diuk, Schapiro, Cordova, &amp; Botvinick, [<reflink idref="bib52" id="ref180">52</reflink>] ), participants were presented with a set of “landmarks” (e.g., post office, school) and learned their adjacency relations within a fictive town. These adjacencies were based on a graph with strong community structure, with one bottleneck location linking two clusters of nodes with dense internal connections. (Importantly, the graphs themselves were never shown to the participants.) Once the adjacency relations among the town's landmarks had been learned, participants were then asked to make “deliveries” in the town, navigating each time from a specified point of origin to a specified goal, and receiving a reward that varied inversely with the number of steps taken to complete the delivery. Before beginning these deliveries, however, the subject was asked to select one landmark as a location for a “bus stop,” understanding that he or she could “jump” to this location from any other during the deliveries, potentially reducing the number of steps taken. Without knowledge of the specific upcoming delivery assignments, the optimal bus‐stop choice corresponds to the bottleneck, and participants overwhelmingly selected this location. Follow‐up experiments demonstrated that participants focused in on bottleneck locations even without the scaffolding provided by the bus‐stop choice, clearly using these locations as subgoals in planning (see Solway et al., [<reflink idref="bib149" id="ref181">149</reflink>] ).</p> <p>How might control processes identify task bottlenecks as useful subgoals? This is a non‐trivial learning problem, as bottlenecks are not necessarily marked by any superficial distinguishing features (as indeed they were not in the experiment just described). One possible method for identifying bottlenecks, and the community structure they imply, was proposed in a recent study by Schapiro, Rogers, Cordova, Turk‐Browne, and Botvinick ([<reflink idref="bib136" id="ref182">136</reflink>] ). This work focused on sequences generated by a random walk in the graph shown in Fig. [NaN] (left). The graph shows obvious community structure, with three clusters of nodes separated by three bottleneck edges. Schapiro et al. ([<reflink idref="bib136" id="ref183">136</reflink>] ) pointed out that the community structure of the graph could be identified from the sequences generated from it, if an effort was simply made to predict each item in those sequences. To illustrate the point, Schapiro et al. implemented a three‐layer neural network (Fig. [NaN] , center), with one input unit and one output unit for each node in the graph. The network was trained, when given an input representing a single node, to produce an output indicating the nodes that could come next in the sequence (as illustrated in the figure). Schapiro et al. found that, following training, the model's internal or hidden representations directly revealed the community structure of the underlying graph; nodes lying within the same community were represented more similarly than nodes lying in different communities (Fig. [NaN] , right). An fMRI study provided evidence that this same effect arises in neural event representations. Here, Schapiro et al. assigned a distinctive visual stimulus to each vertex of the graph in Fig. [NaN] , and participants viewed these stimuli in sequences generated based on a random walk through the underlying graph (the graph itself was never shown). Multivoxel pattern analysis revealed regions of frontal and temporal cortex within which the patterns of activity induced by individual stimuli displayed the similarity relations illustrated in Fig. [NaN] (right).</p> <hd id="AN0097523350-14">Capacity limitations in cognitive control</hd> <p>The idea that control adapts to the structure of the task environment may provide an explanation not only for why control is powerful but also for another of its most characteristic features: The remarkable limitation that people have in the capacity to simultaneously carry out more than one control‐demanding task at a time (i.e., to multitask). From the outset, this capacity limitation was considered to be a defining feature of controlled processing (e.g., Posner &amp; Snyder, [<reflink idref="bib125" id="ref184">125</reflink>] ; Shiffrin &amp; Schneider, [<reflink idref="bib144" id="ref185">144</reflink>] ), and it continues to be central to debates concerning multitasking that have broad social significance (e.g., the use of cell phones while driving). While capacity constraints are assumed by most theories of control, few have addressed the nature or source of this limitation. Typically these are thought to arise from some form of structural constraint or central “resource” limitation, though the specific nature of the resource involved is rarely specified. Some theories (e.g., Anderson, [<reflink idref="bib3" id="ref186">3</reflink>] ; Baddeley, [<reflink idref="bib8" id="ref187">8</reflink>] ) have linked the constraints on control to the long recognized capacity limits of working memory (Miller, [<reflink idref="bib102" id="ref188">102</reflink>] ). However, these too stipulate rather than explain the limitation. Furthermore, the notion of a resource limitation is startling if one considers that control processes are supported by PFC, a structure that occupies approximately one‐third of the human neocortex and comprises about 3 billion neurons.</p> <p>An alternative approach to understanding limitations in the capacity for control is to consider that these may arise from functional (computational) rather than structural constraints. In particular, they may reflect another form of metaoptimization as control mechanisms adapt to the representational structure of the task environment. The latter often involves multiplexed representations; that is, sets of representations that are used for different purposes by different tasks. This introduces the problem of cross talk, in which two tasks try to make use of the same representational resources for different purposes at the same time. This idea has roots in early “multiple resources” theories of attention (Allport, [<reflink idref="bib2" id="ref189">2</reflink>] ; Logan, [<reflink idref="bib89" id="ref190">89</reflink>] ; Navon &amp; Gopher, [<reflink idref="bib107" id="ref191">107</reflink>] ; Wickens, [<reflink idref="bib159" id="ref192">159</reflink>] ). These argued that the degradation in behavior observed when people try to multitask may be due to cross talk in processing, rather than the constrained resources of a single, centralized control mechanism. A classic example contrasts the ability to echo an auditory stream while simultaneously typing visually presented text, with the ability to simultaneously read aloud and take dictation. The former pair is relatively easy to learn, while the latter is considerably more difficult (Shaffer, [<reflink idref="bib141" id="ref193">141</reflink>] ). The multiple resources explanation suggests that echoing and copy‐typing involve non‐overlapping processing pathways (one auditory‐phonological‐verbal and the other visual‐orthographic‐manual). In contrast, reading and dictation involve the shared use of both phonological and orthographic representations (see Fig. [NaN] ). In such situations, control may be needed to serialize processing, so that cross talk does not arise within the shared resources. Note that, on this account, the limitation (and serial processing) arises from cross talk within a set of “local” resources rather than from a capacity constraint on a centralized control resource. Control helps solve the cross talk problem, although its consistent engagement under such circumstances (and therefore close association with them) could engender the misinterpretation that control is a source of, rather than a solution to capacity constraints on performance. Continuing debate about the centralized versus distributed nature of capacity constraints has centered on the psychological refractory period (Pashler, [<reflink idref="bib115" id="ref194">115</reflink>] ; Welford, [<reflink idref="bib158" id="ref195">158</reflink>] ) observed for dual task performance—a seemingly immutable degradation in performance when participants are required to perform two tasks at the same time. While one line of argument maintains that the universality of this observation is evidence of a centralized capacity limitation, at least one modeling effort (using the EPIC production system architecture; Meyer &amp; Kieras, [<reflink idref="bib100" id="ref196">100</reflink>] , [<reflink idref="bib101" id="ref197">101</reflink>] ) has explained many of the observed phenomena in terms of the scheduling problems induced by more local forms of cross talk.</p> <p>The PDP framework offers an alternative perspective that may help reconcile seemingly opposing views on this issue. This builds on the ideas introduced by the Stroop model and generalized by GAT. The Stroop model provides an example of how control mechanisms can solve the problem of cross talk. However, it does so in the isolated context of a single pair of processing pathways. These map two sources of information (colors and words) onto a single set of (verbal) responses. In the brain, of course, these would be intertwined with many others pathways (for example, responding to colors by making other kinds of motor responses, such as stopping when seeing red; see Fig. [NaN] ). Furthermore, to the extent that the representations in PFC are relatively abstract (e.g., the dimension “color”), having been learned under the pressure for generalization (i.e., to span a variety of potential tasks as discussed above), then activating them runs the risk of facilitating pathways that are currently irrelevant to task performance. This is not a problem so long as there are no inputs driving the flow of activity along those other pathways. However, if another task is being performed, then there is once again the risk cross talk.</p> <p>More generally, it may be that there is a tension between the flexibility of control (related to the generality of control representations) and the risk of cross talk. The severity of this tension would, in turn, be closely related to the density of mappings and overlap of the pathways responsible for the performance of different tasks. That is, the control system may face yet another trade‐off, between multitasking (executing multiple processes at once) and multiplexing (using the same representations for different tasks). A simple analogy is the problem faced in designing a network of train tracks and then scheduling the trains: Increasing the directness of connections between sources and destinations necessarily increases the density of tracks and therefore the number of crossing points. Managing these, in turn, requires limiting the number of trains that are running at the same time or carefully scheduling their transit (e.g., by restricting the number of green lights that can be simultaneously lit at crossing points). By analogy, dense overlap of representations in the brain (multiplexing) may be an efficient way of encoding information (and serving concomitant functions, such as constraint satisfaction and similarity‐based inference); however, this brings with it the potential for interference due to cross talk and thus may limit the number of control representations that can be activated at once in PFC (multitasking). This difficulty, coupled with the pressures to develop control representations that are general and thereby flexible (i.e., have the potential to impact a wide range of representations involved in processing) would contribute to constraints in the number of control representations that can be safely activated at once. Accordingly, the control system may have adapted to the task environment by imposing its own limitations on the number of active representations, to avoid the perils of cross talk. From this vantage, while capacity constraints might lie within the control system itself, these could be viewed as a response to the characteristics of the processing architecture over which they preside, reflecting an optimization that favored flexibility in control and the efficiency of multiplexing representations over the capacity for multitasking. This is consistent with the EVC theory of cognitive control (Shenhav et al., [<reflink idref="bib143" id="ref198">143</reflink>] ). As discussed above, this proposes that—in deciding whether and which controlled processes to implement—the control system takes into account the potential costs associated with their execution. Such costs include the potential for interference from cross talk that arises from simultaneously engaging multiple processes.</p> <p>Recent modeling work has quantitatively addressed these ideas. Feng et al. ([<reflink idref="bib57" id="ref199">57</reflink>] ) examined, in networks of the sort shown in Fig. [NaN] , the extent to which increasing pathway overlap (multiplexing) incurred interference costs (from cross talk), and how this impacted the optimal control policy—that is, the number of tasks that could be performed at once (multitasking), taking account of the performance of each. Specifically, they simulated networks of various sizes with multiple pathways, each of which was subject to control and, for each network configuration, determined the control policy that maximized aggregate performance (i.e., the number of tasks to which control should be allocated that maximized reward rate over the entire network). They then examined how this varied as a function of pathway overlap and network size. They found that introducing even modest amounts of pathway overlap dramatically constrained the capacity for multitasking, and that this effect was largely insensitive to the size of the network. This supports the idea that capacity constraints in control may, in fact, reflect an optimization: In the face of pathway overlap and multiplexing (which have their own value), it is optimal for the system to favor restrictions in the allocation of control to a limited number of processes. This work conforms to the two central tenets of the PDP approach that we have emphasized: It explains capacity constraints as arising from an interaction between the environment (in this case, one that includes the rest of the brain), the structure of the control architecture, and the optimization of performance; and, in so doing, it offers to explain rather stipulate the capacity constraints of control.</p> <hd id="AN0097523350-15">The neural environment</hd> <p>We have been considering, from various angles, the idea that control functions may be shaped to dovetail with the statistical structure of the task environment. This idea accords with an important tenet of the PDP approach, namely that processing mechanisms are shaped by an interaction between general purpose learning algorithms and the structure of the environment in which the system must learn to function. However, classic work in the PDP tradition adds an important extension to this idea: In complex systems, processing within any subsystem will be shaped not only by external inputs but also by the ways in which those inputs are processed by the rest of the system (see Plaut et al., [<reflink idref="bib122" id="ref200">122</reflink>] ). The implications of this point are particularly stark in the case of cognitive control, as control is more or less defined by the fact that its most direct interface is not with the external environment, but with the rest of the processing system. The point is immediately evident from the neuroanatomy underlying cognitive control, in the sense that the key structure underlying cognitive control, the PFC, bears no direct connections to primary sensory or motor areas. The “environment” for prefrontal control mechanisms is, essentially, the rest of the brain. It is thus not only the structure of the external task environment but also the properties of the brain structures that are “controlled” that sets the terms of the metaoptimization problem faced by the PFC. This extends from perceptual and semantic representations in occipital and temporal cortex, to sensorimotor maps in parietal cortex and representations of actions and habits in premotor cortex and basal ganglia. One might say that these are the “keys of the piano” on which the PFC must learn to compose its pieces; accordingly, the characteristics of these keys shape the representations that develop within the PFC itself (see Dayan, [<reflink idref="bib48" id="ref201">48</reflink>] for further discussion).</p> <p>Included in this internal environment is the episodic memory system, critical components of which (e.g., the hippocampus) are housed within the medial temporal lobes. This system supports fast (e.g., one‐shot) learning and has dense connections with the PFC. One important function served by these is the use of context and control mechanisms in the PFC to guide retrieval of information from episodic memory. This idea has been formalized in mathematical models (e.g., Howard &amp; Kahana, [<reflink idref="bib75" id="ref202">75</reflink>] ; Polyn &amp; Kahana, [<reflink idref="bib123" id="ref203">123</reflink>] ) as well as in a PDP model (Polyn, Norman, &amp; Kahana, [<reflink idref="bib124" id="ref204">124</reflink>] ), and it has received support from neuroimaging data (Jenkins &amp; Ranganath, [<reflink idref="bib77" id="ref205">77</reflink>] ). However, the interaction may also run in the reverse direction: The PFC may use the episodic memory system to “cache” control representations for future use, which could serve a valuable role in prospective memory and planning behavior. Planning often involves the need to schedule a control‐demanding behavior in the future, at a time well beyond the interval over which the control representation(s) can be actively maintained in the PFC. Cohen and O'Reilly ([<reflink idref="bib40" id="ref206">40</reflink>] ) proposed that the control system may exploit an alternative prospective memory strategy, which is to call to mind the relevant control representations (i.e., activate them in the PFC) and associate them (by way of hippocampally mediated episodic memory) with the environmental circumstances under which they should be elicited. The control system could then rely on the episodic memory system to retrieve the relevant control representations when the circumstances are appropriate, to be maintained actively in working memory during the behavioral epoch over which control is actually needed. This would provide a mechanism for what Gollwitzer ([<reflink idref="bib67" id="ref207">67</reflink>] ) described as the process of forming “implementation intentions” in the context of goal pursuit.</p> <p>Note that this use of episodic memory to “bind” a control representation to a novel context implements a form of flexibility that is closely akin to, if not identical with the substitutability of symbols discussed above, and thus may reflect another means by which the control system achieves flexibility. The decision to rely on episodic memory to retrieve control representations when they are needed, rather than actively maintain them, is also another example of the trade‐off between reactive versus proactive control referred to earlier. In the context of planning and prospective memory, the trade‐off in costs favors reactive rather than proactive control. The point to be made here is that this trade‐off represents another instance of an optimization problem faced by the control system (e.g., in the PFC) in its deployment of resources in the rest of the brain.</p> <p>If control mechanisms are shaped by the rest of the processing system, it seems likely that the converse is also true, and that neural systems for perception, action, attention, and memory are shaped by their interaction with control. This kind of coevolution has been an important motif in PDP models in other domains (see, e.g., Plaut et al., [<reflink idref="bib122" id="ref208">122</reflink>] ), and we suspect the relevant principles continue to hold for control and the systems with which it interacts. To provide a simple illustration, we revisit the Stroop model from Cohen et al. ([<reflink idref="bib36" id="ref209">36</reflink>] ). Here, control serves to switch among parallel pathways from perception to action, one subserving color naming, the other word reading. Clearly, the appropriate set of control (task) representations, and their connections with the rest of the system, directly reflects the specific structure of these input–output pathways. However, the pathways themselves only make functional sense if there are control inputs to regulate the flow of activation. Less obviously, the involvement of control may also determine which pathway comes to dominate the other, that is which task is “automatic” and which “controlled.” If, as we have recently proposed, control is costly (see Botvinick, [<reflink idref="bib17" id="ref210">17</reflink>] , [<reflink idref="bib18" id="ref211">18</reflink>] ; Kool et al., [<reflink idref="bib82" id="ref212">82</reflink>] ; McGuire &amp; Botvinick, [<reflink idref="bib82" id="ref213">82</reflink>] ), then it makes sense to automatize the task that occurs most frequently, as this minimizes the frequency with which control will be demanded. In this sense, the association between task frequency and automaticity, as highlighted by Cohen et al. ([<reflink idref="bib36" id="ref214">36</reflink>] ), may be thought of as a cost‐minimizing metaoptimization.</p> <hd id="AN0097523350-16">Are control mechanisms structurally distinct from other processing mechanisms?</hd> <p>We began this section by raising the question of how control assumes its form, and how this responds to the structure of the task environment. To this point, we have focused on the representations involved in control. However, another closely related question, foreshadowed by the previous section, is how the mechanisms responsible for control fit within the overall physical architecture of the processing system itself. From the beginning, models of cognitive control have assumed that control is not only functionally distinct from other domains of processing (cf. the earlier discussion of controlled vs. automatic processing), but that it is also architecturally distinct, occupying its own, dedicated apparatus in the overall information‐processing system (e.g., within the PFC, basal ganglia, and brainstem). Even PDP models (such as the Stroop model), which reject the idea that control requires qualitatively distinct types of mechanisms, nevertheless allow that control may rely on specific structures. This assumption appears to be strongly supported by convergent neuroscientific evidence, which suggests that critical elements of control functions are localized within portions of the dorsolateral prefrontal cortex (Miller &amp; Cohen, [<reflink idref="bib103" id="ref215">103</reflink>] ), and closely associated portions of the medial frontal and superior parietal cortex (Duncan, [<reflink idref="bib54" id="ref216">54</reflink>] ; Duncan &amp; Owen, [<reflink idref="bib55" id="ref217">55</reflink>] ), basal ganglia (O'Reilly &amp; Frank, [<reflink idref="bib109" id="ref218">109</reflink>] ), and brainstem (Aston‐Jones &amp; Cohen, [<reflink idref="bib6" id="ref219">6</reflink>] ; Braver &amp; Cohen, [<reflink idref="bib31" id="ref220">31</reflink>] ).</p> <p>While models of control have generally assumed the architectural segregation of control functions, it has rarely been considered “why” such segregation might exist: What are its computational implications, and what phylogenetic or ontogenetic forces might be responsible for it? To motivate the question, it is worth noting that at least some of the functions attributed to cognitive control do not, in fact, strictly require that it be architecturally segregated. For example, Botvinick and Plaut ([<reflink idref="bib27" id="ref221">27</reflink>] ) presented a model addressing aspects of hierarchically structured behavior generally considered to rely on prefrontal‐based cognitive control mechanisms (see, e.g., Fuster, [<reflink idref="bib62" id="ref222">62</reflink>] ). However, this model involved no structural module dedicated to control. Instead, the context‐appropriate control of behavior arose out of the learned dynamics of a structurally undifferentiated group of neuron‐like units (see Fig. [NaN] , top right). Representational hierarchy embedded in the same structural elements was shown to be sufficient for hierarchical behavior, without requiring architectural segregation.</p> <p>If architectural segregation is not strictly necessary for control, then what are we to make of the neuroscientific data suggesting that control functions so frequently seem to rely on specific, identifiable structures in the brain? Botvinick ([<reflink idref="bib17" id="ref223">17</reflink>] , [<reflink idref="bib18" id="ref224">18</reflink>] ) examined this question in a follow‐up to the simulations reported in Botvinick and Plaut ([<reflink idref="bib27" id="ref225">27</reflink>] ). This work built on Fuster's ([<reflink idref="bib62" id="ref226">62</reflink>] ) characterization of the prefrontal cortex as occupying the apex of a hierarchy of cortical regions. Botvinick ([<reflink idref="bib17" id="ref227">17</reflink>] , [<reflink idref="bib18" id="ref228">18</reflink>] ) reimplemented the neural network model from Botvinick and Plaut ([<reflink idref="bib27" id="ref229">27</reflink>] ) but introduced an architectural structure mirroring Fuster's hierarchy (see Fig. [NaN] , bottom). Following training on a hierarchical task, units at the apex of the hierarchy were found to play a disproportionate role in the representation of temporally extended context. Thus, although this spatial differentiation of function was not computationally necessary—as demonstrated by Botvinick and Plaut ([<reflink idref="bib27" id="ref230">27</reflink>] )—it nonetheless emerged, when the system architecture assumed a particular initial structure (see also Paine &amp; Tani, [<reflink idref="bib114" id="ref231">114</reflink>] ).</p> <p>Related findings were reported by Reynolds and O'Reilly ([<reflink idref="bib129" id="ref232">129</reflink>] ). Here, a hierarchical pattern of connectivity was imposed among the working‐memory modules or “stripes” of the O'Reilly and Frank ([<reflink idref="bib109" id="ref233">109</reflink>] ) and Rougier et al. ([<reflink idref="bib133" id="ref234">133</reflink>] ) gating models. This architectural constraint led units higher in the hierarchical structure spontaneously to assume a role in representing temporal context. As in the Botvinick ([<reflink idref="bib17" id="ref235">17</reflink>] , [<reflink idref="bib18" id="ref236">18</reflink>] ) model, learning exploited architectural hierarchy to develop spatial differentiation of function. Most recently, Kriete, Noelle, Cohen, and O'Reilly ([<reflink idref="bib84" id="ref237">84</reflink>] ) have shown that such hierarchical structure may contribute critically to the flexibility of the control system, allowing the system to discover a form of indirection and thus approximate the representational power of fully symbolic systems.</p> <p>The Botvinick ([<reflink idref="bib17" id="ref238">17</reflink>] , [<reflink idref="bib18" id="ref239">18</reflink>] ) and Reynolds and O'Reilly ([<reflink idref="bib129" id="ref240">129</reflink>] ) models push beyond the usual assumption that control is architecturally distinct from other domains of processing, to understand the conditions under which such segregation might arise. However, there is also the subtly different question of whether this segregation is in some way computationally advantageous. Returning to the heuristic assumption that control is shaped by a process of optimization, it seems natural to ask whether architecturally separating control representations from other representations may serve this optimization.</p> <p>In considering this question, it may be useful to return to the idea that control provides a basis set for representing naturally occurring tasks. As in other domains (e.g., vision, motor function, spatial cognition, and language), this basis set must carve naturalistic tasks at their joints, embodying patterns of statistical independency and covariation. This was illustrated by the Rougier et al. ([<reflink idref="bib133" id="ref241">133</reflink>] ) model, which was able to extract the relevant basis set—an explicit representation of abstract stimulus dimensions—for flexible task performance and generalization. Rougier et al. showed that the ability to learn these representations and to exhibit generalization relied on specific architectural and functional specializations of the control system that are thought to exist in the brain, including a segregated PFC layer with dense recurrent connectivity capable of actively maintaining control representations, and an adaptive gating mechanism capable of updating those representations when the task changed. As noted above, Kriete et al. ([<reflink idref="bib84" id="ref242">84</reflink>] ) extended this argument to show that including a bias toward hierarchical structure allows the system to develop the capacity for indirection, and thus substantially increase its capacity for flexible generalization.</p> <p>These examples help illustrate how the influence of architectural biases on representational structure can have normative value. It is tempting to speculate that evolution has programmed the human brain with architectural biases that favor the development of a prefrontal cortex, its connections with basal ganglia, and dopaminergic neuromodulation because these permit more efficient and effective extraction of representations that can subserve task control in as general a manner as possible.</p> <p>Another speculation is inspired by work applying the PDP framework to quite a different area: Semantic cognition. Rogers and McClelland ([<reflink idref="bib131" id="ref243">131</reflink>] ) have argued that the representation of concepts requires a “hub region,” positioned so as to collate inputs from a wide range of domain‐specific sources (different sensory modalities, high‐level motor representations, language centers, etc.), and capable of discovering patterns of coherent covariation among these sources. A candidate neuroanatomic structure for this role, Rogers and McClelland propose, lies in anterior temporal cortex. Analogous computational considerations would seem likely to apply in the domain of cognitive control. Control, by its very nature, requires bidirectional connections with a very wide variety of domain‐specific processors (related, again, to perceptual modalities, motor representations, and language, as well as episodic memory, reward and motivation, and conceptual knowledge). And indeed PFC is, by no coincidence, one of the most widely connected structures in the brain. However, to adaptively control the structures it interacts with, the PFC must do more than simply connect with them. As we have sought to emphasize, it must discover a set of representations appropriate to the task of control. Like Rogers' and McClelland's semantic hub, the PFC must identify patterns of coherent covariation, including feature dimensions, conditional dependencies among stimulus features, and hierarchical relationships, among other formal structures, some of which may play out only over time. The anatomical position of the PFC as a hub region may be important to its ability to discover powerful representations for control.</p> <hd id="AN0097523350-17">Conclusion</hd> <p>Our review has focused on three critical challenges faced by efforts to understand the computational mechanisms underlying cognitive control: How is control executed? How do control mechanisms adapt to changes in performance and the environment? And what structural and functional specializations characterize the mechanisms subserving control? The last question is the most difficult and, as yet, least fully addressed. Unpacked, it points to questions about the nature of control representations, how these emerge, and how their emergence depends on and interacts with the task environment and the rest of the brain. These questions define the current frontiers of research on cognitive control and the function of prefrontal cortex and associated structures.</p> <p>We have argued that such research will profit from a normative approach that has proven to be productive in other areas of cognitive and neuroscientific research. This approach assumes that brain mechanisms have evolved to optimize their function by adapting to the features of the environment to which they must respond, and optimizing the balance in trade‐offs that are inherent to any processing system (such as flexibility vs. efficiency; multitasking vs. multiplexing; and possibly others beyond the scope of this review, such exploration vs. exploitation—see Aston‐Jones &amp; Cohen, [<reflink idref="bib6" id="ref244">6</reflink>] ). This approach seems particularly well suited for understanding a set of mechanisms the very purpose of which can be defined in terms of optimization: Control mechanisms can be viewed as optimally parameterizing task processes to maximize rewards. As with other mechanisms in the brain, the terms of this optimization problem are set by the environment. In the case of control, this includes not only the external environment (over which tasks must operate) but also the internal environment of the brain itself—that is, the characteristics of the brain mechanisms over which control must operate. Perhaps the most interesting challenge for research on control is the problem of metaoptimization: How the functional organization of control takes shape within its specific external and neural environment, assuming a form that supports adaptive performance across task domains. We identified several directions in which such efforts are headed, working toward an understanding of how control representations and processes themselves take shape.</p> <p>At the heart of these efforts is the challenge to explain how control mechanisms emerge and function in a self‐organizing manner. The PDP approach provides a natural framework within which to meet this challenge. We have outlined work illustrating how such interactions may give rise to a functional architecture that includes segregated components specialized for control. However, a concern sometimes voiced in discussions about PDP models of control is that structural segregation and functional specialization run counter to the grain of the PDP approach. These could be construed as violating the dictum that one should avoid stipulating what one seeks to explain. We deeply appreciate the discipline that this tenet of the PDP approach has brought to the model building enterprise. However, like any form of discipline, it can be overly restrictive if applied too aggressively. As we hope to have illustrated, the study of control and its neural implementation has presented us with two strong observations: Empirically, the brain appears to have specialized apparatus closely associated with the capacity for control; and, computationally, the capacity for control seems to profit directly from certain forms of functional specialization and structural segregation that happen to be observed empirically. As our review indicates, recent models have begun to reveal how relatively low‐level specializations (e.g., recurrent connectivity supporting active maintenance, and reinforcement learning supporting adaptive gating) give rise to the higher level phenomena of interest (e.g., the capacity for flexible allocation of control). In this regard, not only do these models avert concerns about directly stipulating what is to be explained, but they adhere closely to the other fundamental goal of the PDP approach: to understand how the behavior of interest emerges from of an interaction between the native structure of the information‐processing system, the structure of the behavioral domain, and domain‐general learning and decision‐making mechanisms that seek to optimize function. In these respects, we believe that the PDP approach is alive and well within the domain of cognitive control.</p> <hd id="AN0097523350-18">Acknowledgments</hd> <p>This project was made possible through the support of a grant from the National Science Foundation (CRCNS 1207833, MMB), the National Institute of Mental Health (R01MH098815‐01, MMB), and the John Templeton Foundation (JDC and MMB). The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the John Templeton Foundation.</p> <ref id="AN0097523350-19"> <title>References</title> <blist> <bibl id="bib1" idref="ref99" type="bt">1</bibl> <bibtext>Alexander, W. H., &amp; Brown, J. W. ( 2011 ). Medial prefrontal cortex as an action‐outcome predictor. Nature Neuroscience, 14, 1338 – 1344. </bibtext> </blist> <blist> <bibl id="bib2" idref="ref189" type="bt">2</bibl> <bibtext>Allport, D. A. ( 1982 ). Attention and performance. In G. I. Claxton (Ed.), New directions in cognitive psychology (pp. 112 – 153 ). London : Routledge &amp; Kegan Paul. </bibtext> </blist> <blist> <bibl id="bib3" idref="ref24" type="bt">3</bibl> <bibtext>Anderson, J. ( 1983 ). The architecture of cognition. Cambridge, MA : Harvard University Press. </bibtext> </blist> <blist> <bibl id="bib4" idref="ref47" type="bt">4</bibl> <bibtext>Anderson, J. ( 1993 ). The adaptive character of thought. Hillsdale, NJ : Erlbaum. </bibtext> </blist> <blist> <bibl id="bib5" idref="ref48" type="bt">5</bibl> <bibtext>Anderson, J. R., Carter, C. S., Fincham, J. M., Qin, Y., Susan, M., Ravizza, S. M., &amp; Rosenberg‐Lee, M. ( 2008 ). Using fMRI to test models of complex cognition. Cognitive Science, 32, 1323 – 1348. </bibtext> </blist> <blist> <bibl id="bib6" idref="ref219" type="bt">6</bibl> <bibtext>Aston‐Jones, G., &amp; Cohen, J. D. ( 2005 ). An integrative theory of locus coeruleus‐norepinephrine function: Adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403 – 450. </bibtext> </blist> <blist> <bibl id="bib7" idref="ref6" type="bt">7</bibl> <bibtext>Atkinson, R. C., &amp; Shiffrin, R. M. ( 1968 ). Human memory: A proposed system and its control processes. In K. W. Spence &amp; J. T. Spence (Eds.), The psychology of learning and motivation (pp. 89 – 195 ). New York : Academic Press. </bibtext> </blist> <blist> <bibl id="bib8" idref="ref10" type="bt">8</bibl> <bibtext>Baddeley, A. D. ( 1986 ). Working memory. New York : Clarendon Press. </bibtext> </blist> <blist> <bibl id="bib9" idref="ref11" type="bt">9</bibl> <bibtext>Baddeley, A. D., &amp; Hitch, G. ( 1974 ). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory, vol. 8 (pp. 47 – 89 ). New York : Academic Press. </bibtext> </blist> <blist> <bibl id="bib10" idref="ref121" type="bt">10</bibl> <bibtext>Balci, F., Simen, P., Niyogi, R., Saxe, A., Hughes, J. A., Holmes, P., &amp; Cohen, J. D. ( 2011 ). Acquisition of decision making criteria: Reward rate ultimately beats accuracy. Attention, Perception, &amp; Psychophysics, 73, 640 – 657. </bibtext> </blist> <blist> <bibl id="bib11" idref="ref59" type="bt">11</bibl> <bibtext>Banich, M. T., Milham, M. P., Atchley, R., Cohen, N. J., Webb, A., Wszalek, T., Kramer, A. F., Liang, Z.‐P., Barad, V., Gullett, D., Shah, C., &amp; Brown, C. ( 2000 ). Prefrontal regions play a predominant role in imposing an attentional “set”: Evidence from fMRI. Cognitive Brain Research, 10 ( 1–2 ), 1 – 9. </bibtext> </blist> <blist> <bibl id="bib12" idref="ref143" type="bt">12</bibl> <bibtext>Bell, A. J., &amp; Sejnowski, T. J. ( 1997 ). The “independent components” of natural scenes are edge filters. Vision Research, 37, 3327 – 3338. </bibtext> </blist> <blist> <bibl id="bib13" idref="ref92" type="bt">13</bibl> <bibtext>Berlyne, D. E. ( 1957 ). Uncertainty and conflict: A point of contact between information‐theory and behavior‐theory concepts. Psychological Review, 64 ( 6 ), 329 – 339. </bibtext> </blist> <blist> <bibl id="bib14" idref="ref137" type="bt">14</bibl> <bibtext>Bertsekas, D. P., &amp; Tsitsiklis, J. N. ( 1996 ). Neuro‐dynamic programming. Belmont, MA : Athena Scientific. </bibtext> </blist> <blist> <bibl id="bib15" idref="ref123" type="bt">15</bibl> <bibtext>Bogacz, R., Brown, E. T., Moehlis, J., Hu, P., Holmes, P., &amp; Cohen, J. D. ( 2006 ). The physics of optimal decision making: A formal analysis of models of performance in two‐alternative forced choice tasks. Psychological Review, 113 ( 4 ), 700 – 765. </bibtext> </blist> <blist> <bibl id="bib16" idref="ref122" type="bt">16</bibl> <bibtext>Bogacz, R., Hu, P. T., Holmes, P., &amp; Cohen, J. D. ( 2010 ). Do humans produce the speed‐accuracy tradeoff that maximizes reward rate? Quarterly Journal of Experimental Psychology, 63 ( 5 ), 863 – 891. </bibtext> </blist> <blist> <bibl id="bib17" idref="ref102" type="bt">17</bibl> <bibtext>Botvinick, M. ( 2007a ). Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function. Cognitive, Affective and Behavioral Neuroscience, 7, 356 – 366. </bibtext> </blist> <blist> <bibl id="bib18" idref="ref211" type="bt">18</bibl> <bibtext>Botvinick, M. ( 2007b ). Multilevel structure in behavior and in the brain: A computational model of Fuster's hierarchy. Philosophical Transactions of the Royal Society, Series B: Biological Sciences, 362, 1615 – 1626. </bibtext> </blist> <blist> <bibl id="bib19" idref="ref27" type="bt">19</bibl> <bibtext>Botvinick, M. ( 2008 ). Hierarchical models of behavior and prefrontal function. Trends in Cognitive Sciences, 12, 201 – 208. </bibtext> </blist> <blist> <bibl id="bib20" idref="ref176" type="bt">20</bibl> <bibtext>Botvinick, M. ( 2012 ). Hierarchical reinforcement learning and decision making. Current Opinion in Neurobiology, 22, 956 – 962. </bibtext> </blist> <blist> <bibl id="bib21" idref="ref94" type="bt">21</bibl> <bibtext>Botvinick, M. M., Braver, T. S., Carter, C. S., Barch, D. M., &amp; Cohen, J. D. ( 2001 ). Conflict monitoring and cognitive control. Psychological Review, 108 ( 3 ), 624 – 652. </bibtext> </blist> <blist> <bibl id="bib22" type="bt">22</bibl> <bibtext>Botvinick, M., Buxbaum, L., &amp; Jax, S. ( 2009 ). Toward an integrated account of object and action selection: A computational analysis and empirical findings from reaching‐to‐grasp and tool use. Neuropsychologia, 47, 671 – 683. </bibtext> </blist> <blist> <bibl id="bib23" idref="ref88" type="bt">23</bibl> <bibtext>Botvinick, M., Cohen, J. D., &amp; Carter, C. S. ( 2004 ). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, 8, 539 – 546. </bibtext> </blist> <blist> <bibl id="bib24" idref="ref165" type="bt">24</bibl> <bibtext>Botvinick, M. M., Niv, Y., &amp; Barto, A. C. ( 2009 ). Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition, 113, 262 – 280. </bibtext> </blist> <blist> <bibl id="bib25" idref="ref93" type="bt">25</bibl> <bibtext>Botvinick, M. M., Nystrom, L., Fissell, K., Carter, C. S., &amp; Cohen, J. D. ( 1999 ). Conflict monitoring vs. selection‐for‐action in anterior cingulate cortex. Nature, 402 ( 6758 ), 179 – 181. </bibtext> </blist> <blist> <bibl id="bib26" idref="ref162" type="bt">26</bibl> <bibtext>Botvinick, M., &amp; Plaut, D. C. ( 2002 ). Representing task context: Proposals based on a connectionist model of action. Psychological Research, 66, 298 – 311. </bibtext> </blist> <blist> <bibl id="bib27" idref="ref163" type="bt">27</bibl> <bibtext>Botvinick, M., &amp; Plaut, D. C. ( 2004 ). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111 ( 2 ), 395 – 429. </bibtext> </blist> <blist> <bibl id="bib28" idref="ref164" type="bt">28</bibl> <bibtext>Botvinick, M., &amp; Plaut, D. C. ( 2009 ). Empirical and computational support for context‐dependent representations of serial order: Reply to Bowers, Damian, and Davis (2009). Psychological Review, 116, 998 – 1002. </bibtext> </blist> <blist> <bibl id="bib29" idref="ref145" type="bt">29</bibl> <bibtext>Braun, D. A., Mehring, C., &amp; Wolpert, D. M. ( 2010 ). Structure learning in action. Behavioural Brain Research, 206, 157 – 165. </bibtext> </blist> <blist> <bibl id="bib30" idref="ref69" type="bt">30</bibl> <bibtext>Braver, T. S., Barch, D. M., &amp; Cohen, J. D. ( 1999 ). Cognition and control in schizophrenia: A computational model of dopamine and prefrontal function. Biological Psychiatry, 46 ( 3 ), 312 – 328. </bibtext> </blist> <blist> <bibl id="bib31" idref="ref110" type="bt">31</bibl> <bibtext>Braver, T. S., &amp; Cohen, J. D. ( 2000 ). On the control of control: The role of dopamine in regulating prefrontal function and working memory. In S. Monsell &amp; J. Driver (Eds.), Attention and performance XVIII; control of cognitive processes (pp. 713 – 737 ). Cambridge, MA : MIT Press. </bibtext> </blist> <blist> <bibl id="bib32" idref="ref65" type="bt">32</bibl> <bibtext>Braver, T. S., Cohen, J. D., &amp; Servan‐Schreiber, D. ( 1995 ). A computational model of prefrontal cortex function. In D. S. Touretzky, G. Tesauro, &amp; T. Q. Leen (Eds.), Advances in neural information processing systems, vol. 7 (pp. 141 – 149 ). Cambridge, MA : MIT Press. </bibtext> </blist> <blist> <bibl id="bib33" idref="ref127" type="bt">33</bibl> <bibtext>Braver, T. S., Gray, J. R., &amp; Burgess, G. C. ( 2007 ). Explaining the many varieties of working memory variation: Dual mechanisms of cognitive control. In A. R. A. Conway, C. Jarrold, M. C. Kane, A. Miyake, &amp; J. N. Towse (Eds.), Variation in working memory (pp. 76 – 106 ). New York : Oxford University Press. </bibtext> </blist> <blist> <bibl id="bib34" idref="ref95" type="bt">34</bibl> <bibtext>Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., &amp; Cohen, J. D. ( 1998 ). Anterior cingulate cortex, error detection, and the on‐line monitoring of performance. Science, 280, 747 – 749. </bibtext> </blist> <blist> <bibl id="bib35" idref="ref78" type="bt">35</bibl> <bibtext>Carter, C. S., Macdonald, A. M., Botvinick, M., Ross, L. L., Stenger, V. A., Noll, D., &amp; Cohen, J. D. ( 2000 ). Parsing executive processes: Strategic vs. evaluative functions of the anterior cingulate cortex. Proceedings of the National Academy of Sciences of the United States of America, 97 ( 4 ), 1944 – 1948. </bibtext> </blist> <blist> <bibl id="bib36" idref="ref38" type="bt">36</bibl> <bibtext>Cohen, J. D., Dunbar, K., &amp; McClelland, J. L. ( 1990 ). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97, 332. </bibtext> </blist> <blist> <bibl id="bib37" idref="ref51" type="bt">37</bibl> <bibtext>Cohen, J. D., Aston‐Jones, G., &amp; Gilzenrat, M. S. ( 2004 ). A systems‐level perspective on attention and cognitive control: Guided activation, adaptive gating, conflict monitoring, and exploitation vs. exploration. In M. I. Posner (Ed.), Cognitive neuroscience of attention (pp. 71 – 90 ). New York : Guilford Press. </bibtext> </blist> <blist> <bibl id="bib38" idref="ref116" type="bt">38</bibl> <bibtext>Cohen, J. D., Braver, T. S., &amp; Brown, J. W. ( 2002 ). Computational perspectives on dopamine function in prefrontal cortex. Current Opinion in Neurobiology, 12, 223 – 229. </bibtext> </blist> <blist> <bibl id="bib39" idref="ref53" type="bt">39</bibl> <bibtext>Cohen, J. D., Braver, T. S., &amp; O'Reilly, R. C. ( 1996 ). A computational approach to prefrontal cortex, cognitive control, and schizophrenia: Recent developments and current challenges. Philosophical Transactions of the Royal Society of London Series B (Biological Sciences), 351 ( 1346 ), 1515 – 1527. </bibtext> </blist> <blist> <bibl id="bib40" idref="ref206" type="bt">40</bibl> <bibtext>Cohen, J. D., &amp; O'Reilly, R. C. ( 1996 ). A preliminary theory of the interactions between prefrontal cortex and hippocampus that contribute to planning and prospective memory. In M. Brandimonte, G. O. Einstein, &amp; M. A. McDaniel (Eds.), Prospective memory: Theory and applications (pp. 267 – 295 ). Hillsdale, NJ : Erlbaum. </bibtext> </blist> <blist> <bibl id="bib41" idref="ref55" type="bt">41</bibl> <bibtext>Cohen, J. D., Perlstein, W. M., Braver, T. S., Nystrom, L. E., Noll, D. C., Jonides, J., &amp; Smith, E. E. ( 1997 ). Temporal dynamics of brain activation during a working memory task. Nature, 386, 604 – 608. </bibtext> </blist> <blist> <bibl id="bib42" idref="ref52" type="bt">42</bibl> <bibtext>Cohen, J. D., &amp; Servan‐Schreiber, D. ( 1992 ). Context, cortex and dopamine: A connectionist approach to behavior and biology in schizophrenia. Psychological Review, 99, 45 – 77. </bibtext> </blist> <blist> <bibl id="bib43" idref="ref161" type="bt">43</bibl> <bibtext>Cole, M. W., Etzel, J. A., Zacks, J. M., Schneider, W., &amp; Braver, T. S. ( 2011 ). Rapid transfer of abstract rules to novel contexts in human lateral prefrontal cortex. Frontiers in Human Neuroscience, 5, 142. </bibtext> </blist> <blist> <bibl id="bib44" idref="ref90" type="bt">44</bibl> <bibtext>Cole, M. W., Yeung, N., Freiwald, W., &amp; Botvinick, M. ( 2010 ). Conflict over anterior cingulate cortex: Between‐species differences in cingulate may support enhanced cognitive flexibility in humans. Brain, Behavior and Evolution, 75, 239 – 240. </bibtext> </blist> <blist> <bibl id="bib45" idref="ref167" type="bt">45</bibl> <bibtext>Collins, A. G., &amp; Frank, M. J. ( 2013 ). Cognitive control over learning: Creating, clustering, and generalizing task‐set structure. Psychological Review, 120 ( 1 ), 190 – 229. </bibtext> </blist> <blist> <bibl id="bib46" idref="ref26" type="bt">46</bibl> <bibtext>Cooper, R. P. ( 2010 ). Cognitive control: Componential or emergent? Topics in Cognitive Science, 2 ( 4 ), 598 – 613. </bibtext> </blist> <blist> <bibl id="bib47" idref="ref71" type="bt">47</bibl> <bibtext>Cooper, R., &amp; Shallice, T. ( 2000 ). Contention scheduling and the control of routine activities. Cognitive Neuropsychology, 17, 297 – 338. </bibtext> </blist> <blist> <bibl id="bib48" idref="ref72" type="bt">48</bibl> <bibtext>Dayan, P. ( 2007 ). Bilinearity, rules and prefrontal cortex. Frontiers in Computational Neuroscience, 1, 1 – 14. </bibtext> </blist> <blist> <bibl id="bib49" idref="ref67" type="bt">49</bibl> <bibtext>Dehaene, S., &amp; Changeux, J. P. ( 1989 ). A simple model of prefrontal cortex function in delayed‐ response tasks. Journal of Cognitive Neuroscience, 1, 244 – 261. </bibtext> </blist> <blist> <bibl id="bib50" idref="ref68" type="bt">50</bibl> <bibtext>Dehaene, S., &amp; Changeux, J. P. ( 1992 ). The Wisconsin card sorting test: Theoretical analysis and modeling in a neuronal network. Cerebral Cortex, 1, 62 – 79. </bibtext> </blist> <blist> <bibl id="bib51" idref="ref73" type="bt">51</bibl> <bibtext>Dehaene, S., &amp; Changeux, J.‐P. ( 1997 ). A hierarchical neuronal network for planning behavior. Proceedings of the National Academy of Sciences, 94, 13293 – 13298. </bibtext> </blist> <blist> <bibl id="bib52" idref="ref180" type="bt">52</bibl> <bibtext>Diuk, C., Schapiro, A., Cordova, N., &amp; Botvinick, M. ( 2013 ). Divide and conquer: Task decomposition and hierarchical reinforcement learning in humans. In G. Baldassare &amp; M. Mirolli (Eds.), Intrinsically motivated cumulative learning in natural and artificial systems (pp. 271 – 292 ). Berlin : Springer‐Verlag. </bibtext> </blist> <blist> <bibl id="bib53" idref="ref16" type="bt">53</bibl> <bibtext>Duncan, J. ( 1986 ). Disorganization of behaviour after frontal lobe damage. Cognitive Neuropsychology, 3, 271 – 290. </bibtext> </blist> <blist> <bibl id="bib54" idref="ref216" type="bt">54</bibl> <bibtext>Duncan, J. ( 2010 ). The multiple‐demand (MD) system of the primate brain: Mental programs for intelligent behaviour. Trends in Cognitive Science, 14, 172 – 179. </bibtext> </blist> <blist> <bibl id="bib55" idref="ref217" type="bt">55</bibl> <bibtext>Duncan, J., &amp; Owen, A. M. ( 2000 ). Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends in Neurosciences, 23, 475 – 483. </bibtext> </blist> <blist> <bibl id="bib56" idref="ref82" type="bt">56</bibl> <bibtext>Falkenstein, M., Hohnsbein, J., &amp; Hoorman, J. ( 1995 ). Event related potential correlates of errors in reaction tasks. In G. Karmos, M. Molnar, V. Csepe, I. Czigler, &amp; J. E. Desmedt (Eds.), Perspectives of event‐related potentials research (pp. 287 – 296 ). Amsterdam : Elsevier Science B.V. </bibtext> </blist> <blist> <bibl id="bib57" idref="ref199" type="bt">57</bibl> <bibtext>Feng, S. F., Schwemmer, M., Gershman, S. J., &amp; Cohen, J. D. ( 2014 ). Multitasking vs. multiplexing: Toward a normative account of limitations in the simultaneous execution of control‐demanding behaviors. Cognitive, Affective and Behavioral Neuroscience, 14, 129 – 146. </bibtext> </blist> <blist> <bibl id="bib58" idref="ref144" type="bt">58</bibl> <bibtext>Fiser, J., Berkes, P., Orban, G., &amp; Lengyel, M. ( 2010 ). Statistically optimal perception and learning: From behavior to neural representations. Trends in Cognitive Sciences, 14, 119 – 130. </bibtext> </blist> <blist> <bibl id="bib59" idref="ref111" type="bt">59</bibl> <bibtext>Frank, M. J., Loughry, B., &amp; O'Reilly, R. C. ( 2001 ). Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive, Affective and Behavioral Neuroscience, 1, 137 – 160. </bibtext> </blist> <blist> <bibl id="bib60" idref="ref70" type="bt">60</bibl> <bibtext>Frank, M. J., Seeberger, L. C., &amp; O'Reilly, R. C. ( 2004 ). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306 ( 5703 ), 1940 – 1943. </bibtext> </blist> <blist> <bibl id="bib61" idref="ref62" type="bt">61</bibl> <bibtext>Fuster, J. M. ( 1980 ). The prefrontal cortex. New York : Raven Press. </bibtext> </blist> <blist> <bibl id="bib62" idref="ref14" type="bt">62</bibl> <bibtext>Fuster, J. M. ( 1985 ). The prefrontal cortex, mediator of cross‐temporal contingencies. Human Neurobiology, 4, 169 – 179. </bibtext> </blist> <blist> <bibl id="bib63" idref="ref15" type="bt">63</bibl> <bibtext>Fuster, J. M. ( 1989 ). The prefrontal cortex (2nd ed.). New York : Raven. </bibtext> </blist> <blist> <bibl id="bib64" idref="ref20" type="bt">64</bibl> <bibtext>Fuster, J. M., &amp; Alexander, G. E. ( 1971 ). Neuron activity related to short‐term memory. Science, 173, 652 – 654. </bibtext> </blist> <blist> <bibl id="bib65" idref="ref83" type="bt">65</bibl> <bibtext>Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., &amp; Donchin, E. ( 1993 ). A neural system for error detection and compensation. Psychological Science, 4, 385 – 390. </bibtext> </blist> <blist> <bibl id="bib66" idref="ref21" type="bt">66</bibl> <bibtext>Goldman‐Rakic, P. S. ( 1987 ). Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. In F. Plum (Ed.), Handbook of physiology, section 1 (pp. 373 – 417 ). Bethesda, MD : American Physiological Society. </bibtext> </blist> <blist> <bibl id="bib67" idref="ref207" type="bt">67</bibl> <bibtext>Gollwitzer, P. M. ( 1993 ). Goal achievement: The role of intentions. European Review of Social Psychology, 4, 141 – 185. </bibtext> </blist> <blist> <bibl id="bib68" idref="ref79" type="bt">68</bibl> <bibtext>Gratton, G., Coles, M. G. H., &amp; Donchin, E. ( 1992 ). Optimizing the use of information: Strategic control of activation of responses. Journal of Experimental Psychology, 121, 480 – 506. </bibtext> </blist> <blist> <bibl id="bib69" idref="ref146" type="bt">69</bibl> <bibtext>Graziano, M. S. A., &amp; Aflalo, T. N. ( 2007 ). Mapping behavioral repertoire onto the cortex. Neuron, 56 ( 2 ), 239 – 251. </bibtext> </blist> <blist> <bibl id="bib70" idref="ref147" type="bt">70</bibl> <bibtext>Griffiths, T., Tenenbaum, J., &amp; Kemp, C. ( 2006 ). Theory‐based Bayesian models of inductive learning and reasoning. Trends in Cognitive Sciences, 10, 309 – 318. </bibtext> </blist> <blist> <bibl id="bib71" idref="ref1" type="bt">71</bibl> <bibtext>Hebb, D. O. ( 1949 ). The organization of behavior: A neuropsychological theory. New York : John Wiley &amp; Sons. </bibtext> </blist> <blist> <bibl id="bib72" idref="ref107" type="bt">72</bibl> <bibtext>Hochreiter, S., &amp; Schmidhuber, J. ( 1997 ). Long short‐term memory. Neural Computation, 9 ( 8 ), 1735 – 1780. </bibtext> </blist> <blist> <bibl id="bib73" idref="ref84" type="bt">73</bibl> <bibtext>Holroyd, C. B., &amp; Coles, M. G. H. ( 2002 ). The neural basis of human error processing: Reinforcement learning, dopamine, and the error‐related negativity. Psychological Review, 109, 679 – 709. </bibtext> </blist> <blist> <bibl id="bib74" idref="ref85" type="bt">74</bibl> <bibtext>Holroyd, C. B., Yeung, N., Coles, M. G. H., &amp; Cohen, J. D. ( 2005 ). A mechanism for error detection in speeded response time tasks. Journal of Experimental Psychology: General, 134 ( 2 ), 163 – 191. </bibtext> </blist> <blist> <bibl id="bib75" idref="ref202" type="bt">75</bibl> <bibtext>Howard, M. W., &amp; Kahana, M. J. ( 2002 ). A distributed representation of temporal context. Journal of Mathematical Psychology, 46, 269 – 299. </bibtext> </blist> <blist> <bibl id="bib76" idref="ref100" type="bt">76</bibl> <bibtext>Ide, J. S., Shenoy, P., Angela, J. Y., &amp; Chiang‐shan, R. L. ( 2013 ). Bayesian prediction and evaluation in the anterior cingulate cortex. The Journal of Neuroscience, 33 ( 5 ), 2039 – 2047. </bibtext> </blist> <blist> <bibl id="bib77" idref="ref205" type="bt">77</bibl> <bibtext>Jenkins, L. J., &amp; Ranganath, C. ( 2010 ). Prefrontal and medial temporal lobe activity at encoding predicts temporal context memory. The Journal of Neuroscience, 30 ( 46 ), 15558 – 15565. </bibtext> </blist> <blist> <bibl id="bib78" idref="ref129" type="bt">78</bibl> <bibtext>Jimura, K., Locke, H. S., &amp; Braver, T. S. ( 2010 ). Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts. Proceedings of the National Academy of Sciences of the United States of America, 107 ( 19 ), 8871 – 8876. </bibtext> </blist> <blist> <bibl id="bib79" idref="ref36" type="bt">79</bibl> <bibtext>Kahneman, D., &amp; Treisman, A. ( 1984 ). Changing views of attention and automaticity. In R. Parasuraman, D. R. Davies, &amp; J. Beatty (Eds.), Varieties of attention (pp. 29 – 61 ). New York : Academic Press. </bibtext> </blist> <blist> <bibl id="bib80" idref="ref60" type="bt">80</bibl> <bibtext>Knight, R. T. ( 1997 ). Distributed cortical network for visual attention. Journal of Cognitive Neuroscience, 9, 75 – 91. </bibtext> </blist> <blist> <bibl id="bib81" idref="ref132" type="bt">81</bibl> <bibtext>Kool, W., &amp; Botvinick, M. ( 2014 ). A labor‐leisure tradeoff in cognitive control. Journal of Experimental Psychology: General, 143, 131 – 141. </bibtext> </blist> <blist> <bibl id="bib82" idref="ref130" type="bt">82</bibl> <bibtext>Kool, W., McGuire, J., Rosen, Z., &amp; Botvinick, M. M. ( 2010 ). Decision making and the avoidance of cognitive demand. Journal of Experimental Psychology: General, 139, 665 – 682. </bibtext> </blist> <blist> <bibl id="bib83" idref="ref134" type="bt">83</bibl> <bibtext>Kool, W., Wang, G., McGuire, J., &amp; Botvinick, M. ( 2013 ). Neural and behavioral evidence for an intrinsic cost of self‐control. PLoS ONE, 8, p. e72626. </bibtext> </blist> <blist> <bibl id="bib84" idref="ref237" type="bt">84</bibl> <bibtext>Kriete, T., Noelle, D. C., Cohen, J. D., &amp; O'Reilly, R. C. ( 2013 ). Indirection and symbol‐like processing in the prefrontal cortex and basal ganglia. Proceedings of the National Academy of Sciences USA, 110, 16390 – 16395. </bibtext> </blist> <blist> <bibl id="bib85" idref="ref25" type="bt">85</bibl> <bibtext>Laird, J. E., Newell, A., &amp; Rosenbloom, P. S. ( 1987 ). SOAR: An architecture for general intelligence. Artificial Intelligence, 33, 1 – 64. </bibtext> </blist> <blist> <bibl id="bib86" idref="ref77" type="bt">86</bibl> <bibtext>Laming, D. ( 1979 ). Choice reaction performance following an error. Acta Psychologica, 43, 199 – 224. </bibtext> </blist> <blist> <bibl id="bib87" idref="ref4" type="bt">87</bibl> <bibtext>Lashley, K. S. ( 1951 ). The problem of serial order in behavior. In L. A. Jeffress (Ed.), Cerebral mechanisms in behavior: The Hixon symposium (pp. 112 – 136 ). New York : Wiley. </bibtext> </blist> <blist> <bibl id="bib88" idref="ref12" type="bt">88</bibl> <bibtext>Lhermitte, F. ( 1983 ). “Utilization behaviour” and its relation to lesions of the frontal lobes. Brain, 106, 237 – 255. </bibtext> </blist> <blist> <bibl id="bib89" idref="ref190" type="bt">89</bibl> <bibtext>Logan, G. D. ( 1985 ). Skill and automaticity: Relations, implications, and future directions. Canadian Journal of Psychology, 39, 367 – 386. </bibtext> </blist> <blist> <bibl id="bib90" idref="ref80" type="bt">90</bibl> <bibtext>Logan, G. D., Zbrodoff, N. J., &amp; Fostey, A. R. W. ( 1983 ). Costs and benefits of strategy construction in a speeded discrimination task. Memory and Cognition, 11, 485 – 493. </bibtext> </blist> <blist> <bibl id="bib91" idref="ref64" type="bt">91</bibl> <bibtext>Luria, A. R. ( 1969 ). Frontal lobe syndromes. In P. J. Vinken &amp; G. W. Bruyn (Eds.), Handbook of clinical neurology (pp. 725 – 757 ). New York : Elsevier. </bibtext> </blist> <blist> <bibl id="bib92" idref="ref13" type="bt">92</bibl> <bibtext>Luria, A. R. ( 1973 ). The working brain. New York : Basic Books. </bibtext> </blist> <blist> <bibl id="bib93" idref="ref149" type="bt">93</bibl> <bibtext>MacDonald, A. W., Cohen, J. D., Stenger, V. A., &amp; Carter, C. S. ( 2000 ). Dissociating the role of dorsolateral prefrontal cortex and anterior cingulate cortex in cognitive control. Science, 288, 1835 – 1837. </bibtext> </blist> <blist> <bibl id="bib94" idref="ref37" type="bt">94</bibl> <bibtext>MacLeod, C. M. ( 1991 ). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109 ( 2 ), 163 – 200. </bibtext> </blist> <blist> <bibl id="bib95" idref="ref42" type="bt">95</bibl> <bibtext>MacLeod, C. M., &amp; Dunbar, K. ( 1988 ). Training and Stroop‐like interference: Evidence for a continuum of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 126 – 135. </bibtext> </blist> <blist> <bibl id="bib96" idref="ref141" type="bt">96</bibl> <bibtext>Marr, D. ( 1982 ). Vision: A computational approach. San Francisco, CA : Freeman &amp; Co. </bibtext> </blist> <blist> <bibl id="bib97" idref="ref30" type="bt">97</bibl> <bibtext>McClelland, J. L., Botvinick, M. M., Noel, D., Plaut, D. C., Rogers, T. T., Seidenberg, M., &amp; Smith, L. ( 2010 ). Letting structure emerge: Connectionist and dynamical systems approaches to understanding cognition. Trends in Cognitive Sciences, 14, 348 – 356. </bibtext> </blist> <blist> <bibl id="bib98" idref="ref166" type="bt">98</bibl> <bibtext>McClelland, J. L., McNaughton, B. L., &amp; O'Reilly, R. C. ( 1995 ). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102, 419 – 457. </bibtext> </blist> <blist> <bibl id="bib99" idref="ref135" type="bt">99</bibl> <bibtext>McGuire, J., &amp; Botvinick, M. ( 2010 ). Prefrontal cortex, cognitive control, and the registration of decision costs. Proceedings of the National Academy of Sciences, 107, 7922 – 7926. </bibtext> </blist> <blist> <bibl id="bib100" idref="ref46" type="bt">100</bibl> <bibtext>Meyer, D. E., &amp; Kieras, D. E. ( 1997a ). A computational theory of executive control processes and human multiple‐task performance: Part 1. Basic mechanisms. Psychological Review, 104, 3 – 65. </bibtext> </blist> <blist> <bibl id="bib101" idref="ref197" type="bt">101</bibl> <bibtext>Meyer, D. E., &amp; Kieras, D. E. ( 1997b ). A computational theory of executive control processes and human multiple‐task performance: Part 2. Accounts of Psychological Refractory‐Period Phenomena. Psychological Review, 104, 749 – 791. </bibtext> </blist> <blist> <bibl id="bib102" idref="ref188" type="bt">102</bibl> <bibtext>Miller, G. A. ( 1956 ). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81 – 97. </bibtext> </blist> <blist> <bibl id="bib103" idref="ref28" type="bt">103</bibl> <bibtext>Miller, E. K., &amp; Cohen, J. D. ( 2001 ). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167 – 202. </bibtext> </blist> <blist> <bibl id="bib104" idref="ref5" type="bt">104</bibl> <bibtext>Miller, G. A., Galanter, E., &amp; Pribram, K. H. ( 1960 ). Plans and the structure of behavior. New York : Holt, Rinehart &amp; Winston. </bibtext> </blist> <blist> <bibl id="bib105" idref="ref150" type="bt">105</bibl> <bibtext>Milner, B. ( 1963 ). The effects of different brain lesions on card sorting: The role of the frontal lobes. Archives of Neurology, 9 ( 1 ), 90 – 100. </bibtext> </blist> <blist> <bibl id="bib106" idref="ref112" type="bt">106</bibl> <bibtext>Montague, P. R., Dayan, P., &amp; Sejnowski, T. J. ( 1996 ). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16 ( 5 ), 1936 – 1947. </bibtext> </blist> <blist> <bibl id="bib107" idref="ref191" type="bt">107</bibl> <bibtext>Navon, D., &amp; Gopher, D. ( 1979 ). On the economy of the human processing system. Psychological Review, 86, 214 – 255. </bibtext> </blist> <blist> <bibl id="bib108" idref="ref9" type="bt">108</bibl> <bibtext>Norman, D. A., &amp; Shallice, T. ( 1986 ). Attention to action: Willed and automatic control of behavior. In R. J. Davidson, G. E. Schwartz, &amp; D. Shapiro (Eds.), Consciousness and self‐regulation: Advances in research and theory (pp. 1 – 18 ). New York : Plenum Press. </bibtext> </blist> <blist> <bibl id="bib109" idref="ref218" type="bt">109</bibl> <bibtext>O'Reilly, R. C., &amp; Frank, M. J. ( 2005 ). Making working memory work: A computational model of learning in prefrontal cortex and basal ganglia. Neural Computation, 18, 283 – 328. </bibtext> </blist> <blist> <bibl id="bib110" type="bt">110</bibl> <bibtext>O'Reilly, R. C., &amp; Frank, M. J. ( 2006 ). Making working memory work: A computational model of learning in prefrontal cortex and basal ganglia. Neural Computation, 18, 283 – 328. </bibtext> </blist> <blist> <bibl id="bib111" idref="ref29" type="bt">111</bibl> <bibtext>O'Reilly, R. C., Herd, S. A., &amp; Pauli, W. M. ( 2010 ). Computational models of cognitive control. Current Opinion in Neurobiology, 20, 257 – 261. </bibtext> </blist> <blist> <bibl id="bib112" idref="ref154" type="bt">112</bibl> <bibtext>O'Reilly, R. C., Noelle, D. C., Braver, T. S., &amp; Cohen, J. D. ( 2002 ). Prefrontal cortex in dynamic categorization tasks: Representational organization and neuromodulatory control. Cerebral Cortex, 12, 246 – 257. </bibtext> </blist> <blist> <bibl id="bib113" idref="ref151" type="bt">113</bibl> <bibtext>Owen, A. M., Roberts, A. C., Polkey, C. E., Sahakian, B. J., &amp; Robbins, T. W. ( 1991 ). Extra‐dimensional versus intra‐dimensional set shifting performance following frontal lobe excisions, temporal lobe excisions or amygdalo‐hippocampectomy in man. Neuropsychologia, 29 ( 10 ), 993 – 1006. </bibtext> </blist> <blist> <bibl id="bib114" idref="ref231" type="bt">114</bibl> <bibtext>Paine, R. W., &amp; Tani, J. ( 2005 ). How hierarchical control self‐organizes in artificial adaptive systems. Adaptive Behavior, 13, 211 – 225. </bibtext> </blist> <blist> <bibl id="bib115" idref="ref194" type="bt">115</bibl> <bibtext>Pashler, H. ( 1984 ). Processing stages in overlapping tasks: Evidence for a central bottleneck. Journal of Experimental Psychology: Human Perception and Performance, 10, 358 – 377. </bibtext> </blist> <blist> <bibl id="bib116" idref="ref17" type="bt">116</bibl> <bibtext>Passingham, R. ( 1993 ). The frontal lobes and voluntary action. Oxford, England : Oxford University Press. </bibtext> </blist> <blist> <bibl id="bib117" idref="ref87" type="bt">117</bibl> <bibtext>Paus, T., Koski, L., Caramanos, Z., &amp; Westbury, C. ( 1998 ). Regional differences in the effects of task difficulty and motor output on blood flow response in the human anterior cingulate cortex: A review of 107 PET activation studies. NeuroReport, 9 ( 9 ), R37 – R47. </bibtext> </blist> <blist> <bibl id="bib118" idref="ref139" type="bt">118</bibl> <bibtext>Peshkin, L., Meuleau, N., &amp; Kaelbling, L. ( 1999 ). Learning policies with external memory. In Sixteenth International Conference on Machine Learning (pp. 307 – 314 ). San Francisco : Morgan Kaufman. </bibtext> </blist> <blist> <bibl id="bib119" idref="ref19" type="bt">119</bibl> <bibtext>Petrides, M., &amp; Milner, B. ( 1982 ). Deficits on subject‐order tasks after frontal‐ and temporal‐lobe lesions in man. Neuropsychologia, 20, 249 – 262. </bibtext> </blist> <blist> <bibl id="bib120" idref="ref105" type="bt">120</bibl> <bibtext>Pezzulo, G., &amp; Castelfranchi, C. ( 2009 ). Thinking as the control of imagination: A conceptual framework for goal‐directed systems. Psychological Research, 75, 559 – 577. </bibtext> </blist> <blist> <bibl id="bib121" idref="ref31" type="bt">121</bibl> <bibtext>Plaut, D., &amp; McClelland, J. L. ( 2000 ). Stipulating versus discovering representations. Behavioral and Brain Sciences, 23, 489 – 491. </bibtext> </blist> <blist> <bibl id="bib122" idref="ref159" type="bt">122</bibl> <bibtext>Plaut, D. C., McClelland, J. L., Seidenberg, M. S., &amp; Patterson, K. ( 1996 ). Understanding normal and impaired word reading: Computational principles in quasi‐regular domains. Psychological Review, 103, 56 – 115. </bibtext> </blist> <blist> <bibl id="bib123" idref="ref203" type="bt">123</bibl> <bibtext>Polyn, S. M., &amp; Kahana, M. J. ( 2008 ). Memory search and the neural representation of context. Trends in Cognitive Sciences, 12 ( 1 ), 24 – 30. </bibtext> </blist> <blist> <bibl id="bib124" idref="ref204" type="bt">124</bibl> <bibtext>Polyn, S. M., Norman, K. A., &amp; Kahana, M. J. ( 2009 ). A context maintenance and retrieval model of organizational processes in free recall. Psychological Review, 116 ( 1 ), 129 – 156. </bibtext> </blist> <blist> <bibl id="bib125" idref="ref7" type="bt">125</bibl> <bibtext>Posner, M. I., &amp; Snyder, C. R. R. ( 1975 ). Attention and cognitive control. In R. L. Solso (Ed.), Information processing and cognition: The Loyola Symposium. Hillsdale, NJ : Erlbaum Associates. </bibtext> </blist> <blist> <bibl id="bib126" idref="ref148" type="bt">126</bibl> <bibtext>Pouget, A., &amp; Sejnowsky, T. J. ( 1997 ). Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience, 9, 222 – 237. </bibtext> </blist> <blist> <bibl id="bib127" idref="ref76" type="bt">127</bibl> <bibtext>Rabbitt, P. M. ( 1966 ). Errors and error correction in choice‐response tasks. Journal of Experimental Psychology, 71, 264 – 272. </bibtext> </blist> <blist> <bibl id="bib128" idref="ref120" type="bt">128</bibl> <bibtext>Ratcliff, R. ( 1978 ). A theory of memory retrieval. Psychological Review, 83, 59 – 108. </bibtext> </blist> <blist> <bibl id="bib129" idref="ref172" type="bt">129</bibl> <bibtext>Reynolds, J. R., &amp; O'Reilly, R. C. ( 2009 ). Developing PFC representations using reinforcement learning. Cognition, 113, 281 – 292. </bibtext> </blist> <blist> <bibl id="bib130" idref="ref113" type="bt">130</bibl> <bibtext>Roesch, M. R., Calu, D. J., &amp; Schoenbaum, G. ( 2007 ). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience, 10, 1615. </bibtext> </blist> <blist> <bibl id="bib131" idref="ref243" type="bt">131</bibl> <bibtext>Rogers, T. T., &amp; McClelland, J. L. ( 2004 ). Semantic cognition: A parallel distributed processing approach. Cambridge, MA : MIT Press. </bibtext> </blist> <blist> <bibl id="bib132" idref="ref3" type="bt">132</bibl> <bibtext>Rosenblatt, F. ( 1958 ). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 6, 386 – 408. </bibtext> </blist> <blist> <bibl id="bib133" idref="ref155" type="bt">133</bibl> <bibtext>Rougier, N. P., Noelle, D. C., Braver, T. S., Cohen, J. D., &amp; O'Reilly, R. C. ( 2005 ). Prefrontal cortex and the flexibility of cognitive control: Rules without symbols. Proceedings of the National Academy of Sciences of the United States of America, 102 ( 20 ), 7338 – 7343. </bibtext> </blist> <blist> <bibl id="bib134" idref="ref101" type="bt">134</bibl> <bibtext>Rushworth, M. F. S., &amp; Behrens, T. E. J. ( 2008 ). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11, 389 – 397. </bibtext> </blist> <blist> <bibl id="bib135" idref="ref74" type="bt">135</bibl> <bibtext>Salinas, E. ( 2004 ). Fast remapping of sensory stimuli onto motor actions on the basis of contextual modulation. Journal of Neuroscience, 24, 1113 – 1118. </bibtext> </blist> <blist> <bibl id="bib136" idref="ref182" type="bt">136</bibl> <bibtext>Schapiro, A., Rogers, T., Cordova, N., Turk‐Browne, N., &amp; Botvinick, M. ( 2013 ). Neural representations of events arise from temporal community structure. Nature Neuroscience, 16, 486 – 492. </bibtext> </blist> <blist> <bibl id="bib137" idref="ref114" type="bt">137</bibl> <bibtext>Schultz, W., Dayan, P., &amp; Montague, P. R. ( 1997 ). A neural substrate of prediction and reward. Science, 275 ( 5306 ), 1593 – 1599. </bibtext> </blist> <blist> <bibl id="bib138" idref="ref117" type="bt">138</bibl> <bibtext>Seamans, J. K., &amp; Yang, C. R. ( 2004 ). The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Progress in Neurobiology, 74, 1. </bibtext> </blist> <blist> <bibl id="bib139" idref="ref2" type="bt">139</bibl> <bibtext>Selfridge, O. G. ( 1988 ). Pandemonium: A paradigm for learning ,“ in J. A. Anderson (Ed.), Foundations for Research (pp. 115 – 122 ). Cambridge, MA : MIT Press. </bibtext> </blist> <blist> <bibl id="bib140" idref="ref118" type="bt">140</bibl> <bibtext>Servan‐Schreiber, D., Printz, H., &amp; Cohen, J. D. ( 1990 ). A network model of catecholamine effects: Gain, signal‐to‐noise ratio, and behavior. Science, 249, 892 – 895. </bibtext> </blist> <blist> <bibl id="bib141" idref="ref193" type="bt">141</bibl> <bibtext>Shaffer, L. H. ( 1975 ). Multiple attention in continuous verbal tasks. In P. M. A. Rabbitt &amp; S. Dornic (Eds.), Attention and performance V (pp. 157 – 167 ). London : Academic Press. </bibtext> </blist> <blist> <bibl id="bib142" idref="ref18" type="bt">142</bibl> <bibtext>Shallice, T. ( 1982 ). Specific impairments of planning. Philosophical Transactions: Biological Sciences, 298, 199 – 209. </bibtext> </blist> <blist> <bibl id="bib143" idref="ref75" type="bt">143</bibl> <bibtext>Shenhav, A., Botvinick, M. M., &amp; Cohen, J. D. ( 2013 ). The expected value of control: An integrative theory of anterior cingulate cortex function. Neuron, 79 ( 2 ), 217 – 240. </bibtext> </blist> <blist> <bibl id="bib144" idref="ref8" type="bt">144</bibl> <bibtext>Shiffrin, R. M., &amp; Schneider, W. ( 1977 ). Controlled and automatic information processing: II. Perceptual learning, automatic attending, and a general theory. Psychological Review, 84, 127 – 190. </bibtext> </blist> <blist> <bibl id="bib145" idref="ref126" type="bt">145</bibl> <bibtext>Simen, P., Cohen, J. D., &amp; Holmes, P. ( 2006 ). Rapid decision threshold modulation by reward rate in a neural network. Neural Networks, 19, 1013 – 1026. </bibtext> </blist> <blist> <bibl id="bib146" idref="ref124" type="bt">146</bibl> <bibtext>Simen, P., Contreras‐Ros, D., Buck, C., Hu, P., Holmes, P., &amp; Cohen, J. D. ( 2009 ). Reward rate optimization in two‐alternative decision making: Empirical tests of theoretical predictions. Journal of Experimental Psychology: Human Perception and Performance, 35, 1865 – 1897. </bibtext> </blist> <blist> <bibl id="bib147" idref="ref177" type="bt">147</bibl> <bibtext>Simsek, O. ( 2008 ). Behavioral building blocks for autonomous agents: description, identification, and learning. PhD thesis, University of Massachusetts Amherst. </bibtext> </blist> <blist> <bibl id="bib148" idref="ref178" type="bt">148</bibl> <bibtext>Simsek, O., Wolfe, A., &amp; Barto, A. ( 2005 ). Identifying useful subgoals in reinforcement learning by local graph partitioning. Proceedings of the Twenty‐Second International Conference on Machine Learning (ICML 05). Madison, WI : Omnipress. </bibtext> </blist> <blist> <bibl id="bib149" idref="ref179" type="bt">149</bibl> <bibtext>Solway, A., Diuk, C., Cordova, N., Yee, D., Barto, A., Niv, Y., &amp; Botvinick, M. (in press). Optimal behavioral hierarchy. PLOS Computational Biology. </bibtext> </blist> <blist> <bibl id="bib150" idref="ref33" type="bt">150</bibl> <bibtext>Stroop, J. R. ( 1935 ). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, p. 643. </bibtext> </blist> <blist> <bibl id="bib151" idref="ref61" type="bt">151</bibl> <bibtext>Stuss, D. T., &amp; Benson, D. F. ( 1986 ). The frontal lobes. New York : Raven. </bibtext> </blist> <blist> <bibl id="bib152" idref="ref138" type="bt">152</bibl> <bibtext>Sutton, R. S., &amp; Barto, A. G. ( 1998 ). Reinforcement learning: An introduction. Cambridge, MA : MIT Press. </bibtext> </blist> <blist> <bibl id="bib153" idref="ref142" type="bt">153</bibl> <bibtext>Taylor, M. E., &amp; Stone, P. ( 2009 ). Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10, 1633 – 1685. </bibtext> </blist> <blist> <bibl id="bib154" idref="ref109" type="bt">154</bibl> <bibtext>Todd, M., Niv, Y., &amp; Cohen, J. D. ( 2008 ). Learning to use working memory in partially observable environments through dopaminergic reinforcement. Advances in Neural Information Processing Systems, Vol. 20. Cambridge, MA : MIT Press. </bibtext> </blist> <blist> <bibl id="bib155" idref="ref81" type="bt">155</bibl> <bibtext>Tzelgov, J., Henik, A., &amp; Berger, J. ( 1992 ). Controlling Stroop effects by manipulating expectations for color words. Memory and Cognition, 20, 727 – 735. </bibtext> </blist> <blist> <bibl id="bib156" idref="ref104" type="bt">156</bibl> <bibtext>Ullsperger, M., Von Cramon, Y., Bylsma, L., &amp; Botvinick, M. ( 2005 ). The conflict adaptation effect: It's not just priming. Cognitive, Affective and Behavioral Neuroscience, 5, 467 – 472. </bibtext> </blist> <blist> <bibl id="bib157" idref="ref152" type="bt">157</bibl> <bibtext>Weinberger, D. R., Berman, K. F., &amp; Daniel, D. G. ( 1991 ). Prefrontal cortex dysfunction in schizophrenia. In H. S. Levin, H. M. Eisenberg, &amp; A. L. Benton (Eds.), Frontal lobe function and dysfunction (pp. 276 – 285 ). New York : Oxford University Press. </bibtext> </blist> <blist> <bibl id="bib158" idref="ref195" type="bt">158</bibl> <bibtext>Welford, A. T. ( 1952 ). The ‘psychological refractory period' and the timing of high‐speed performance—a review and a theory. British Journal of Psychology: General Section, 43 ( 1 ), 2 – 19. </bibtext> </blist> <blist> <bibl id="bib159" idref="ref192" type="bt">159</bibl> <bibtext>Wickens, D. D. ( 1984 ). Processing resources in attention. In R. Parasuraman, D. R. Davies &amp; J. Beatty (Eds.), Varieties of attention (pp. 63 – 102 ). New York : Academic Press. </bibtext> </blist> <blist> <bibl id="bib160" idref="ref86" type="bt">160</bibl> <bibtext>Yeung, N., Botvinick, M. M., &amp; Cohen, J. D. ( 2004 ). The neural basis of error detection: Conflict monitoring and the error‐related negativity. Psychological Review, 111 ( 4 ), 931 – 959. </bibtext> </blist> <blist> <bibl id="bib161" idref="ref125" type="bt">161</bibl> <bibtext>Yeung, N., &amp; Monsell, S. ( 2003 ). Switching between tasks of unequal familiarity: The role of stimulus‐attribute and response‐set selection. Journal of Experimental Psychology: Human Perception and Performance, 29 ( 2 ), 455 – 469. </bibtext> </blist> <blist> <bibl id="bib162" idref="ref108" type="bt">162</bibl> <bibtext>Zipser, D., Kehoe, B., Littlewort, G., &amp; Fuster, J. ( 1993 ). A spiking network model of short‐term active memory (1993). Journal of Neuroscience, 13, 3406 – 3420. </bibtext> </blist> </ref> <p>Graph: A PDP model of the S troop e ffect (after Cohen, Dunbar, &amp; McClelland, 36 ), implementing the basic principles of g uided a ctivation t heory ( GAT ). Units correspond to representations of the different stimulus features, responses, associations between them, and control representations that select between the two pathways through top–down biasing. Thicker connections in the word‐reading pathway denote strong connection strengths assumed to have arisen from greater practice with this task. In this model, the control representations correspond to the stimulus dimensions relevant to performance of the S troop task. However, other control representations are assumed to exist that can bias (select) pathways appropriate for the performance of other tasks (see text and Fig.  ).</p> <p>Graph: Expanded PDP model showing mechanisms for the adaptive, self‐regulation of cognitive control. This includes a conflict monitoring mechanism (proposed to be served by anterior cingulate cortex) that modulates the activity of control representations, and an adaptive gating mechanism (proposed to be served by brainstem dopaminergic nuclei) that regulates the updating of control representations in prefrontal cortex. In this version of the gating model, afferents to prefrontal cortex are regulated directly by dopaminergic projections. However, alternative models have been proposed in which dopamine serves to train a gating mechanism implemented by the basal ganglia (see text).</p> <p>Graph: Top left: Schematic of the gating model proposed by O'Reilly and Frank ( 110 ), during performance of a task requiring maintenance of the stimuli “1” and “A” in working memory. At the point shown, a “1” has already occurred and has been gated into a prefrontal ( PFC ) stripe via a pathway through the striatum, substantia nigra ( SN r), and thalamus (thal). At the moment shown, an “A” stimulus occurs (Stim) and is gated into another PFC stripe. Two levels of context are thus represented. Top right: The network studied by Botvinick and Plaut ( 27 ). Arrows indicate all‐to‐all connections. Only a subset of units in each layer is depicted. Botvinick and Plaut ( 27 ) applied this network architecture to the problem of learning subtask hierarchies in goal‐directed behavior involving object manipulation. The resulting model developed internal representations that reflected the temporal relations among coherent action subsequences. When these internal representations were degraded, the model committed errors informed by the hierarchical structure of the target task, matching patterns seen in empirical studies of everyday slips of action. Bottom: The hierarchical model studied by Botvinick ( 18 ).</p> <p>Graph: Left: The task studied by Collins and Frank ( 45 ). During initial training, participants learned to respond to four stimuli (red triangle, red circle, yellow triangle, yellow square) with unique actions (A1‐4). Later, triangles and circles in two new colors (blue and green) were introduced. Participants who had learned to treat color as a task cue showed better transfer to the new color (blue) whose shape‐response associations matched those earlier linked to another color (red). Right: State space of the Tower of Hanoi task. The Tower of Hanoi involves a set of disks of varying sizes, each of which can be placed on any of three posts. The task is to move the disks one by one to reach a specified goal configuration, with the constraint that no disk can ever be placed on top of a smaller one. The figure builds up the task's state space starting with a single disk. The blue graph reflects the fact that, in the single‐disk case, there are three possible game states (graph nodes), with transitions possible between every pair (edges). The green graph shows what happens when a second (larger) disk is added. Note that this situation involves an asymmetric precondition constraint: The smaller blue disk can be moved to any post without reference to the position of the green disk, but the position of the blue disk strongly constrains where the larger green disk can be moved (and whether it can legally be moved at all). This asymmetry induces an interesting hierarchical structure in the state‐space graph. Specifically, the graph comprises three triangular cliques, throughout each of which the green disk occupies a specific post. Adding a third, still‐larger red disk adds further asymmetric constraints: Movements of the green disk are unconstrained by the red disk, but the green disk (along with the blue disk) strongly limits the moves available to the red disk. As shown in the red graph, these relations induce a further layer of structure in the state space. The three‐disk graph contains three large clusters corresponding to the three possible red‐disk positions, each of which subsumes three clusters reflecting the three possible green‐disk positions. The resulting structure illustrates the general point that networks of asymmetric preconditions naturally induce hierarchical task structures, comprising clusters or “communities” of states separated by state space bottlenecks (see Schapiro et al., 136 ).</p> <p>Graph: Left: The graph studied by Schapiro et al. ( 136 ). Center: A neural network trained to predict the next node to be visited in a random walk on the graph. Right: A multidimensional scaling rendering of the internal representations arising in the neural network for inputs corresponding to each of the 15 nodes in the graph. Colors correspond to those in the left panel. Adapted from Schapiro et al. ( 136 ), with permission.</p> <p>Graph: Examples of dual task performance. Left panel: The two tasks do not draw upon any representations in common (i.e., processing pathways are non‐overlapping), and thus there is no cross talk. Right panel: Both tasks draw upon the same representations (there is pathway overlap), and thus multitasking is subject to cross talk.</p> <p>Graph: An extension of the S troop model (see Fig.  ) illustrating the problem of cross talk associated with multitask performance. Additional stimuli (sounds), responses (stop and go), and control representations (for the sound stimulus dimension, and motor responses) have been added to the model, allowing several new tasks to be performed. (Note that tasks now require both dimension and response control representations to be activated. This was implicit in the original S troop model; see Cohen et al., 36 , Simulation 6 for a discussion, and Botvinick, Buxbaum, &amp; Jax, 22 ; for related ideas). Tasks involving pathways that do not overlap can be performed without cross talk (e.g., word reading and go‐no‐go to a sound). However, several tasks overlap and these cannot be performed without the risk of cross talk. These require that the control representations associated with only one task be active at a time. For example, go‐no‐go to sounds (control representations: “do” and “sound”) cannot be performed at the same time as color naming (control representations: “say” and “color”) without risking cross talk. This is due to the generality of the color control representation, which makes it useful for both the color naming and go‐no‐go tasks. This illustrates the tension between generality of control representations and overlap of pathways.</p> <aug> <p>By Matthew M. Botvinick and Jonathan D. Cohen</p> </aug> |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ1035921 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Botvinick%2C+Matthew+M%2E%22">Botvinick, Matthew M.</searchLink><br /><searchLink fieldCode="AR" term="%22Cohen%2C+Jonathan+D%2E%22">Cohen, Jonathan D.</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Cognitive+Science%22"><i>Cognitive Science</i></searchLink>. Aug 2014 38(6):1249-1285. – Name: Avail Label: Availability Group: Avail Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 37 – Name: DatePubCY Label: Publication Date Group: Date Data: 2014 – Name: SourceSuprt Label: Sponsoring Agency Group: SrcSuprt Data: National Science Foundation<br />National Institute of Mental Health (DHHS) – Name: NumberContract Label: Contract Number Group: NumCntrct Data: CRCNS 1207833<br />R01MH098815-01 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Evaluative – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Cognitive+Ability%22">Cognitive Ability</searchLink><br /><searchLink fieldCode="DE" term="%22Guidelines%22">Guidelines</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Processes%22">Cognitive Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Brain+Hemisphere+Functions%22">Brain Hemisphere Functions</searchLink><br /><searchLink fieldCode="DE" term="%22Computers%22">Computers</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Science%22">Cognitive Science</searchLink> – Name: DOI Label: DOI Group: ID Data: 10.1111/cogs.12126 – Name: ISSN Label: ISSN Group: ISSN Data: 0364-0213 – Name: Abstract Label: Abstract Group: Ab Data: Cognitive control has long been one of the most active areas of computational modeling work in cognitive science. The focus on computational models as a medium for specifying and developing theory predates the PDP books, and cognitive control was not one of the areas on which they focused. However, the framework they provided has injected work on cognitive control with new energy and new ideas. On the occasion of the books' anniversary, we review computational modeling in the study of cognitive control, with a focus on the influence that the PDP approach has brought to bear in this area. Rather than providing a comprehensive review, we offer a framework for thinking about past and future modeling efforts in this domain. We define control in terms of the optimal parameterization of task processing. From this vantage point, the development of control systems in the brain can be seen as responding to the structure of naturalistic tasks, through the filter of the brain systems with which control directly interfaces. This perspective lays open a set of fascinating but difficult research questions, which together define an important frontier for future computational research. – Name: AbstractInfo Label: Abstractor Group: Ab Data: As Provided – Name: DateEntry Label: Entry Date Group: Date Data: 2014 – Name: AN Label: Accession Number Group: ID Data: EJ1035921 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1035921 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1111/cogs.12126 Languages: – Text: English PhysicalDescription: Pagination: PageCount: 37 StartPage: 1249 Subjects: – SubjectFull: Cognitive Ability Type: general – SubjectFull: Guidelines Type: general – SubjectFull: Models Type: general – SubjectFull: Cognitive Processes Type: general – SubjectFull: Brain Hemisphere Functions Type: general – SubjectFull: Computers Type: general – SubjectFull: Cognitive Science Type: general Titles: – TitleFull: The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Botvinick, Matthew M. – PersonEntity: Name: NameFull: Cohen, Jonathan D. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Type: published Y: 2014 Identifiers: – Type: issn-print Value: 0364-0213 Numbering: – Type: volume Value: 38 – Type: issue Value: 6 Titles: – TitleFull: Cognitive Science Type: main |
| ResultId | 1 |
