Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Contents lists available at ScienceDirect

Neuroscience and Biobehavioral Reviews

journal homepage: www.elsevier.com/locate/neubiorev

Neural mechanisms of attending to items in working memory T

a,⁎ b,d b c a,b
Sanjay G. Manohar , Nahid Zokaei , Sean J. Fallon , Tim P. Vogels , Masud Husain
a
Nuffield Department of Clinical Neurosciences, University of Oxford, OX3 9DU, United Kingdom
b
Department of Experimental Psychology, University of Oxford, United Kingdom
c
Centre for Neural Circuits and Behaviour, University of Oxford, United Kingdom
d
Oxford Centre for Human Brain Activity, University of Oxford, United Kingdom

A R T I C LE I N FO A B S T R A C T

Keywords: Working memory, the ability to keep recently accessed information available for immediate manipulation, has
Working memory been proposed to rely on two mechanisms that appear difficult to reconcile: self-sustained neural firing, or the
Attention opposite—activity-silent synaptic traces. Here we review and contrast models of these two mechanisms, and then
Neural networks show that both phenomena can co-exist within a unified system in which neurons hold information in both
Hebbian plasticity
activity and synapses. Rapid plasticity in flexibly-coding neurons allows features to be bound together into
Attractor network
objects, with an important emergent property being the focus of attention. One memory item is held by per-
sistent activity in an attended or “focused” state, and is thus remembered better than other items. Other, pre-
viously attended items can remain in memory but in the background, encoded in activity-silent synaptic traces.
This dual functional architecture provides a unified common mechanism accounting for a diversity of perplexing
attention and memory effects that have been hitherto difficult to explain in a single theoretical framework.

1. Introduction unfocused items which are considered to be stored but in a non-privi-

leged state (Lewis-Peacock et al., 2012; Sprague et al., 2016). In con-
Our capacity to hold and manipulate information over delays of a trast, unfocused items are decoded better after their latent representa-
few seconds has long been thought to be subserved by the persistent tion is re-activated (Rose et al., 2016; Wolff et al., 2017). These findings
firing of neurons during the delay (Funahashi, 2017; Fuster and suggest that both active and inactive representations may coexist in
Alexander, 1971). However, a number of recent studies have instead WM, and items can move between these two states (LaRocque et al.,
proposed “activity-silent” working memory, in which synaptic weights 2014; Postle, 2016; Zokaei et al., 2014b). Computational neural models
hold information during the delay, even in the absence of neuronal of both active (Compte et al., 2000; Zenke et al., 2015) and silent (Mi
firing (Silvanto, 2017; Sreenivasan et al., 2014; Mongillo et al., 2008; et al., 2017; Mongillo et al., 2008) WM have been separately postulated,
Stokes, 2015). This dispute comes at a time when it is also becoming but neither type of model on their own accounted for shifts of attention
clear that working memory (WM) is not a homogeneous store. When we within WM. In Section 1 we review these models. In Section 2 we
hold multiple items in WM, strong attentional effects are apparent. For propose a model to account for both persistent activity and silent sy-
example, people are faster and more accurate to recall the last item naptic storage, that reproduces several neural and behavioral results
encoded, or the last item that was brought to mind (Chun et al., 2011; regarding the focus of attention within memory, and makes new tes-
Oberauer, 2002; Souza and Oberauer, 2016; Zokaei et al., 2014a). First, table predictions. Section 3 discusses some open questions regarding
we review how active and silent working memory have previously been models with this dual functional architecture.
modelled independently, and second, we build a simple neural model in
which sustained firing and activity-silent working memory are re- 1.1. Models of synaptic WM without sustained activity
conciled as attended and unattended items within memory.
One item in memory, termed the ‘focus of attention’, appears to be Rapid synaptic plasticity at the millisecond scale has been used to
in a privileged state. An item may enter the focus of attention when it is explain how a pattern of inputs can be remembered (Fiebig and
newly encountered, or if it becomes relevant for subsequent decisions Lansner, 2017; Sandberg et al., 2003). In these synaptic models, si-
or actions (Olivers et al., 2011). The identity of the focused item is multaneously-activated neurons become more strongly connected.
decodable using functional MRI and is susceptible to TMS, unlike the Whereas some models have utilized short-term facilitation (Mongillo

⁎
Corresponding author.
E-mail address: sanjay.manohar@ndcn.ox.ac.uk (S.G. Manohar).

https://doi.org/10.1016/j.neubiorev.2019.03.017
Received 21 January 2019; Received in revised form 18 March 2019; Accepted 23 March 2019
Available online 26 March 2019
0149-7634/ © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license
(http://creativecommons.org/licenses/BY/4.0/).
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

et al., 2008), others have proposed Hebbian plasticity, which requires (Parthasarathy et al., 2017; Rigotti et al., 2013). The bottleneck also
coincident firing of presynaptic and postsynaptic neurons (Fiebig and allows such models to predict interference errors, and may also account
Lansner, 2017; Sandberg et al., 2003). Short term facilitation permits for some attentional effects on decodability (Schneegans and Bays,
weakly-encoded activity patterns to spontaneously reactivate, allowing 2017a) but they cannot reinstate information that becomes fully un-
partial readout of those patterns (Trübutschek et al., 2017). decodable from activity. With the exception of the binding pool which
Plasticity has long been demonstrated in cortical neuron receptive includes token- or pointer-like representations (Swan and Wyble,
fields (Edeline et al., 1993) and may arise through a variety of synaptic 2014), these models account for WM primarily as perceptual storage, in
mechanisms (Zucker, 1989; Tsodyks and Markram, 1997; Fischer et al., sensory brain areas. They do not explain how other brain areas read out
1998; Dittman et al., 2000; Jensen et al., 1996; Malsburg, 1981). In or decode the stored information. Models involving spatial feature maps
particular Hebbian learning rules allow associative mappings to be (Schneegans and Bays, 2017b) account also for the privileged role of
formed between neurons that are co-active, so that when a partial spatial features, but they would require an analogous ‘map’ of temporal
pattern is later presented, the original combination of active neurons context to account for sequential same-location items.
can be re-activated, by associative recall. Such short-term Hebbian An attractive common feature of several of these active-storage
plasticity has been demonstrated in pyramidal neurons, is dependent on models is that the statistics of recall errors are accurately explained by
postsynaptic NMDA receptors, has a rapid onset after brief stimulation interference, governed by the proximity structure of features within
(e.g. after just 25 spikes over 500 ms) and may persist for up to 15 min each dimension (Oberauer and Lin, 2017). Thus, if two items are nearby
(Malenka, 1991). Stronger stimulation may lead to facilitation over on a feature dimension e.g. space or time, they are more likely to be
longer time scales, which may underlie associative episodic memory confused – as supported by behavioural data. In fact, this general result
(Burgess and Hitch, 2005; Rizzuto and Kahana, 2001), or long term of Oberauer and Lin applies both to associative context models and the
memory, which can provide the synaptic backdrop to support an active fixed conjunctive neuron models.
WM (Litwin-Kumar and Doiron, 2014, 2014; Zenke et al., 2015). Rapid
plasticity in auto-associative networks can also account for serial recall 1.3. A new model of WM using a plastic attractor
of sequences of items (Fiebig and Lansner, 2017; Howard and Kahana,
2002) – including serial order effects such as primacy and recency The present work unites persistent activity attractors with silent
(Farrell and Lewandowsky, 2002) – because new information may use synaptic storage. In our new class of memory model, both active and
up free space, or overwrite old information (Matthey et al., 2015; silent representations are essential to WM. We propose that persistent
Sandberg et al., 2003). activation serves as the focus of attention that encodes recent activity
One model of this kind uses associative plasticity, not between the patterns into synapses. Rapid plasticity in flexibly-coding neurons al-
co-occuring features themselves, but between the features and a sepa- lows features to be bound together into objects, with an emergent
rate ‘context vector’. Such temporal context models have been used to property being that the last item is maintained actively. Recent, pre-
explain episodic retrieval (Howard and Kahana, 2002). In these models, viously-attended items are preserved instead in synaptic traces. They
input patterns co-occurring in time are bound by Hebbian plasticity to a are in a non-privileged state but, importantly, can be re-activated by
temporal context — a vector which varies depending on the input itself. partial information.
In these synaptic models, the physiological meaning of a neuron’s We propose that attention arises from the interaction between two
firing depends upon its input and output connections. Plasticity in these distinct types of neural representation: fixed feature neurons, and freely-
models could therefore lead to neurons whose activity represents dif- conjunctive neurons (Fig. 1A). Feature neurons may be sensory, motor or
ferent things on different trials – a property that we characterize here as conceptual. They have fixed receptive fields or tuning curves – as ob-
flexible coding. Such models may therefore generate novel testable served in posterior cortical areas. In contrast, the freely-conjunctive
predictions about neurophysiological data. However these models do neurons can rapidly change their connection weights with the feature
not produce stable persistent-activity states in feature-selective neu- cells, and therefore their activity does not represent a fixed feature or
rons, which has long been considered a hallmark of WM (Funahashi, item in memory. Instead, through rapid plasticity on each trial, a con-
2017). junctive cell will come to encode a conjunction of simultaneously active
features, by forming a transient reciprocal associative mapping to fea-
1.2. Models involving sustained neural firing ture-selective neurons.
Persistent activity arises by mutual excitation between feature and
In contrast, in sustained activity models, items are held in WM by conjunction neurons. The conjunction neurons form a limited-capacity
virtue of delay-period activity (Compte et al., 2000; Funahashi, 2015; store that can hold many kinds of information in one place. Thus, our
Funahashi et al., 1989), which relies on positive feedback to allow model bridges the gap between neuron-level descriptions and the psy-
stimulus-induced activity to persist or resonate, leading to an “at- chological notion of a general-purpose register, sometimes termed a
tractor” state. (Chumbley et al., 2008; Wimmer et al., 2014; Zipser “memory slot” (Cowan, 2010; Luck and Vogel, 1997), a concept which
et al., 1993). Although such active maintenance may also depend upon has not as yet been characterized at the level of single prefrontal neu-
rapid changes in synaptic weights (Hansel and Mato, 2013; Pereira and rons. Such registers are difficult to explain unless individual neurons
Wang, 2015), the neurons generally retain their selectivity over time. can encode different types of WM content at different times. Our model
These models do not generally allow memory recall from a silent in- permits this by allowing rapid synaptic changes so that conjunctive
active state. neurons can represent many kinds of information, depending on the
Several non-plastic models have been proposed, in which features recent context.
are bound by persistent activity in fixed conjunctive neurons. Fixed We suggest that two lines of evidence point to such conjunction
conjunctions may involve a spatial map (e.g. Schneegans and Bays, neurons being located in prefrontal cortex (PFC): firstly, PFC is highly
2017a), neurons with mixed selectivity (Matthey et al., 2015; active in memory and manipulation (Eriksson et al., 2015; Postle et al.,
Schneegans and Bays, 2017b) or a “binding pool” (Bowman and Wyble, 2006), yet secondly, information is not always easy to decode
2007; Swan and Wyble, 2014). In all these hard-wired models, in- (Christophel et al., 2012; Cogan et al., 2017; Kamiński et al., 2017).
formation is stored only in the activation of neurons – not in their sy- Although WM contents can undoubtedly be decoded from many PFC
napses. They must therefore overcome a combinatorial problem by neurons, about 60% of prefrontal neurons appear to be nonselective,
employing lower resolutions (over low-level features) for the con- and even for those that are selective, they often show less than a 50%
junctive neurons. These models predict that binding neurons should modulation of their firing rate by information in WM (Miller et al.,
exhibit mixed selectivity, as observed in prefrontal cortex 1996; Parthasarathy et al., 2017). This apparently-nonselective

2
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Fig. 1. Conjunctive neurons form a plastic attractor to support attention and working memory.
A Two populations of neurons are distinguished based on their inputs. Posterior neurons (green) encode sensory-motor features, whereas prefrontal neurons (blue)
are “conjunctive”: i.e. they are able to rapidly increase or decrease their synaptic connectivity with patterns of feature neurons, using a Hebbian associative rule. We
simulated 12 feature-selective neurons (f) and 4 freely-conjunctive neurons (c). An active combination of neurons (pink) causes strengthening of synapses in both
directions, producing a stable attractor across brain areas. c=conjunctive cells, f=feature cells. W = synaptic weights, i=sensory input.
B Sequence of proposed neuronal events during attention, encoding and retrieval in working memory. 1. Sensory input activates features. In this case a vertical red
bar located at the top left of the display activates separate feature neurons tuned to orientation, color and location. 2. Features excite conjunctive neurons, which
compete. 3. The winning conjunction drives sustained activity. 4. New input to the system (in this case an oblique purple bar at bottom left) disrupts current firing
activity, but synaptic weightings remain. 5. Probe feature (in this case red colour) re-activates the original conjunctive unit that encoded the red vertical bar. 6.
Conjunctive unit re-activates original features, completing recall.

component of prefrontal activity could reflect transient and flexible response to the combination of active features. In the example shown in
coding by conjunctive units. Fig. 1 the conjunction units learn rapidly to encode combinations of
We first aim to provide a single common mechanism accounting for color, orientation and location (Fig. 1B.2). During encoding into WM,
a diverse range of perplexing attention and memory effects. Second, we the winning conjunctive unit sustains the activity of all co-active fea-
attempt to explain neurophysiological data where items in memory ture neurons through mutual excitation. This strengthens synapses in
initially produce persistent activity, which then falls “silent” when at- both directions through rapid Hebbian plasticity, further stabilizing the
tention shifts to new information (Konecky et al., 2017), and why active pattern. Once a conjunctive unit succeeds in reciprocally acti-
sometimes “inverted” representations of unattended information may vating a set of feature units, we say that attention is focused on the ac-
be decoded. Third, we aim to explain why many imaging studies con- tivated features, binding the features of a compound stimulus into a
clude that attention and working memory are “distributed” processes perceptual object.
involving both prefrontal and sensory brain areas (Christophel et al., The reciprocal feature-to-conjunctive synapses keep the novel
2017; Gayet et al., 2017, 2017; Xu, 2017) that also explain how WM combination of features persistently active, even when the stimulus is
enables us to encode and execute task rules. In our simulations, we no longer present (Fig. 1B.3).
chose to examine the extreme situation where conjunctive neurons are When a new stimulus arrives, a new pattern of sensory input de-
fully nonselective for features. This limiting scenario is of course im- stabilizes internal activity, shifting activity away from the attractor
plausible, since no single prefrontal neuron could receive input from carved by the first object. A new conjunction may win and form another
every feature neuron. However we argue that it is a highly illustrative attractor state by plasticity, which in our model amounts to shifting the
paradigmatic case. In reality prefrontal neurons will necessarily have focus of attention to the newly activated feature pattern. Crucially,
some degree of selectivity, but here we focus only on characterizing the however, synapses between the previous object’s constituent features
novel concept of how rapid plasticity can give rise to flexible coding, and one particular conjunctive unit remain strengthened even after
and therefore we model purely conjunctive neurons as distinct from those neurons fall silent (Fig. 1B.4). Thus, presenting any one feature of
feature-selective neurons. a previously attended object (e.g. color, as shown in Fig. 1) will act as a
memory probe, re-activating the corresponding conjunction neuron
(Fig. 1B.5), and therefore also the other features that were associated
2. Simulation of a generic feature binding model with it (Fig. 1B.6). The object’s features are therefore recalled by auto-
associative pattern completion, which brings them back into an at-
2.1. Operation of the model tended, foreground state. Separate objects must always be encoded
sequentially, which we suggest is plausible in light of the empirically
When a stimulus is perceived (Fig. 1B; Movie S1), conjunctive observed attentional bottleneck in feature binding (Reynolds and
neurons compete through lateral inhibition to become active in

3
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Fig. 2. Predicting visuospatial WM capacity, encoding and decay.

A To simulate WM performance, four objects are presented sequentially, by activating feature neurons (f, activity depicted as a heatmap from dark blue to red)
indicating the color, orientation and location of each item. Conjunctive units (c) are shown below as four differently-colored traces. Conjunctive units compete to
become active for each object. One conjunctive unit wins for each object, driving activity that persists even after input is removed (yellow parts of heatmap). At the
time of the probe, a single feature is stimulated, triggering pattern completion. Recall is accurate if the orientation of the corresponding item is re-activated. Two
example trials are shown; note that different patterns of conjunctive units are activated on different trials even for the same stimuli, depending on trial history.
Example 1: good encoding. Example 2: weak encoding of the second item. Two conjunctive neurons with similar recent preferences compete to encode object 2
(arrowhead). When it is probed, item 4 is reported instead.
B & C When more items are encoded in the model, recall accuracy is reduced, as observed in data (adapted from Luck and Vogel, 1997).
D & E The last item encoded in the model is recalled better than others, as it remains active in the focus of attention during the delay period, matching observed serial
order curves. Figure adapted from (Gorgoraptis et al., 2011) indicates the probability of reporting the target item as calculated by fitting the distribution of responses
in a similar task.
F&G Shorter encoding durations reduce modelled recall accuracy. Data from a similar task (adapted from Bays et al., 2011b) where adding items reduced both initial
encoding rate and asymptote. The model qualitatively reproduces the interaction observed in human performance.
H & I The model predicts faster memory decay when more items are stored. This matches the empirical interaction between memory-set size and delay. Data adapted
from (Pertzov et al., 2016) shows the modelled probability of reporting the target. Note that at very short delays, model recall was more accurate than in human data.

4
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Desimone, 1999). distinguishing feature to each object (Fig. S7). Controversially, some
To demonstrate the power of the model, we simulated a common studies indicate that objects form fundamental units (Hardman and
visuospatial WM task (Fig. 2A) in which participants remember the Cowan, 2015; Luria and Vogel, 2011), whereas others suggest show
orientations of a set of colored bars (e.g. Gorgoraptis et al., 2011; that the features of an object can be forgotten independently (Bays
Pertzov et al., 2016). Neurons were modelled as firing-rate units et al., 2011a; Fougnie and Alvarez, 2011; Wang et al., 2017). Our model
obeying a Hebbian plasticity rule (see Methods). Memory items were predicts a mixture of feature-based and object-based forgetting (Fig.
composed of combinations of features, and up to four unique items S17).
were presented sequentially to the feature units. After a delay, we
probed one of the items by activating its color-feature alone, and re- 2.3. Serial order effects
cording whether its orientation was subsequently re-activated. Re-
markably, just four color, orientation, location and conjunctive neurons When we remember a sequence of objects, we recall the first and
each are needed to explain a wide range of behavioral and neurophy- last objects better (primacy and recency). Our model can reproduce
siological data, which no models have yet captured (Table S1). both of these effects. Simulated performance (Fig. 2D) matched the
Crucially both the activation and learning equations were im- serial position curve obtained in WM experiments (Fig. 2E). The si-
plemented continuously over a block of trials, with blank input in be- mulation suggests that neutrally, primacy benefits arise because the
tween trials, so that encoding, recall and interference from the previous first object in a trial does not need to compete with ongoing persistent
trial all arose naturally from the way stimuli were presented. We tuned activity from a previous item (Fig. 1B4). In our model this relies on the
the model to perform at levels comparable to humans at this task (see fact that, at the start of each trial, feature units are inhibited but pre-
Methods). vious synaptic weights are not erased – though there is no explicit
For clarity, here we elected to keep the model’s operation almost signal to forget items from the previous trial. Recency benefits arose for
identical for all the simulations, even though the experimental data we two reasons. First, the finally-encoded item did not incur retroactive
match come from a variety of tasks and measures. Although it is pos- interference from subsequent items, whereas previous items are cor-
sible to adjust the numbers of features, synaptic and learning para- rupted by interference when subsequent items are encoded. Second, the
meters and timings to reproduce each individual experiment, this per- final item remains in an active state rather than a silent state during the
mits many degrees of freedom. Thus we believe that showing that a delay. Note that our plasticity rule has no explicit temporal decay.
single generic model can qualitatively reproduce all the effects makes Because capacity limits are generated through interference, we only
clear the capabilities and limitations of the basic model. Moreover, we require that the plasticity lasts longer than the memory delay (Fig. S11).
wished to emphasise that features within the model could also map to Serial position effects are strongly disrupted when items share features
non-visuospatial attributes – such as motor or conceptual representa- (Fig. S16).
tions.
First we confirm the network can qualitatively produce standard 2.4. Encoding and maintenance
working memory effects; second, we demonstrate the novel features of
the model regarding the focus of attention; third, we show that the The time-course of encoding was interrogated by presenting items
model accounts for neural data from multi-item WM; and finally, we for brief durations, and demonstrated exponential saturation with an
make some novel predictions and show that model could be extended to asymptote dependent on the number of items encoded. In a similar
implement task rules. empirical study (Bays et al., 2011b), memory precision (1/standard
deviation of response angular error) followed a similar pattern. In that
2.2. Capacity limits and serial order in WM study, the probability of choosing the target was not calculated, but
their reported precision appears to correspond well to our model’s
A key feature of WM is its limited capacity. The more items held in probability of reporting the correct target orientation (Fig. 2F&G). Ac-
memory, the less accurately they are remembered (Luck and Vogel, curacy and precision are not guaranteed to be equivalent measures
1997; Bays and Husain, 2008). Simulated recall accuracy (Fig. 2B) however.
matched the set-size effect from classical visuospatial WM experiments Simulations demonstrated that memory deteriorates faster when
(Fig. 2C). This is because each additional stimulus competes for con- increasing numbers of items are remembered (Fig. 2H&I), as shown in a
junctive neurons, and may corrupt or overwrite synaptic traces of recent study (Pertzov et al., 2016). This arises because a greater pro-
previously-seen objects. Whether a previous item is overwritten is de- portion of items are held in an unattended state. Unattended items are
termined by how well the currently-active features match the existing more vulnerable to interference, because their synapses are gradually
synaptic weights, which are themselves continuously subject to Heb- weakened over time. This occurs not because of any specific decay rule,
bian rules. Therefore in our model, capacity is limited by interference but rather because the plasticity rule operates continuously to alter all
between items in memory, similar to several previous psychological synaptic weights, and this ‘erodes’ the representations that are not
models (Howard and Kahana, 2002), in line with convergent evidence currently active, such that all non-attended features become more
from multiple WM domains (Almeida et al., 2015; Farrell et al., 2016; homogeneously connected to the non-attended conjunctive neurons
Oberauer and Lewandowsky, 2014). Note that with our canonical ex- (see Supplementary Video). Memory items therefore interfere with each
ample parameters, accuracy is lower than the illustrated data because other during the delay. Our model also makes the strong prediction that
the model chooses between four options rather than two, but varying an item stored in an attended state (e.g. the final item in a sequence) is
the model parameters can make it arbitrarily more accurate (Fig. more robust to decay over time. For very short delays, the last item was
S8,S9). Moreover the capacity limit is not simply determined by the recalled even better (akin to an “iconic” effect, Fig. S13), an effect that
number of conjunctive neurons, and can be adjusted by tuning the level was not seen in human data.
of inhibition if more conjunctive neurons are used (Fig. S14).
Importantly the model predicts the counterintuitive finding that 2.5. Shifting the focus of attention
storing extra features on different dimensions within a single object
either occurs automatically (Allen et al., 2006) or else may incur a An important advance over other models, is the ability of our model
smaller cost than a separately-encoded feature (Luck and Vogel, 1997) – to re-activate a previous item by bringing it into the focus of attention.
although other studies have demonstrated that extra features do impose The logic here is that sensory input can guide attention by pattern-
costs (Oberauer and Eichenberger, 2013). Our model predicts that completion. In behavioral experiments, an “incidental” task inserted
primacy effects may be stronger when adding an irrelevant but into the memory delay can shift attention to one of the items in memory

5
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Fig. 3. Shifting the focus of attention in WM.

A Experiment (Zokaei et al., 2014a) where participants remembered two items, each comprising three features: color, location and orientation. During the retention
interval, a color was shown, and as a secondary task, the location of the corresponding object had to be recalled. At the end of the delay, a color was shown which
could indicate the same (“congruent”) or different (“incongruent”) object than the one tested during the delay. Participants then reported the orientation of the
corresponding object. Reproduced under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) license (https://creativecommons.org/licenses
/by/3.0) from Fig. 1A of Zokaei et al. (2014a), The Journal of Neuroscience. January 1, 2014. 34(1);158–162.
B Similar events were simulated, with an incidental cue (IC) during the delay. If the first object was cued, then persistent delay activity shifted to the cued item.
C&D The model predicts that the item in the focus of attention before recall is reported more accurately, matching data. Probability of target from mixture model
fitted to data of Zokaei et al. 2014.
E Decoding direction of object 2 from feature-selective units during the delay, on trials where the first item was cued (IC). Decodability is low but still above chance
after the cue, with below-chance performance of a cross-decoder trained on trials where the second item was cued (full analysis Fig. S12).

(Fig. 3A) (Zokaei et al., 2014a) bringing it into the foreground. We si- 2.6. Recall
mulated “retro-cueing” one of the items during the memory delay by
presenting one of its features for a brief period, which brought that item After the probe feature was activated, it took a number of time steps
back into the focus of attention (Fig. 3B). The external cue could thus for the conjunction and response feature units to become active. We
re-activate a memory item which was previously encoded silently. Note measured this time to obtain reaction time predictions, which varied
that this simulation illustrates how feature-selective units can exhibit inversely with accuracy similar to empirical data (Fig. S1).
task-dependent modulation because they also receive non-sensory input The process of recall may also be susceptible to interference, be-
through rapidly-plastic synapses from the conjunctive units. cause it effectively uses pattern completion to re-activate the other
Recall of the incidentally-cued item improved, compared to the features of the corresponding object. In particular, the memory probe
uncued item (Fig. 3C), matching experimental data (Fig. 3D). This at- itself can interfere with recall, for example if it contains a feature on the
tentional shifting also explains how cues that indicate which item will dimension that needs to be reported (Fig. S2), in line with empirical
be probed (predictive retro-cues, Rose et al., 2016) improve perfor- probe-interference effects (Souza et al., 2016). Items in the focus of
mance. attention are protected from probe interference, presumably because
they do not need to be brought back from an inactive state (Makovski
et al., 2008; Wang et al., 2018; Tabi et al. 2019) Interference of another

6
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Fig. 4. Introducing a pulse of excitation during the delay period.

A After presenting two items, during the delay all feature neurons
f received an excitatory input pulse i=+1, consequently acti-
vating conjunction neurons.
B&C We tried decoding the identity of each of the two stimuli
from feature neuron activity. Although the first object was not
decodable without the pulse, it became transiently distinguish-
able (*) after the pulse. This matches the observed increase in
decodability after TMS (Rose et al., 2016).
D&E Stronger pulses altered model performance, abolishing the
benefit for the second item, which was in the focus of attention.
The pulse disrupted persistent activity, re-instating competition
between conjunctive neurons. This results in randomly re-se-
lecting which of the stable states of the plastic attractor is active.
The prediction matches observed effects of TMS targeting motion-
selective cortex (probability of selecting the target in mixture
model fitted to data from (Zokaei et al., 2014a).

kind arises when recalling items as a whole series: often the preceding from feature-selective neurons (Fig. S4) are in keeping with human and
or following item is reported instead (Smyth, 1996; Solway et al., nonhuman physiological data demonstrating that only the attended
2012). Although our simulations probe a single item at a time, they still WM item is decodable using standard techniques (Konecky et al., 2017;
demonstrate such “transposition errors”, where consecutively presented Lewis-Peacock et al., 2012; Sprague et al., 2016). Second, evoking
objects are confused (Fig. S3). neural activity by stimulation can restore decodability from EEG signals
(Rose et al., 2016; Wolff et al., 2017). We simulated transcranial
2.7. Neural encoding of items in WM magnetic stimulation (TMS) by an indiscriminate pulse of activation to
feature neurons (Fig. 4A), and decoded one feature dimension from
Three major predictions emerge about neural decoding. First, an feature-selective units (Fig. 4B). If the model’s color and orientation
emergent property of our framework is that sustained activity re- feature dimensions are considered as mapping to spatial location and
presents a single item held in memory (Funahashi, 2017), but not stimulus category respectively, then the simulation matches the effects
multiple items (Lara and Wallis, 2014). We used a linear decoder to of TMS on decoding (Fig. 4C) (Rose et al., 2016), or if they are instead
extract information about one feature of one of the items in WM, after mapped to spatial location and orientation, then the model’s results
items had been encoded. The predictions of the model for decodability reproduces the effects of a high-energy visual pulse (Wolff et al., 2017).

7
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Simulating a stronger pulse of stimulation disrupted attention, but not confirms that each conjunctive neuron’s activity represents different
synapses. This worsened recall of the attended item, yet contrarily things, as its synaptic weights change. Such a system can flexibly en-
improved unattended items (Fig. 4D&E), precisely as demonstrated code a broad variety of novel information rapidly, without incurring the
empirically (Zokaei et al., 2014a). combinatorial explosion that haunts previous fixed-selectivity models
We were initially surprised to note that even when the second item (Matthey et al., 2015; Postle et al., 2006).
is in the focus of attention, during the delay period, decoding for the
first item is still above chance. We therefore employed ‘cross-decoding’ 2.8. Simulation of task sets
to examine whether an item is encoded in the same way when it is
attended vs unattended (Fig. S12). The accuracy with which a classifier The same system can also implement stimulus-response rules, if
could decode an item from the activity of feature neurons fell con- some feature neurons represent motor plans. In this case, we encode a
siderably below chance when training on attended and testing on un- task rule by attending to a stimulus and a motor plan together. For
attended representations, and vice versa. Thus, unattended items were example, if a left-hand movement plan is activated while a red color-
encoded in an ‘inverse’ pattern to the attended items. Why should this feature is simultaneously activated, they will be encoded together into
be? In our model, feature neurons of items not in the focus of attention working memory. The conjunction of sensory features with a motor
are inhibited by the mutual competition in that layer. This led to a non- plan creates a task-set mapping (Duncan et al., 2012). Later, that sti-
specific inhibition of the unattended three features in each dimension. mulus can also re-activate the corresponding motor plan by pattern-
Moreover, conjunctive neurons for unfocused objects are also inhibited completion, triggering the movement – so that the stimulus generates a
by competition, leading to selective inhibition of neural pattern corre- response. Task sets can therefore be rapidly formed by sequentially
sponding to the unattended item. Remarkably, several studies in recent attending to stimulus-response pairs (Curtis and D’Esposito, 2003), and
months have suggested this “representational inversion” phenomenon deciding on an action is simply the motor analogue of WM recall.
can be observed in human imaging data (van Loon et al., 2018; To simulate stimulus-response mapping, we presented the task rules
Rademaker et al., 2018; Yu and Postle, 2018). sequentially, each consisting of a pairing between one color and one
Third, the model predicts that decoding from prefrontal cortex is response (Fig. S6A). Then on each subsequent trial, a single color from
unreliable (Lee and Baker, 2016). This is because the concept of a re- the set was shown, and the response was recorded. The model re-
ceptive field breaks down for conjunctive neurons. The same activity produces Hick’s law, in which response times are longer in situations
can have different meanings on different trials, dependent on residual when more response options are possible in the current task set (Fig.
synaptic weights from previous trials. Such neurons should show much S6B) (Proctor and Schneider, 2017). It also produces faster reaction
stronger representations over short timescales. We predict this will times when the response is repeated from the previous trial (Fig. S6C),
manifest behaviorally, with better recall for a feature combination in line with experimental evidence (Schvaneveldt and Chase, 1969).
present on the previous trial (Fig. S5), because the same conjunction In this situation, the role of prefrontal conjunctions can be viewed as
unit will be re-used. Moreover, neural activity patterns in conjunction controlling representations in posterior cortex, i.e. routing information
neurons predict stimuli strongly if we consider data only from con- from perceptual to motor representation as governed by task sets held
tiguous pairs of trials, compared to data from temporally-separated trials in working memory, a role classically assigned to executive/supervisory
(Fig. 5A), and the pattern similarity should be even lower when inter- attention. Critically the model predicts that, because the task rules are
vening stimuli involve a recombination of the features (Fig. 5B–D). This held in WM across many trials rather than being repeatedly

Fig. 5. Conjunctive unit representations are stable over short timescales.

Conjunctive units change their selectivity over short periods. If selectivity were stable, neural patterns should be similar when the stimulus is the same. We compared
similarity of the pattern of an earlier trial, to trial n, during the delay periods of a series of 1-item trials.
A) The similarity of the conjunctive neurons’ delay activity pattern is calculated for trials where the stimuli were identical (blue line) or different (red line). Patterns
were more similar when stimuli were the same, compared to when stimuli were different, indicating “classical encoding” at least for nearby trials. This classical
behavior decreased with the temporal distance between trials. Since we modelled the extreme case where neurons are purely conjunctive, with no feature selectivity,
consistency of pattern is completely abolished after about 6 trials.
B-D) The model predicts that interference reduces pattern similarity over time by overwriting the synaptic weights. If the objects in intervening trials share one
feature with the nth trial object, but mismatch on the other feature dimension, then we say the conjunction between the two feature dimensions is “violated”.
B) When the intervening trial contained a violation, the patterns on the n-2 and nth trials reflected the stimuli much more weakly, indicating interference or
overwriting of the original conjunction.
C and D) Trials 3-back and 4-back were similarly examined, this time asking how many intervening conjunction violations occurred. The more overwriting that
occurred between the n-3 and nth trials, the less classical encoding could be observed.

8
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

overwritten, the current stimulus and response (i.e. the active task rule) are traditional, place-coded (“labelled-line”) units. They are selective
are consistently decodable from conjunctive neurons, until the rules because they have some fixed, non-plastic inputs (or in the case of
change (Fig. S6D). This contrasts with WM storage, where frequent motor units, fixed outputs). But if plasticity modifies both the input and
overwriting leads to poor decoding, and may explain why task rules output synapses of a neuron, the meaning or interpretation of a neu-
have generally been easier to decode from PFC (Reverberi et al., 2012; ron’s firing will also change. This is simply because neurons code in-
Sakai, 2008). formation only in virtue of their inputs and outputs. Plasticity therefore
begets a new category of flexibly-coding neurons, where the informa-
2.9. Simpler models tion signaled by firing is protean and dependent on the history on each
trial. Decoding the fine-grained identity of stimuli from prefrontal
To investigate the necessity of various components of our model, we cortex is unreliable compared to posterior sensorimotor regions (Cogan
compared the full model with three variants with simpler assumptions. et al., 2017; Lee and Baker, 2016), because the idea of a receptive field
First, we examined a model that used non-Hebbian short-term facil- breaks down. Standard decoding methods assume trial-to-trial stability
itation at synapses between the feature and conjunctive neurons. With of activation patterns to represent a given feature, and so do not
the small number of fully-connected neurons in our model, this is un- measure the sequential effects we predict.
able to generate stable persistent activity because facilitation is not This flexible coding scheme is crucial for our model to generate two
synapse-selective (Fig. S10A). Second, we removed plasticity from just phenomena. First, it permits sustained activity that is guided dynami-
the conjunctive-to-feature neurons. This network was able to produce cally by task sets or objects in memory, which we postulate corresponds
stable persistent activity, but was unable to re-activate appropriate to attentional interactions between frontal and temporo-parietal re-
features during recall, because the reciprocal synapses back to the gions. Second, because individual neurons can encode different things
feature neurons did not develop appropriate selectivity (Fig. S10B). at different times, information must compete to be encoded by any
Third, we examined a model without conjunctive neurons, but with conjunctive neuron – thus leading to a capacity limit for general-pur-
Hebbian plasticity directly between feature neurons. This model was pose information storage, observed in both WM and attention. This may
able to produce sustained activity, and could shift attention between help resolve a long-standing theoretical debate on whether working
items in memory. It accounts for set size and some aspects of the serial memory consists of pointers, or activated long-term memory (Norris,
position curves, but did not produce interference between memory 2017): conjunctive neurons act as pointers that activate long-term
items during the delay (Fig. S10C). Moreover, without conjunctive memories. If the inputs to feature-selective neurons are governed by
neurons, we are unable to predict prefrontal activation during WM long-term memory, then their activity may convey abstract conceptual,
tasks, or the possibility of activity without apparent selectivity. configural or statistical information about the world, which could then
Conjunctive neurons also potentially allow for extending the model to also be flexibly controlled by conjunctive neurons. For example, it
support gating of distractors and internally-driven shifts of attention, would be possible to store arbitrary associations–for example, “if you
without invoking extensive prewired connectivity between feature-se- see a platypus, touch your nose with your left hand”, by holding the
lective neurons and prefrontal cortex or thalamus. For these reasons, we neurons active during the encoding of the rule that are selective for
conclude that a combination of Hebbian plasticity and flexibly con- platypuses, moving the left hand, and targeting the nose.
junctive neurons are critical components for our particular model. The conjunctive neurons in our model mathematically resemble
temporal contexts (Howard and Kahana, 2002), but those models have
3. Discussion focused on behavioral effects in list recall. They do not explicitly keep
one item in a privileged state, and make few direct neural predictions.
The model of freely-conjunctive neurons presented here accounts Unlike the temporal context model, our model does not account for
for both sustained firing and activity-silent synaptic traces in WM working and episodic memory in a single framework.
(Silvanto, 2017; Stokes, 2015), and consequently makes a range of
testable behavioral and neural predictions (Table S1). This neuronal 3.2. Relaxing the model’s assumptions
framework provides a parsimonious mechanism for feature binding,
general-purpose memory ‘slots’, and task sets. The model reproduces In this study we deliberately chose to study the simplest possible
classical WM effects of capacity, serial order, encoding rate, temporal model that could support conjunctive neurons. The very small number
decay (Fig. 2), reaction times, and transposition errors (Figs. S1&3), as of neurons, and their simple learning and dynamics, makes it much
well as the ability to switch attention between items within memory – a easier to see how they interact to generate the novel predictions.
phenomenon that evades most current models (Fig. 3). At a neural Moreover it is much more transparent where the model can or cannot
level, it explains why it is difficult to decode memory contents from match existing data. Naturally there are many directions in which the
prefrontal activity, why only the item in the focus of attention can be model needs to be extended, to fully reproduce the phenomena ob-
decoded elsewhere (Fig. S4). Further it explains why decodability can served in real neurons. A number of its assumptions can plausibly be
be restored by re-focusing an unattended item, or after a perturbation relaxed.
such as transcranial magnetic stimulation (TMS) or bottom-up input
(Rose et al., 2016; Wolff et al., 2017), which presumably re-activate the 3.3. Pure flexible and stable representations
conjunctive neurons and thus an object’s features through synaptic
traces (Fig. 4). The model also makes strong novel predictions about For simplicity we have treated conjunctive neurons as “pure”: i.e.
probe interference, trial-to-trial effects (Figs. S2&5), and disruption of that they are homogeneous and domain-general, resulting in inability to
neural pattern similarity by intervening stimuli (Fig. 5). decode information across many trials. This architecture parallels the
psychological notion of a memory slot. However it is certainly im-
3.1. Flexible neural codes plausible because all-to-all connections between PFC and feature-se-
lective neurons are not feasible. Moreover, how can we then explain
One strength of our model is that it allows pattern completion using studies that do demonstrate decoding of WM from prefrontal areas? In
flexible attractors, potentially providing a mechanism for mapping in- reality, we envisage that each conjunctive neuron is likely to receive
formation in WM to appropriate responses, via changes in the focus of inputs from only a subset of feature neurons. In order for conjunctive
attention. neurons to bind features into objects, these inputs must at least include
To support flexible attractor states, we postulated two distinct multiple feature dimensions and multiple features in each dimension.
modes of neural representation (Fig. 1). First, feature-selective neurons The model is therefore potentially compatible with the presence of

9
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

mixed selectivity (Rigotti et al., 2013), which would provide a back- explains how a single architecture could hold generic, content-general
ground of weak input selectivity based on the presence or absence of information in WM.
connections, upon which rapid plasticity is superimposed. This means
that the variable selectivity predicted by the model (Fig. 5) would not 3.4. Internal control over attentional shifts
be as strong in real neurons, and indeed single-unit studies do suggest
considerable stability in monkey prefrontal cortex, at least when We have assumed that attentional shifts are externally cued.
averaged over many trials (Freedman et al., 2001; Rainer et al., 1998). Endogenous shifts of attention are not modelled. One way of im-
We note that stable mixed selectivity, even without plasticity, could in plementing internally-generated attentional modulation would be to
some situations produce binding and capacity limits (Matthey et al., de-stabilize the persistent activity by adding delayed suppression, or
2015). However without additional mechanisms, it would presumably refractoriness, to the competitive conjunctive neurons. The result would
not account for attentional shifts, activity-silent storage, or apparent be that, after an object is attended, its activity is extinguished after a
control over posterior cortical areas, and moreover it complicates many delay, leading to a transient and unstable focus of attention. Akin to
accounts of how other brain areas would ‘read-out’ WM contents. some models of visual attention guidance (Itti and Koch, 2001), at-
Further, there may also be significant topography in conjunctive tention may then be successively re-deployed towards weaker-re-
cells connectivity, which we have neglected. For example, different presented features in WM. This would be needed to account for three
regions of prefrontal cortex may be specialized for remembering dif- key phenomena: (a) rehearsal, in which attention moves sequentially
ferent kinds of information (Romanski, 2004). This may have two de- between items during a memory delay, (b) the ability to free-recall WM
sirable consequences. First, aspects of the attended object – especially items in order, and (c) to permit serial encoding of a simultaneously-
information that is highly topographical in posterior areas, such as presented memory array. Our model currently relies on each object to
stimulus category and spatial location – would be consistently decod- be presented or attended sequentially, like the temporal context model
able from prefrontal cortex (Lee and Baker, 2016) but will be modu- (Howard and Kahana, 2002).
lated by relevance (Kornblith and Tsao, 2017). Second, conjunctive Although WM maintenance commonly engages PFC, evidence from
neurons in different prefrontal subregions may connect preferentially to neuropsychology and functional imaging suggests PFC’s role includes
visual, motor or auditory cortex, which could account for the separ- cognitive control, WM manipulation, and response selection, rather
ability of visuospatial and phonological WM and also their overlap than simply WM storage (Bechara et al., 1998; D’Esposito and Postle,
(Morey et al., 2011). One concern with freely-conjunctive neurons is 1999; Rowe et al., 2000; Thompson-Schill et al., 2002), and it remains
that, in order to allow truly arbitrary information or instructions to be to be tested whether the conjunctive neurons we propose can perform
stored, they would need an implausibly large array of hard-wired in- such functions. For example, we cannot account for the ability to “gate
puts. These concerns might be allayed by including topography, e.g. out” distractors, and prevent them from being encoded in WM. How
intermediate layers of flexible but more domain-specific conjunctive could irrelevant distractors be ignored, while still allowing relevant in-
neurons. puts to capture attention? To achieve this, conjunctive units would
Some studies show that more than one item can be prioritized and themselves need to be under higher-level control. The current model,
recalled better. Does this indicate that attention can highlight more with only one layer of conjunction units, does not explain higher order
than one item at a time (Cowan, 2011)? Some recent findings suggest control of attention, since sufficiently-strong bottom-up stimuli that
there may be two or more “foci of attention” (Christophel et al., 2018; match a conjunction will always tend to re-activate that conjunction
Sutterer et al., 2018). Our model may still be compatible with some of and thus capture the focus attention. The conjunction and feature
these findings, as it predicts graded benefits for more than one item. neurons together simply act as a “matched filter”, amplifying patterns
Recently-focused items have facilitated synaptic weights, even though that have recently been active (Chrysikou et al., 2014; Hayden and
only one item is technically held in an active state. An alternative Gallant, 2013). The model also cannot yet perform n-back tasks, where
strategy might be to directly permit multiple foci of attention within a decision must be made regarding items presented earlier in a se-
this model. To do this, conjunctive neurons could be partially segre- quence. After an item is presented, attention seems to shift back to
gated, which in theory could generate more than one focus of attention previously-presented items (Greene et al., 2015). Perhaps gating vs
(although this would require reduction of inhibition between features, granting access to working memory by preventing this might be con-
and would significantly disrupt encoding). trolled by interactions between prefrontal cortex and the basal ganglia
We treated “features” as just simple perceptual attributes, but we (Badre, 2012; Chatham et al., 2014).
believe that our class of feature-selective neurons could include any In some studies, activity-silent representations have been associated
aspect of the world that is encoded in a stable way, including those with so-called non-conscious WM. In this phenomenon, the identity of a
aspects that incorporate long-term knowledge, such as object identity, subliminal masked stimulus can be guessed above chance despite re-
category, or even linguistic information such as word meanings. These ports that no stimulus was seen, and despite an intervening distractor
attributes are likely to be encoded stably in posterior cortical areas, in (Soto et al., 2011). This non-conscious storage has been accounted for
contrast to the temporary combinations of information represented in by storage in synaptic weights (Trübutschek et al., 2017), in line with
an ephemeral way – e.g. for online manipulation – as typified by our non-conscious episodic memory (Chong et al., 2014). This contrasts
conjunctive neurons. The current simulations used only a single, rapid with our model, however, in which information in synapses can be fully
learning rate, but it remains to be studied how this could be reconciled reactivated and brought back into the focus of attention, and would
with longer-term learning. thus presumably be reportable.
Biological sensory neurons encode perceptual features on a con-
tinuous domain, with overlapping neural selectivities; however the 3.5. Location of conjunctive neurons and their plasticity
present model used only a few discrete features. It would therefore be
important to confirm that our model could also be extended to con- Conjunctive-coding neurons might not be confined to prefrontal
tinuous feature domains, to predict the range of proximity phenomena cortex. Other regions that play a role in working memory, such as the
accounted for by other models (Oberauer and Lin, 2017). Moreover, hippocampus, basal ganglia or thalamus, might also contain freely
unlike our model, visual representations in posterior cortex are ar- conjunctive neurons. Moreover, there may be a continuum or overlap of
ranged spatially, such that space is a fundamental component of all mechanisms subserving working memory and episodic memory (Fiebig
other feature representations. In its present form, our model does not and Lansner, 2014). Quite unlike the long-lasting episodic encoding
account for the unique role of space in visual WM (Pertzov and Husain, proposed in the hippocampus, however, the volatile synaptic weights
2014; Wang et al., 2016), but we argue that its simpler form better we propose would produce strong but evanescent trial-by-trial

10
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

selectivity changes (Fig. 5). A more intriguing possibility is that both Christophel, T.B., Iamshchinina, P., Yan, C., Allefeld, C., Haynes, J.-D., 2018. Cortical
freely-conjunctive and stable-feature neurons are actually present in the specialization for attended versus unattended working memory. Nat. Neurosci. 21,
494.
same brain regions, with a spectrum between highly-plastic and stably- Chrysikou, E.G., Weber, M.J., Thompson-Schill, S.L., 2014. A matched filter hypothesis
coding neurons. for cognitive control. Neuropsychologia 62, 341–355.
The Hebbian rule we use could share mechanisms with long-term Chumbley, J.R., Dolan, R.J., Friston, K.J., 2008. Attractor models of working memory and
their modulation by reward. Biol. Cybern. 98, 11–18.
potentiation (LTP). Stimuli too weak to elicit LTP can still elicit sy- Chun, M.M., Golomb, J.D., Turk-Browne, N.B., 2011. A taxonomy of external and internal
napse-specific potentiation that decays over minutes, sometimes termed attention. Annu. Rev. Psychol. 62, 73–101.
“short-term potentiation” or “relatively short-lasting LTP” (Malenka Cogan, G.B., Iyer, A., Melloni, L., Thesen, T., Friedman, D., Doyle, W., Devinsky, O.,
Pesaran, B., 2017. Manipulating stored phonological input during verbal working
and Nicoll, 1993; Frey and Morris, 1998). This potentiation differs from memory. Nat. Neurosci. 20, 279–286.
post-tetanic potentiation and short-term facilitation (Jin et al., 2011) in Compte, A., Brunel, N., Goldman-Rakic, P.S., Wang, X.-J., 2000. Synaptic mechanisms
that it includes a postsynaptic component (Huang et al., 1992). This and network dynamics underlying spatial working memory in a cortical network
model. Cereb. Cortex 10, 910–923.
kind of rapid-onset, postsynaptic-dependent plasticity is sufficient for
Cowan, N., 2010. The magical mystery four how is working memory capacity limited, and
our WM model to operate (Fig. S11), irrespective of whether it decays why? Curr. Dir. Psychol. Sci. 19, 51–57.
over minutes or not. Fewer studies have quantified postsynaptic po- Cowan, N., 2011. The focus of attention as observed in visual working memory tasks:
tentiation in PFC, but those that have show similar effects: a single making sense of competing claims. Neuropsychologia 49, 1401–1406.
Curtis, C.E., D’Esposito, M., 2003. Persistent activity in the prefrontal cortex during
500 ms train produces 130% facilitation lasting 20 min, which is NMDA working memory. Trends Cogn. Sci. 7, 415–423.
dependent and modulated by dopamine (Huang et al., 2004). D’Esposito, M., Postle, B.R., 1999. The dependence of span and delayed-response per-
In summary, a single architecture captures both persistent activity formance on prefrontal cortex. Neuropsychologia 37, 1303–1315.
Dittman, J.S., Kreitzer, A.C., Regehr, W.G., 2000. Interplay between facilitation, de-
attractors and silent synaptic memory. We introduce a new scheme of pression, and residual calcium at three presynaptic terminals. J. Neurosci. 20,
transient flexible neuronal coding, that can support many empirical 1374–1385.
phenomena (Tables S1/2) including the “focus of attention”, and gen- Duncan, J., Schramm, M., Thompson, R., Dumontheil, I., 2012. Task rules, working
memory, and fluid intelligence. Psychon. Bull. Rev. 19, 864–870.
erates numerous testable neural predictions. Edeline, J.-M., Pham, P., Weinberger, N.M., 1993. Rapid development of learning-in-
duced receptive field plasticity in the auditory cortex. Behav. Neurosci. 107, 539.
Acknowledgments Eriksson, J., Vogel, E.K., Lansner, A., Bergström, F., Nyberg, L., 2015. Neurocognitive
architecture of working memory. Neuron 88, 33–46.
Farrell, S., Lewandowsky, S., 2002. An endogenous distributed model of ordering in serial
Research was funded by an MRC Clinician Scientist Fellowship to recall. Psychon. Bull. Rev. 9, 59–79.
SGM, a Wellcome Trust Principal Fellowship to MH and NIHR Oxford Farrell, S., Oberauer, K., Greaves, M., Pasiecznik, K., Lewandowsky, S., Jarrold, C., 2016.
A test of interference versus decay in working memory: varying distraction within
Biomedical Research Centre. TPV is funded by a Sir Henry Dale
lists in a complex span task. J. Mem. Lang. 90, 66–87.
Fellowship from the Wellcome Trust. SGM conceived the model, ran Fiebig, F., Lansner, A., 2014. Memory consolidation from seconds to weeks: a three-stage
simulations, and wrote the manuscript. TV advised on modelling, NZ, neural network model with autonomous reinstatement dynamics. Front. Comput.
SJF and MH co-wrote the manuscript. We thank Prof. Mark Stokes and Neurosci. 8, 64.
Fiebig, F., Lansner, A., 2017. A spiking working memory model based on Hebbian short-
Dr. Nicholas Myers for comments on a draft, Dr Eva Feredoes and Dr term potentiation. J. Neurosci. 37, 83–96.
Yoni Pertzov whose discussions inspired some of the concepts, and the Fischer, M., Kaech, S., Knutti, D., Matus, A., 1998. Rapid actin-based plasticity in den-
detailed feedback of several reviewers and editors. dritic spines. Neuron 20, 847–854.
Fougnie, D., Alvarez, G.A., 2011. Object features fail independently in visual working
memory: evidence for a probabilistic feature-store model. J. Vis. 11.
Appendix A. Supplementary data Freedman, D.J., Riesenhuber, M., Poggio, T., Miller, E.K., 2001. Categorical representa-
tion of visual stimuli in the primate prefrontal cortex. Science 291, 312–316.
Frey, U., Morris, R.G.M., 1998. Synaptic tagging: implications for late maintenance of
Supplementary material related to this article can be found, in the hippocampal long-term potentiation. Trends Neurosci. 21, 181–188.
online version, at doi:https://doi.org/10.1016/j.neubiorev.2019.03. Funahashi, S., 2015. Functions of delay-period activity in the prefrontal cortex and
017. mnemonic scotomas revisited. Front. Syst. Neurosci. 9, 2.
Funahashi, S., 2017. Working memory in the prefrontal cortex. Brain Sci. 7.
Funahashi, S., Bruce, C.J., Goldman-Rakic, P.S., 1989. Mnemonic coding of visual space
References in the monkey’s dorsolateral prefrontal cortex. J. Neurophysiol. 61, 331–349.
Fuster, J.M., Alexander, G.E., 1971. Neuron activity related to short-term memory.
Science 173, 652–654.
Allen, R.J., Baddeley, A.D., Hitch, G.J., 2006. Is the binding of visual features in working
Gayet, S., Paffen, C.L.E., Van der Stigchel, S., 2017. Visual working memory storage re-
memory resource-demanding? J. Exp. Psychol. Gen. 135, 298.
cruits sensory processing areas. Trends Cogn. Sci. 22, 189–190.
Almeida, R., Barbosa, J., Compte, A., 2015. Neural circuit basis of visuo-spatial working
Gorgoraptis, N., Catalao, R.F.G., Bays, P.M., Husain, M., 2011. Dynamic updating of
memory precision: a computational and behavioral study. J. Neurophysiol. 114,
working memory resources for visual objects. J. Neurosci. 31, 8502–8511.
1806–1818.
Greene, C.M., Kennedy, K., Soto, D., 2015. Dynamic states in working memory modulate
Badre, D., 2012. Opening the gate to working memory. Proc. Natl. Acad. Sci. U. S. A. 109,
guidance of visual attention: evidence from an n-back paradigm. Vis. cogn. 23,
19878–19879.
546–560.
Bays, P.M., Husain, M., 2008. Dynamic shifts of limited working memory resources in
Hansel, D., Mato, G., 2013. Short-term plasticity explains irregular persistent activity in
human vision. Science 321, 851–854.
working memory tasks. J. Neurosci. Off. J. Soc. Neurosci. 33, 133–149.
Bays, P.M., Wu, E.Y., Husain, M., 2011a. Storage and binding of object features in visual
Hardman, K., Cowan, N., 2015. Remembering complex objects in visual working memory:
working memory. Neuropsychologia 49, 1622–1631.
do capacity limits restrict objects or features? J. Exp. Psychol. Learn. Mem. Cogn. 41,
Bays, P.M., Gorgoraptis, N., Wee, N., Marshall, L., Husain, M., 2011b. Temporal dynamics
325–347.
of encoding, storage, and reallocation of visual working memory. J. Vis. 11, 6.
Hayden, B.Y., Gallant, J.L., 2013. Working memory and decision processes in visual area
Bechara, A., Damasio, H., Tranel, D., Anderson, S.W., 1998. Dissociation of working
V4. Front. Neurosci. 7.
memory from decision making within the human prefrontal cortex. J. Neurosci. 18,
Howard, M.W., Kahana, M.J., 2002. A distributed representation of temporal context. J.
428–437.
Math. Psychol. 46, 269–299.
Bowman, H., Wyble, B., 2007. The simultaneous type, serial token model of temporal
Huang, Y.Y., Colino, A., Selig, D.K., Malenka, R.C., 1992. The influence of prior synaptic
attention and working memory. Psychol. Rev. 114, 38.
activity on the induction of long-term potentiation. Science 255, 730–733.
Burgess, N., Hitch, G., 2005. Computational models of working memory: putting long-
Huang, Y.-Y., Simpson, E., Kellendonk, C., Kandel, E.R., 2004. Genetic evidence for the
term memory into context. Trends Cogn. Sci. 9, 535–541.
bidirectional modulation of synaptic plasticity in the prefrontal cortex by D1 re-
Chatham, C.H., Frank, M.J., Badre, D., 2014. Corticostriatal output gating during selec-
ceptors. Proc. Natl. Acad. Sci. U. S. A. 101, 3236–3241.
tion from working memory. Neuron 81, 930–942.
Itti, L., Koch, C., 2001. Computational modeling of visual attention. Nat. Rev. Neurosci. 2,
Chong, T.T.-J., Husain, M., Rosenthal, C.R., 2014. Recognizing the unconscious. Curr.
194–203.
Biol. 24, R1033–R1035.
Jensen, O., Idiart, M.A., Lisman, J.E., 1996. Physiologically realistic formation of auto-
Christophel, T.B., Hebart, M.N., Haynes, J.-D., 2012. Decoding the contents of visual
associative memory in networks with theta/gamma oscillations: role of fast NMDA
short-term memory from human visual and parietal cortex. J. Neurosci. 32,
channels. Learn. Mem. 3, 243–256.
12983–12989.
Jin, I., Kandel, E.R., Hawkins, R.D., 2011. Whereas short-term facilitation is presynaptic,
Christophel, T.B., Klink, P.C., Spitzer, B., Roelfsema, P.R., Haynes, J.-D., 2017. The dis-
intermediate-term facilitation involves both presynaptic and postsynaptic protein
tributed nature of working memory. Trends Cogn. Sci. 21, 111–124.
kinases and protein synthesis. Learn. Mem. 18, 96–102.

11
S.G. Manohar, et al. Neuroscience and Biobehavioral Reviews 101 (2019) 1–12

Kamiński, J., Sullivan, S., Chung, J.M., Ross, I.B., Mamelak, A.N., Rutishauser, U., 2017. 585–590.
Persistently active neurons in human medial frontal and medial temporal lobe sup- Rizzuto, D.S., Kahana, M.J., 2001. An autoassociative neural network model of paired-
port working memory. Nat. Neurosci. 20, 590–601. associate learning. Neural Comput. 13, 2075–2092.
Konecky, R.O., Smith, M.A., Olson, C.R., 2017. Monkey prefrontal neurons during Romanski, L.M., 2004. Domain specificity in the primate prefrontal cortex. Cogn. Affect.
sternberg task performance: full contents of working memory or most recent item? J. Behav. Neurosci. 4, 421–429.
Neurophysiol jn.00541.2016. Rose, N.S., LaRocque, J.J., Riggall, A.C., Gosseries, O., Starrett, M.J., Meyering, E.E.,
Kornblith, S., Tsao, D.Y., 2017. How thoughts arise from sights: inferotemporal and Postle, B.R., 2016. Reactivation of latent working memories with transcranial mag-
prefrontal contributions to vision. Curr. Opin. Neurobiol. 46, 208–218. netic stimulation. Science 354, 1136–1139.
Lara, A.H., Wallis, J.D., 2014. Executive control processes underlying multi-item working Rowe, J.B., Toni, I., Josephs, O., Frackowiak, R.S.J., Passingham, R.E., 2000. The pre-
memory. Nat. Neurosci. 17, 876–883. frontal cortex: response selection or maintenance within working memory? Science
LaRocque, J.J., Eichenbaum, A.S., Starrett, M.J., Rose, N.S., Emrich, S.M., Postle, B.R., 288, 1656–1660.
2014. The short- and long-term fates of memory items retained outside the focus of Sakai, K., 2008. Task set and prefrontal cortex. Annu. Rev. Neurosci. 31, 219–245.
attention. Mem. Cognit. 43, 453–468. Sandberg, A., Tegnér, J., Lansner, A., 2003. A working memory model based on fast
Lee, S.-H., Baker, C.I., 2016. Multi-voxel decoding and the topography of maintained Hebbian learning. Netw. Bristol Engl. 14, 789–802.
information during visual working memory. Front. Syst. Neurosci. 2. Schneegans, S., Bays, P.M., 2017a. Restoration of fMRI decodability does not imply latent
Lewis-Peacock, J.A., Drysdale, A.T., Oberauer, K., Postle, B.R., 2012. Neural evidence for working memory states. J. Cogn. Neurosci. 29, 1977–1994.
a distinction between short-term memory and the focus of attention. J. Cogn. Schneegans, S., Bays, P.M., 2017b. Neural architecture for feature binding in visual
Neurosci. 24, 61–79. working memory. J. Neurosci. 3413–3925.
Litwin-Kumar, A., Doiron, B., 2014. Formation and maintenance of neuronal assemblies Schvaneveldt, R.W., Chase, W.G., 1969. Sequential effects in choice reaction time. J. Exp.
through synaptic plasticity. Nat. Commun. 5 ncomms6319. Psychol. 80, 1.
Luck, S.J., Vogel, E.K., 1997. The capacity of visual working memory for features and Silvanto, J., 2017. Working memory maintenance: sustained firing or synaptic mechan-
conjunctions. Nature 390, 279–280. isms? Trends Cogn. Sci. 21, 152–154.
Luria, R., Vogel, E.K., 2011. Shape and color conjunction stimuli are represented as bound Smyth, M.M., 1996. Serial order in spatial immediate memory. Q. J. Exp. Psychol. Sect. A
objects in visual working memory. Neuropsychologia 49, 1632–1639. 49, 159–177.
Makovski, T., Sussman, R., Jiang, Y.V., 2008. Orienting attention in visual working Solway, A., Murdock, B.B., Kahana, M.J., 2012. Positional and temporal clustering in
memory reduces interference from memory probes. J. Exp. Psychol. Learn. Mem. serial order memory. Mem. Cognit. 40, 177–190.
Cogn. 34, 369–380. Soto, D., Mäntylä, T., Silvanto, J., 2011. Working memory without consciousness. Curr.
Malenka, R.C., 1991. Postsynaptic factors control the duration of synaptic enhancement Biol. 21, R912–R913.
in area CA1 of the hippocampus. Neuron 6, 53–60. Souza, A.S., Oberauer, K., 2016. In search of the focus of attention in working memory: 13
Malenka, R.C., Nicoll, R.A., 1993. NMDA-receptor-dependent synaptic plasticity: multiple years of the retro-cue effect. Atten. Percept. Psychophys. 1–22.
forms and mechanisms. Trends Neurosci. 16, 521–527. Souza, A.S., Rerko, L., Oberauer, K., 2016. Getting more from visual working memory:
Malsburg, Cvonder, 1981. The correlation theory of brain function. In: Domany, P.E., van retro-cues enhance retrieval and protect from visual interference. J. Exp. Psychol.
Hemmen, P.D.J.L., Schulten, P.K. (Eds.), Models of Neural Networks. Springer, New Hum. Percept. Perform.
York), pp. 95–119. Sprague, T.C., Ester, E.F., Serences, J.T., 2016. Restoring latent visual working memory
Matthey, L., Bays, P.M., Dayan, P., 2015. A probabilistic palimpsest model of visual short- representations in human cortex. Neuron 91, 694–707.
term memory. PLoS Comput. Biol. 11, e1004003. Sreenivasan, K.K., Curtis, C.E., D’Esposito, M., 2014. Revisiting the role of persistent
Mi, Y., Katkov, M., Tsodyks, M., 2017. Synaptic correlates of working memory capacity. neural activity during working memory. Trends Cogn. Sci. 18, 82–89.
Neuron 93, 323–330. Stokes, M.G., 2015. ‘Activity-silent’ working memory in prefrontal cortex: a dynamic
Miller, E.K., Erickson, C.A., Desimone, R., 1996. Neural mechanisms of visual working coding framework. Trends Cogn. Sci. 19, 394–405.
memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167. Sutterer, D.W., Foster, J.J., Adam, K.C.S., Vogel, E.K., Awh, E., 2018. Item-specific delay
Mongillo, G., Barak, O., Tsodyks, M., 2008. Synaptic theory of working memory. Science activity demonstrates concurrent storage of multiple items in working memory.
319, 1543–1546. BioRxiv, 382879.
Morey, C.C., Cowan, N., Morey, R.D., Rouder, J.N., 2011. Flexible attention allocation to Swan, G., Wyble, B., 2014. The binding pool: a model of shared neural resources for
visual and auditory working memory tasks: manipulating reward induces a trade-off. distinct items in visual working memory. Atten. Percept. Psychophys. 76, 2136–2157.
Atten. Percept. Psychophys. 73, 458–472. Tabi, Y.A., Husain, M., Manohar, S.G., 2019. Recall cues interfere with retrieval from
Norris, D., 2017. Short-term memory and long-term memory are still different. Psychol. visuospatial working memory. Brit. J. Psychol. https://doi.org/10.1111/bjop.12374.
Bull No Pagination Specified. [Epub ahead of print] (Jan 2019).
Oberauer, K., 2002. Access to information in working memory: exploring the focus of Thompson-Schill, S.L., Jonides, J., Marshuetz, C., Smith, E.E., D’Esposito, M., Kan, I.P.,
attention. J. Exp. Psychol. Learn. Mem. Cogn. 28, 411–421. Knight, R.T., Swick, D., 2002. Effects of frontal lobe damage on interference effects in
Oberauer, K., Eichenberger, S., 2013. Visual working memory declines when more fea- working memory. Cogn. Affect. Behav. Neurosci. 2, 109–120.
tures must be remembered for each object. Mem. Cognit. 41, 1212–1227. Trübutschek, D., Marti, S., Ojeda, A., King, J.-R., Mi, Y., Tsodyks, M., Dehaene, S., 2017. A
Oberauer, K., Lewandowsky, S., 2014. Further evidence against decay in working theory of working memory without consciousness or sustained activity. eLife 6,
memory. J. Mem. Lang. 73, 15–30. e23871.
Oberauer, K., Lin, H.-Y., 2017. An interference model of visual working memory. Psychol. Tsodyks, M.V., Markram, H., 1997. The neural code between neocortical pyramidal
Rev. 124, 21. neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. U. S.
Olivers, C.N.L., Peters, J., Houtkamp, R., Roelfsema, P.R., 2011. Different states in visual A. 94, 719–723.
working memory: when it guides attention and when it does not. Trends Cogn. Sci. van Loon, A.M., Olmos-Solis, K., Fahrenfort, J.J., Olivers, C.N.L., 2018. Current and future
15, 327–334. goals are represented in opposite patterns in object-selective cortex. eLife 7.
Parthasarathy, A., Herikstad, R., Bong, J.H., Medina, F.S., Libedinsky, C., Yen, S.-C., 2017. Wang, B., Cao, X., Theeuwes, J., Olivers, C.N.L., Wang, Z., 2016. Location-based effects
Mixed selectivity morphs population codes in prefrontal cortex. Nat. Neurosci. underlie feature conjunction benefits in visual working memory. J. Vis. 16, 12.
Pereira, J., Wang, X.-J., 2015. A tradeoff between accuracy and flexibility in a working Wang, B., Cao, X., Theeuwes, J., Olivers, C.N., Wang, Z., 2017. Separate capacities for
memory circuit endowed with slow feedback mechanisms. Cereb. Cortex N. Y. 1991 storing different features in visual working memory. J. Exp. Psychol. Learn. Mem.
25, 3586–3601. Cogn. 43, 226.
Pertzov, Y., Husain, M., 2014. The privileged role of location in visual working memory. Wang, B., Theeuwes, J., Olivers, C.N.L., 2018. When shorter delays lead to worse mem-
Atten. Percept. Psychophys. 76, 1914–1924. ories: task disruption makes visual working memory temporarily vulnerable to test
Pertzov, Y., Manohar, S., Husain, M., 2016. Rapid forgetting results from competition interference. J. Exp. Psychol. Learn. Mem. Cogn. 44, 722–733.
over time between items in visual working memory. J. Exp. Psychol. Learn. Mem. Wimmer, K., Nykamp, D.Q., Constantinidis, C., Compte, A., 2014. Bump attractor dy-
Cogn. namics in prefrontal cortex explains behavioral precision in spatial working memory.
Postle, B.R., 2016. How does the brain keep information “in mind”? Curr. Dir. Psychol. Nat. Neurosci. 17, 431–439.
Sci. 25, 151–156. Wolff, M.J., Jochim, J., Akyürek, E.G., Stokes, M.G., 2017. Dynamic hidden states un-
Postle, B.R., Ferrarelli, F., Hamidi, M., Feredoes, E., Massimini, M., Peterson, M., derlying working-memory-guided behavior. Nat. Neurosci. 20, 864–871.
Alexander, A., Tononi, G., 2006. Repetitive transcranial magnetic stimulation dis- Xu, Y., 2017. Reevaluating the sensory account of visual working memory storage. Trends
sociates working memory manipulation from retention functions in the prefrontal, Cogn. Sci. 21, 794–815.
but not posterior parietal. Cortex. J. Cogn. Neurosci. 18, 1712–1722. Yu, Q., Postle, B.R., 2018. Different states of priority recruit different neural codes in
Proctor, R.W., Schneider, D.W., 2017. Hick’s law for choice reaction time: a review. Q. J. visual working memory. BioRxiv, 334920.
Exp. Psychol. 1–56. Zenke, F., Agnes, E.J., Gerstner, W., 2015. Diverse synaptic plasticity mechanisms
Rademaker, R.L., Chunharas, C., Serences, J.T., 2018. Simultaneous representation of orchestrated to form and retrieve memories in spiking neural networks. Nat.
sensory and mnemonic information in human visual cortex. BioRxiv, 339200. Commun. 6, 6922.
Rainer, G., Asaad, W.F., Miller, E.K., 1998. Memory fields of neurons in the primate Zipser, D., Kehoe, B., Littlewort, G., Fuster, J., 1993. A spiking network model of short-
prefrontal cortex. Proc. Natl. Acad. Sci. U. S. A. 95, 15008–15013. term active memory. J. Neurosci. Off. J. Soc. Neurosci. 13, 3406–3420.
Reverberi, C., Görgen, K., Haynes, J.-D., 2012. Compositionality of rule representations in Zokaei, N., Manohar, S., Husain, M., Feredoes, E., 2014a. Causal evidence for a privileged
human prefrontal cortex. Cereb. Cortex 22, 1237–1246. working memory state in early visual cortex. J. Neurosci. 34, 158–162.
Reynolds, J.H., Desimone, R., 1999. The role of neural mechanisms of attention in solving Zokaei, N., Ning, S., Manohar, S., Feredoes, E., Husain, M., 2014b. Flexibility of re-
the binding problem. Neuron 24, 19–29. presentational states in working memory. Front. Hum. Neurosci. 8, 853.
Rigotti, M., Barak, O., Warden, M.R., Wang, X.-J., Daw, N.D., Miller, E.K., Fusi, S., 2013. Zucker, R.S., 1989. Short-term synaptic plasticity. Annu. Rev. Neurosci. 12, 13–31.
The importance of mixed selectivity in complex cognitive tasks. Nature 497,

12