THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3111-169

Printed in the United States ol America

Computation and cognition:

issues in the foundations
of cognitive science
Zenon W. Pylyshyn
'Center for Advanced Study in the Behavioral Sciences. Stanford, Calif.
94305

Abstract: The computational view of mind rests on certain intuitions regarding the fundamental similarity between computation and cognition.
We examine some of these intuitions and suggest that they derive from the fact that computers and human organisms are both physical systems
whose behavior is correctly described as being governed by rules acting on symbolic representations. Some of the implications of this view are
discussed. It is suggested that a fundamental hypothesis of this approach (the "proprietary vocabulary hypothesis") is that there is a natural
domain of human functioning (roughly what we intuitively associate with perceiving, reasoning, and acting) that can be addressed exclusively
in terms of a formal symbolic or algorithmic vocabulary or level of analysis.
Much of the paper elaborates various conditions that need to be met if a literal view of mental activity as computation is to serve as the basis
for explanatory theories. The coherence of such a view depends on there being a principled distinction between functions whose explanation
requires that we posit internal representations and those that we can appropriately describe as merely instantiating causal physical or biological
laws. In this paper the distinction is empirically grounded in a methodological criterion called the "cognitive impenetrability condition."
Functions are said to be cognitively impenetrable if they cannot be influenced by such purely cognitive factors as goals, beliefs, inferences, tacit
knowledge, and so on. Such a criterion makes it possible to empirically separate the fixed capacities of mind (called its "functional
architecture") from the particular representations and algorithms used on specific occasions. In order for computational theories to avoid being
ad hoc, they must deal effectively with the "degrees of freedom" problem by constraining the extent to which they can be arbitrarily adjusted
post hoc to fit some particular set of observations. This in turn requires that the fixed architectural function and the algorithms be
independently validated. It is argued that the architectural assumptions implicit in many contemporary models run afoul of the cognitive
impenetrability condition, since the required fixed functions are demonstrably sensitive to tacit knowledge and goals. The paper concludes with
some tactical suggestions for the development of computational cognitive theories.

Keywords: cognitive science; artificial intelligence; computational models; computer simulation; cognition; mental representation; mental
process; imagery; philosophical foundations; functionalism; philosophy of mind

1. Introduction and summary precisely what constitutes the core of the approach and what
constraints it imposes on theory construction.
The view that cognition can be understood as computation is In this essay I shall present what I consider some of the
ubiquitous in modern cognitive theorizing, even among those crucial characteristics of the computational view of mind and
who do not use computer programs to express models of defend them as appropriate for the task of explaining
cognitive processes. One of the basic assumptions behind this cognition. As in the early stages of many scientific endeavors,
approach, sometimes referred to as "information processing the core of the approach is implicit in scientists' intuitions
psychology," is that cognitive processes can be understood in about what are to count as relevant phenomena and as
terms of formal operations carried out on symbol structures. It legitimate explanations of the underlying processes. Yet as we
thus represents a formalist approach to theoretical explana- tease out the central assumptions, we will find room for
tion. In practice, tokens of symbol structures may be depicted refinement: not everything that is intuitively cognitive will
as expressions written in some lexicographic notation (as is remain so as the theory develops, nor will all processes turn
usual in linguistics or mathematics), or they may be physi- out to be appropriate for explaining cognitive phenomena.
cally instantiated in a computer as a data structure or an We begin with an informal discussion of the position that
executable program. certain types of human behavior are determined by repre-
The "information processing" idiom has been with us for sentations (beliefs, tacit knowledge, goals, and so on). This,
about two decades and represents a substantial intellectual we suggest, is precisely what recommends the view that
commitment among students of cognition. The fields that mental activity is computational. Then we present one of the
share this view (notably, segments of linguistics, philosophy of main empirical claims of the approach - namely, that there is
mind, psychology, artificial intelligence, cultural anthropolo- a natural domain of inquiry that can be addressed at a
gy, and others) have been increasingly looking toward some privileged algorithmic level of analysis, or with a proprietary
convergence as the "cognitive sciences." Several journals vocabulary.
devoted to that topic now exist (including, to some extent, The remainder of the paper elaborates various require-
BBS), and a Cognitive Science Society has just been formed. ments for constructing adequate explanatory theories on this
There remains, however, considerable uncertainty regarding basis. First, however, we need to analyse the notions of

91980 Cambridge University Press 0140-525X/80/010111-59S4.00/0 111

Pylyshyn: Computation and cognition
"cognitive phenomenon" and "cognitive process" in the light "metameric" equivalence classes of colors, rather than over
of the evolving understanding of cognition as computation. physically specifiable properties of light, since they hold
Hence we take a detour in our discussion (sections 5 and 6) to regardless of how the particular color is produced by the
examine the fundamental distinction between 1) behavior environment - e.g., whether, say, the perceived yellow is
governed by rules and representations, and 2) behavior that is produced by radiation of (roughly) 580 nm, a mixture of
merely the result of the causal structure of the underlying approximately equal energy of radiations of 530 nm and 650
biological system. A primary goal of the paper is to defend nm, by a mixture of any complex radiations that metamer-
this as a principled distinction, and to propose a necessary ically match the latter two wavelengths, or even by direct
(though not sufficient) empirical criterion for it. This meth- electrical or chemical stimulation of the visual system. We
odological criterion (called the cognitive impenetrability might never be able to specify all the possible physical stimuli
condition) supports the maxim that to be explanatory, a that produce a particular perceived color, and yet the laws of
theoretical account should make a principled appeal to a color mixture shall hold if stated over perceived color.
computational model. Exhibiting an algorithm and a repre- Hochberg (1968) presents a variety of such examples, showing
sentative sample of its behavior is not enough. One must, in that the regularities of perception must be stated over
addition, separate and independently justify two sources of its perceived properties - i.e., over internal representations.
performance: its fixed (cognitively impenetrable) functional Similarly, when a particular event can be given more than
capacities (or its "functional architecture"), and the "effec- one interpretation, then what determines behavioral regular-
tiveness principles" underlying its capacity to perform as ities is not its physical properties, or even some abstract
claimed. function thereof, but rather each agent's particular interpre-
tation. The classical illustrations are ambiguous stimuli, such
as the Necker cube, the duck-rabbit or the profiles-vase
2. The cognitive vocabulary and folk psychology illusions, or ambiguous sentences. Clearly, what people do
(e.g., when asked what they see or hear) depends upon which
Most people implicitly hold a sophisticated and highly reading of the ambiguity they take. But all physical events are
successful cognitive theory; that is, they can systematize, intrinsically ambiguous, in the sense that they are subject to
make sense of, and correctly predict an enormous range of various interpretations; so psychological regularities will
human behavior. Although textbook authors are fond of always have to be stated relative to particular readings of
pointing out the errors in folk psychology, it nonetheless far stimuli (i.e., how they are internally represented).
surpasses any current scientific psychology in scope and Finally, if we include goals as well as beliefs among the
general accuracy. What is significant about this is the types of representations, it becomes possible to give an
mentalistic vocabulary, or level of description, of folk account of a wide range of additional regularities. Behavior
psychology, and its corresponding taxonomy of things, behav- that is goal-directed is characterized by such properties as
iors, events and so on. equin'nality (i.e., its termination can only be characterised in
Whatever the shortcomings of folk psychology (and there terms of the terminating state, rather than in terms of the
are plenty, to be sure), the level of abstractness of its concepts path by which the system arrived at that state - see the
(relative to those of physics), and particularly its appeal to the discussion in Newell and Simon 1972).
way situations are represented in the mind (i.e., its appeal to It is no accident that the systematicity of human behavior is
what human agents think, believe, infer, want, and so on, as captured in a vocabulary that refers to internal representa-
opposed to the way they actually are), seems precisely suited tions, for these are the terms in which we conceptualize and
to capturing just the kinds of generalizations that concern plan our actions in the first place. For example, as I write this
cognitive psychology - e.g., the nature of our intellectual paper, I produce certain movements of my fingers and hand.
abilities, the mechanisms underlying intellectual perfor- But I do that under control of certain higher level goals, or, as
mances, and the causal and rational antecedents of our some people prefer to put it, I execute the behaviors under a
actions. certain "intended interpretation." I intend to make certain
It seems overwhelmingly likely that explanations of cogni- statements by my behavior. I do not intend to make marks on
tive phenomena will have to appeal to briefs, intentions, and paper, although clearly I am doing that too. Although this
the like, because it appears that certain regularities in human hierarchical aspect of the behavioral description is part of my
behavior can only be captured in such terms and at that level conceptualization of it, rather than an intrinsic part of the
of abstraction. For example, when a person perceives danger, behavior itself, yet it is critical to how the behavior must be
he will generally set about to remove himself from the source treated theoretically if the theory is to capture the systema-
of that danger. Now, this generalization has an unlimited ticity of my actions. A theory that took my behavior to be an
variety of instances. Thus, generally, if a person knows how to instance of finger-movement could not account for why,
get out of a building, and believes the building to be on fire, when my typewriter broke, I proceeded to make quite
then generally he will set himself the goal of being out of the different movements using pencil and paper. This is an
building, and use his knowledge to determine a series of instance of the "equifinality" property associated with goal-
actions to satisfy this goal. The point is that even so simple a directed behavior.
regularity could not be captured without descriptions that use Of course, to say that folk psychology has nonfortuitously
the italicized mentalistic terms (or very similar ones), because settled on some of the appropriate terms for describing our
there is an infinite variety of specific ways of "knowing how cognitive activity is not to say that it is good scientific theory.
to get out of the building," of coming to "believe that the It may be that this set of terms needs to be augmented or
building is on fire," and of satisfying the goal of being out of pruned, that many of the beliefs expressed in folk psychology
the building. For each combination of these, an entirely are either false or empirically empty, and that many of its
different causal chain would result if the situation were explanations are either incomplete or circular. But its most
described in physical or strictly behavioral terms. Conse- serious shortcoming, from the point of view of the scientific
quently the psychologically relevant generalization would be enterprise, is that the collection of loose generalizations that
lost in the diversity of possible causal connections. This makes up this informal body of knowledge is not tied together
generalization can only be stated in terms of the agent's into an explicit system. The way in which sets of generaliza-
internal representation of the situation (i.e. in mentalistic tions are tied together in developed sciences is through a
terms). For a different example, the laws of color mixture are theory that shows how the generalizations are derivable in
properly stated over perceived color, or what are called some appropriate idealization from a smaller set of deeper

112 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
universal principles (or axioms). The categories of the deeper etc, and five is indeed the sum of two and three). In other
principles are typically quite different from those found in words, from the definition of the symbols (numerals) as
the broader generalization (e.g. pressure and temperature are representations of numbers, and from the definition of the
reduced to aspects of kinetic energy in molecular theory.) "PLUS" as representing a certain abstract mathematical
operation, it follows that some state of the machine after
reading the expression will correspond to a state that
3. Representation and computation represents the value of the function and (because of a further
definition of the implicit printout function) causes the
There are many characteristics that recommend computation printout of the appropriate answer.
as the appropriate form in which to cast models of cognitive This is true of computation generally. We explain why the
processes. I have discussed some of them in Pylyshyn (1978a). machine does something by referring to certain interpreta-
For example, the hierarchical character of programs and the tions of its symbols in some intended domain. This is, of
abstraction represented by the "information processing" level course, precisely what we do in describing how (and why)
of analysis make it the ideal vehicle for expressing functional people do what they do. In explaining why a chess player
models of all kinds. These were the aspects of computation moves some piece onto a certain square, we refer to the type
that led Miller, Galanter, and Pribram (I960) to propose the of piece it is in terms of its role in chess, to the player's
basic iterative loop (TOTE, or Test-Operate-Test-Exit) as the immediate goal, and to the rules of chess. As I suggested
fundamental building block for psychological theory - to earlier, this way of describing the situation is not merely a
replace the reflex arc and even the cybernetic energy way of speaking or an informal shorthand reference to a more
feedback loop. precise functional description. Furthermore, it is not, like the
What I wish to focus on here is what I take to be the most functional description referred to earlier, an abstraction over
fundamental reason why cognition ought to be viewed as a set of physical properties. There is a fundamental difference
computation. That reason rests on the fact that computation is between a description of a computer's operation cast in terms
the only worked-out view of process that is both compatible of its states (i.e., equivalence classes of physical descriptions)
with a materialist view of how a process is realized and that and one cast in terms of what it is about, such as in the
attributes the behavior of the process to the operation of rules illustrative example above. The fundamental difference is
upon representations. In other words, what makes it possible that the former refers to intrinsic properties of the device,
to view computation and cognition as processes of fundamen- while the latter refers to aspects of some entirely different
tally the same type is the fact that both are physically realized domain, such as chess. The former can be viewed as a
and both are governed by rules and representations [see syntactic description, while the latter is semantic, since it
Chomsky, this issue]. Furthermore, they both exhibit the same refers to the represented domain.
sort of dual character with respect to providing explanations This dual nature of mental functioning (referred to
of how they work - and for the same reason. traditionally as the functional or causal, and the intentional)
As a physical device, the operation of a computer can be has been a source of profound philosophical puzzlement for a
described in terms of the causal structure of its physical long time (e.g. Putnam 1978). The puzzle arises because,
properties. The states of a computer, viewed as a physical while we believe that people do things because of their goals
device, are individuated in terms of the identity of physical and beliefs, we nonetheless also assume, for the sake of unity
descriptions, and its state transitions are therefore connected of science and to avoid the extravagance of dualism, that this
by physical laws. By abstracting over these physical proper- process is actually carried out by causal sequences of events
ties, it is possible to give a functional description of the that can respond only to the intrinsic physical properties of
device. This is a description of the systematic relations that the brain. But how can the process depend both on properties
hold over certain (typically very complex) classes of physical of brain tissue and on some other quite different domain, such
properties - such as the ones that correspond to computa- as chess or mathematics? The parallel question can of course
tionally relevant states of the device. While the transitions equally be asked of computers: How can the state transitions
over states defined in this way are no longer instances of in our example depend both on physical laws and on the
physical laws (i.e., there is no law relating state n and state m abstract properties of numbers? The simple answer is that this
of an IBM machine, even though it is wired up so that state m happens because both numbers and rules relating numbers
always follows state n), they are nonetheless reducible to some are represented in the machine as symbolic expressions and
complex function of various physical laws and of the physical programs, and that it is the physical realization of these
properties of states n and m. Such a functional description of representations that determines the machine's behavior. More
the device might, for example, be summarized as a finite state precisely, the abstract numbers and rules (e.g. Peano's
transition diagram of the sort familiar in automata theory. axioms) are first expressed in terms of syntactic operations
We shall see below, however, that this is not an adequate over symbolic expressions or some notation for the number
functional description from the point of view of understand- system, and then these expressions are "interpreted" by the
ing the device as a computer. built-in functional properties of the physical device. Of
On the other hand, if we wish to explain the computation course, the machine does not interpret the symbols as
that the device is carrying out, or the regularities exhibited by numbers, but only as formal patterns that cause the machine
some particular programmed computer, we must refer to to function in some particular way.
objects in a domain that is the intended interpretation or the Because a computational process has no access to the actual
subject matter of the computations, such as, for example, the represented domain itself (e.g., a computer has no way of
abstract domain of numbers. Thus, in order to explain why distinguishing whether a symbol represents a number or letter
the machine prints out the symbol " 5 " when it is provided or someone's name), it is mandatory, if the rules are to
with the expression "(PLUS 2 3)," we must refer to the continue to be semantically interpretable (say as rules of
meaning of the symbols in the expression and in the printout. arithmetic), that all relevant semantic distinctions be
These meanings are the referents of the symbols in the mirrored by syntactic distinctions - i.e., by features intrinsic
domain of numbers. The explanation of why the particular to the representation itself. Such features must in turn be
symbol "5" is printed out then follows from these semantic reflected in functional differences in the operation of the
definitions (i.e., it prints out "5" because that symbol device. That is what we mean when we say that a device
represents the number five, "PLUS" represents the addition represents something. Simply put, all and only syntactically
operator applied to the referents of the other two symbols, encoded aspects of the represented domain can affect the way

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 113

Pylyshyn: Computation and cognition
a process behaves. This rather obvious assertion is the representational states may be followed by the same subse-
cornerstone of the formalist approach to understanding the quent state, or that rules tend to be invoked in certain
notion of process. Haugeland (1978) has made the same point, systematic ways in relation to one another. Indeed, one could
though in a slightly different way. It is also implicit in not represent individual rules in the state transition notation,
Newell's (1979) "physical symbol system" hypothesis. Many since individual rules affect a large (and in principle
of the consequences of this characteristic of computation and unbounded) set of different states. For example, the rule that
of this way of looking at cognition are far-reaching, however, specifies that one remove oneself from danger must be
and not widely acknowledged [for a discussion, see Fodor, this potentially evokable by every belief state a part of whose
issue]. representational content corresponds to danger. By repre-
By separating the semantic and syntactic aspects of senting this regularity once for each such state, one misses the
cognition, we reduce the problem of accounting for mean- generalization corresponding to that one rule. In addition, of
ingful action to the problem of specifying a mechanism that course, the fact that there are an unbounded number of
operates upon meaningless symbol tokens and in doing so possible thoughts and representational states makes it-
carries out the meaningful process being modelled (e.g. mandatory that the symbolic encoding of these thoughts or
arithmetic). This, in turn, represents an important break- states be combinatoric - i.e., that they have a recursive
through because, while one can see how the formal process syntactic structure. I mention all this only because there have
can be realized by causally connected sequences of events (as been some who have proposed that we not view the content of
in a computer), no one has the slightest notion of how states in terms of some articulation of what they represent
carrying out semantically interpreted rules could even be (e.g. Davidson 1970) - i.e., that we avoid postulating an
viewed as compatible with natural law [c.f. Fodor, this issue]. internal syntax for representations, or a "mentalese."
That is, as far as we can see there could be no natural law The syntactic, representation-governed nature of compu-
which says, for example, that when (and only when) a device tation thus lends itself to describing cognitive processes in
is in a state that represents the number five, and applies the such a way that their relation to causal laws is bridgeable, at
operation that represents the successor function, it will go into least in principle. But beyond that, the exact nature of the
the state that represents the number six. Whatever the device that instantiates the process is no more a direct concern
functional states that are being invoked, there is nothing to to the task of discovering and explaining cognitive regularities
prevent anyone from consistently and correctly interpreting than it is in computation - though in both cases specifying the
the same states and the same function as representing fixed functional architecture of the underlying system is an
something quite different - say, producing the name of the essential component of understanding the process itself.
next person on a list, or the location of an adjacent cell in a Given that computation and cognition can be viewed in these
matrix, or any other consistent interpretation. Indeed, the common abstract terms, there is no reason why computation
very same physical state recurs in computers under circum- ought to be treated as merely a metaphor for cognition, as
stances in which very different processes, operating in quite opposed to a hypothesis about the literal nature of cognition.
different domains of interpretation, are being executed. In In spite of the widespread use of computational terminology
other words, the machine's functioning is completely inde- (e.g., terms like "storage," "process," "operation"), much of
pendent of how its states are interpreted (though it is far from this usage has had at least some metaphorical content. There
clear whether this would still remain so if it were wired up has been a reluctance to take computation as a literal
through transducers to a natural environment). For that description of mental activity, as opposed to being a mere
reason we can never specify the behavior of a computer heuristic metaphor. In my view this failure to take compu-
uniquivocally in terms of a semantically interpreted rule such tation literally has licensed a wide range of activity under the
as the one cited above (which referred to the domain of rubric of "information processing theory," some of it repre-
numbers). This is what people like Fodor (1978), Searle senting a significant departure from what I see as the core
(1979), or Dreyfus (1979) mean when they say that a ideas of a computational theory of mind.
computer does not know what it is doing. Taking a certain characterization literally carries with it
The formalist view requires that we take the syntactic far-reaching consequences. The history of science contains
properties of representations quite literally. It is literally true numerous examples of the qualitative changes that can come
of a computer that it contains, in some functionally discern- about when a community accepts a certain characterization
able form (which could even conceivably be a typewritten as applying literally to phenomena. The outstanding example
form, if someone wanted to go through the trouble of of this is the case of geometry. Our current scientific
arranging the hardware that way), what could be referred to conception of physical space is a projection of Euclidean
as a code or an inscription of a symbolic expression, whose geometry onto the observations of mechanics. But plane
formal features mirror (in the sense of bearing a one-to-one geometry was well known and widely used by the Egyptians
correspondence with) semantic characteristics of some repre- in surveying. Later it was developed into an exquisitely
sented domain, and which causes the machine to behave in a elegant system by the Greeks. Yet for the Egyptians it was a
certain way. Because of the requirement that the syntactic way of calculating - like an abacus - while for the Greeks it
structure of representations reflect all relevant semantic was a demonstration of the perfect Platonic order. It was not
distinctions, the state transition diagram description of an until two millenia later that Galileo began the conceptual
automaton is an inadequate means of expressing the func- transformation that eventually resulted in the view, that is so
tional properties of a computational system. Individual states commonplace today that virtually no vestige remains of the
must be shown as factored into component parts rather than Aristotelian ideas of natural motions and natural places.
as being distinct atomic or holistic entities. Some functionally Everyone imagines space to be that empty, infinitely
distinguishable aspects of the states much correspond to extended, isotropic, three-dimensional receptacle, whose exis-
individual terms of the representing symbol structure, while tence and properties are quite independent of the earth or
other aspects must correspond to such additional properties as any other objects. Such a strange idea was literally unthink-
the control state of the device (which determines which able before the seventeenth century. In fact, not even Galileo
operation will be carried out next) and the system's relation to completely accepted it. For him a straight line was still bound
the representation (e.g., whether the representation corre- to the earth's surface. It was not until Newton that the task of
sponds to a belief or a goal). Components of the syntactic "geometrization of the world" was completed (to borrow a
expressions must be functionally factorable, otherwise we phrase from Butterfield 1957).
could not account for such regularities as that several distinct
The transformation that led to the reification of geometry -

114 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
to accepting the formal axioms of Euclid as a literal what the target model would have encoded them as in its
description of physical space - profoundly affected the course own encoding scheme. This step is, furthermore, one that
of science. Accepting a system as a literal account of reality must be carried out "on line" each time an operation is to be
enables scientists to see that certain further observations are mimicked in the mimicking model - it cannot be done once
possible and others are not. It goes beyond merely asserting and for all in advance by the constructor itself, except in the
that certain thinks happen "as if" some unseen events were uninteresting case where there are only a finite number of
taking place. In addition, however, it imposes severe restric- stimuli to be discriminated (not merely a finite number of
tions on a theory-builder, because he is no longer free to codes). But finding out which real stimuli generate a certain
appeal to the existence of unspecified similarities between his code in a model is precisely to determine the semantic
theoretical account and the phenomena he is addressing - as extension of a representation - something that a formal
he is when speaking metaphorically. It is this latter degree of system clearly cannot do.
freedom that weakens the explanatory power of computation Similarly, in describing the principles of operation of a
when it is used metaphorically to describe certain mental model, it is common to characterize them in terms of the
functions. If we view computation more abstractly as a represented domain. For example, when it is claimed that
symbolic process that transforms formal expressions that are "mental rotation" of images proceeds by small angular steps
in turn interpreted in terms of some domain of representation (e.g. Kosslyn and Shwartz 1977; Anderson 1978), or that
(such as the numbers), we see that the view that mental "mental scanning " of images proceeds by passing through
processes are computational can be just as literal as the view adjacent places, one is appealing to properties of the repre-
that what IBM computers do is properly viewed as compu- sented domain. Phrases like "small angular steps" and
tation. "adjacent places" can only refer to properties of the stimulus
Below I shall consider what is entailed by the view that or scene being represented. They are semantically interpreted
mental activity can be viewed literally as the execution of notions. The representation itself does not literally have small
algorithms. In particular I shall suggest that this imposes angles or adjacent places (unless one wishes to claim that it is
certain constraints upon the theory construction enterprise. If laid out spatially in the brain - a logical possibility that most
we are to view cognition as literally computation, it then people shun); it only represents such properties. Of course, it
becomes relevant to inquire how one can go about developing is possible to arrange for a process to transform a represen-
explanatory theories of such cognitive processes in selected tation of a stimulus into a representation of the same stimulus
domains and also to consider the scope and limit of such a in a slightly different orientation, or to focus on a part of the
view. In other words, if computation is to be taken seriously as representation that corresponds to a place in the stimulus that
a literal account of mental activity, it becomes both relevant is adjacent to another specified place, but in neither case is
and important to inquire into the sense in which computa- this choice of transformation principled [see Kosslyn et al.:
tional systems can be used to provide explanations of mental "On the demystification of mental imagery," BBS 2(4) 1979].
processes. Before turning to the general issue of what is In other words, there is nothing about the representation itself
entailed by a literal view of cognition as computation, I shall (e.g. its syntax or format) that requires one class of transfor-
conclude this discussion of the formalist nature of this mation rather than another - as is true in the situation being
approach with one or two examples, suggesting that even represented, where to physically go from one point to
proponents of this view sometimes fail to respect some of its another, one must pass through intermediate adjacent points
fundamental constraints. One still sees the occasional lapse or else violate universal laws of (classical) physics. Thus the
when the psychological significance or the psychological existence of such a constraint on permissible transformations
principles implicated in some particular computer model are in the imaginal case is merely stipulated. The choice of the
being described. Thus, semantic properties are occasionally transformation in that case is like the choice of a value for a
attributed to representations (for an animated discussion, see free empirical parameter - i.e., it is completely ad hoc - in
Fodor 1978), and even more commonly, principles of process- spite of the appeal to what sounds like a general principle.
ing are stated in terms of the semantics of the representa- However, because the principle as stated applies only to the
tions. semantic domain, one is still owed a corresponding principle
For example, in arguing for the indeterminacy of forms of that applies to the representational or syntactic domain - i.e.,
representation, Anderson (1978) proposed a construction by which applies in virtue of some intrinsic property of the
means of which a system could be made to mimic the representation itself. This issue is discussed in greater detail in
behavior of another system while using a form of represen- Pylyshyn (1979c).
tation different from that used by the mimicked system.
While I have already argued (Pylyshyn 1979b) that much is
wrong with Anderson's case, one flaw that I did not discuss is 4. Algorithmic models and the proprietary
that the mimicking model is required (in order to be vocabulary hypothesis
constructable as in Anderson's "existence proof") to have
access to the semantics of the original representations (i.e., to We begin our discussion of how a literal computational view
the actual objects being referred to) in the system it is can provide an explanatory account of cognition by examin-
mimicking - which, as we have seen, is not possible according ing what sorts of questions a computational model might be
to the computational view. This occurs in Anderson's
expected to address. One of the discoveries that led to the
construction, because in order to decide what representation
to generate next, the mimicking model must first determine development of computer science as a discipline is that it is
what the mimicked model would have done. It does this by possible to study formal algorithmic processes without regard
first determining what representation the original system to how such processes are physically instantiated in an actual
would have had at that point and what new representation it device. What this means is that there is an interesting and
would have transformed it into. Then it is in a position to reasonably well-defined set of questions concerning compu-
infer what representation it should generate in its mimicking tational processes (e.g., what is the fastest or the least
process. But the only way the mimicking system could be memory-consuming way of realizing a certain class of
assured of finding out what representation the target model functions?), the answers to which can be given without regard
would have had (and the way actually invoked in the to the material or hardware properties of the device on which
proposed construction) is to allow it to find out which stimuli these processes are to be executed. This is not to suggest, of
correspond to its current representation, and then to compute course, that issues of material realization are uninteresting, or
even that they are irrelevant to certain other questions about

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 115

Pylyshyn: Computation and cognition
computation. It simply implies that in studying computation like the notion of a well-formed sentence, it will itself evolve
it is possible, and in certain respects essential, to factor apart as the larger theoretical system develops. The tacit assump-
the nature of the symbolic process from properties of the tion, of course, has been that this domain coincides with what
physical device in which it is realized. It should be noted that we pretheoretically think of as the domain of cognition - i.e.,
the finding that there is a natural domain of questions with the phenomena associated with language, thought,
concerning the behavior of a system, which is independent of perception, problem-solving, planning, commonsense reason-
the details of the physical structure of the device - providing ing, memory, and so on. However, there may also turn out to
only that the device is known to have a certain general be significant departures from the pretheoretical boundaries.
functional character - is a substantive one. Prior to this After all, we cannot be sure in advance that our commonsense
century there was no reason to believe that there was even a taxonomy will be coextensive with the scope of a particular
coherent notion of process (or of mechanism) apart from that class of theory - or that we have pretheoretically carved
of physical process. The new notion initially grew out of Nature exactly at her joints.
certain developments in the study of the foundations of One of the main points of this article will be that the
mathematics, especially with the work of Alan Turing proprietary level hypothesis, together with certain intrinsic
(1936). requirements on the use of algorithmic models to provide
Because of the parallels between computation and cogni- theoretical explanations, leads to certain far-reaching conse-
tion noted earlier, the corresponding view has also been quences. For the next two sections, however, I shall make a
applied to the case of mental processes. In fact, such a view somewhat extensive digression to establish a number of
has a tradition in philosophy that, in various forms, predates background points. Discussions of cognition and computation
computation - though it has received a much more precise frequently founder on certain preconceived views regarding
formulation in recent years in the form known as the what a cognitive phenomenon is and what a cognitive process
representational theory of mind, one of the most developed is. Thus, for example, cognitive science is sometimes criti-
versions of which can be found in Fodor (1975). One way of cized for allegedly not being able to account for certain types
of phenomena (e.g. emotions, consciousness); at other times,
stating this view is to put it in terms of the existence of a
attempts are made to provide explanations of certain obser-
privileged or a proprietary vocabulary in which the structure vations associated with thinking (e.g. reaction time, effect of
of mental processes can be couched - viz. the vocabulary practice) in terms of ad hoc mixtures of biological and
needed to exhibit algorithms. computational mechanisms. In many of these cases what is
More precisely, the privileged vocabulary claim asserts that happening is that inappropriate or unclear ideas concerning
there is a natural and reasonably well-defined domain of the notion of cognitive phenomenon or cognitive process are
questions that can be answered solely by examining 1) a being invoked, to the detriment of theoretical clarity. Conse-
canonical description of an algorithm (or a program in some quently it is important to our story that we first try to clarify
suitable language - where the latter remains to be specified), these notions in the following digression.
and 2) a system of formal symbols (data structures, expres-
sions), together with what Haugeland (1978) calls a "regular
scheme of interpretation" for interpreting these symbols as
expressing the representational content of mental states (i.e.,
as expressing what the beliefs, goals, thoughts, and the like are 5. What is a cognitive phenomenon?
about, or what they represent). Notice that a number of issues
have been left unresolved in the above formulation. For Many things happen when people are engaged in thinking or
example, the notion of a canonical description of an algorithm problem-solving. For example, they may ask questions or
is left open. We shall return to this question in section 8. Also, report being puzzled, frustrated, or following false leads, they
we have not said anything about the scheme for interpreting may get up and walk around or jot down notes or doodles;
the symbols - for example, whether there is any indetermi- their attentiveness to environmental events may deteriorate;
nacy in the choice of such a scheme or whether it can be various physiological indicators such as skin resistance,
uniquely constrained by empirical considerations (such as peripheral blood flow, or skin temperature (as measured by
those arising from the necessity of causally relating represen- GSR, plethysmograph, or thermometer) may change sys-
tations to the environment through transducers). This ques- tematically, and finally, they may report what they believe is
tion will not be raised here, although it is a widely debated a solution. In addition, these various events may occur over
issue on which a considerable literature exists (e.g. Putnam time in certain systematic ways. Now, one might ask which, if
1978). any, of these observations could be explained by examining a
A crucial aspect of the assumption that there exists a fixed canonical description of an algorithm, or, taking the position
formal vocabulary for addressing a significant set of psycho- that algorithmic accountability defines the now technical
logical questions is the view that such questions can be notion "cognitive phenomenon," one might ask which of the
answered without appealing to the material embodiment of above reports represents an observation of a cognitive
the algorithm, and without positing additional special analy- phenomenon?
tical functions or relations which themselves are not to be It seems clear that this question cannot be answered
explained in terms of algorithms. In fact, as I shall argue without bringing in other considerations. In particular, we
presently, one can take the position that this proprietary must know how the observations are being interpreted in
vocabulary or level of description defines the notion of relation to the algorithm. For example, a cognitive model (of
cognitive phenomenon in the appropriate technical sense the kind I have been speaking about) would not account for
required for explanatory theories. It should, however, be people's ability to move their limbs in certain ways (e.g. in
emphasized that such a definition is presented in the spirit of certain directions and at certain speeds) or for the spectral
a broad empirical hypothesis. This hypothesis would be properties of the sounds they make, although it ought to be
falsified if it turned out that this way of decomposing the able to account in some general way for what people do (i.e.
domain of cognitive psychology did not lead to any progress what intended actions they carry out) in moving their limbs,
or any significant insights into human thought and human or for what they say in making those sounds. Thus, even when
rationality. Thus when I spoke of this level as specifying a a phenomenon is clearly implicated in the cognitive activity,
domain of questions, it was assumed that such a domain is a its relevance is restricted to the case in which it is appro-
natural one that (at least to a first approximation) can be priately described or interpreted (and, as we noted earlier, the
identified independently of particular algorithms - though, appropriate interpretation will generally correspond to what

116 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
the people themselves intended when they performed the construction and validation of computational models has
action). developed, there has evolved a general (though usually tacit)
But what of the other observations associated with the agreement that the kind of input-output equivalence that
problem-solving episode, especially the physiological indica- Turing proposed as a criterion for intelligence is insufficient
tors, task-independent performance measures (such as as a condition on the validity of such models. It was obvious
measures of distractability), and the temporal pattern asso- that two quite different algorithms could compute the same
ciated with these observations? Here the case has typically not input-output function. It gradually became clear, however,
been as clear, and people's intuitions have differed. Thus the that there were ways to distinguish empirically among
manner in which we interpret measures such as the galvanic processes that exhibited the same input-output function,
skin response (GSR) and time-of-occurence in relation to the providing certain additional methodological assumptions
algorithm depends on various methodological assumptions, were made. In particular, these additional assumptions
and these are just beginning to be articulated as the required that one view certain aspects of the organism's
methodological foundations of cognitive science evolve. behavior not as a response computed by the cognitive process,
For example, it is clear that a cognitive model will not have but rather as an independent indicator of some property of
values for blood flow, skin resistance, or temperature among the algorithm by which the response was computed. There
its symbolic terms, because the symbols in this kind of model are, in fact, independent reasons for wanting to distinguish
must designate mental representations (i.e., they must desig- between the kind of behavior that is directly attributable to
nate mental structures that have representational content - the symbolic process, and other kinds of behavior that are not
such as thoughts, goals, and beliefs). The calculation of viewed as "outputs" of the cognitive process. We shall return
temperature and resistance in this case does not itself to these below when we examine the complementary notion
represent a cognitive process, and the values of these of a cognitive process - a notion that is as much in a state of
parameters are simply intrinsic physical magnitudes - they evolution as that of a cognitive phenomenon.
themselves do not represent anything (see the further discus- It has become customary in cognitive science to view
sion of this in section 6, below). However, such measures can reaction time in the same way that we view measures such as
be (and often are) taken as indices of certain aggregate the GSR or plethysmograph records, or measures of distract-
properties of the process. For example, they might be taken as ability - namely as an index, or an observable correlate, of
indices of what could be thought of as "processing load," some aggregate property of the process. In particular,
where the latter is theoretically identified with, say, the size reaction time is frequently viewed as an index of what might
and complexity of the data structures on which the model is be called "computational complexity, " which is usually taken
operating. Whether such an interpretation of these observa- to correspond to such properties of the model as the number
tions is warranted depends on the success of the ancilliary of operations carried out. A process that merely computed
hypotheses in accounting for the relation among observations time as a parameter value would not account for reaction
in the past. In other words, the justification of subsidiary time viewed in this particular way, since the parameter
methodological assumptions is itself an empirical question. would not express the computational complexity of the
There is no room here for a priori claims that a cognitive process.
theory must account for these (or any other observations, and Measures such as reaction time, when interpreted in this
hence that certain phenomena are necessarily (or analytical- way, are extremely important to cognitive science, because
ly) "cognitive." they provide one possible criterion for assessing strong
Consider the following example, in which the developing equivalence of processes. Thus, all processes that, for each
methodology of cognitive science has led to a gradual shift in input, produce 1) the same output, and 2) the same measure
the way an important aspect of observed behavior is inter- of computational complexity, as assessed by some indepen-
preted. The example concerns what is probably the most dent means, are referred to as complexity-equivalent. The
widely used dependent measure in cognitive psychology, complexity-equivalence relation is a refinement of the input-
namely, reaction time. This measure has sometimes been output or the weak equivalence relation. However, it need not
interpreted as just another response, to be accounted for by a be identical to what we would call the relation of strong
cognitive model in the same way that the model accounts for equivalence, since we must allow for the possibility that
such response properties as which button was pressed. Since future methodologies or other theoretical considerations may
Donders's (1969) pioneering work (carried out in the 1860s), refine the relation even further.
it has also been widely interpreted as a more-or-less direct Nonetheless the complexity-equivalence relation remains a
measure of the duration of mental processes [see Wasserman central one in cognitive science, and reaction time measures
and Kong: "The Absolute Timing of Mental activities BBS remain one of the primary methods of assessing complexity.
2(2) 1979]. I have argued (e.g. in commenting on the Since complexity is a relative quality (e.g., it makes no sense
Wasserman and Kong paper) that neither of these interpre- to speak of the absolute complexity of a computation, only of
tations is correct in general - that, in general, reaction time its complexity in relation to other computations that utilize
can neither be viewed as the computed output of a cognitive the same hypothetical operations), we are only concerned
process itself, nor as a measure of mental duration. with measures of complexity up to a linear transform. Thus
If reaction time were thought of as simply another we would distinguish between two lookup processes if the
response, then it would be sufficient if our computational complexity of one increased linearly with the number of
model simply calculated a predicted value for this reaction items stored while the complexity of the other was indepen-
time, given the appropriate input. But clearly that would not dent of, or increased as the logarithm of, the number of items.
suffice if the computation is to be viewed as modelling the However, we would not necessarily discriminate between two
cognitive process. Contemporary cognitive theorists would hypothetical processes if their complexity varied in the same
not view a system that generated pairs of outputs, interpreted way with the input, but one process took a constant number of
as the response and the time taken, as being an adequate steps more than the other or required a fixed amount of
model of the underlying process, no matter how well these storage capacity more than the other, unless we had some
outputs fit the observed data. independent calibration that enabled us to translate this
difference into, say, absolute reaction-time predictions.
The reason for this intuition is significant. It is related to
what people understand by the concept of a cognitive process, This idea of appealing to plausible methodological assump-
and to the evolving concept of strong equivalence of tions (which must in the long run themselves be justified by
processes, to which we will return in section 6.1. As the the success of the models to which they lead) in order to make

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 117

Pylyshyn: Computation and cognition
finer discriminations among theoretical systems is exactly the algorithm was implemented in it; some might arise from
parallel to the situation that has developed in linguistics over variable delay physical effects, such as the distance that a
the last two decades. Linguistic judgments made by compe- moveable arm had to travel in making a disk access in that
tent speakers are not simply taken as part of the linguistic implementation; and some could even depend on physical
corpus. The methodological assumption is that these judg- properties of the noncomputational environment, as would be
ments provide an independent source of constraint on the case if real-time interrupts could occur. None of these
linguistic structures, defining the analogous relation of strong properties bears on the nature of the algorithm, since they
equivalence of grammars (cf. Chomsky 1975). could be quite different for a different realization of the
This view of the role of reaction-time measures takes it for identical algorithm. Consequently, in this case, measuring
granted that such measures cannot be interpreted as a direct such times would not help to distinguish different candidate
observation of the duration of a mental process. A mental algorithms. That is why time measurements alone cannot be
process does not possess the intrinsic property of duration, any taken as measurements of a property of the algorithmic
more than it possesses the property of location, size, mass, process in the computer case. And for precisely the same
electrical resistance, or concentration of sodium ions. Of reasons they cannot be taken literally as measurements of
course, the underlying brain events do have these properties, mental durations - only as indirect (as possibly false) indica-
and we can, at least in principle, measure them. But one must tors of such things as processing complexity, to be used
be careful in drawing conclusions about properties of the judiciously along with other indirect sources of evidence in
cognitive process on the basis of such measurements. inferring underlying mental processes.
Recall that we are examining the claim that there is a The other case in which the observations may tell us little
domain, roughly coextensive with the range of phenomena about the cognitive process itself arises when the preliminary
covered by our intuitive notion of cognition, that can be determinant of the behavior in question is what Newell and
accounted for solely in terms of a canonical description of the Simon (1972) call the "task demands" (which, incidentally,
symbol manipulation algorithm. Suppose we have some are not to be confused with such "experimental demand"
observations (duration being a case in point) of properties of factors as Kosslyn, Pinker, Smith, and Schwartz 1979 have
the physical instantiation of the algorithm. The question is, tried to rule out in their defense of their interpretation of the
can such evidence be treated as the measurement of a "mental scanning" results). Consider, for example, the vari-
cognitive property? We have already seen how, with the aid ous observations associated with certain operations on mental
of ancillary methodological hypotheses, it can be used as images. A large number of these investigations (e.g. Shepard
evidence in favor of one or another putative cognitive process 1978; Kosslyn, in press) have proceeded by measuring the
or algorithm. We are now asking whether such measurements time it takes to imagine a certain mental action, such as
can be viewed as anything more than fallible correlates, rotating an image or scanning with one's attention between
whose validity in each case depends upon whether or not the two points on a mental image. But what the analysis of these
ancillary hypothesis holds. results has consistently failed to do is to distinguish two
I have argued (e.g. in my commentary on the Wasserman different tasks that could be subsumed under the same
and Kong paper, BBS 2(3) 1979, and in Pylyshyn 1979b) that general instructions to the subject. The first is the task of
the answer to this question must in general be no. There are simply using a certain form of representation to solve the
many situations in which measurements of properties of the problem. The second is to imagine actually seeing certain of
underlying physical events may tell us little about the the problem-solving events taking place. In the latter case one
algorithm. It may, instead, tell us either about the way in would expect various incidental properties of the real events
which the process is physically (i.e. neurophysiologically) to be duplicated in the imagining. For example, suppose you
instantiated, or it may tell us about the nature of the task are asked to imagine two different events: call them E) and
itself. We will briefly consider these two cases since they E2. If you know that, in reality, E, takes, say, twice as long as
reveal an important general point concerning the relations E2) and if you interpret the task as requiring you to imagine
among cognitive phenomena, the task being carried out, the yourself seeing the events happening (as seems reasonable,
method the person is using, the fixed functional properties (or given the ambiguity of the instructions in this respect), then
functional architecture) of the cognitive system, and the surely the task requires, by its very definition, that you spend
biological or physical properties of some particular (token) more time in the act of imagining E, than in the act of
instantiation of that solution process. imagining E2 (in fact, twice as long - if you are capable of the
Measurement such as reaction time are particularly psychophysical task of generating a time interval correspond-
unlikely to tell us much about the nature of the underlying ing to a particular known magnitude). The crucial difference
biological mechanism in the case of what is often called a between these two tasks is that quite different success criteria
"higher-level cognitive function," in which the processing is apply in the two cases; although "using an image" does not
not as closely tied to anatomical structures as it is, say, in require, as a condition for having carried out the task, that
certain parts of perception or motor coordination. While in one reproduce incidental characteristics of some imagined
many cases there would, no doubt, be a correspondence event such as its relative duration, the task of "imagining
between the duration of some correlated physical measure- seeing it happen" does demand that this be done.
ments and such purely algorithmic properties as the number This is, in fact, part of a very general difficulty that
of steps carried out, which particular steps were carried out, permeates the whole range of interpretations of imagery
whether parts of the algorithm were carried out serially or in research findings that appeal to such things as "analogue
parallel, and so on, this is not always the case. The explanation media." I discuss this issue in detail elsewhere (Pylyshyn
of some of the time differences (and every explanation of the 1979c). For the present purposes we might simply note that a
absolute time taken) must appeal to some properties of the variety of determinants of observed phenomena, other than
physical realization that are unique to that particular instan- the structure of the cognitive process or the functional
tiation and therefore irrelevant to the algorithmic or process architecture of the underlying system, are possible. In order
explanation in general. Such duration data may not make a for observations such as, say, that reaction time varies linearly
valid discrimination among putative algorithms. with the imagined distance, to have a bearing upon the
Using a computer as an example, we can readily see that cognitive theory itself (i.e., upon the structure of the algo-
some of the time differences might arise from the fact that a rithm or the functional architecture of the model), it would
signal had farther to travel in some particular (token) occasion be necessary to further show that under certain prescribed
because of the way the machine was wired up and the way conditions (such as when subjects report that their answers

118 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
come from examining their image) the relation between believe that the acquisition and exercise of such skills can be
distance and time is a necessary one (i.e., it is not determined enhanced by purely cognitive means. For example, imagined
by beliefs, tacit knowledge, or certain specific goals - such as or "mental" practice is known to improve certain motor skills.
to reproduce aspects of the event as it would have been Furthermore, a careful functional analysis of a skill such as
perceived). To my knowledge this has not been demon- juggling, in terms of its intrinsic hierarchical structure, can
strated - and furthermore, some preliminary evidence from lead to methods for verbally instructing novices, which cuts
our own laboratory suggests that it is false in the case of the learning time to a fraction of what it would otherwise take
mental scanning (see Pylyshyn 1979c). Clearly, in the case of (Austin 1974). Clearly, cognitive processes are relevant to
a situation such as imagining a heavy stone being rolled as motor skills. Equally clearly, however, certain aspects of their
opposed to a light object being flipped over, any differences execution are purely biological and physical. How this
in the time taken would be attributed to the subject's problem will naturally factor will depend on what the facts
knowledge of the referent situation - i.e. to the task demand. are: it is not a question of which we can issue a verdict in
It is not clear how many of the reported time functions for advance.
mental operations fall into this category. I suspect that not The same goes for moods, emotions, and the like. It. is very
only mental scanning, but also such findings as the relation likely that there will be certain aspects of these phenomena
between "size" of mental images and time to report details is that will resist functional analysis, and therefore computa-
explainable in this way, although perhaps some of the mental tional explanation (as there have been aspects that have
rotation results are not - but then it appears that the latter are historically resisted every kind of analysis). For instance, the
not explainable as holistic analogue processes either (cf. conscious feeling accompanying moods may not be suscepti-
Pylyshyn 1979a). ble to such analysis, except insofar as it has a cognitive
Whatever the ultimate verdict on the correct explanation component (e.g., insofar as it leads to people noticing that
of the observed reaction time functions, considerations such as they are experiencing a certain mood). It would not even be
these do illustrate that certain distinctions need to be made surprising if the pervasive effects that moods have on
before we are free to interpret the observations. For example, cognitive activity (which Haugeland mentions) turn out to be
if we distinguish, as I suggested, between the case of applying outside the scope of computational explanation. We know
operations to visual images of objects and the case of that there are global changes in various aspects of cognitive
imagining yourself seeing operations physically applied to activity that accompany biological development and endo-
real objects, then it would follow that in at least the second of crine changes. Such changes seem more akin to variations in
these cases the reaction time functions should to some degree the underlying functional architecture (a concept to which
(depending on subjects' capacity to carry out the required we shall return shortly) than to changes in the algorithm.
psychophysical task of generating appropriate outputs) Variations in such inner environments do not connect with
mirror the real time functions of the corresponding events. mental representations the way variations in the perceived
Furthermore, on this view it is also not at all surprising that environment do: We do not perceive our hormonal levels and
we cannot imagine (in the sense of imagining ourselves thus come to have new beliefs. The relation in that case does
seeing) such things as four-dimensional space. Certainly this not appear to be of the appropriate type: It is biological and
fact tells us nothing about the nature of the representation: It causal, rather than rational or cognitive. On the other hand,
is a matter of definition that we cannot imagine ourselves we cannot rule out the possibility that a broader view of
seeing what is in principle not seeable! The important point is computation might make it possible to subsume even these
that, to the extent that task-demand factors can be viewed as facts into a computational theory. The real mystery of
the primary determinants of observations, such observations conscious experience is that the fact that it appears to elude
cannot be taken as having a bearing on the structure of our both functionalist (i.e. computational) and naturalist (i.e.
cognitive model. identity theory) accounts. A possible approach to this
We thus have another instance of the principle that dilemma (and one on which I have speculated in Pylyshyn
whether or not some measurement turns out to be an 1979d) might be to allow cognitive states to have access (via
observation of a "cognitive phenomenon" by our definition internal transduction) to biological states (e.g., imagine a
depends on how we interpret it. I have no doubt that we will computer with a register whose contents correspond to its
refine our notion of "cognitive phenomenon," just as we internal temperature or line voltage). Neither providing for a
refined our notion of "physical phenomenon" in the course of computational system to appeal directly to arbitrary causal
the development of our theories. It should not come as a shock properties of its physical instantiation, nor allowing it to alter
to us, therefore, if certain ways of viewing phenomena its own functional architecture presents any difficulties in
ultimately end up as irrelevant to cognitive theory, even principle. In fact, we routinely do change the functional
though they might have been part of our pretheoretic notion architecture of computers by means of programs - for exam-
of cognition. As Chomsky (1964) has reminded us, it is no less ple, by a process called compiling. Whether such methods
true of cognitive science than of other fields that we start off will allow us to capture aspects of such global effects as those
with the clear cases and work out, modifying our view of the mentioned above is a question for future theories to
domain of the theory as we find where the theory works. answer.
For example, Haugeland (1978) considers it a serious
shortcoming of the computational view that it appears unable
to explain such things as skills and moods. There are two 6. What is a cognitive process?
assumptions here, for which little evidence exists at present,
one way or the other. The first is that no aspect of skills or 6.1 Cognitive simulation and strong equivalence. There
moods can be accounted for computationally. The second is is a systematic ambiguity in the use of the term "computer
that, under the appropriate interpretation, these are phenom- simulation " that occasionally causes confusion. Computers
ena that we know a priori to be cognitive. Consider the case are said to simulate economic cycles, traffic flow, chemical
of skills. The popular view is that, at least in the case of motor reactions, and the motion of celestial bodies. They are also
skills, competence is built up by repeated practice (given the said to simulate aspects of human behavior as well as
inherent talent) without any intellectual intervention. In fact, cognition. It is important, however, to understand the
it is often believed that thinking about such skills impedes essential difference between the latter sense of simulation (as
their fluency (recall the story about the centipede that was a psychological model) and the former. When we simulate,
asked which foot it moved first). But there is now reason to say, the motion of planets, the only empirical claim we make

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 119

Pylyshyn: Computation and cognition
is that the coordinate values listed on the printout correspond plethysmograph, thermometer, and timing records in the
to the ones that will actually be observed under the specified latter category. Although, clearly, we could in principle
conditions. Which algorithm is used to compute these values compute values of these paramenters, and this computation
is irrelevant to the veridicality of the simulation. In other itself would be some sort of process, yet it would not be a
words, for this purpose we do not distinguish among cognitive process. In computing these values we would, at
algorithms that compute the same input-output function. We best, be simulating the function of the biological mechanism,
have refered to such algorithms as being weakly not literally executing its algorithm on an emulated
equivalent. architecture. The intermediate states of this process would not
The case of cognitive simulation, however, is quite be viewed as having empirical cognitive referents. To the
different. As we have already noted, weak equivalence is not extent that the body could be said to "compute" GSR, it
a sufficient condition for validity of a model. Intuitively, we clearly does not do it symbolically. If it could be said to
require that the algorithm correspond in much greater detail compute at all, it would be by analogue means (we shall have
to the one the person actually uses. The difficulty is, of course, more to say below about the important question of when an
that we cannot directly observe this algorithm, so we must organism or device symbolically computes a function, as
discriminate among weakly equivalent processes on the basis opposed to merely instantiating or exhibiting that
of other considerations. We have already suggested how function).
observations such as reaction time can be used as evidence for Another closely related reason why strong equivalence is
deciding among weakly equivalent algorithms. Such criteria demanded of cognitive models, but not of models of physical
help us to select a set of what I have referred to as processes, is that the former are assumed to be governed by
complexity-equivalent processes. Other criteria, such as those rules acting upon symbolic representations. While we do not
that provide evidence for intermediate states and for assume that planets have a symbolic representation of their
factorable subcomponents of the process (cf. Pylyshyn 1978a), orbits (or of the laws governing their trajectory), we do claim
as well as endogenous criteria, such as minimizing that the appropriate explanation of cognitive processes must
redundancy through appropriate subroutinizing, and further appeal to the organism's use of rules and explicit symbolic
methodological criteria yet to be developed, define a sense of representations. The distinction between behavior being
equivalence I have referred to earlier as strong evidence. governed by symbolic representations and behavior being
There is an important reason why strong equivalence is merely exhibited by a device in virtue of the causal structure
relevant to cognitive models, though not to other uses of of that device is one of the most fundamental distinctions in
computer simulation, such as those mentioned earlier. The cognitive science. We shall therefore devote the next section
reason was hinted at when we spoke of modelling the to examining that distinction.
algorithm that the person "actually uses." The idea is that the
appropriate way to functionally characterize the mental 6.2 Representation-governed behavior. The question of
activities that determine a person's behavior is to provide an whether we should explain the behavior of a certain organism
initial representational state - interpreted as representing or device by ascribing to it certain explicit symbolic rules and
beliefs, tacit knowledge, goals and desires, and so on - and a representations, or whether we should simply describe its
sequence of operations that transform this initial state, dispositions to respond (i.e. its intrinsic input-output
through a series of intermediate states, into the commands function) in any way we find convenient (including
that are finally sent to the output transducers. All the appealing to exactly the same rules, but without the
intermediate states, on this view, are also representations assumption that they are represented anywhere other than in
which, in the model, take the form of expressions or data the theorist's notebook, and consequently without any
structures. Each of these has psychological significance: it concern for strong equivalence), is a central philosophical
must be interpretable as a mental representation. Thus all issue in the foundations of cognitive science. To some extent
intermediate states of the model constitute claims about the the issue is a conceptual one and relates to the question of
cognitive process.
whether psychology has a special (and less than objective)
Contrast this with the case of simulating planetary motion. status among the sciences (for example, Putnam 1978 has
Clearly in this case no empirical claims whatsoever are being argued that cognitive theories are necessarily "interest
made about the intermediate states of the computation itself: relative" rather than being empirically determinate - see,
only the outputs are interpreted as referring to the modelling however, Chomsky, this issue). In addition to this more
domain. The ambiguity we spoke of earlier arises because of general question, however, there is also a straightforward
the use of the phrase "computer simulation" to refer to two empirical side to this issue, to which I now turn.
quite different activities. In the case of planetary motion, Elsewhere (Pylyshyn 1978b) I have sketched some general
chemical processes, traffic flow, and so on, what we call conditions under which it would be reasonable to speak of
simulation is only a way of computing a series of values of a behavior as being governed by rules and representations. The
function, while in the psychological case, simulation refers to general position is that whenever behavior is sufficiently
the execution of a hypothesized algorithm in a simulated (in plastic and stimulus-independent, we can at least assume that
the first sense) mechanism. In the computer industry the it is somehow mediated by internal functional states. Such
technical term for this kind of mimicry is emulation: one states may be further viewed as representational, or
computer is sometimes made to emulate the functional epistemic, if certain other empirical conditions hold. For
capacities or functional architecture of another, so that example, we would describe the behavior as being governed
programs originally written for the first can be run directly by representations and rules if the relation between
on the second. Thus, modelling cognitive processes must environmental events and subsequent behavior, or the
proceed in two phases. The first is to emulate the functional relations among functional states themselves, could be shown
architecture of the mind, and the second is to execute the to be, among other things, a) arbitrary with respect to natural
hypothetical cognitive algorithm on it (not to simulate the
laws, b) informationally plastic, or c) functionally
behavior of the cognitive process, for the algorithm represents
a literal proposal for the structure of this process). transparent. We elaborate briefly on these conditions
below.
Notice that this discussion is relevant to the distinction The relation between an event and the ensuing behavior
made earlier, between observations directly attributable to would be said to be arbitrary if there were no necessary
the cognitive algorithm and ones that must be attributed to intrinsic connection between them. More precisely, a relation
physical properties of the device, where we included GSR, between an environmental event, viewed in a certain way,

120 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
and a behavioral act, is said to be arbitrary if, under that means that the states of the process are representational, and
particular description of the event and the behavior, the that this representational content is hypothesized to be the
relation does not instantiate a natural law. For example, there same as the content of the mental states (i.e. tacit knowledge,
can be no nomological law relating what someone says to you goals, beliefs) being modelled. Thus the computational states
and what you will do, since the latter depends on such things do not represent biological states, unless by this we mean to
as what you believe, what inferences you draw, what your suggest that the person being modelled is thinking about
goals are (which might include obeying totally arbitrary biology. The process is, however, realized or carried out by
conventions), and, perhaps even more important, how you the fixed functional capacities provided by the biological
perceive the event (what you take it to be an instance of). substrate. These functional capacities are called the func-
Since all physical events are intrinsically ambiguous, in the tional architecture. The decision as to whether or not a
sense that they can be seen in very many different ways, each particular function should be explained by positing a cogni-
of which could lead to very different behavior, the tive process-i.e., by appeal to rules and representations-
nonlawfulness of the link between these events and rests on certain empirical considerations. One sufficient
subsequent behavior seems clear. Systematic but nonlawlike (though not necessary) condition for a function; being
relations among functional states are generally attributed to determined by representations is that it be influencible by
the operation of rules rather than natural laws. beliefs and goals-i.e., that it be cognitively penetrable.
The condition that I referred to as informational plasticity Hence only cognitively impenetrable functions constitute the
will play an important role in later discussion. For the present fixed functional capacities, or the functional architecture, out
I introduce it to suggest that epistemic mediation is of which cognitive processes are composed. We shall elabo-
implicated whenever the relation between environmental rate upon these conditions further in sections 8, 9, and 10.
events and behavior 'can be radically, yet systematically,
varied by a wide range of conditions that need have no more
in common than that they provide certain information, or
that they allow the organism to infer that a certain state of
affairs holds - perhaps one which, if it were actually 7. Computational models and explanations
perceived, would also produce the same behavior. For
example, seeing that the building you are in is on fire, We return to the theme raised in section 4, namely the
smelling smoke coming in through the ventilation duct, or question of how a computational model, in the form of a
being told by telephone that the buildings is on fire, can all symbol-processing algorithm, can be used to provide an
lead to similar behavior, and this behavior might be radically explanation of cognitive phenomena. In general, what is
different if you believed yourself to be performing in a play commonly referred to as a model can be used in various ways
at the time. in explanation. These range from using it merely as an
The third condition, that of transparency of relations illustration or metaphorical object, to using it as a constitutive
among representations, is, in a sense, the converse of the part of the theory. Computational models have been partic-
second condition. Whereas informational plasticity reflects ularly free in the way the explanatory burden has been shared
the susceptibility of the process between stimulus and between the model and the theorist. There has, in other
response to cognitive influences, the transparency condition words, been very little constraint on what a theorist is
reflects the multiple availability of rules governing relations warranted to say about an algorithm. But if the algorithm is to
among representational states. Wherever quite different be taken as a literal description of a mental process, then
processes appear to use the same set of rules, we have a prima much stronger constraints must be imposed on how it is used
facie reason for believing that there is a single explicit in the explanatory account. In particular, the appeal to the
representation of the rules, or at least a common shared algorithm must be principled.
subprocess, rather than independent identical multiple Notice that the parallel problem is not as serious in the case
processes. The case can be made even stronger, however, if it of mathematical models in psychology, or models in physics.
is found that whenever the rules appear to change in the Here a clear separation is made among the fixed universal
context of one process, the other processes also appear to constants, empirical parameters estimated from the data,
change in a predictable way. In that case we would say variables, and functional forms. Although in formulating the
(borrowing a term from computer science) that the rules were model the theorist is relatively free to choose a function
being used in an "interpretive" or transparent, rather than a (subject to available mathematical notation and technique, as
"compiled" or opaque mode. well as to certain other intrinsic constraints, such as the
Such seems to be the case with grammatical rules, since we general prevalence of linear and inverse-square law effects),
appear to have multiple access to these rules. We appeal to thereafter the theorist is, to a considerable extent, accountable
them to account for production, for comprehension, and for for the way estimates of parameter values are revised in
linguistic judgments. Since these three functions must be response to observations. This is done by monitoring the
rather thoroughly coordinated (e.g., a rule first available only degrees of freedom in the model. To prevent a theorist from
in comprehension soon becomes effective in the other two arbitrarily revising the cognitive model, a notion similar to
functions), it seems a reasonable view that they are explicitly this accountability is needed. While I don't see an exact
represented and available as a symbolic code (as I have analogue to the degrees-of-freedom ledger emerging in
argued elsewhere; Pylyshyn 1976). In other words, there are cognitive science, I think there is an analogue to the
cases, such as grammar, in which rules can be used not only technique of factoring the sources of variability in the system.
within some specific function, but in some circumstances they Part of the requirement that the appeal to the computational
can even be referred to or mentioned by other parts of the model be principled, then, is the demand that various sources
system. These cases argue even more strongly that the system of the model's behavior be factored apart and independently
could not simply be behaving as if it used rules, but must in justified. This is, in fact, an instance of an extremely general
fact have access to a symbolic encoding of the rules. [Other scientific maxim, namely, that a central goal of explanatory
arguments for the representational theory of mind are offered theories is to factor a set of phenomena, a problem, or a
by Fodor (1975; and this issue) and Chomsky (this issue).] system into the most general and perspicuous components,
principles, or subsystems.
To summarize, then, a cognitive process is distinguished
from a process in general inasmuch as it models mental Apart from the general application of this principle in, say,
events, rather than merely simulating some behavior. This factoring apart knowledge of a domain, various problem-

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 121

Pylyshyn: Computation and cognition
specific heuristics, and resource-limited performance mech- spatial character of images. We shall examine these briefly
anisms, there are two major classes of factors that a compu- later.
tational theory must explicitly recognize when appealing to The second class of factors - those relevant to answering
an algorithmic model. I refer to these as a) the functional the question of why the algorithm exhibits the ability it does
architecture, or the fixed capacities of the system, and b) the (or the ability claimed) - represents a further application of
functional requisites, or the effectiveness principles relating the injunction that explanatory appeals to a computational
the system to the task. Partitioning the explanatory appeal to model must be principled. It is also an attempt to deal with
the computational model in this way places two special the (often justified) criticism that a hypothesized algorithm is
burdens on the theory-building enterprise. The first is that the ad hoc in the sense that there is no independent reason for it
basic functional mechanisms or computational building- to be the way it is, as opposed to some other way, other than
blocks out of which the model is constructed must be because it was designed to duplicate a certain set of
independently justified. The second is that one must be able observations. Earlier I characterized design decisions, moti-
to specify the characteristics of the model and the task in vated solely by the need to account for certain observations, as
virtue of which the system is capable of carrying out that task being equivalent to fixing the value of a free empirical
or producing the behavior we are explaining. In other words, parameter. Such decisions are unmotivated by general prin-
both the underlying architecture of the computational model, ciples or constraints on processes, except perhaps ones that are
and the properties of the algorithm or task that enables the stated in terms of properties of the represented domain,
system to successfully carry it out, must be explicitly which, we have argued, makes them descriptive but not
addressed in the explanatory story, and both must be explanatory. Explanatory principles must characterize the
independently justified. operation of a system in terms of endogenous structural or
The first factor, regarding assumptions about the underly- functional properties of that system.
ing functional architecture, is the primary concern of most of The nature of the requirement, that we be able to state the
the remainder of this paper, and hence a detailed discussion principles in virtue of which the algorithm is capable of
will be postponed until the next section. What is at issue is achieving the claimed skill, is best illustrated by the following
that the success of an algorithm in accounting for a certain example, based on the work of Shimon Ullman (1979). It is
domain of behavior must not be due merely to some quite known that the shape of a moving object can be perceived
fortuitous property of a particular computational architec- even in highly impoverished circumstances - such as when
ture, which itself cannot be independently justified. For the presentation consists of a silhouette projection of a
example (Winston 1975), some interest was once aroused by rotating unfamiliar wire shape, or even random unconnected
the fact that a certain artificial intelligence "blocks world" elements on the transparent surface of the rotating forms.
vision system [based on Guzman's (1968) program SEE] This perceptual ability was first studied by Wallach and
seemed to exhibit an effect very similar to the Muller-Lyer O'Connell (1953) under the title "kinetic depth effect." Now,
illusion (i.e., it took a line with arrows on its ends like to suppose someone designed an algorithm (perhaps one using
be shorter than a line with forks on its ends like > <), and statistical methods) that recognized shape successfully in over
thus to provide a possible account of this illusion. This ninety percent of the cases presented to it. Could we consider
particular effect was due to the fact that the system's this algorithm to be a model of the underlying process? If we
line-recognizing procedure used a diameter-limited scanner, answer this question in the affirmative, we must then ask the
which scanned the line looking for evidence of certain types further question: In virtue of what principle does it have the
of terminating vertices. Evidence for an "arrow" vertex claimed competence to perceive form from motion? In fact, it
accumulates sooner in the scan than does evidence for a turns out that the appropriate order to ask these two questions
"fork" vertex, because the secondary arrow lines enter the is the opposite to the one given above. For it is only by trying
scan region even before the actual point of intersection does. to discern the principle governing the alleged ability that we
Thus the system recognizes the end of the line segment earlier can be sure that the system does indeed have that ability -
in scanning toward it when the segment terminates with an that it does more than to "account for variance" or to mimic
arrow than when it terminates with a fork, hence yielding a some segment of the behavior without embodying the
lower estimate of its length in the former case. By adjusting competence in question.
the diameter of the scan region, this method can account for The problem of inferring shape from motion is not solvable
some quantitative properties of the illusion reasonably well. in general without bringing in additional constraints, because
Now in this example it is very clear that the phenomenon is three-dimensional shape is underdetermined by its two-
associated with the assumption of a diameter-limited line dimensional orthographic projection. But the imposition of
scan. Thus, whether this particular account of the illusion is extrinsic constraints on this inference should be, as we have
classed as a valid serendipitous finding or merely a fortuitous emphasized, principled. In particular, it should be such as to
coincidence depends very much on whether the assumption guarantee a unique solution in all cases where a unique
concerning the mechanism, or the architectural property of interpretation occurs perceptually. Ullman (1979) proposed
the detector, can survive empirical scrutiny. that the constraint be viewed as a warranted assumption
Although this example may seem fanciful, it is nonetheless made by the interpretation scheme. He calls it the rigidity
the case that every computational model, if it is taken assumption, because it enjoins the system to interpret the
literally, must make assumptions about the mechanism. These moving elements in the two-dimensional projection as origi-
are frequently not taken to be empirical hypotheses, since it nating from a rigid body in motion, and to fail if such an
can easily escape our notice that some of the system's interpretation is not possible (i.e., as a first approximation, not
performance is attributable to certain assumed architectural to produce an interpretation of the elements as belonging to
features. Our experience with a rather narrow range of an elastic or fluid medium). This is analogous to an assump-
possible computational architectures can blind us to the fact tion of grammatically made in interpreting natural
that our algorithms are relative to such architectures (as we language. In that case the system would be asked to interpret
shall see in section 8). Furthermore, when the assumptions are a string as an instance of a structure generated by the
exposed and analysed, they do not always seem so plausible. A grammar, and to fail if such an interpretation is not possible.
particular case worth mentioning concerns the architectural Of course, in both cases a more elaborate model might
assumptions underlying the successful use of lists and two- produce analyses of the deviant cases as well (i.e. nonrigid
dimensional matrices to model aspects of reasoning and the and nongrammatical interpretations). However, this would

122 THE BEHAVIORAL AND BRAIN SCIENCES (1980). 3

 Pylyshyn: Computation and cognition
not be done by abandoning the assumption completely, but This level of description lies somewhere between the
rather by considering systematic departures from the strict physical and the representational, in the sense that, unlike the
form of the assumption (even ungrammatical sentences must physical description, it is independent of the particular
be analysed in terms of grammatical rules, rather than simply physical laws that characterize any one physical realization of
in terms of such considerations as what is usually true of the the system, and, unlike the usual algorithmic or rules-
referents of the terms, otherwise we could not understand a plus-representations description, it is an uninterpreted rule
sentence about anything unexpected). schema that can be exploited, by an appropriate choice of
The rigidity assumption is warranted by two further initial expression and interpretation scheme, to actually carry
findings. The first is the mathematical result (called the out some intended algorithm. It thus serves as the interface
structure from motion theorem), showing that a set of four between the two. The physics of the device, together with a
noncoplanar points in a rigid configuration is uniquely specified mapping from classes of physical states onto
determined by three distinct orthographic views. The second expressions, defines the functional architecture. In the
finding is that the interpretation is in fact computable by a computer case we can then view the role of such utility
reasonable locally-parallel procedure, given the sorts of data software as assemblers, loaders, compilers, interpreters, and
available to the human visual system, and under approxi- operating systems as providing various realizations of this
mately those conditions in which people do perceive three- mapping, and hence as defining (or emulating) different
dimensional shape. functional architectures.
The lesson of this example is that by first understanding The notion of functional architecture has a special role to
what the demands of the task are, it is possible to specify play in computer science, where it is sometimes referred to as
constraints that must be met by any algorithm capable of "the architecture of the underlying virtual machine." When
successfully carrying out that task. This, in turn, makes it writing programs for some particular computer, program-
possible to answer the question why a certain algorithm has mers only have the resources provided by some particular
the claimed ability. Without this extra stage of theoretical programming language available to them. They are (to a first
analysis we would be unable to say why some particular approximation - neglecting such practical factors as cost and
algorithm appeared to work on the set of problems on which resource limits) unconcerned about how the real physical
we tried it. What is even more serious, however, is that we device operates at the hardware level, since what they can do
could not even say, with any degree of precision or confi- with the system is fixed by the functional specification of the
dence, what the class of tasks was that the algorithm could language. This specification consists of the sorts of things that
handle. While this might not be considered a shortcoming if are contained in the language user's manual - e.g., a list of the
we were merely concerned to account for experimental available operations and what they do, restrictions on how the
variance (i.e. for observational adequacy), it would be a operations can be put together, how the contents of memory
serious defect if our goal was to provide a theoretical can be accessed, how arguments are to be passed to functions,
explanation of some domain of competence (i.e., if we were how control is transferred, and so on. This functional
concerned with explanatory adequacy). If we know neither specification defines the programmer's virtual machine,
what the scope of a model's abilities is, nor what the principles which a programmer cannot distinguish from a real machine
are in virtue of which it behaves as it does, we do not have the without approaching the device in a quite different mode
basis for explanation. (e.g., looking in the systems manual or examining the switches
To develop the analogous point with respect to the other and lights on the console). As far as the programmer is
class of requirements on the explanatory use of computational concerned the device may well be wired to function exactly
models (viz., that the functional architecture be separately as the language manual specifies (indeed, contemporary
constrained), we shall first examine the notion of functional computers are frequently designed to execute programs in
architecture itself. LISP or PASCAL or some other high-level programming
language at very nearly the level of real hardware). Thus,
even though there may in fact be several layers of program
8. The influence of architecture and notation interpretation between the programmer and the actual
on processes hardware, only the properties of the virtual machine archi-
tecture are relevant to the user, because that is the only level
Computation is generally understood to be completely inde- to which the user has uniform access (i.e., any changes to the
pendent of the particular way it is physically realized. After machine's functioning can be made only by utilizing the
all, a program can be executed by an unlimited variety of facilities defined by this virtual machine).
quite different physical devices, operating on quite different For the cognitive scientist a similar distinction between the
physical principles and in radically different physical media, functional architecture of the "cognitive virtual machine "
such as mechanical, optical, acoustical, fluid, or any other and the mental algorithm is important, though the principle
conceivable substance (even including a group of trained for distinguishing the two must be stated in a slightly
pigeons!). On the other hand, the way in which the device different form, since it will ultimately depend on empirical
functions is critical in determining whether it is capable of criteria. The functional architecture is, by assumption, that
executing an algorithm. The design of a physical system that part of the system that is fixed in a certain respect (which we
can function as a computer is no simple matter. But in view of shall specify in the next section when we examine the notion
the unboundedness of the variety of physical forms it can of cognitive architecture more closely), and that is also,
take, one might well ask what it is about the structure of the hopefully, universal to the species. Mental algorithms are
device, or class of devices, that makes it a computer? To viewed as being executed by this functional architecture. In
answer this question we must recognize a level of description view of the fact that a valid cognitive model must execute the
of the device intermediate between its description as a same algorithms as those carried out by subjects, and in view
physical system (governed by the appropriate physical laws of the fact that (as we shall see below) which algorithms can
that determine its physical state transitions) and its descrip- be carried out (keeping in mind that algorithms are indi-
tion as a representational system (in which its behavior is viduated according to the criterion of strong equivalence)
governed by the rules and representations it embodies). This depends on the functional architecture of the device, and
furthermore, in view of the fact that electronic computers
is the level we have been calling the functional architec-
clearly have a very different functional architecture from
ture.

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 123

Pylyshyn: Computation and cognition
that of minds, we would expect that in constructing a other words, it can compute the same lookup function, but
computer model the mental architecture will first have to be not with the same complexity profile, and hence not by using
emulated (i.e. itself modelled) before the mental algorithm an algorithm that is complexity-equivalent to the hash-coding
can be implemented. algorithm. Of course, it could be made to compute the
Consider, for example, how we would go about using a function by simulating the individual steps of the register
computational system as a cognitive model. First of all, in machine's algorithm, but in that case the Turing machine
order to describe an algorithm so it can be viewed as a literal would be emulating the architecture of the register machine
model of a cognitive process, we must present it in some and executing the algorithm in the emulated architecture, a
standard or canonical form or notation. Typically this means very different matter from computing it directly by the
formulating it as a program in some programming language, Turing machine. The distinction between executing an
but it might also include graphical presentation (as a algorithm and emulating a functional architecture is crucial
flowchart) or even a discursive natural language description. to cognitive science, as we have already remarked, because it
Now, what is typically overlooked when we do this is the relates directly to the question of which aspects of the
extent to which the class of algorithms that can even be computation can be taken literally as part of the model, and
considered is conditioned by the assumptions we make which aspects are mere technical implementation details,
regarding what basic operations are possible, how these may necessitated by the fact that at the level of actual hardware,
interact, how operations are sequenced, what data structures production-model electronic computers have a functional
are possible, and so on. Such assumptions are an intrinsic part architecture different from that of brains.
of our choice of descriptive formalism, since the latter defines Examples of the architecture-specificity of algorithms
what we are calling the functional architecture of the could be easily multiplied. For example, a register machine
system. that has arithmetic operations and predicates among its
What is remarkable is that the range of computer archi- primitive operations (and hence can use numerals as names -
tectures available for our consideration is extremely narrow, or, as they are more frequently called, "addresses") makes a
compared with what could in principle be considered. variety of additional algorithms possible, including binary
Virtually all the widely available architectures are basically of search (in which the set of remaining options is reduced by a
the Von Neumann type, or closely related to it. This goes for fraction with each comparison, as in the game "twenty
both hardware (which uses serially organized, stored-list questions"). The existence of arithmetic primitives as part of
programs and location-addressable memory) and software the architecture also means that it is possible to specify a total
(see a discussion of the latter in Backus 1978). Because our ordering on names, and hence to primitively partition certain
experience has been with such a rather narrow range of search spaces (as in an n-dimensional matrix data structue), so
architectures, we tend to associate the notion of computation, that search can be confined within a region while other
and hence of algorithm, with the class of algorithms that can regions are literally not considered - items in those regions
be realized by architectures in this limited class. For example, are not even checked and discarded, as they would have to be
we tend to think of flow diagrams as a neutral way of if they were merely part of an unordered set of items. Such
exhibiting algorithms. This is the idea behind the TOTE unit algorithms could not be implemented on a Turing machine
of Miller, Galanter, and Pribram (1960). But flow diagrams architecture.
(and TOTE units) are totally inappropriate as a way of As we go to more unusual architectures, other algorithms -
characterizing algorithms implemented on unconventional with quite different complexity profiles - become possible.
(non-Von-Neumann) architectures - such as, for example, the For example, Scott Fahlman (1979) has proposed a design for
less familiar architecture of production systems, Planner-like an architecture (realizable only with unconventional hard-
languages, or predicate calculus-based programming systems ware) that computes set intersection as a primitive operation
like PROLOG (see Bobrow and Raphael 1974 for a discussion (in time independent of set size). He argues that many
of some of these languages). If we use the criterion of strong otherwise complex combinatorial algorithms required for
equivalence (even just the psychologically important subset of symbolic pattern recognition (i.e., for access to stored data
this equivalence relation, which I referred to as complexity- through multiple descriptions) become simple in such an
equivalence) to individuate algorithms, we will find that architecture. In other words, because this architecture allows
different architectures are in general not capable of executing interesting new classes of algorithms to be implemented, the
strongly equivalent algorithms. locus of the difficulty of giving a computational account of a
This point is best illustrated by considering examples of certain task domain is dramatically shifted. Fahlman also
several simple architectures. The most primitive machine argues that the resulting complexity profiles of certain
architecture is no doubt the original binary-coded Turing memory-retrieval processes in this architecture are more like
machine introduced by Turing (1936). Although this machine those of "natural intelligence." Although he does not use the
is universal, in the sense that it can be programmed to methodology of the psychological laboratory (viz. reaction
compute any computable function, anyone who has tried to times), the goal is similar. Along the same lines, Mitch Marcus
write procedures for it will attest to the fact that most (1977) has also proposed certain general architectural features
computations are extremely complex. More importantly, of the processor associated with grammatical analysis, which
however, the complexity varies with such things as the task have the interesting consequence that they provide a princi-
and the nature of the input in ways that are quite different pled account of certain linguistic universals in terms of
from the case of machines with a more conventional archi- general architectural constraints.
tecture. For example, the number of basic steps required to As we have already remarked, even when information-
look up a string of symbols in such a Turing machine processing theorists make no claims about the form of the
increases as the square of the number of strings stored. On the functional architecture, and attempt merely to develop
other hand, in what is called a register architecture (in which models of certain cognitive processes, they cannot in fact
retrieving a symbol by name or by "reference" is a primitive avoid making certain tacit assumptions about the underlying
operation) the time complexity can, under certain conditions, architecture. Furthermore, the implicit adoption of some
be made independent of the number of strings stored. A particular architecture carries with it certain further assump-
register architecture can execute certain algorithms (e.g. the tions about the nature of mental functions. To see exactly
hash-coding lookup algorithm) that are impossible in the what is being further assumed tacitly when a certain
Turing machine - in spite of the fact that the Turing machine architecture is adopted, it is useful to look at properties of the
can be made to be weakly equivalent to this algorithm. In architecture in a more abstract way, and to ask what it is

124 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
about it that makes it possible to carry out certain functions in John is shorter than Fred. Who is tallest?"), because all they
certain ways. A fruitful way to view this question is in terms have to do is arrange the three items in order (either in a list
of the formal or mathematical type of the primitive opera- or in an image) and read the answer off. But, of course, the
tions built into the virtual machine. For example, the fact that one can solve the problem this way does not entail
primitive relation that we call reference (or the operation of that tacit knowledge of formal properties (e.g. transitivity) of
retrieving a symbol given another symbol that serves as its the relation "taller than" is not needed, since the decision to
name) is of the formal type asymmetric. It can be used to represent "taller" by something like "further on the list" must
define operations for computing other relations such as, for have been based on the implicit recognition that the two
example, the operation we might call AFTER, which relations were of the same formal type (a list would not, for
computes a relation that is irreflexive, antisymmetric, tran- example, have been suitable to represent the relation "is
sitive, and connected over some specified set of names. Such a married to"). Furthermore, while ordering three names in a
syntactic operation in the functional architecture can be list and then examining the list for the position of a particular
freely interpreted, at the level of interpreted rules, as any name may seem straightforward and free from logical
relation of the same formal type, for example the relation of deduction, a little thought will show that the ability to carry
being of higher rank, or older, or being a superset of (a out this operation mentally, as distinct from physically,
relation frequently used in semantic hierarchies known as ISA presupposes a great deal about the available primitive mental
trees). In making such an interpretation we automatically operations. For example, in the mental case, if we have the
inherit the formal properties of these built-in primitive items A, B, and C, and we place A and B in a certain order
relations. Thus we do not need to explicitly represent and then add C next in the sequence, we must assume (in this
properties such as the reflexivity, symmetry, transitivity, example) that: a) placing C next to B leaves the relation
noncyclicality, and so on, of the relation in the interpreted between A and B unchanged, and b) the relation of A to C
domain - i.e., we do not need to represent symbolically a rule (with B between them) is the same with respect to the
such as "If X is older than Y, and Y is older than Z, then X is relevant represented aspect (e.g. tallness) as that between A
older than Z," if we have chosen to represent "older" by a and B. But such assumptions are justifiable only if the agent in
primitive relation of the architecture that is of the appro- question implicitly understands the logical properties of
priate formal type. succession. Consequently, even if list operations are part of
In an architecture that has what we call an "arithmetic the functional architecture (which, as we saw earlier, assumes
unit" (really only a set of functional properties that are useful that the architecture incorporates primitive operations of the
for, among other things, representing the syntactic rules appropriate formal type), one is still not entitled to assume
relating numerals under certain mathematically interpretable that the use of this capacity requires no further appeal to tacit
operations), as well as an ordering relation over symbols, there knowledge of logical constructs. Furthermore, as Piaget has
is an even richer set of formal types to exploit in constructing suggested, it may well be that either the availability of the
an interpreted system of rules. For example, certain formal relevant primitive operation or the tacit knowledge relevant
properties of the Peano axioms for arithmetic, as well as the to its appropriate use may develop with the maturation of the
metric axioms, can be modelled in terms of these primitive organism. If this were so, we might then wish to represent a
"arithmetic" relations. This means that, in a sense, certain logical property like transitivity as an explicit logical rule,
aspects of metric scales are available in such an architecture. rather than building it into the architecture [see Brainerd:
In fact, since the axioms of Euclidean geometry have a model "The Stage Question in Cognitive-Developmental Theory"
in the real numbers, such an architecture allows us to choose BBS 1(2) 1978].
an interpretation scheme that makes geometric properties To take another timely example, matrix data structures
available without the need to represent geometric axioms have frequently been used to represent the spatial properties
symbolically (e.g., if we interpret pairs of numerals as of images (e.g. Kosslyn and Schwartz 1977; Funt 1977). This
locations of points in space and use the "arithmetic" opera- is a convenient way to represent spatial layout, partly because
tions to define distances and movements through this space). we tend to think of matrices in spatial terms anyway. In
If we interpret primitive relations in this way, a variety of addition, however, this structure seems to make certain
spatial and metric properties can be represented, changed, consequences available without any apparent need for certain
and their consequences inferred without the need for deductive steps involving reference to knowledge of geome-
symbolic computation. try. For example, when we represent the locations of
This sort of exploitation of the functional architecture of imagined places in our model by filling in cells of a matrix,
computational systems is central to computer science, as we we can "read off" facts such as which places are adjacent,
have already noted. From the point of view of cognitive which places are "left of" or "right of" or "above" or
science, however, it is not only important to choose functional "below" a given place, and which places are "in between" a
properties of the computational architecture in such a way as given pair of places. Furthermore, when a particular object is
to accomplish certain tasks efficiently, but it is equally moved to a new place, its spatial relations to other places need
important to be explicit about why it works, and to justify not be recomputed. In an important sense this is implicit in
these crucial properties independently. That is, it is important the data structure. Such properties make the matrix a much
for the use of computational models in an explanatory mode, more natural representation than, say, a list of assertions
rather than simply a performance mode, that we not take specifying the shape of objects and their locations relative to
certain architectural features for granted simply because they other objects.
happen to be available in our computer language. We must
first explicitly acknowledge that certain noncomputational But, as in the case of the apparently noninferential
properties originate with certain assumed properties of the consequences of using lists, such properties of matrices arise
functional architecture, and then we must attempt to empir- from the existence of certain formal properties of particular
ically motivate and justify such assumptions. Otherwise functional architectures. These properties would not, for
important features of our model may be left resting on instance, be available in a Turing machine architecture. In
adventitious and unmotivated assumptions. order for a matrix data structure with the desired properties
to be realizable, the architecture must provide at least the
For example, people have occasionally suggested that primitive capacity to address the content of a representation
subjects do not need to have knowledge of concepts such as, by place - i.e., it must be possible to name a location and to
say, transitivity, in making certain inferences, as in the ask for the content of a named location. This itself may
three-term series problems ("John is taller than Mary and require, for instance, what is known as a register architecture

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 125

Pylyshyn: Computation and cognition
(or some other kind of location-addressable store). Further- formal properties built into a notation, or the greater the
more, it must be possible in this architecture to primitively number of primitively fixed formal properties of the func-
generate the names of places adjacent to a given place (i.e., it tional architecture that must be exploited, the weaker the
must be possible to do this without appealing to other expressive power, and the more constrained will be the
representations or to tacit knowledge of geometry or anything resulting computational system. This is because we no longer
else that would involve intermediate inferential steps). This is have the option of changing such properties at will. In
necessary in order to allow the representation to be choosing a particular notation or architecture, we are making
"scanned." In addition there must be primitive predicates a commitment concerning which aspects of the functions are
that, when applied to names, evaluate the relative directions to be viewed as the free parameters that are tailored to fit
of places corresponding to those names (e.g. two-place specific situations, and which are to be viewed as the fixed
predicates such as "RIGHT-OF" must be primitive in the properties shared by all functions in the class of models that
architecture). This, in turn, implies that there are at least two can be constructed using that notation. The more constrained
independent, implicit, total orderings over the set of names. a notation or architecture, the greater the explanatory power
In addition, if the relative distance between places is to be of resulting models. It provides a principled rationale for why
significant in this representation, then there might be further the model takes one particular form, as opposed to other
primitive operations that can be applied to place names so as logically possible ones. Recall that the lack of such a rationale
to evaluate, say, relative size (e.g. the predicate "LARGER- was one of the features that made some computational models
THAN"). ad hoc. One goal in developing explanatory cognitive models,
This whole array of formal properties is available in all then, would be to fix as many properties as possible by
common computer architectures, because they all use numer- building them into the fixed functional architecture. Oppos-
ical expressions for register (i.e. place) names and have ing this goal, however, is the need to account for the
built-in primitive arithmetic operations. But these are part of remarkable flexibility of human cognition. We shall see in
such architectures for reasons that have nothing to do with the section 10 that this character of cognition provides the
needs of cognitive science. When these features are exploited strongest reason for attributing much of its manifested
in building cognitive models, we are tacitly assuming that behavior to tacit knowledge of various kinds rather than to
such operations are part of the functional architecture of the the sorts of fixed functional properties that have frequently
mind - an assumption that clearly needs to be justified. been proposed.
Arguments have rarely been provided for any such proposals.
The only suggestions of an argument for such architectural
features that I have seen are due to Piaget, who has been 9. Cognitive functional architecture
concerned with abstract formal characteristics of cognition,
and Brouwer (1964) and Nicod (1970), who, for quite The architecture-algorithm distinction is central to the
different reasons, proposed that succession be viewed as a project of using computational models as part of an expla-
cognitive primitive. nation of cognitive phenomena. The core notion of strong
The general point concerning the intimate relation equivalence depends upon there being a common architec-
between virtual machine architecture and process is exactly ture among processes. Processes can only be compared if their
the same as the observation that different notations for a grain or "level of aggregation" [to use Newell and Simon's
formal system can lead to different expressive powers and (1972) phrase] is the same. In fact, even the formal semantic
even different axioms for the same system. For example, if we properties of programs, as developed by Scott and Strachey
use conventional notation for algebraic expressions, we need (1971) are relative to an abstract model of computation that,
to explicitly state facts about the associativity and precedence in effect, specifies an appropriate grain for the analysis.
of arithmetic operators, whereas in Polish notation we do not Furthermore, the privileged vocabulary claim (described in
need to represent such properties explicitly, because, in a section 4) asserts that cognitive phenomena can be accounted
sense, they are implicit in the notation. Similarly, proposi- for solely by appealing to the symbolic representations (i.e the
tional logic normally contains axioms for commutativity and algorithm and its associated data structures). Thus, any
for the complementarity expressed in de Morgan's principles. differences among such phenomena arise solely from the
However, if we use the disjunctive normal form for logical structure of these symbol systems - from the way component
expressions, such axioms need not be stated, because, as in the parts are put together. Thus, no distinctions among phenom-
algebraic case, they are also implicit in the notation. Mechan- ena that we would classify as cognitive distinctions can be due
ical theorem-proving exploits a variety of such intrinsic to such things as differences in the way the primitive
formal properties of both the notation (e.g. using the operations that constitute the algorithm themselves function.
disjunctive normal form) and the virtual machine architec- Furthermore, if differences in cognitive processing between
ture. For example, such theorem-provers can be made much individuals are to be explained within our computational
more efficient and natural in certain domains by representing framework, it is necessary that the basic functional architec-
sets of propositions in the form of "semantic nets," which ture be universal.
exploit the formal properties of the reference relation Mental architecture can be viewed as consisting of just
available in typical register machines (i.e in the usual Von those functions or basic operations of mental processing that
Neumann architectures). are themselves not given a process explanation. Thus they are
From the point of view of cognitive science, the notation functions instantiated in the biological medium. Unlike
we choose is important for reasons that go beyond questions cognitive functions in general, they are, on the one hand, the
of efficiency or naturalness. Because we claim that behavior is primitive functions appealed to in characterizing cognition,
governed by symbolically encoded rules and representations, and on the other hand, they are functions that are themselves
the exact format or notation that is used to encode these explainable biologically, rather than in terms of rules and
representations consitutes a claim about mental structures, representations. Thus we see that the architecture-algorithm
and hence an empirical claim. Formats, like functional distinction parallels one we made earlier between functions
architectures, become empirically decidable in principle if symbolically computed. Since we gave, as two of the
we admit the relevance of the criterion of strong equivalence, conditions for the appropriateness of the latter way of
and if we develop appropriate methodologies based on this characterizing a function, that the relation between anteced-
criterion. ent conditions and subsequent behavior be arbitrary and
It is important to recognize that the greater the number of informationally plastic (or cognitively penetrable), these

126 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
criteria will consequently be relevant to distinguishing with cognitive phenomena. Hence the proprietary vocabu-
between functions attributable to the architecture and func- lary hypothesis and the representation-governed behavior
tions attributable to the structure of the algorithm and the arguments converge on the same principled distinction,
cognitive representations. This is our answer to Wittgenstein's which we take as defining the architecture/algorithm bound-
"third man" puzzle. We do not need a regress of levels of ary.
interpreters, with each one interpreting the rules of the Condition (2) can also be viewed in a way that brings out its
higher level and each in turn following its own rules of relation to the distinction between automatic and attentional
interpretation. Nor do we need to view our computational processes, which has attracted considerable interest recently
model as consisting of a cascade of "intentional instantia- (e.g. Posner and Snyder 1975; Schneider and Shiffrin 1977).
tions," as Haugeland (1978) does. Only one uniform level of Condition (2) entails the claim that functions attributable to
rules is followed, since only one symbolic level of cognitively the functional architecture can consume only constant
interpreted representations and of cognitive process is resources. In other words, they must not include what are
involved. However complex the remaining functions are, referred to as "attentional" processes. For any differences in
they are considered to be instantiated by the underlying resource use must arise from different execution traces of
biological medium - i.e., they are part of the functional cognitive processes, or different algorithmic sequences.
architecture. The reader will note a certain inevitable Hence functions that exhibit such differences do not qualify
circularity in the above discussion of the relation between as part of the fixed architecture. I have discussed this issue in
cognition and the architecture/algorithm boundary. On the connection with Anderson's (1978) behavioral mimicry claim
one hand, cognitive phenomena are understood as those (Pylyshyn, 1979b), where I pointed out some of the conse-
which, under the appropriate interpretation of observations, quences of allowing varying complexity (e.g. as indexed by
can be accounted for solely by examining the algorithm. On reaction time) to be exhibited by primitive operations (or
the other hand, the distinction between what is attributable to individual steps) in a process.
the algorithm and what is attributable to the functional We thus conclude that a basic methodological distinguish-
architecture is to be decided on the basis of whether certain ing mark of functions that are part of the basic fixed
phenomena (namely cognitive ones) can be accounted for functional architecture of the mind is that they cannot be
while keeping the architecture fixed. This circularity is not, influenced in their operation by what might be called
however, vicious. The basic core intuitions concerning what informational or cognitive factors, nor do variations in their
constitutes a cognitive phenomenon will not have to be operation need to be posted in order to account for observed
revised willy-nilly every time a new algorithm is formulated. cognitive phenomena. Such functions and properties remain,
While the domain of the theory will evolve gradually to to use a phrase that I have adopted in other contexts (e.g.
accommodate the evolving theoretical system, yet at any time Pylyshyn 1979a; 1979b), cognitively impenetrable.
there will be phenomena clearly identifiable as cognitive. The criterion of cognitive impenetrability serves as the
These are the ones that will adjudicate whether or not some litmus by which we can decide whether a function is a fixed,
function is legitimately viewed as part of the mental archi- built-in, causally explainable, primitive operation or property
tecture that is to serve as the fixed primitive basis for in terms of which cognitive processes are to be described, or
constructing specific algorithms. whether it will itself require a computational or process
We may therefore summarize the conditions under which explanation. It is clear why such a criterion is needed.
a specific function or behavioral property may be attributed Without it one could not distinguish between a literal and a
to properties of the fixed functional architecture as follows: 1) metaphorical appeal to a computational model. By providing
If the form of the hypothetical function or the putative a principled boundary between the "software" and the
property can be systematically influenced by purely cognitive "hardware," or between functions that must be explained
factors, such as changes in instructions or in the information- mentally (i.e. computationally), and those that can only be
bearing aspects of the context, or any other condition that explained biologically, one can factor the explantion into the
clearly leads to differences in goals, beliefs, interpretations, fixed biological components and the more variable symbolic
and so on; or 2) if we must postulate variations in the form of or rule-governed components. Like the factoring of fixed
the hypothetical function or in the putative property in order universal constants from particular conditions specific to the
to account for certain systematic differences in observations case at hand, which occurs in all physical explanations, or the
of cognitive phenomena; then such a function or property factoring of linguistic universals (also taken to represent fixed
may not be attributed to properties of the fixed functional properties of mind), from language-specific rules and
architecture (or the "medium"). Consequently, if it is not comprehension algorithms, which occurs in models of linguis-
attributable to a property of the functional architecture, it tic competence, such factoring is essential for accounts to
must be given a cognitive account that appeals to the achieve explanatory adequacy.
structure of the process and to the content and form of the Although, as I have repeatedly stressed, the explanation of
representations. This cognitive process model itself will, as we how the primitive properties of the functional architecture
noted earlier, have to acknowledge explicitly the contribution are realized will ultimately be given in biological or physical
of the task demands in setting the goals and in accounting for terms, this should not be interpreted as meaning either that
the skill exhibited by the model. such properties must be stated in a biological vocabulary, or
Conditions (1) and (2) above are required not only because that they are to be inferred from biological observations. It
of our proprietary vocabulary hypothesis, but also by the has sometimes been claimed (e.g. Anderson 1978; Palmer
arguments raised earlier regarding when it is appropriate to 1978) that only biological data could help us decide between
invoke rules and representations to explain observed behav- certain theoretical proposals for fixed properties of the
ior. Notice how closely conditions (1) relates to the informa- cognitive system - such as between analogical and proposi-
tional plasticity criterion of rule-governed behavior. The tional forms of representation. It should be clear from the
informal notion of informational plasticity refers to the current discussion of functional architecture that this is not so.
property of a system in virtue of which certain aspects of its Although the architecture represents a crucial interface
behavior can be systematically altered by information, and between properties requiring a cognitive process account and
hence by how the organism encodes stimuli or interprets those requiring a biological account, the actual description of
events. We argued that the explanation of such behavior this architecture is a functional one. It simply specifies the
should appeal to rules and representations. Such a property is, primitive functions or fixed symbol-manipulation operations
of course, also one of the clearest signs that we are dealing of the cognitive system. Furthermore, the point of the

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 127

Pylyshyn: Computation and cognition
cognitive impenetrability condition is to provide a purely biological and technological systems). However, he had too
functional methodology for deciding whether a putative limited a view of how topographic factors could operate.
property qualifies as belonging in the caregory of architec- Perhaps because of the types of artifacts with which he was
ture or in the category of cognitive process. familiar, Kohler took the spatial distribution of constraints to
Finally, before concluding this section I must again be the primary focus of the distinction. What this fails to
reiterate the very strong contrast between the position I have recognize, however, is that in certain systems (e.g. computers)
been describing and psychophysical dualism. According to the functional structure can still change radically, even when
the present view, two distinct types of explanation of human the topographical structure remains fixed. Functionally, the
behavior are possible. One of these is naturalistic (i.e., it part-whole relation in this kind of system is such that global
appeals directly to intrinsic properties of the organism) and discontinuities in function are produced by appropriate local
the other cognitive (i.e., it appeals to internal representations). changes in structure. Such propagation of local changes to
I have tried to sketch some general criteria under which each produce systematic global effects is what Kohler believed
is appropriate. But for many people there still remains the would require a different sort of system - one governed
uneasy question of why it should be the case that a certain primarily by nonanatomical dynamic factors such as field
class of systems (e.g. certain organisms and computers) admit effects [see also Puccetti & Dykes: "Sensory Cortex and the
of these two analyses, while other (perhaps equally complex) Mind-Brain Problem" BBS 1(3) 1978].
systems (e.g space vehicles) do not. Since the minutest details Although Kohler may even have been right in his analysis
of the operation of all systems are clearly governed by causal of the structural nature of biological as opposed to artifactual
physical laws, one might ask what distinguishes these two systems, the realization of computational processes in topo-
classes of system. graphically fixed systems shows that this particular dimension
In its most general form this question runs into the problem of difference is not a logical prerequisite for representation-
of the intentionality of human cognition - a problem that is governed behavior. Rather, what seems necessary is just what
clearly beyond the scope of the present essay. This is the was hinted at above - namely, that the fixed factors, regard-
question of what it is about a mental representation or a less of whether they are due to the topography or to laws of
thought that makes it a thought about some particular thing interaction of forces within prescribed boundary conditions
or to any other fixed constraints, enable a certain kind of
rather than another. For example, since the Lowenheim-
radical second-order flexibility. That flexibility might be
Skolem theorem assures us that any consistent formal system characterized in terms of the global alteration of function that
has a model in the integers, any such formal system could be can be effected through local changes, or in terms of the
just as legitimately viewed as representing, say, the natural existence of instantaneous functional networks that can be
numbers as anything else we care to name. In his accompa- varied over and above the fixed topographical network, or in
nying target article, Fodor (this issue) suggests that an some other way. I do not know of a satisfactory way of
essential methodological strategy of cognitive psychology is to precisely specifying this function in abstract terms, but I
factor away the intentionality question and develop models believe that it amounts to the requirement that the functional
that are concerned primarily with coherence. Interesting architecture be universal in Turing's sense - i.e., that it be
questions still remain, even if we somehow factor away the capable of computing any computable function or of simu-
difficult issue of intentionality from the issue of epistemic lating any Turing machine. Whatever the correct general
mediation. However, even in that case we can still ask what it specification of this function is, it must distinguish between
is about certain systems that makes them candidates for an the long-term, structurally fixed functions that we have called
epistemic mediation account that appeals to representations the functional architecture, and the instantaneously alterable
and rules. functions that are necessary to sustain rule-governed behavior
Among the exogenous factors to which we have already and to enable behavior to change radically in response to such
alluded is the human imperative to explain cognitive behav- transitory effects as the inferences that the system makes
iors in terms that parallel the form in which we plan and upon receipt of new information. Only a system that has this
conceive of our own behavior. Perhaps as a direct conse- character can be a candidate for a rule-governed system.
quence of this it also seems that the appropriate regularities in What further requirements it must meet beyond that I cannot
our behavior are to be found when it is described in terms of say, but clearly this particular aspect of rule-governed
what Kenneth Pike (1967) referred to as an emic as opposed behavior does not raise any problems for a materialist view of
to an etic taxonomy (i.e., intensional as opposed to extension- mind.
al, conceptual as opposed to objective, or perceptual as
opposed to physical). Of course this applies only when the
phenomena to be explained are themselves cast in such
terms - as they must be in the case of meaningful human 10. The psychological relevance of the notion
behavior. Thus, to explain how the mouth and tongue move of functional architecture
and how acoustical energy patterns are generated, we appeal
to the taxonomy and laws of physics and perhaps biology, In this section I shall discuss some of the implications of the
whereas to explain what someone was saying at the time and present view for the development of cognitive theory. In the
their choice of words, we must appeal to cognitive concepts next section I will consider the traditional importance of
and processes. something like the notion of functional architecture in
But even if we accept such reasons for turning to a psychology and examine several "fixed functions" that might
representational account of human behavior, there still be viewed as proposed properties of such an architecture.
remains the question of what intrinsic properties humans and The analysis we have been giving can be viewed as setting
computers share that make behavior fitting such an account out a particular program of research - namely that of
possible. Many years ago Kohler (1929) offered some sugges- designing a cognitive virtual machine, or rather a system or
tions on this question. He distinguished between processes programming language having a functional architecture
that were determined by anatomically fixed, or as he called appropriate for implementing cognitive algorithms. Such a
them, "topographical factors," and those that were deter- machine (which, of course, would have to be emulated on
mined by nonlocalizable or "dynamic factors" such as "forces available computers) would, among other things, display the
and other factors inherent in the processes of the system." I appropriate resource-limited constraints characteristic of
believe that Kohler was on the right track in this analysis human processing. For example, it might conceivably be
(which he used in arguing for certain differences between possible to implement various algorithms on such a machine

128 THE BEHAVIORAL AND BRAIN SCIENCES (1980). 3

 Pylyshyn: Computation and cognition
for carrying out essentially the same function but with mind that every computational model commits one to
different trade-offs. It might, for instance, be possible to have implicit assumptions concerning the underlying architecture
an algorithm that attended to (i.e. whose behavior depended can keep one from making at least some of the more
on) a larger number of symbolic expressions, in exchange for problematic assumptions. As one example of this approach,
requiring more steps (thus trading off speed and memory Pylyshyn, Elcock, Marmor, and Sander (1978) report some
load), or one that took fewer steps but failed to attend to some highly preliminary results of a research project aimed at
of the potentially relevant data (thus perhaps exhibiting a understanding aspects of perceptual-motor coordination as
speed/accuracy trade-off). these are involved in drawing diagrams and making infer-
As we have already pointed out, such an architecture ences from what is seen. This project was primarily concerned
would also make possible the goal of designing algorithms with certain design issues that arise when one avoids certain
that were strongly equivalent to ones used by humans. technically simple, but in our view cognitively implausible,
Computational complexity profiles of processes, as the latter ways of dealing with the problem of representing spatial
ranged over various systematic changes in inputs, could relations in a computational model, especially in the context
simply be read off from properties of the execution of the of perceptual-motor coordination [see Gyr et al.: "Motor-
algorithms such as the number of primitive steps they took in Sensory Feedback and Geometry of Visual Space" BBS 2(1)
each case. Such complexity features would now be empiri- 1979]. When experimental data concerning some particular
cally significant, rather than merely incidental properties of human cognitive function were not available, we followed the
the model. These could then be compared with various strategy of adopting the least powerful mechanism we could
hypothesized empirical correlates of human processing devise to carry out that function.
complexity - such as reaction times, perhaps. In the ideal case For example, instead of maintaining a matrix structure as a
every operation and every feature of algorithms implemented global representation of the spatial layout, our system only
on such an architecture would constitute an empirical used a qualitative representation of spatial relations, together
hypothesis, just as every theorem derivable from Newtonian with a limited number of pointers to specific places on the
axioms constitutes an empirical prediction. One would not say retina. Coordination with the motor system was accomplished
of an algorithm executable on this architecture that the mind through a minimal mechanism, which was able to maintain a
cannot compute such a function since, by hypothesis, process- "cross-modality" binding between visual and kinesthetic
ing constraints such as resource limitations, or any other fixed spaces for only two places. We thought of these as "fingers,"
universal limitation in processing capacity, would have been whose position (when they were on the retina) could be seen,
incorporated into the architecture. Thus every algorithm that and whose location in proprioceptive space could continue to
could be executed on this virtual machine would now be be sensed even when they were off the retina. Thus we had a
considered a humanly possible cognitive process, just as any primitive ability to hold on to off-retinal locations in order to
grammar generatable by an ideal universal grammar (in glance back to them. This "two-finger" mechanism made
Chomsky's sense) is a possible structure of a human (or possible a quite general drawing capability, which extended
humanly accessible) language. beyond the foveal region. The general strategy was to opt for
This fact itself has the interesting consequence that it minimal mechanisms wherever possible, even at the cost of a
eliminates a certain asymmetry between what appear to be computationally awkward system, on the grounds that such
two radically different ways in which the rules and repre- mechanisms committed one to the weakest presuppositions
sentations in a model could be modified. On the one hand about the underlying architecture, and hence ones that could
there are the changes that come about as a consequence of easily be revised upward without the need to radically
various "inputs" to the model. These produce only orderly redesign the system. A number of such minimal mechanisms
and psychologically appropriate modifications in representa- were proposed and are described in the Pylyshyn et al. (1978)
tional states. On the other hand there are the changes that paper. This is an instance of the application of the principle of
come about when the programmer intervenes by adding or "least committment" to the design project. Newell's
deleting rules or inserting new representations. Since in production-system architecture proposal, referred to above,
conventional programming systems these changes are not and Marr and Nishihara's (1977) minimal mechanism for
constrained in any way by psychological considerations, they rotating the principle axes of a three-dimensional model to
could result in the most bizarre and humanly inaccessible bring it into congruence with a given two-dimensional retinal
algorithms being specified. With the functional architecture pattern, could both be viewed as examples of this strate-
appropriately constrained, however, this distinction in large gy-
part disappears, since all and only cognitively possible In addition to these more ambitious goals of applying the
(though for various reasons perhaps not actual) algorithms idea of a cognitive virtual machine directly to the design task,
and representations are permitted, regardless of their gene- one can also appeal to it in deciding the soundness of certain
sis. proposals concerning the nature of cognitive processing. It is
The most concerted effort at designing a cognitive virtual useful to have a general criterion for distinguishing among
machine (or at least at developing design specifications for an classes of mechanisms when these are proposed as compo-
appropriate functional architecture) is currently being nents of a cognitive model or a cognitive explanation. For
pursued by Allen Newell. The present form of this design is example, it is useful to have a principled basis for deciding
called a "production system." There are some interesting whether a certain cognitive phenomenon ought to be viewed
reasons for this particular choice, which we shall not go into as arising from the nature of some analogue representational
here. They relate, in part, to Newells concern with modelling medium, as frequently claimed, or whether it will require an
certain resource-consuming properties of computation, which explanation in terms of the nature of the symbolic cognitive
in conventional systems are not visible to the user but remain process itself. In the latter case the properties we observe
a part of the "backstage" activity of the virtual machine - might be seen as arising from tacit knowledge, rather than
namely, the control structure itself. The reader is referred to from the nature of the mechanism or medium, as we
Newell (1973; in press) for a technical discussion of these suggested earlier was the case for the "mental scanning"
issues. phenomena.
However, we should note that the view expressed here on Recall that one of the features of analogues was that they
the importance of the functional architecture does not were nonsymbolic and noncomputational. Analogue repre-
commit one to the project of designing a complete cognitive sentations, as generally conceived, are not articulated
virtual machine as the first step. Indeed, simply having in symbolic expressions, and analogue processes are not viewed

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 129

Pylyshyn: Computation and cognition
as rule-governed symbolic computations. They can thus be it agreed with our informal observation of certain aggregate
viewed as characteristics of the functional architecture. I (i.e. statistical) regularities of behavior, as well as of behavior
believe that it is this quality of incorporating fixed constraints in certain highly controlled situations.
into the architecture, and therefore of weakening the expres- It is no accident that the controlled situations that were
sive power and consequently increasing the explanatory value investigated from the conditioning perspective universally
of the models, that makes analogue systems particularly involved the suppression of what we have been calling
attractive in cognitive science. It is not the fact that such cognitive factors. The experimental paradigm was always
systems may be more efficient or more natural, nor even that contrived in such a way that beliefs, goals, inferences, or
they may be continuous or holistic in some sense, that ought to interpretations were rendered irrelevant as much as possible.
be their attraction. None of these are essential properties of Furthermore, critical observations were invariably expressed
analogues: analogue processes don't have to be efficient, nor as frequencies or probabilities, thus averaging over any
must they be continuous, as opposed to quantized, to retain remaining cognitive effects or strategies. However, cognitive
their analogue character (e.g., imagine your favorite analogue factors inevitably still left their effect in the case of research
model approximated by a discrete but finely grained quan- involving human subjects. In those cases, as Brewer (1974) has
tization). What they do seem to require is that the formal eloquently argued, the most plausible explanation of human
property that characterizes their function be explainable as conditioning phenomena is one given in terms of change in
being instantiated or exhibited, rather than as being symbol- belief. In other words, the most straightforward explanation
ically computed by the operation of rules on symbolic of what reinforcers do is that they inform the subject of the
representations. Thus, whether some particular function contingent utilities. Thus, for example, the same effects can
should be explained as a noncomputational or as an analogue typically be produced by other ways of persuasively provid-
process is the same kind of question we raised earlier in this ing the same information (such as explaining to subjects the
essay; namely, whether the appropriate explanation of some conditions under which they will or will not be shocked and
behavior is one that appeals to rules and representations or backing up the story by showing them the wiring).
merely to primitive dispositions. An examination of a number However, I don't doubt that there are ways of incorporat-
of proposed analogue mechanisms suggests that, as formu- ing such results into the conditioning account (especially since
lated (i.e. in their simplest and most attractive form), they all individuation criteria for "stimulus," "response," or "rein-
exhibit some degree of cognitive penetration and hence are forcer " are unspecified within the theory, allowing our
not eligible as part of the noncomputational function of the informal "folk psychology" to come to our aid in describing
system. Examples such as those discussed by Pylyshyn (1979a; the situation appropriately and thus smuggling knowledge of
1979c) considerably weaken the case for the existence of a cognitive factors into the predictions). Yet, whatever
large degree of nonsymbolic processing in cognition, at least approach one takes to explaining such phenomena, Brewer's
of the kind that has frequently been proposed in discussions review and analysis of a large number of studies makes it
of, say, imagery phenomena. Every case of cognitive pene- clear that even the simplest and most paradigmatic cases of
tration is an instance in which we are forced to relax the conditioning in humans (e.g. avoidance conditioning of finger
hypothesized fixed functional constraints. While it will withdrawal to shock, or conditioning of the eyeblink reflex)
always be possible in principle to build very flexible infer- exhibit cognitive penetration - i.e., they are radically, yet
ential processes, which at various arbitrary points in their systematically, influenced by what the subjects believe (or by
operation turn to highly constrained subsystems (as, for what they are told or shown). Thus even these simplest cases
example, in the suggestion put forth by Kosslyn et al. 1979, do not demonstrate that conditioning is a primitive function
for a way to retain the holistic analogue rotation in the face of of the fixed architecture. As before (e.g. the mental rotation
the data I reported), it is much more problematic to justify case mentioned earlier), attempts to retain this function as a
such hybrid models. Such proposals work only by subverting primitive might well be possible at the cost of considerably
the main motivation for positing such fixed architectural weakening the claims that can be made about the role of
capacities as, say, the ability to "mentally rotate" an image - conditioning-e.g., by relegating it to the role of one small
namely the greater explanatory power inherent in the element in a large cognitive process involving rules, repre-
weakest or least expressive system. In such hybrid models we sentations, goals, inferences, and so on. But then the burden of
would need to have strong independent reasons for retaining proof falls on those who posit this mechanism to show that it is
the (now redundant) constrained subprocesses. still necessary, given the flexibility of the remainder of the
system that has to be posited in any case.
An intrinsic part of the conditioning account - and the only
11. The search for fixed architectural functions part that could conceivably explain novel behavior, is the
notion of generalization. A more sophisticated contemporary
If we take a broad view of the notion of functional version of the generalization gradient is the similarity space.
architecture, we can recognize that proposals for components Just as conditioning requires dimensions along which it can
of such an architecture are not infrequent in psychological generalize to new stimuli, so some contemporary theories that
theorizing - in fact, they have often characterized the differ- appeal to prototypes require dimensions of similarity in order
ences among psychological schools. For example, the domi- to account for the relations among prototypes as well as
nant assumption, from the time of the British empiricists until between prototypes and various exemplars or instances.
about fifteen years ago (and even today in some quarters), Among those theoretical approaches that seek to avoid the
was that the principal built-in functional capacity of orga- complexities of inferential and problem-solving processes in
nisms was their ability to form associative links. For this accounting for such phenomena as recognition and classifi-
assumption to be predictive it was further necessary to specify cation (e.g. Rosch 1973), the most common proposal is the
the sorts of entities over which the associations could be functional similarity space. Even Quine (1977) speaks of a
formed (e.g., for behaviorists these had to be behaviorally biologically endowed "quality space." Though no account is
defined), as well as conditions on the formation and evocation given of how such a space might be realized (presumably
of these links (e.g. conditions such as contiguity, reinforce- neurophysiologically), or how the location of novel stimuli in
ment, generalization gradients, and so on). Such a hypothe- the space is determined, it can still be viewed as a possible
tical capacity, like many of the more contemporary nonbe- component of the functional architecture. Thus it is appro-
priate once again to inquire whether this view can be
haviorist proposals [cf. Bindra: "How Adaptive Behavior is
empirically sustained.
Produced" BBS 1(1) 1978], was intuitively appealing because

130 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3

 Pylyshyn: Computation and cognition
The first thing to notice is that such a view cannot be penetration demonstrated in studies of selective attention. It
applied to stimuli such as sentences, since it is clear that there seems as though every time one investigator has proposed an
is an unbounded variety of ways in which sentences can attention-selection filter of one sort, another investigator has
relevantly differ - i.e., there is no finite set of dimensions or found evidence that the operation of that filter was sensitive
categories of comparison that can exhaustively locate the to aspects of the information that the filter was supposed to
meaning of a sentence in a similarity space. While it seems have eliminated, and that therefore must have been getting
likely that the same is true of visual patterns, the existence of through (see Norman 1969 for a review of some of these
at least some quantitative dimensions of similarity (e.g. size, results). That is why, in an early model of selective attention,
orientation, color-distance) makes it not as clear as in the Norman (1968) found it necessary to include a factor he
language case. On the other hand, the very strong interactions called "pertinence," which affects the recognition threshold
among such putative dimensions - demonstrated repeatedly of the postfilter processing. As the name suggests, the
by the Gestaltists - could also be cited against the view that introduction of the pertinence factor is nothing but an
these can be treated as orthogonal dimensions in a similarity admission that such processing is cognitively penetrable.
space. One of the clearest and simplest demonstrations of the The various proposals that go by the title of "direct
inadequacy of the similarity space view, however, comes perception," as developed by J. J. Gibson (1979) and others,
from the work of one of the pioneers of the multidimensional can be viewed as proposals for functions that are part of the
scaling technique. Shepard (1964) showed that when stimuli fixed functional architecture of the perceptual system. The
varied along several dimensions, judgments of their similarity denial by this school of any "epistemic mediation" in
could yield ratings that conform to a Euclidean metric along perception makes it particularly clear that they view func-
any of the dimensions of variation, depending on what tions, such as those that are said to "pick up information"
subjects are instructed to attend to. But when the subjects concerning perceptual invariants, as being instantiated by the
were not given specific attention instructions and were left to functional architecture. But such proposals have generally not
attend to whatever they wished, the resulting data failed to withstood the test being proposed here: the detection of
conform to any metric (Euclidean or nonEuclidean) in the everything from distance information to one's grandmother
number of dimensions along which the stimuli varied. appears to be cognitively penetrable.
Shepard concluded that subjects' noticing strategy deter- Howard (1978) has shown that even the perception of so
mined the similarity structure of their judgements - and these obvious a property of a display as the horizontality of a
were free to move from one possible similarity space to colored fluid in a transparent container is strongly influenced
another at will. In other words, the similarity space function is by knowledge of the relevant physical principle. In his
itself cognitively penetrable. studies, conservatively conducted using trick forms of three-
Once again, one could probably get around this sort of dimensional photographs and motion pictures, with the
counterexample by allowing a cognitive component to pre- method of random presentation, Howard made the startling
analyse the stimulus prior to the use of the similarity space. discovery that over half the population of undergraduate
But as in the previous case, in which such added cognitive subjects he tested were unable to recognize anomalous
processes had to be posited, it is no longer clear what function stimuli, despite the fact that some of the mistakenly classified
the space would now be serving, other than as an ad hoc pictures depicted deviations of fluid levels as much as thirty
parameter to increase the precision of prediction. For, if we degrees off horizontal, and despite the fact that the percep-
need a cognitive process to analyse and oversee the recogni- tual discriminability of the orientations involved was clearly
tion, we already have a mechanism that, at least in principle, above threshold, as evidenced by other methods of assess-
is capable of accounting for the similarity structure of the set ment. For example, the same subjects who failed the
of stimuli. Again the burden would be on those who posit such horizontality test could readily report when the surface of the
a fixed function to show exactly what principled role it plays fluid was not parallel to shelves visible in the background.
(not just in providing similarity judgements, but as a general What was even more surprising and relevant to the present
representational system). discussion was that postexperimental interviews, scored
The latter result is only one of a large number of cases in blindly by two independent judges, revealed that every
psychophysics in which attempts to posit fixed psychophysi- subject who recognized all anomalous stimuli that were out by
cal functions have run into difficulty because they turned out at least five degrees could clearly articulate the principle of
to be cognitively penetrable. Almost any psychophysical fluid level invariance, whereas no subject who failed to
judgement that requires the subject to attend selectively to recognize such stimuli as deviant gave even a hint of
certain specific aspects of the stimulus, while ignoring other understanding the relevant principle. What is especially
aspects, is likely to be cognitively penetrable. One of the best surprising in these studies was that evidence of knowledge of
known examples of a simple function that turned out to be the relevant principle was obtainable by verbal probing.
cognitively penetrable is the simple sensory threshold. It was Usually the inference that tacit knowledge is involved is
shown to be penetrable by subjects' beliefs concerning the much less direct - as is the case for phonological rules and for
utilities of alternative responses - a finding that generated at least some instances of syntactic rules.
considerable interest in the theory of signal detectability (a In a similar vein I have argued for the cognitive penetra-
decision-theoretic and therefore cognitive analysis) as an bility of visual (or "imaginal") memory (Pylyshyn 1973;
alternative to the threshold function. 1978c), of the proposed analogue function of "mental
Similarly, it has been argued (e.g. Segall, Campbell, and rotation" of images (Pylyshyn 1979a), and of the spatiality of
Herskowits 1963) that visual phenomena such as the Muller- images as inferred from "mental scanning" studies (Pylyshyn
Lyer illusion are also penetrable, though not as directly as the 1979c; also in section 5 above).
threshold function. The argument is that in time the function The point of these examples is not to suggest that all mental
responsible for the illusion can be influenced by cognitive functions are cognitively penetrable. Indeed, I do not see how
experience (in particular by experience with large three- a computational view of cognition could be developed if that
dimensional rectangular shapes such as buildings). The "new were so. (I don't mean that it would be impossible; only that it
look" movement in perception in the 1950's (e.g. Bruner would require a rather different conception of functional
1957) was also built upon the recognition that a great deal of architecture, and hence of algorithmic process.) The point is
the perceptual process is penetrable. Among the better known merely to suggest that many of the more constraining (and
experimental results in cognitive psychology that could be hence potentially more explanatory) proposals fail because of
viewed in this way are the various instances of cognitive the flexibility of human cognition.

THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3 131

Pylyshyn: Computation and cognition
Of course, there are numerous examples of proposals that happens when certain small changes are made to an object
have not failed the test of penetrability (at least not yet). Most (e.g. in the Shepard and Feng 1972 case, what happens when
of these involve more peripheral functions, and most of them a single fold is made in a sheet of paper - see the discussion of
have been studied only relatively recently. For example, the this interpretation in Pylyshyn 1978c), or subjects may simply
work of Marr (1979) and associates provide some rather clear have been in the habit of doing it in some particular way for
examples of complex processes that do not appear to be one reason or another. The point is that mere statistical
penetrable. These involve early visual processes (e.g. deriva- regularities need not tell us anything about the nature of
tion of what Marr calls the "raw primal sketch" from mental structures. More significant theoretical questions arise
incoming visual information, combining information from when we inquire into which of these regularities are invio-
two retinal inputs to derive stereoptically encoded structures, lable because they arise from fixed properties of mind - i.e.
derivation of certain textural and form information from from the functional architecture.
retinally local patterns, and so on). At the motor end there has Rather than placing the emphasis on explaining regularities
similarly been considerable success in demonstrating cogni- that may well reflect little more than aggregate averages over
tively impenetrable functions (the earliest of which goes back various habits of thought, we ought to be far more impressed
to the work by Bernstein 1967). [See Roland, BBS 1(1) with the extreme flexibility that thought can exhibit. For
1978.] example, we ought to take seriously the fact that there seems
It is rather more difficult to point to examples of more to be no specifiable limit to what the human mind can
central processes that are clear instances of impenetrable imagine or think. As George Miller recently remarked to me,
architectural functions. Perhaps this is because central cogni- the salient property of mental life is surely the fact that we
tive processes are more like deductions - in which virtually can will it to do practically anything we wish: given the
any two concepts can be connected by some inference path appropriate goals and beliefs, we can alter our behavior and
our thoughts to a remarkable extent by a mere act of will.
under the appropriate circumstances. Thus I suspect that the
Although psychology has typically focused on the things we
semantic net proposals (including the organization of the cannot do well or on the habitual patterns of behavior we
lexicon in Morton's (1970) Logogen model) could be shown to display, one should not lose sight of the fact that a psycho-
be penetrable, though I know of no direct evidence bearing logical theory of any interest will also have to account for
on this question at the moment. My own suspicions are that the fact that most of these patterns and limitations can be
the functions that will be considered part of the functional overcome, and hence that they tell us little about the
architecture of such higher level processes as thinking and underlying cognitive system. Thus, however uncomfortable
common sense reasoning will be of two distinct kinds. On the may be the possibility that very many of the functions that
one hand there will be extremely primitive elementary have been studied by psychologists are not fixed mecha-
symbol processing operations, though they are unlikely to be nisms - in the sense that they are cognitively penetrable - we
the sorts of operations found in contemporary serial digital must be prepared to recognize that what is universal and
computers. On the other hand there will be extremely fixed about the human mind may be very different from the
abstract constraints on data structures and on the control sorts of gross functions that we readily infer from the patterns
system. Further, the primitive functions needed may be quite we observe in behavior. It could well turn out - and indeed it
different in different parts of the cognitive system, though seems extremely likely, judging from the sorts of consider-
there may well be common resource allocation mechanisms ations explored in this paper - that the major problem in
(e.g. a common type of control structure). understanding how cognition proceeds will be to explain how
The apparent lack of highly constrained functional prop- the vast tacit knowledge at our disposal is organized and how
erties of the sort that our folk psychology might lead us to it is brought to bear in determining our thoughts, our
expect should not surprise us. This is one of the areas where imaginings, and our actions. To do this will require that we
we are seeking deeper explanatory principles and hence pay special attention to formalisms adequate to the task of
where folk psychology is least likely to be of service. The representing the relevant knowledge, rather than primarily
comparable case in which some success has been achieved in addressing the issue of how typical behavior might be
seeking highly constrained and universal properties is in the generatable. This, in turn, presupposes that we take seriously
case of linguistic universals or universal grammar. There it such distinctions as between competence and performance
has become obvious that properties of this sort could only be (see Chomsky 1964; Pylyshyn 1977) or, to use McCarthy and
found at extremely high levels of abstractness, and at a Hayes's (1969) term, the distinction between the epistemo-
considerable deductive distance from our intuitions and logical and the heuristic problems of intelligence. One
observations. There is no reason to expect the situation to be considerable advantage that the computational view of
any different in the case of other areas of cognition. cognition gives us is the potential to explore these more
Furthermore, to expect that observations such as those abstract issues formally and to work with longer deductive
associated with the study of mental imagery will provide a chains between observations and explanatory principles.
direct route to the functional architecture of mind is to vastly
underestimate the flexibility of mental processes. Even in
those cases in which, out of habit or for some other reason "Present address: Departments of Psychology and Computer
(such as, for example, a certain typical way of interpreting Science, The University of Western Ontario, London, Ontar-
the task demands of imagery tasks), it turns out that people io, Canada N6A 5C2
very frequently do things in certain ways when they image,
this need not be of any special theoretical significance, since it ACKNOWLEDGMENTS
may not be due to any general properties of mind, but This paper was written while the author was a visiting fellow at the
perhaps to certain long-standing habits. For example, when Center for Cognitive Science, Massachusetts Institute of Technology.
we find that people typically solve problems by imagining The support of M.I.T. and the Social Science and Humanities
that they are viewing a sequence of events in more detail than Research Council of Canada is gratefully acknowledged.
is actually needed to find the solution, this fact itself could be I also wish to acknowledge my indebtedness to Allen Newell who
due to some relatively uninteresting reason (from a theoret- provided the inspiration for several of (he ideas developed here. In
ical standpoint). For example, it may be that this is what particular the notion of mental architecturefirstarose from Newell's
subjects believed they were supposed to do; or it may simply work on production systems and its importance became clear to me
reflect a logical way of decomposing the task so as to make after some discussions with Newell at a workshop at Carnegie-Mellon
use of knowledge of certain elementary facts, such as what University in 1973.

132 THE BEHAVIORAL AND BRAIN SCIENCES (1980), 3