Computer

semiotics

Peter Bgh Andersen Department of Information and Media Science, University of Aarhus, Niels Juelsgade 84, DK-8200 rhus, Denmark. Abstract This paper presents semiotics as a framework for u n d e r s t a n d i n g and designing computer systems as sign systems. Although semiotic methods can be applied to all levels of computer systems, they view computer systems under a particular perspective, n a m e l y as targets of interpretations. When we need to see c o m p u t e r systems as automata, semiotics has little to offer. The main f o c u s of the paper is semiosis, the process of sign formation a n d interpretation. The paper discusses different semiotic paradigms, and advocates the European structuralist paradigm in c o m b i n a t i o n with the American Peircean tradition. Programming is described a s a process of sign-creation, and a semiotic approach to p r o gramming is compared to the object-oriented method. T h e importance of the work situation as a context of interpretation is emphasized

Keywords: Semiotics, design, work programming, computer-based sign

language,

object-oriented

1.

Introduction

Semiotics, "the science of the life of signs within society" a s Saussure (1966) defined it, is a general theoretical framework f o r analyzing and understanding a diverse range of p h e n o m e n a : language, film, theater, pictures, architecture, clothings, gestures, etc. Their common denominator is that they are used as signs; t h e y stand for something else than themselves. This paper sketches a possible discipline of computer semiotics (Andersen 1990a, Figge 1991). It is a discipline that analyze computer systems and their context of use under a specific perspective, namely as signs that users interpret to m e a n something.

2 Peter Bgh Andersen Within this perspectives. global perspective, we can define four local

Program development Interface design

Multimedia applications

Signs as art(ifacts)

Systems Description

Signs Signs as as system

Signs as behavior

Signs Signs as knowledge

Computer Support for Collaborative Work Work analysis Organizational analysis Technology assessment

Cognitive science Cognitive ergonomics

Fig. 1. A map of computer semiotics. Adapted from Halliday 1978. The center is signs as system. The individual is considered as a creator, interpreter and referent of signs, as a user and r e p r o d u c e r of a common meaning potential and code, utilizing the results of a semiotic labor done by others. The focus of this box is sign systems as social phenomena with a structure that cannot be changed a t will. Systems analysis, design and implementation aims at creating computer based sign systems that will typically be used by a whole organisation. Signs as knowledge. The individual is considered as a n assemblage of parts: his biological psychophysiological nature, t h e psychological mechanisms that enable the individual to learn, u s e and understand signs. In traditional linguistics these issues a r e treated by psycholinguistics. In the case of computer-based signs, the role is taken over by cognitive science and cognitive ergonomics that study what goes on in the mind and body of t h e individual. Signs as behavior. The individual is considered as a single, indivisible entity, and the focus is on his interactions with t h e environment, especially that part which consists of c o m m u n i c a t i o n with other individuals. Sociolinguistics and pragmatics work with these questions. Parts of the new field of Computer S u p p o r t e d Collaborative Work fills the box.

Semiotics as a basis for a humanistic computer science

Signs as art(ifacts). The individual is considered as an i n n o v a t o r of code and meaning potential, as an explorer and inventor o f signs. In this box, we should distinguish between code-creation i n general (signs as artifacts) and its particular variants: to invent a new style of art (signs as art) is not the same as inventing n e w designs of articles for everyday use. Part of the design p r o c e s s belong here, in particular design that aims a creating new kinds o f systems. The new multimedia technology requires skills of t h e designer that allows us to remove the parenthesis and talk a b o u t signs as art. Semiotics methods have been suggested and used by several authors. (Goldkuhl & Lyytinen 1982, Winograd & Flores 1 9 8 6 ) regard computers a tools for performing speech acts like promising, requesting, etc, predominantly viewing signs a s behavior. (Rasmussen 1986) is a good representative for " t h e signs as knowledge" perspective; he uses semiotics for understanding the relation between the sign types needed by a control room operator and the kind of cognition he is engaged in. Signs as systems are treated by (Piotrowsky 1990, Decls 1 9 8 9 , Andersen 1990a). Decls describes machine architecture from a semiotic point of view, while Piotrowsky and Andersen use t h e structuralist school of glossematics as a theoretical basis f o r understanding computer systems. (Boland 1991) presents hermeneutic methods for understanding users' interpretation o f the interface, and (Nadin 1988, Andersen 1991a) address aesthetic issues of interface design. I believe semiotics to be a global perspective on c o m p u t e r systems; all levels of a system can be treated semiotically, but like all perspectives it can only treat a subset of the total set o f relevant properties systematically. In this sense it is c o m p l e m e n tary to the mathematical perspective of natural science, since issues that can be treated systematically in one, is typically o u t s i d e the range of the other. The reason for this is that the two traditions are biased i n different ways. Whereas natural science focuses on the mechanical aspects of the system those aspects that can be treated as a n automaton semiotics focuses on the interpretative aspects. Semiotics must necessarily view computer systems as sign-vehicles whose main function is to be perceived and interpreted by s o m e group of users. It has nothing to say about data in itself, only in i t s capacity of being interpreted and used as a source of knowledge o r guide for action. If you want reliable methods for calculating t i m e complexity of algorithms or proving correctness of p r o g r a m s , semiotics will be a disappointment.

4 Peter Bgh Andersen Semiotic theory can only make scientific statements a b o u t phenomena that are used as signs to stand for something else f o r somebody. Although some semioticians will claim that all h u m a n behavior is sign behavior, I personally believe there are m a n y aspects of behavior that cannot sensibly be classified in this way. Tacit knowledge is one example if it means: knowledge that c a n n o t be expressed in any way. Neither is physical work whose purpose is not communication but production of commodities directly analyzable by semiotic methods, although valuable i n f o r m a t i o n about the workers' conception of their work can be gained b y analyzing the way they talk about it. The statement about the globality of semiotic analysis entails that it is not confined to "soft" areas like interface design and u s e r friendliness, but that it can also treat aspects of the technical construction of computer systems. At first this might sound implausible. After all, a computer is a machine, and we certainly do not want to use semiotics as a basis for constructing other kinds of machines like cars, v a c u u m cleaners, and lawn-mowers. However, computer systems are not ordinary machines, a s s e m bled by means of bolts and screws. They are symbolic m a c h i n e s constructed and controlled by means of signs. The interface o f systems is of course one obvious example of a computer b a s e d sign. A sign stands for something to somebody in some r e s p e c t , and since the interface of a flight reservation system stands f o r flights and seats to the clerk, this interface is clearly a sign. Using the system involves interpretation and manipulation of text a n d pictures. But underneath the interface, in the intestines of the system, w e find other signs. The system itself is specified by a program t e x t (that is a sign since it stands for the set of possible program executions to the programmer). The actual execution involves a c o m piler or interpreter that controls the computer by means of t h e program text, and since the compiler is a text standing for the s e t of permissible program texts, the compiler is also a sign in f a c t it is a meta-sign that in some versions very much r e s e m b l e s an ordinary grammar. If we continue this descent through the different layers of t h e system, passing through the operating system and the assembly code, down to the actual machine code, we will encounter signs most of the way. As we reach the machine code things m a y possibly change. The machine code is unique in that its s h a p e physically influences the machine without any mediating layers. Chunks of machine-code may not stand for anything else t h a n

Semiotics as a basis for a humanistic computer science

themselves, so we seem to have arrived at a stage where signs become mere signals. But even if it can be argued that the bottom layer of a c o m p u t e r system does not exhibit sign behavior, everything on top of that is clearly used as signs by some group of professionals. There a r e always texts that must be interpreted as statements or p r e s c r i p tions about some present or future state of the system. As w e change level, the concepts signified by the texts change. On t h e lower levels, the meaning of the signs are related to the physical parts of the machine, like registers and storage cells . As we a s c e n d , the texts are interpreted differently, we move away from a physical interpretation, and new software concepts appear, like r u n - t i m e stacks, heaps, and variables . A total picture of the whole system will depict semiotic activities from the top down to the very bottom of the system. A c o m p u t e r system can be seen as a complex network of signs, and every level contains aspects that can be treated semiotically. This globality thesis is characteristic of the approach illustrated in this paper. Since semiosis , sign creation and sign i n t e r p r e t a t i o n , takes places everywhere in the system, the paper does not only address user interpretations of systems (Section 4) but also t a k e s an interest in the formal construction of the computer-based signs that enter into this semiosis (Sections 3.2 - 3.3). Both users a n d programmers interpret the system, and the clash between t h e s e two kinds of interpretations is an interesting topic. In fact, t h e main purpose of the paper is to contribute to a framework f o r connecting systems interpretation with systems design. Another characteristics is the emphasis on empirical r e s e a r c h and analysis. We know so little about sign systems and sign u s a g e that speculative methods and constructed examples should be u s e d with caution. Finally, this paper is primarily interested in signs as systems, t h a t is: as properties of a social group, not of the individual m i n d . However, I see no reason to exclude the cognitive traditions treating signs as a vehicle for individual cognition, nor signs as a means for interacting with other people or objects. These perspectives do not exclude but supplement each other. The interest in signs as social phenomena, in empirical work, a n d in the technical construction of systems necessitates a discussion of suitable theoretical frameworks. This is done in the next section.

6 Peter Bgh Andersen

2. Which Semiotic Theory?

There are many kinds of semiotic and linguistic theories. In order to be useful as the theoretical framework we need m u s t satisfy the following criteria, adapted from (Bannon 1989): it must be applicable to linguistic field work, not only constructed examples and must respect actual language u s a g e as the basis of analysis, because we want to use it for building systems that must work in a real environment;

it must view language as a social phenomenon used f o r communicating and coordinating work, not only as a phenomenon of the individual mind, because most work is social, involving cooperation and communication; it must be founded on a rigorous theory, because otherwise the research process will quickly degenerate; it must give understanding of creative use of signs, a n understanding parts of which must be formalizable, since design of computer systems is a creative process, and creative use of computer-based signs involves formalization.

In the following I shall reject two popular paradigms, t h e generative and the logical one, and advocate a third one, t h e European structuralist tradition.

2.1. The Generative Paradigm

The generative paradigm is relevant because it (or variants of it, like the augmented transition networks or case-grammars) h a s shown very immediate uses as a basis for programming n a t u r a l language interfaces or machine-translation systems. In fact, t h e theory of formal languages and automata it helped building h a s become a standard part of a computer science curricula. The generative paradigm was founded by Noam Chomsky (Chomsky 1957, 1965, 1968, 1976, 1980) in 1957) and was originally built around one simple question: which algorithms a r e needed to determine whether a string is a member of a formal language, i.e. a possibly infinite set of strings? By applying descriptive techniques of hitherto unprecedented rigor, Chomsky established a set of formal grammars, and argued - against the dominating behavioristic tradition of the time that natural language exhibited a complexity that required a much more c o m p l e x grammar than posited by the behaviorists. Since Chomsky at t h e same time offered new and more precise descriptive t e c h n i q u e s

Semiotics as a basis for a humanistic computer science

and a mentalistic, non-behavioristic ideology to go with them, h i s ideas revolutionized the linguistic world in a few years. The revolution consisted in insisting on explicit description, s o that only those sentences actually specified by a set of rules c o u n t as described. The grammars are modelled on formal logical p r o o f s , and consist of an axiom and a set of rules. A grammar is said t o generate all and only those strings that can be derived from t h e axiom by applying a sequence of one or more rules to it. T h e grammars are defined so precisely that it becomes possible to give mathematical proofs of the descriptive capacity of different k i n d s of grammars (Hopcroft and Ullman 1969). As can be imagined, this added rigor was seen as a progress b y many linguists. But with time it became clear that the descriptions produced ran a serious risk of drowning possible insights i n enormous masses of rules, so Chomsky initiated a s e r i o u s discussion about the quality of the descriptions. This d e v e l o p m e n t is important since it shows that although precision may b e desirable, it does not automatically provide insight in the subject. Chomsky's strategy in recent years has been to attribute as m u c h of the linguistic description as possible to a universal g r a m m a r , trying to show that many linguistic facts do not belong to t h e description of individual languages at all, but characterizes t h e human faculty of language, the amount of universal rules being a measure of the insight gained. Developing formal characterizations of the human faculty of language has become an important aim o f the school. Even though this short description does not do full justice to t h e theory, it can be concluded that the generative paradigm is n o t what we are looking for. Although generative grammar is built on a formalized rigorous body of theory, it is focused on the individual language user, not on the social process of communication. It is only concerned with the mental faculties, not the physical skills, and it provides few bridges from the linguistic to the non-linguistic domains. In addition, the most usual source of data is examples and counter-examples constructed by linguists, not a u t h e n t i c speech. Finally, the paradigm has very little to say about creative use of language, linguistic inventions, which is an important part o f design. The reason is that the theory basically sees language as a rule-defined set of sentences. The critique of the generative paradigm is summarized by (Halliday 1985):

8 Peter Bgh Andersen A language is not a well-defined system, and cannot be e q u a t e d with "the set of all grammatical sentences", whether that set is conceived of as finite or infinite. Hence a language cannot b e interpreted by rules defining such a set. A language is a semiotic system (...)- what I have often called a "meaning potential"(...). Linguistics is about how people exchange meanings by "languaging". Halliday 1985: 7

2.2 The logical Paradigm: Language as Reasoning

The next possibility is the logical paradigm which is really o l d e r than the generative one. It was founded by Frege, systematized i n the thirties and forties by Rudolf Carnap and Ludwig Wittgenstein and later given a linguistic turn by e.g. Hans Reichenbach (Reichenbach 1947). However, only in recent time, with the logical grammars of Richard Montague (Montague 1976), it was fashioned in such a way as to count as a kind of linguistic theory. Like p a r t s of generative grammar, the logical theory of language has a l s o become an integrated part of computer science. Since an important part of the logical paradigm consists i n translating natural language sentences into logical formulas o n which rules of inference can operate, it can be used as a theoretical basis for constructing language understanding systems. One way of constructing such systems is to represent knowledge i n terms of logical statements, to translate queries into c o r r e s p o n d i n g logical formulas, and then let the system try to prove the q u e r y from the knowledge base. The main objection towards using logical grammar is that it is not a linguistic theory at all, since it does not describe factual b u t idealized linguistic behavior. This is a drawback if we want to u s e the analysis to learn about the concepts through which a profession understands its work, assuming that a good understanding of this is a prerequisite for designing u s a b l e systems. We want to analyze language from within ("immanently"), not as a projection of something else. If we use logic as our p o i n t of departure, we defeat this purpose, since logic and n a t u r a l language semantics are two different systems, developed for different aims; for example, logic uses two main conjunctions, , o f t e n translated and , and , translated or . However, the logical conjunctions and their linguistic counterparts behave differently i n many respects. Natural language or normally has an exclusive sense, meaning "one out of several", while means "one or both o f two". It is of course possible to define an exclusive o r - c o n j u n c t i o n

Semiotics as a basis for a humanistic computer science

but this is beside the point. The point is that logical or is n o n exclusive because it is convenient for constructing logical proofs t o have two conjunctions that obey certain rules, e.g. de Morgans law , (A B) (A B). But since language users do not spend t h e i r time constructing proofs, their semantic system cannot apriori b e expected to contain the same semantic elements. Logicians perform other tasks than other language users, so t h e i r professional language is under constraints absent in other kinds o f language. An immanent structuralist semantics cannot apriori accept t h e logical concepts like quantifiers " x, $ x, connectors , , etc. a s basic semantic units, since the purpose of the semantic research is precisely to establish the identity of these self-same units. The only way of establishing the units is to investigate whether they have a function in language. Using logic as a method for describing t h e semantics of natural language is like trying to use the tonal scale t o describe its phonology. In both cases, we impose distinctions o n language from activities that are different from those p e r f o r m e d by language users. Here logic, there music. Therefore, a l t h o u g h logic grammar may masquerade as a sort of linguistic theory b y using linguistic terms like syntax, semantics, and lexicon, it is not a linguistic discipline. From this point of view, logic and logic grammar should b e viewed as a professional activity that seeks to discover general laws of valid inferences. Montague grammar can be seen as a g o o d method for comparing results from formal logic to results f r o m semantics, comparisons that can yield valuable insights in b o t h areas, just as comparisons between a linguistic analysis of c o l o r terms and a psychological description of our visual discrimination capabilities can do. To summarize: in spite of their popularity in linguistic work with computers, both the generative and logical paradigms have b e e n rejected as a general theoretical framework, not because they a r e bad theories, but simply because their general goals are different from the ones we want to accomplish. This does not mean a wholesale rejection of the approaches a n d insights they represent, only a delimitation of their range o f applicability. Just as logical grammar is useful in systems w h e r e logical proof procedures are relevant, so is generative g r a m m a r relevant in those cases where formal manipulation of s t r u c t u r e d strings of characters is demanded.

10 Peter Bgh Andersen

2.3. The European Structuralist Paradigm: Language as Creation of Meaning

The generative and the logical paradigm - and their m a n y variations - have had a large impact on linguistic debate from t h e sixties and onward. However, in our connection a better c h o i c e seems to be the European structuralist paradigm characterized b y such names as Ferdinand de Saussure, Louis Hjelmslev, Michael Halliday, and Umberto Eco. In particular systemic grammar founded by Halliday is interesting because of the following properties: Its data is to a large degree gathered from authentic language usage in real situations, and tape recordings are often u s e d . Thus the paradigm is well suited for field-work. Language is basically seen as a social phenomenon and is described according to the functions people use it for in r e a l life. There exist theories that relate language to situations a n d social roles and classes. This makes the theory suitable f o r describing communication in work situations. The semantic aspect of language has highest priority. Language is seen as a meaning potential, a set of alternatives from which language users can choose when they create meaning. Although the emphasis is on semantics, there are fairly explicit rules for relating meanings to observable expressions, and t h e lifeline to the traditional body of knowledge about syntax a n d morphology is kept intact. This means that the theory can b e used in practical textual analysis. Because of its emphasis on the language use and its attempts t o relate language to social structures, systemic grammar can s e r v e many, but not all of the needs mentioned in the beginning of t h i s section. One problem is that systemic grammar does not offer s u p port for active design and creative use of signs. A n o t h e r inadequacy is that systemic grammar, with a few exceptions, is n o t used to describe other codes than verbal ones and it is difficult t o judge if it can be generalized to cover non-verbal signs also. Technological developments in recent years has made this a r e a l drawback, since pictorial codes are now commonly used i n interfaces, and multimedia applications, involving both s o u n d , animations and video seem to be just around the corner. One place to look for relevant theories is in European semiotic research because it treats creative sign-usage in a broad s p e c t r u m of codes: literature (Greimas, Eco), photographs (Barthes), a n d

Semiotics as a basis for a humanistic computer science

11

film (Metz) and shares a minimum of theoretical background. As an example, let us take Umberto Eco's "A Theory of Semiotics" (Eco 1977). The book is relevant to design since its second part is devoted to a theory of sign production. The basic concepts in t h i s part is the dichotomy between form and substance that, in a creative artistic context, is important, since most art consists i n giving form to a substance. The sculptor gives form to his s t o n e , the painter to his colors, the dancer to body movements, and t h e poet to linguistic material: We may define as invention a mode of production whereby t h e producer of the sign-function chooses a new material c o n t i n u u m not yet segmented for that purpose and proposes a new way o f organizing (or giving form to) it in order to map within it t h e formal pertinent elements of a content-type. Eco 1977: 245 The book describes and classifies many ways of producing signs. At one end of the scale, we have everyday sign production a n d interpretation where the expression-tokens (e.g. sounds) can easily be assigned to its proper type (e.g. phonemes), and where s t a n d a r d coding rules exist that enable the listener to interpret its meaning. At the other end, we have radical inventions, like e.g. the birth o f impressionism, where the artist strives to perceive reality in a n e w way, and fixes the result of his labor on a canvas in the form of a n expression-token (paint) that does not belong to any established type (unlike a sound that belongs to a definite phoneme) and is n o t referred to by any socially accepted coding rule (like the rules t h a t relate the words and sentences of language to meanings). Although systems designers may not be van Gogh's, they a r e faced with similar problems when building systems: should we b a s e our system on a metaphor that users understand in order to e n s u r e understandability, but running the risk of constructing a s y s t e m that really do not give users new opportunities, or should w e invent new ways of doing and looking at things, risking t h a t nobody will understand it? In fact, it is possible to stretch the analogy even further: it is n o t only sculptors that give form to substance. Designers a n d programmers of computer systems do the same thing, only t h e substance they mould does not consist of stone but of p r o g r a m executions (viz. sequences of system states), and the tools they u s e to shape the substance are not chisels but p r o g r a m m i n g environments. The term structure in the following quotation in f a c t has some similarities to the form concept: A process is a development of a part of the world through transformations during a time interval.

12 Peter Bgh Andersen Structure of a process is limitations on its set of possible states (and thus on its possible sequences of states). In most programming languages, the transformations are prescribed by imperatives (and thought of as actions ) in structure descriptions called programs . Nygrd & Srgrd 1986: 380

3. What Is a Computer-Based Sign? 3.1. The Sign Concept.

If we have decided to look at computer systems as sign-vehicles, the next question is naturally: what is a sign and what is a computer-based sign? This section presents two variants of the sign-concept, n a m e l y the structuralist variant defined by Saussure and later e l a b o r a t e d by Hjelmslev and Eco, and the Peircian variant. The structuralist variant can be depicted as follows:

Content/information Continuum Substance Form Form Substance Continuum Expression/data Expression features of the system (data states and transformations used to signify something) Data transformations and states The topic of the system Content features of the system

Fig. 2. The structuralist sign concept. The structuralist sign is mainly a description of a sign type . It focuses on its social and institutional aspects. The sign has t w o main planes, the expression and the c o n t e n t plane, also called t h e signifier and the signified. Both planes have two aspects, a s u b stance and a f o r m aspect. The substance of the sign is a part of the continuum articulated by the form of a sign occurrence. When a sign token is produced i n some situation, two continua are articulated: significant distinctions are introduced into the expression continuum, e.g. sound in spoken language and screen pixels in computers; t h e s e

Semiotics as a basis for a humanistic computer science

13

form elements are via the sign function correlated with s e mantic form elements, which in their turn establish distinctions i n the content continuum. In both language, the content continuum is the meaning of the sentences, in computer systems it is the d o m a i n of the system. The theoretical role of the continuum is solely that of a n amorphous mass that can be formed. We cannot say anything about it, since in that case the continuum has already been f o r m e d and has turned into substance. Still, it helps us defining t h e concept of a computer-based sign since, according to the analysis in (Piotrowsky 1990), we can identify the expression continuum o f computer-based signs with the storage cells of the computer. If w e disregard the program, the only thing that can be said about t h e cells is that they are different and can be in different states but a r e able to contract meaningful relations when structured by a program. Similarly, like the continuum of the sign concept, t h e y are extremely versatile, being able to manifest widely different kinds of meaning-bearing forms: word processors, spread-sheets, data-bases, etc, etc. I will add the peripherals, e.g. screen a n d loudspeaker, to Piotrowsky's substance concept. Since programming imposes a form onto the cells, p r o g r a m m i n g is really sign-creation. In a flight reservation system the content continuum is airplanes, passengers, flights, departure and destination location and times. This continuum can be articulated in many ways, and a flight reservation system imposes quite special distinctions into it, reflecting the particular tasks of the clerk. For example, t h e number of motors of the plane is non-distinctive, whereas t h e number of seats is.

3.2. Simple Signs.

The two planes of a sign are built out of smaller units. The smallest expression units in language are the phonemes. We can use them t o build syllables and syllables can again be parts of words, which again are parts of tone-groups, utterances. In a similar vein, w e must look for the smallest elements out of which to build computer-based signs. In the following, I present some very r o u g h outlines of the nature of computer-based signs and their m e t h o d s of combination. The prototypical computer-based sign is composed of t h r e e classes of features: A handling feature of a computer-based sign is produced by t h e user and includes key-press, mouse and joystick m o v e m e n t s that cause electrical signals to be sent to the processor.

14 Peter Bgh Andersen A permanent feature of a computer-based sign is generated b y the computer. It is a property of the sign that r e m a i n s constant throughout the lifetime of a sign token, serving t o identify the sign by contrasting it to other signs. Examples: icons in picture based systems, or letter sequences in textual systems. A transient feature of a computer-based sign is also generated b y the computer, but unlike permanent features, it changes as t h e sign token is used. It does not contrast primarily to o t h e r signs, but only internally in the same sign, symbolizing the different states in which the sign referent can be. Examples: location, hilite. In addition we must incorporate the notion of action in t h e definition of the sign types, since our interpretation of a c o m p u t e r base sign also depends upon what it can do to other signs. For example, the fact that the pencil sign can leave black traces on t h e paper sign of the screen enables the pencil and the paper to build a composite computer based sign that can be paraphrased "I a m painting the paper with the pencil". Similarly, a sort c o m m a n d would not be interpreted as "sorting" if it did not change its o b j e c t in a specified way. The following classification of computer-based signs is b a s e d upon three criteria which features does the sign possess? can the sign perform signs? actions that affect features of o t h e r

what is its possible range of meaning?

Semiotics as a basis for a humanistic computer science

15

Fig. 3. Classification of computer based signs. The Interactor exploits features from all three dimensions. It is distinguished from the other signs by permanent features like e.g. size and shape, and during its lifetime it can change t r a n s i e n t properties like e.g. location and color, these changes being functionally dependent upon its handling features. In most cases i t can perform actions that change transient features in other signs. As Fig.4 shows, it is used for signifying tools in tool-like applications, and represents the user in games.

Fig. 4. Interactors. Cursors from Canvass, sliders from Digital Darkroom, and the protagonist from the adventure game Dark Castle Buttons resemble Interactors, only their transient features a r e rudimentary, e.g. consisting of highlighting. Actors , lacking handling features, are autonomous processes that cannot b e interfered with once started. In a word processor, they include operations like sorting, creating a table of contents, or saving t h e text to disc. In games, actors represent the antagonist. Controllers lack transient and handling features but can still act on other signs. Windows are often divided into areas, e.g. at tool palette and a work space, whose borders are controllers that change the c u r s o r Interactor. If the cursor is within the palette it becomes an a r r o w

16 Peter Bgh Andersen for selecting a tool, while it changes into a tool when moved i n t o the work space. As the name indicates, objects are passive i t e m s lacking handling and action features. They are typically acted on b y means of actors or interactors. Examples are signs for work o b j e c t s like pictures and texts. Ghosts are signs that can neither been s e e n nor handled, but still influence other signs. Examples are h i d d e n typographical codes for new line and soft h y p h e n s in w o r d processors. In games ghosts can represent hidden traps. Finally, Layouts lack all features except permanent ones, being m e r e decorations. Examples are constant headings in textual systems, and boxes and colors in graphical systems. It is important to notice that this definition of c o m p u t e r - b a s e d signs is not a definition of interface elements, since a semiotic analysis of the form of computer-based signs will be p e r p e n d i c u l a r upon the standard distinction between functionality and interface. In a semiotic approach separation of interface from functionality is theoretically invalid: on the one hand, there will be features of t h e interface that do not belong the form but to substance, since t h e y play no role in the creation of meaning. On the other hand, a c cording to the argument about the pencil sign or the s o r t command above, many features of the algorithms providing t h e functionality of the system will belong to form, since the m e a n i n g of the signs depends upon them. For example, the sort-sign will incorporate those part of the s o r t algorithm that distinguishes it from e.g. processes that are i n t e r preted as merging-processes. It encompasses what in s o m e traditions are called the invariants of the sort-process. Suppose that x and y are elements that can be ordered in a list. Let C(x) be the contents of the elements, and suppose that an o r d e r is defined on the contents also, e.g. an alphabetical order. Let P(x,y) =def C(x) < C(y) -> x < y be an ordering principle d e m a n d i n g that the two orders must agree. An important part of the m e a n i n g of sorting is that before sorting at least two element did not satisfy P, while afterwards all elements satisfy P. This description w o u l d capture the essentials of the meaning of "sorting" its c o n t e n t form that distinguishes it from other kinds of processes. The specific method of sorting, however, belongs to t h e substance of the sort-sign, since exchanging one method f o r another does not alter the interpretation of the process.

3.3. The semiotics of object-oriented programming.

The concepts of object-oriented programming, OOP, (Kirstensen e t al. 1991, Korson & McGregor 1990, Rumbaugh et al 1991) a r e close to the semiotic concepts presented above. In the object-

Semiotics as a basis for a humanistic computer science

17

oriented paradigm, the system is seen as a model of the application domain. Its components, viz. classes and objects, stand for things, events, and interactions in the domain. Object-oriented analysis can be seen as a componential semantic analysis of the signs w e wish to enter into the system. The characteristic of the object-oriented paradigm, the c o n c e p t s of classes and specialization, is a standard in semantic analysis a t least going back to Aristotle. The hierarchy of classes, known a s the Porphyrian tree, was developed by the semioticians of the m i d dle ages (Eco 1984: 46 ff). The semiotic form-substance distinction has its analogue in t h e object-oriented encapsulation concept. The form describes t h e distinctive properties of a sign the features that contribute t o creation of meaning while its substance describes a p a r t i c u l a r way of realizing the form. In object-oriented programming, t h e interface of an object can be made to correspond to form, while i t s inner data and processes are designed as substance, a p a r t i c u l a r way of implementing the interface concepts: "The fourth guideline requires that to belong to the class, each operator must r e p r e s e n t a behavior of the concept being modeled by the class" (Korson & McGregor 1990: 53). A main difference between the semiotic and the object-oriented approach lies in the treatment of the "modelleling" relation. From a semiotic point of view, the modelling relation between system a n d domain is a very imprecise one in the object-oriented tradition. I agree that the system stands for represents the domain, but a sign does not just stand for something abstractly. It stands f o r something to somebody in some respect. Here the European structuralist and the object-oriented paradigm have a common problem: they lack one i m p o r t a n t concept, namely that of the interpreting person and his r e a c t i o n s to the sign. This is a problem, since we build computer systems i n order for the users to interpret and use it. Peirce's variant of the sign concept can help us clarify this p o i n t : in his framework it does not make sense to say that signs o c c u r except when they are interpreted by some interpreter. Furthermore, in this process the interpreter cannot help reacting to the sign, producing a new sign called an interpretant . T h e r e f o r e Peirce's sign contains three parts: the representamen (analogous t o the expression plane), the object (analogous to the content p l a n e ) , and the interpretant (which has no analogue in the structuralist sign concept). Peirce's sign concept represents an important corrective to t h e structuralist approach and motivates the question: to w h o m is t h e system a model? W h o is going to make which interpretants? T h e

18 Peter Bgh Andersen system as a whole cannot be a sign to the users, since they do n o t normally see its elaborate Porphyrian tree, but only bits and pieces of system executions. But a sign has to have an interpreter to exist, and I believe that the implicit interpreter, the person who sees t h e system standing for something, is the designers and p r o g r a m m e r s . The problem is that these people get paid to produce signs for t h e users, not for themselves! In Section 4 we shall see an example of users that definitely d o not interpret their system by means of classification a n d aggregation. They use space instead. This is an argument for revising the view of design a n d programming. The essence of programming in a semiotic perspective is not to make models for the benefit of the designer; it is rather to use the machine to try to tell people something. This point is well made in a paper by (Nardi and Miller 1991) on u s e r programming of spreadsheets Many spreadsheets are destined from the start for the boardroom or the boss's desk or the auditor's file. In our study, spreadsheet users were very aware of the importance of presenting their spreadsheets to others Laura stated "I usually think in terms of my stuff [her spreadsheets] as being used by somebody else" and users constructed spreadsheets with effective presentation in mind. Nardi and Miller 1991: 170 Making a spreadsheet is programming, and the purpose of t h i s programming is communication sharing domain knowledge a s they call it. Therefore spreadsheets signs must be seen as parts o f other kinds of communication. For example, they enter into c o n versations and influence them: Her comments show that the spreadsheet organizes discussion, as we have seen in the preceding example. Nardi and Miller 1991: 171 But if users enter computer-based signs into conversations, t h e former must fit into the latter. I return to this issue in Section 4. Instead of thinking of model making, I advocate the metaphor o f stage direction. A system is viewed as a kind of theater, i t s executions are performances that are interpreted by an audience, and the designer is a stage director whose success does n o t ultimately depend on the look of the props and set-pieces f r o m backstage, but on their communicative effect on the audience. The typology of computer-based signs presented in t h e preceding section, was designed with this objective in mind. T h e

Semiotics as a basis for a humanistic computer science

19

individual classes are designed so that they have meaningful r o l e s in "computerized" dramas. In a tool system for example, Interactors work well as signs for tools, Objects perform excellent as work material, and Controllers and Layout signs are skilled producers of illusions of spaces and locations. With suitable elaboration and a better empirical foundation, I think the typology could become useful as a general classification of object classes, encouraging the designer to think more consciously about t h e performance he is directing. Another suggestion relates to the level structure of programs. I suggest that system executions are divided into levels, each level being defined by a set of possible plots, actors, set-pieces a n d props. The requirement is that the plots of a level must be of t h e same types, and that only set-pieces and props with a f u n c t i o n relative to the plots are allowed at this level. The program text is divided into similar levels, and the individual level is not allowed t o make distinctions that lacks meaning in its c o r r e s p o n d i n g execution. These principles are intended to make easier for t h e programmer to interpret the program text as a script f o r meaningful performances. The following two examples show consequences of t h e s e principles. Consider first a direct manipulation interface. At t h e user level execution, the plot is about hitting, grasping, dragging, and letting go of material. Hitting can be softer (click) or h a r d e r (double click). If the following line
if ticks-oldticks>10 then return "Pause"

is visible in the code defining clicks and double clicks it is a programming error. The reason is that ticks (returns the c u r r e n t system time) and oldticks (stores the time of the last mouse e v e n t ) are not part of the user-level story, and that replacing 10 with 1 2 does not produce a new story. The line belongs to a level below that tells stories about mouses and clocks. In this example, the semiotic and the object-oriented guidelines probably agree. The next example shows that they can be at c r o s s purposes. Consider classes in an accounting system at the level w h e r e customers withdraw and deposit money and occasionally c h a n g e addresses:

20 Peter Bgh Andersen

Class account Balance: integer; procedure Deposit(CustomerId,Amount);... procedure Withdraw(CustomerId,Amount);... end account Class customer Address: string; procedure ChangeAddress (NewAddress) Address := NewAddress end ChangeAddress end customer

Fig. 5. Account and Customer classes of a banking system. The Account class possesses Deposit and Withdraw o p e r a t i o n s , since they both effect Balance, the internal variable of the class. For a similar reason, Customer contains the ChangeAddress operation. This object design, which follows normal guidelines, entails code like this:
ACustomer.ChangeAddress("A new address"). AnAccount.Deposit(Acustomer,1000).

According to the criteria above, this is bad code, since i t introduces a syntactic distinction between depositing/withdrawing (subject-verb-object) on the one hand, and changing a d d r e s s e s (object-verb-subject) on the other hand which has no interpretation at the current plot level, where the customer is t h e subject of both actions: customers change adresses as well a s deposit money. The underlying principle is that differences o f object structure should have an interpretation in the domain. According to OOP the code is good because it obeys the principle of strong cohesion between operations and properties of a class. The principle is that the Deposit operation should not be d e c l a r e d in the Customer class, because it affects the Balance property o f the Account class. In this case, the OOP principle of operation/data cohesion a n d the semiotic demand for interpretability do not agree. The criteria for judging the two pieces of code is based on one o f the basic principles of sign analysis called the commutation t e s t . The test is used to empirically determine what belongs to the f o r m of a sign and contributes to creation of meaning and what d o e s not: a feature is a part of the expression or content plane of a sign if and only if exchanging the feature for another one causes a change of features on the other plane. Therefore /male/ is a p a r t

Semiotics as a basis for a humanistic computer science

21

of the contents of the word b o y since exchanging it for / f e m a l e / causes the expression to change to girl . /large/ is not a feature o f boy since a small boy is still a boy. On the other hand, /large/ is a part of the meaning of forest , since a small wood is not a forest. The recommendations in the preceding simply boils down to t h e guideline that the program text should pass the commutation t e s t , but a closer inspection of the sign-processes reveals that there a r e at least three different kinds of signs we can apply the test to:
domain contents contents

expression program expression contents

expression execution

Fig. 6. The sign relations of program text, execution, and domain. The execution is intended as a sign for objects and processes i n the domain, but the program text is an ambiguous sign w h o s e contents can b o t h be the execution and the domain. In the b a n k i n g example, the syntax and identifiers of a code like ACustomer. ChangesAddress ("A new address") invites an i n t e r p r e t a t i o n relating to the domain, while the implementation inside t h e Customer object Address := NewAddress refers to variables a n d therefore stands for properties of the execution. It seems to m e that OOP focuses on the program/domain sign as the i m p o r t a n t part of the system the modelling relation and seeks to h i d e the program/execution relation by means of encapsulation. My criticism of this arrangement was that the interpreters of t h e program/domain signs are programmers, while users interpret t h e execution/domain sign. If the programmer is to be a successful director of executions, he should work as carefully with the p r o gram/execution sign as a director works with m o v e m e n t s , gestures, lights, and set-pieces. But this is difficult if t h e program/execution signs are hidden and encapsulated! On the basis of this analysis, I would shift the focus from t h e program/domain relation to the program/execution relation, a n d require of a program text that it clearly expresses the f o r m elements in the execution that contribute to creating t h e execution/domain signs. I conjecture that the typology in Fig. 3 , suitably elaborated, could be the basis of a programmer's p a l e t t e for creating the required signs. This shift of course requires the d e -

22 Peter Bgh Andersen signer and programmer to assume a more humble attitude: we d o not mirror reality in our systems, we only direct a play that t h e users under lucky circumstances can use to gain knowledge a n d insight.

3.4. Composite Signs.

As in language or paintings, the smallest sign-parts are g r o u p e d into larger wholes called syntagmas or phrases . Formally, a syntagma can be described as a sequence of slots that can be filled according to rules that specify what material can be used as filler and whether the slot must be filled or can be empty. Traditionally, we distinguish between three main type o f syntagmas, constellations, subordinations, and interdependences . They can be defined formally in this way: In a constellation the slots can be filled or empty, In a subordination one slot (often called the h e a d ) must b e filled while the other (the modifier ) needs not, and In an interdependence all slots must be filled.

Although the formal definitions have analogies in logic ( a constellation corresponds to a disjunction , a subordination to a n implication , and an interdependence to an equivalence ), the t h r e e syntagmas are not identical to logical constructions, since t h e y have a different semantics. It is is as follows: In a constellation both parts relate to the context; t h e individual part can play the same semantic role as does t h e whole construction. In "I saw a house and a car" both h o u s e and car relate to saw , since it makes sense to say "I saw a house" and "I saw a car". In a subordination only to other meanings in the relate to the text outside tree" large relates only to the head makes a direct c o n n e c t i o n text. The modifier does not directly the subordination. In "I saw a large tree , not to I or s a w .

In an interdependence the effect of the whole cannot b e reduced to the effect of one of its parts, so none of the t w o parts relates individually to the surrounding text. Only t h e i r whole does so. In "I said he came" he c a m e as a whole r e l a t e s to said . It makes no sense to say "I said he" or "I said came".

Constellations express contingent relations, e.g. an o p e n enumeration of elements that can be continued: "I saw a house, a

Semiotics as a basis for a humanistic computer science

23

car, a train, ...". In opposition to this, interdependences e x p r e s s necessary relations, and their parts form a closed set. The r e l a t i o n expressed by "He came" must contain two elements, while that o f "He gave the book to me" needs three. We cannot add or s u b t r a c t elements randomly as with constellations. Finally, subordinations organize the material in subordinate a n d superordinate elements. For example, it follows from the definition that only the head of a subordination is active in creating t h e narrative structure of a text. These concepts can be used as a guide to structure the p r o g r a m code. For example, suppose we are building a teaching s y s t e m always containing conceptual explanations in one window, a n d sometimes examples in another window. Then we could say t h a t the main syntagma of the system is a subordination with t h e Explanation window as the head and Example window as t h e modifier:
Explanation Example

Fig. 7. Subordination. The arrow points from the modifier to the head. Descriptions like Fig. 7 could be entered into programs a s constraints the system must always obey (see Freeman-Benson, Maloney & Borning 1990). Suppose in addition to the two windows above, we have two others, Text and Picture, that must c o n t r a c t interdependence, and suppose furthermore that the user can o p e n and close windows as she wants via menus or buttons. Then t h e following procedures, which are called every time s o m e t h i n g happens to the system, ensure that the constraints are obeyed.
Procedure KeepWatch Subordination Explanation, Examples Interdependence Text, Picture end KeepWatch

24 Peter Bgh Andersen

Procedure Subordination Head, Modifier if Opened(Modifier) then Open(Head) end if if not(Opened(Head)) then Close(Modifier) end if end Subordination Procedure Interdependence Window1,Window2 Subordination Window1, Window2 Subordination Window2, Window1 end Interdependence

Fig. 8. Implementation of subordination and interdependence. The KeepWatch procedure is called every time an event h a p p e n s , maintaining subordination between Explanation and Examples a n d interdependence between Text and Picture. The Subordination procedure checks if the Modifier is open; if so, it opens the Head. If the Head is not opened, it closes the Modifier. Interdependence is implemented as a double Subordination. Like the simple signs, composite signs have a dual definition, o n e part of it referring to the expression plane (the formal definition), the other one to the content plane, and it is an error to i m p l e m e n t a formal subordination that does not also satisfy the semantic r e quirements. Thus, if we make a system where a text window is formally subordinate to a picture window, then the main structure of t h e system must be interpretable solely via the picture window, a n d the text window should only play a secondary role as a commentary to the picture window (Andersen 1990b). As emphasized in the preceding section, the programming p r o c e s s always have one foot in the expression plane (the formal a s p e c t ) and the other foot in the content plane (the interpretation aspect).

Semiotics as a basis for a humanistic computer science

25

4. The context of interpretation.

A computer-based sign acquires part of its meaning from i t s relation to other signs the user sees in the interface; but the u s e situation contains other kinds of signs it must relate to. In fact, i t combines with the global semiotic system of the workplace t h e most important part of which is the work language of the u s e r s , but which also includes e.g. diagrams and plant layout. In order f o r the computer-based sign to amalgamate with its semiotic c o n t e x t of use, the structuring principles underlying the two semiotic systems must be similar. This is not always easy to achieve, since users o f t e n conceptualize and talk about their work in ways that are different from those of the designers. Sometimes the conceptualization is fairly common, sometimes they use distinctions a designer c o u l d never get at except by listening carefully. This section gives an example of both cases. The first one is t h e use of space as a way of talking about computer systems. As an example I quote some of the data from a project w h e r e Berit Holmqvist and I investigated how users interpreted a n e w administrative system at the Postal Giro Office in Sweden (Holmqvist & Andersen 1991). The system was based on of t h e OOP principles of aggregation and classification, but the w o r k e r s disregarded the system and used space and time as principles f o r naming their computerized work processes. The following quotations are from a speaking a loud session, where we asked a worker to tell us what she did while she worked. In her speech work objects can go there, come, come forth, you know, everything goes there this one is no good because then all cards come and she presents her own actions as a journey in a landscape o f cards and tasks, describing the surroundings from her p r e s e n t location: come to, go to, go back to, be, sit because I came to the last card on this one when I sit in "comp" then it is "completing" Her own actions on the work objects either removes them f r o m her: put aside, send back, place there, take out, take away then I put aside then I have to take this signal away or brings it nearer to her: f e t c h I could have fetched it to In addition to the spatial distinction between here and not here , the worker uses another basic contrast, namely between have a n d not have .

26 Peter Bgh Andersen These verbs can be arranged in a semantic system structured b y three sets of features: location versus possession positive versus negative versus modal (result of) process versus state where positive location = here, negative location = there, n o t here. The feature modal means that the sentence does not signify a state of affairs but a wish or an obligation.

location positive negative (result COME, FETCH of) process hmta, komma till, fram BE, SIT state vara,sitta PUT, SEND

possession positive negative GET, TAKE ABANDON verge, inte f LACK inte ha, sakna

modal WANT, MUST vill ha, ska ha

f, f fram, lgga t sidan, ta bort, ta, skicka NOT EXIST finns inte HAVE ha, ha framme

Fig.9. Semantic system of task designations in work-language. For example, put aside differs from get by being concerned with location instead of possession, from c o m e by by leaving the o b j e c t there instead of here , from exist by being a process and not a s t a t e , and from want to have by expressing a state of affairs and not a wish for the future. Kim Halskov Madsen and I got the same result in another p r o j e c t where we looked at librarians' interpretation of their system. Although users use other metaphors, the space metaphor is very frequent. The reason why the desk-top metaphor of Xerox and Apple h a s been so successful is probably that it uses space as the m a i n structuring principle, and the designers of the system at the Postal Giro might have got good design ideas from knowledge of t h e workers' principles of structuring meaning. In this case, the programmers' Porphyrian tree was foreign to t h e users, and should probably not have been let lose in the interface. However, although the spatial metaphor seems to be a general metaphor, we should be cautious not to over-generalize. It is p e r fectly possible that classification and aggregation are n a t u r a l concepts in other professions, e.g. librarianship. We have to verify

Semiotics as a basis for a humanistic computer science

27

these issues empirically, since the complexity of reality exceeds our current theories. The next example from the same research project illustrates t h i s unpredictability. The computer system was built as a mirror image of the paper flow and therefore the distinction between c o m p u t e r based and paper-based material was important. The system and t h e manuals tried to make the distinction systematically, but it t u r n e d out that the users only upheld it in verbs, not in nouns. Verbs for manipulating work objects are divided into t w o different categories, one used about paper ( skriva p, skriva d i t , skriva under, mla, mla ver, [write, sign, paint]) and a n o t h e r used for manipulating the electronic data (sl, sl in, trycka, t r y c k a p, trycka ner, sl fel [hit, press, enter]). But in the nouns t h e y used the same words for both categories: But you understand, you have to put the card [Sw:kort , paper card] in the flier box, if you have put a card [kort, electronic card] in the flier, and then I'll have to look in this flier box. The first occurrence of the word card refers to the paper c a r d , the second to the electronic card, since flier box denotes a physical plastic tray, and only paper cards can be placed there, while flier denotes a file, where only electronic cards can be placed. The reason for this unpredictability is that work language evolves closely connected to tasks. The here and now pressure o f daily work changes the language continually; if a worker needs t o make a verbal distinction in order to do work, she just creates i t out of whatever linguistic material is at hand. Work languages a r e not neat and logical but look more like a patchwork blanket. The use of domain specific concepts as a basis for designing classes is recommended in object-oriented design; the lesson f r o m the preceding examples is that they do not just lay around ready t o pick; it sometimes requires close attention and knowledge of t a s k s and organization to discover the semantic system of t h e workplace. Another point worth of emphasizing is that signs have specific contexts of uses. Concepts do not just spend their life hanging i n the scaffold of the Porphyrian tree, b u t are constantly put to w o r k in conversations (Winograd & Flores 1966). Investigating r e c u r r e n t patterns of conversations gives information about when information is needed and in which form. For example, p r o b l e m solving conversations as the following were recorded in the Postal Giro project E: That is an adding mistake. Y: Yes.

28 Peter Bgh Andersen E: Yes, it is. ... E: Card missing? X: Well, I didn't have the card then, you know, then I put it to one side, for the time being, though we'll sit on it, now we fetch job , enter job number 1 , it was the first job, you see. E: You'll have to destroy it or hand it in the box. X: Right, that's it. E: He's, he's... X: Well, let's see now, then I should, then I'll destroy the whole batch.E: Let's see, hmm, it's sure to be scan, this one's going to be a real mess. ... E: Do it like this, I think: take a little slip, and then put it on the C card, and tell them that it's a C card, then when you get it back, then you put it all in this box here. It bears the hall-mark of rule-governed work since it does n o t aim at inventing methods for solving the problem, but seeks o n e out a fixed set of rules. Its start with a problem definition (adding m i s t a k e ) , followed by attempts to determine the error (card m i s s ing) . Then follows a lengthy method discussion (You'll have t o destroy it) that eventually ends with picking a rule (do it like t h i s ) . The recurrent structure of this type of conversations is d e s c r i b e d in the diagram below:

Rule seeking Problem definition Rule to be followed

Error determination Error type Satisfaction

Method discussion

Semiotics as a basis for a humanistic computer science

29

Fig. 10. Interdependences are represented as double arrows, cf. the implementation in Fig. 8. This diagram tells us which kind of information they handle i n the conversation and describes the relations between types o f information: for example, Problem Definition and Rule to b e Followed always occur together in our data, whereas the Error Determination and Method Discussion were optional. An obvious guideline is to use work language as a basis o f design, not by copying it, but rather as an inspiration for creativity, in the same sense as a filmmaker can use a novel as a basis for a film. In the case of Fig.10, we could base the design of p r o b l e m solving support on a copy of the conversation structure. It will consist of three windows. The main window, Rule Seeking, c o n t a i n s two parts, a List of Problems, and suggestions for Rules to b e Followed. These two items always pop up because our data s h o w s that workers always seem to need this information. The two o t h e r windows, Error Determination and Method Discussion, a r e subordinate to the Rule-Seeking window, which may t h e r e f o r e contain two buttons for opening them. The relation between work process and design is discussed further in (Holmqvist & Andersen 1991, Andersen 1990a).

5.

Summary.

The key concept of this paper is semiosis, the processes w h e r e b y signs emerge and are interpreted. Computer systems are seen as sign-vehicles, and since signs consist of contents as well as expressions, one should n e i t h e r investigate user interpretations without considering the technical structure of the system, nor treat design and i m p l e m e n t a t i o n divorced from the intended interpreters of the system. A c o m p u t e r semiotics must integrate an understanding of programming as a semiotic process with empirical field work investigating the a c t u a l interpretants of the users. It means that the theoretical f r a m e w o r k must contain concepts for understanding formal structures as well as concepts for analyzing the non-formal signs of the users. It h a s been argued that the European structuralist tradition is b r o a d enough to meet these demands, provided its bias t o w a r d s structural descriptions is balanced by a Peircean understanding o f sign processes. In order to suggest possible contributions of a semiotic a p p r o a c h concretely, example solutions based on object-oriented programming has been compared to solutions based on semiotic theory. In some cases they agree, in others they disagree. The t w o

30 Peter Bgh Andersen approaches agree in seeing the computer system as a sign s y s t e m standing for events and objects in the application domain. T h e main difference between them is that object-oriented p r o g r a m m i n g seems to focus on the designer's and programer's concept of t h e domain; the main task is to build a model of the domain which is then later used as a basis for providing information users n e e d . Thus, model first, interfaces afterwards. In opposition to this, the semiotic approach focuses on t h e execution viewed as sign-vehicles users interpret in their context o f work. It does not advocate the opposite course of action, interface first, model afterwards, simply because in this approach interfaces should not be separated from functionality. Instead the recommendation is to view the execution in analogy with a t h e a t e r performance, and to design the classes and objects of the s y s t e m as actors, props and set-pieces in this drama. How different the two approaches really are is difficult to judge. After all, one of the founders of the object-oriented p a r a d i g m , Kristen Nygaard, is rumored to have been fond of p u p p e t - t h e a t e r as a child!

References
Andersen, P. Bgh and B. Holmqvist, (1990a). Narrative computer s y s t e m s . The dialectics of emotion and formalism. In J. F. Jensen, e d i t o r , Computer-kultur computer-medier Computer-semiotik [Computer culture computer media computer semiotics]. Nordic S u m m e r University, Aalborg. Andersen, P. Bgh and B. Holmqvist, (1990b). Interactive Fiction. A r t i f i c i a l intelligence as a mode of sign production. AI and Society, 4(4): 291-313. Andersen, P. Bgh, (1990a). A Theory of Computer Semiotics. Approaches to Construction and Assessment of Computer Cambridge University Press, Cambridge. Semiotic Systems.

Andersen, P. Bgh, (1990b). Towards an aesthetics of hypertext systems. A semiotic approach. In A. Rizk et al, editors, Hypertext: concepts, s y s t e m s , and applications. Cambridge University Press, Cambridge. Andersen, P. Bgh, (1991a). Vector spaces as the basic component o f interactive systems.Towards a computer semiotics. VENUS report no 7. Department of Information and Media Science, University of A a r h u s . Niels Juels gade 84, 8200 Aarhus N. To appear in Hypermedia . Andersen, P. Bgh, (1991b). A semiotic approach to construction and assessment of computer systems. In: Nissen, Klein & Hirschhaim, e d i t o r s , Information Systems research: Contemporary Approaches & E m e r g e n t Traditions, 465-514. North Holland. Bannon, L., (1989). From Cognitive Science to Cooperative Design. In: N. O. Finnemann, editor, Theories and Technologies of the Knowledge S o c i e t y . 33-59. Center for Cultural Research, University of Aarhus: Aarhus.

Semiotics as a basis for a humanistic computer science

31

Boland, J. R., (1991). Information system use as a hermeneutic process. I n : Nissen, Klein & Hirschhaim, editors, Information Systems r e s e a r c h : Contemporary Approaches & Emergent Traditions, 439-458. N o r t h Holland, Amsterdam. Chomsky, N., (1957). Syntactic Structures. Mouton, The Hague. Chomsky, N., (1965). Aspects of the Theory of Syntax. MIT Press, C a m b r i d g e , Mass. Chomsky, N., (1968). Language and Mind. Harcourt, Brace & World, New York. Chomsky, N., (1976). Reflections on Language. Fontana/Collins. Chomsky, N., (1980). On Binding. Linguistic Inquiry 11: 1-46. Decls, J-P., (1989) Intermediate Semiotica 77(1/3): 121-135. representations in the cognitive sciences.

Eco, U., (1984). Semiotics and the philosophy Press, Bloomington. Figge, U. L., (1991). Computersemiotik. 330.

of language. Indiana U n i v e r s i t y

Eco, U., (l977). A Theory of Semiotics. The MacMillan Press, London. Zeitschrift fr S e m i o t i k 13(3/4): 321-

Freeman-Benson, B. N., J. Maloney & A. Borning, (1990). An I n c r e m e n t a l Constraint Solver. Comm. of the ACM, 33(1): 54-63. Goldkuhl, G. & K. Lyytinen, (1982). A language action view of i n f o r m a t i o n systems. SYSLAB report no. 14. Gteborg, Sweden: Department of Computer Sciences, Chalmers University of Technology and t h e University of Gteborg. Halliday, M.A.K., (1976). System and Function in Language. Oxford U n i v e r s i t y Press, Oxford, U.K. Halliday, M.A.K., (1978). Language as Social Semiotic. The Social Interpretation of Language and Meaning. Edward Arnold, London. Hjelmslev, L., (1963). Prolegomena to a Theory of L a n g u a g e . Menasha. T h e University of Wisconsin Press, Winsconsin. Translated from " O m k r i n g Sprogteoriens Grundlggelse". University of Copenhagen 1943, reprinted and published by Akademisk Forlag, Copenhagen 1966. Holmqvist, B. and P. Bgh Andersen, (1987). Work-language technology. Journal of Pragmatics 11: 327-357. and i n f o r m a t i o n

Holmqvist, B. and P. Bgh Andersen, (1991). Language, perspective, and d e sign. In J. Greenbaum and M. Kyng, editors, Design at Work. E a r l b a u m , Hillsdale. Holmqvist, B., (1989). Work-language and perspective. of Information Systems 1: 72-96. Scandinavian Journal

Hopcroft, J. E. and J. D. Ullman, (1969). Formal languages and their relation t o automata. Addison-Wesley Publ. Comp., Reading, Mass. Kaasbll. J., (1986). Intentional development of professional language through computerization. A case study and some theoretical c o n s i d e r a tions. Paper for the conference on System Design for H u m a n Development and Productivity: Participation and Beyond. H u m b o l d t Universitt zu Berlin, Berlin. Korson, T. & J. D. McGregor, (1990). Understanding object-oriented: unifying paradigm. Comm. of the ACM (33/9) : 40-60. A

32 Peter Bgh Andersen

Kristensen, B. B, O. L. Madsen, B. Mller-Pedersen & K. Nygaard, (1991). Object oriented programming in the BETA programming l a n g u a g e . U n i v e r s i t y of Aarhus, Dept. of Computer Science, Aarhus. Mathiassen, L. and P. Bgh Andersen, (l984). Semiotics and informatics: t h e impact of edp-based systems upon the professional language of n u r s e s . In van der Veer, editor, Readings on Cognitive Ergonomics - Mind a n d Computers. 226-247. Springer-Verlag, Berlin. Also in Journal o f Pragmatics 10(1986): 1-26. Montague, R., (1976). Formal Philosophy. Selected papers Montague. Yale University Press, New Haven and London. Nadin, M., (1988). Interface 302. design: A semiotic paradigm. of Richard

Semiotica 69: 269-

Nardi, B. A. and J. R. Miller, (1991). Twinkling lights and nested loops: distributed problem solving and spreadsheet development. Int. J. M a n Machine Studies, 34: 161-184. Nygrd, K. & P. Srgrd, (1987). The persepctive concept in informatics. I n Bjerknes, G. et al., editors, Computers and Democracy. A l d e r s h o t , Avebury. 357-379. Ouellet, P. Semiotics, cognition, and artificial Semiotica. Semiotica 77. 1989. intelligence. Special issue o f

Piotrowsky, D., (1990). Structures Applicatives et Language Naturel. Recherches sur les fondements du modele: "Grammaire Applicative e t Cognitive". Ph.D thesis, Ecole des Hautes Etudes en Sciences Sociales, Paris. Rasmussen, J., (1986). Information processing tion. North-Holland, New York. Rumbaugh, J. et al., (1991). Object-oriented Hall: Englewood Cliffs. and human-machine interac-

Reichenbach, H., (1947). Elements of symbolic logic. New York. modeling and design. P r e n t i c e -

Saussure, F. de. (1966). Course in General Linguistics. McGraw-Hill: New York. Stamper, R., (1991). The Semiotic Framework for Information Systems Research. In Nissen, Klein & Hirschhaim, editors, Information S y s t e m s research: Contemporary Approaches & Emergent Traditions, 515-528. North Holland, Amsterdam. Winograd, T. & F. Flores, (1986). Understanding Ablex: Norwood, New Jersey. Computers and C o g n i t i o n .