AI in CLinical Lab
AI in CLinical Lab
1-15
Measurement
Comparison
t-
Effectuator
Meter
Display
Computer
Input Output
Sensor Software
KeY_bard_
Automatic tool Programmable tool
Feedback Effectuator
Figure 1. Functional elements of an instrumental component. Figure 4. Classification ofsimple instruments intofour categories.
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
(bottom half of the figure) and the open or stand-alone expert systems are able to make decisions based on the
system (total figure). In the stand-alone system, direct accumulated knowledge with which they are programmed
and interactive human control is possible as output is and are therefore commonly included within the definition
displayed for human evaluation. In the embedded system, of AI systems. An excellent introduction to expert systems
cycles of processes are evaluated automatically by is given by Bartels and Weber [6], and Wulkan [7]
intelligent controllers below the surface of the man- provides a recent review of the state of the art in relation
machine interface. to clinical chemistry.
On the basis of whether the input is from man (for example Expert systems possess high-quality, specific knowledge
via a keyboard) or machine (by sensor and associated about some problem area and build upon the problem-
circuitry) and whether output is to man (for example as solving capabilities of human experts. Human expertise
a display) or to machine (by effectuator), one can develop is characterized by the fact that it is perishable, difficult
a basic classification of simple instruments. This is to transfer and to document, hard to predict and expen-
illustrated in figure 4. In a meter, the machine (sensor) sive to obtain, and one of the main aims of expert systems
input provides data that are processed by software and is to overcome, these frailties by acquiring, representing,
relayed as information to a display, while in a personal documenting and preserving expertise in a form that then
computer it is the human operator (for example via a becomes widely available at lower cost. The very process
keyboard) that provides input to the software. Effectuators of realizing and documenting this human expertise may
(for example a motor or some other form of tool) can be well improve upon it, and in some cases, expert systems
monitored automatically by a sensor system (for example use exhaustive or complex reasoning that individual
in robotics), or programmed via a keyboard by a human human experts would be incapable of using. However,
operator. the expert system, like its makers, is not necessarily
infallible.
Figure 4 also shows that software is the central feature of
processing and control in automation. This is of course a Expert systems, like their human counterparts, should
grossly simplified illustration. Real instrumentation is very provide a high level of skill with adequate robustness.
complex, integrating many such simple units into a Unlike most conventional programs, expert systems
complex whole. usually have the ability to work with represented and
uncertain knowledge and the ability to handle very
difficult problems using complex rules. On the other hand,
Intelligence and learning mechanisms if the problem requires creative invention, it is really not
possible to develop an expert system to solve it. Similarly,
With the introduction of programmable software it if the solution requires a significant input of general broad
became possible to introduce great flexibility into the common-sense knowledge, the expert system may not be
system, both in the operational processes and in the able to deal with the problem in a satisfactory way.
feedback loop. As more flexibility and higher level control Differences between conventional software programs and
is introduced into the system it can acquire the capacity expert systems are listed in table 1. Some of the terms
to become ’intelligent’. used in table are discussed below under the section ’the
knowledge base’.
However, it is not always possible or appropriate to
introduce machine intelligence into system operation and The distinction between a conventional software program
control. When the decision sequence is clear, an algorithm and an expert system may not be as clear as table might
can be defined, and a conventional software programme suggest. Not all existing expert systems are able to deal
may be more appropriate. with uncertainty and some operational expert systems use
conventional Bayesian probabilistic methods to deal with
Intelligence is very difficult to define. It has something to
do with the faculty of understanding and the capacity to
know or to learn and much of the basic work on AI is to Table 1. Conventional programs and expert systems compared.
do with the mechanisms of learning. The major advantage
of AI is that it allows learning via the accumulation of
Conventional programs Expert systems
knowledge. Represent and use data Represent and use knowledge
Algorithmic Heuristic
However, learning includes a large proportion of insight Iterate repetitively and Use inferential processes
characterized by good classification of the stimulus and
exhaustively
’silent’ trial and error to reinforce the response. Use a decision sequence May have numerous ways to
Must have all decision rules reach conclusion
Stop if data missing Allow for missing information
Knowledge engineering Derive one solution Derive best decisions possible
consistently
In an attempt to get away from science fiction connotations, Use numbers and equations Can also use concepts
AI technologies are now commonly termed ’knowledge Present a result with Provide uncertain results
confidence limits Rank results by likelihood
engineering’ and the intelligent computer software that
embodies knowledge is called the ’expert system’. Because Appropriate for known, Appropriate for imprecise
accepted ways to solve a information and
it has been rather difficult to develop practical applications problem, especially with conceptual problems
of automatic learning, expert systems often do not include only one solution
the ability to learn by themselves. Nevertheless, such
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
Table 3. Knowledge representation techniques. knowledge bases linked by nodes to higher level knowledge
bases. At each node, a reasoning process is designed and
Facts list all the reasoning procedures together form the inference
--.no uncertainty engine. The inference engine is independent of the
Symbols contents of the knowledge base.
--representing defined concepts
--.uncertainty included in the definition The way in which the inference engine reasons can be
Production rules illustrated by looking at a typical rule. The condition part
--.premise + action part (the premise) has three components, the object, a logic
certainty factor given to conclusion
Structured objects and object-oriented structures operator and an attribute. The action part then specifies
---descriptive statements what will be done if the condition part is found to be
--symbolic, used in morphology and anatomy true; for example the rule;
Associative networks
--entities (nodes)
IF (nuclear size is >40 gm 2)
--relationship between entities (arcs) THEN (compute the total absorbance [A]).
Frames At the start, the system is pointing at the location where
---complex nodes allowing data entry the input ’nuclear size’ is written into the working
--contains slots with attributes
---organized in class hierarchy memory. As this follows an IF statement it is identified
as a component of the condition part of the rule. The
object ’nuclear size’ is matched with an object table. The
logical operator ’is >’ is identified and the value to be
form of knowledge representation. They are used to list attributed is requested from the user. The value is entered
data and equations and the uncertainty attached to the through the user interface into the working memory. The
facts can be precisely defined. Symbols are used to define intierence engine retrieves this value and compares it with
items or concepts. As described above, their uncertainty the value of 40 lam z specified in the rule. If the rule is
lies in the definition itself and not in the symbol. triggered, the action part is interpreted by the inference
Production rules consist of a set of preconditions (the engine and output is provided to execute the THEN part.
premise) and an action part that leads to a conclusion Finally, the inference engine writes the completed
(for example, see O’Connor and McKinney [9]). If the sequence into the working memory, to add to the session
premise is true, the action part is justified and a certainty history.
tiactor is given to the conclusion. It is well known that the higher level of pertbrmance of
Structured objects are used, for example by Bartels et al. experts in any domain is related to their ability to make
[-10], in morphological or anatomical representation that a structured analysis, rather than trying all the alternatives.
are best dealt with in the form of descriptive, declarative Especially in medical diagnosis, powerful hierarchical
statements. In associative (or semantic) networks both the structures lead from symptoms to diagnosis.
entities themselves and the relations between the entities The inference engine typically operates using two sets of
are represented, the former by nodes and the latter by
rules, the first concerned with the search space(s) among
arcs connecting these nodes.
knowledge representation instances within which to look
Frames are analogous to complex nodes in a network (for for solutions and the second concerned with which nodes
example Du Bois et al. [-11]). They allow a more precise to evaluate and in which order.
and comprehensive description of an entity. In a frame,
a data object has slots that contain attributes of the object
There are two basic ways of searching. If the routing
and a slot can activate data gathering procedures if the decisions at each node are highly certain and unambiguous,
a depth first strategy is indicated. If such a strategy turns
information it contains is incomplete. Frames are usually
out to give the wrong answer, the program returns to the
organized in a class hierarchy with information in higher start and follows a new in-depth search. In the breadth
classes inherited by lower classes. Each problem requires
its own knowledge representation and there seems to be first strategy the search space is searched exhaustively at
no consensus concerning the use of the various knowledge
each level to avoid going deep into a blind path. A
reasonable (in relation to time taken and use of memory
representation techniques. Hybrid techniques abound. An
example of a hybrid frame and rule architecture is given space) focus on the problem is retained in a so-called
by Sielaffet al. [12] in ESPRE, a knowledge-based system bounded, depth-first strategy.
to support decisions on platelet transfusion. Once the strategy is indicated, the inference engine will
compare observed facts and rules, keep a record ot" the
sequence and outcome of rule use, resolve conflicts
Inference engine between rules, present the user with a set of alternative
results including degree of certainty and with a detailed
The inference engine implements the problem-solving
explanation of the reasoning sequence. It is important
strategies built up by the knowledge engineer. A common to stress that problem-solving strategies are heavily
technique in developing the expert system is to identify dependent on the representation and inappropriate
the ’primitives’ or independent variables first and then representation can make an expert system fail.
devise a system in which these can be redefined to
represent higher level or dependent knowledge. Each A number of inference (problem-solving) mechanisms can
knowledge based system is thus built of ’primitive’ be recognized and some are listed in table 4.
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
Table 4. Some inference (problem-solving) mechanisms. Unless knowledge already exists and is accepted (for
example in the form of maintenance manuals), knowledge
Heuristic search acquisition, i.e. what to put into the knowledge base and
---.rules of thumb more particularly what to leave out, is a problem. It may
Generate/test be difficult to ensure that all the relevant knowledge has
--.defining search spaces as required been obtained from the human expert and indeed the
Forward/backward chaining knowledge engineer may be in doubt about what is
--pruning decision trees
Recognize/act relevant.
--special conditions requiring specific actions
Constraint directed search Domain experts will always provide their knowledge in a
--main skeleton of acceptability is known specific context, whereas the expert system attempts to
Metarules apply this knowledge to all cases. Knowledge acquisi-
---exhaustive mapping tion tends to increase the size of the knowledge base
dramatically. For example in a thyroid report interpre-
tation expert system, the knowledge base doubled in size
to move from 96 to 99.7 agreement with experts
Heuristic searching is based on experience or ’rules of
thumb’. Some people, for example in maintenance and
(Compton [13]). For this reason the function of a new
expert system should be carefully defined. If the expert
failure detection, are able to diagnose problems more
system is to be used for teaching, it will contain a large
quickly and effectively than. others, because of domain- amount of explanatory functions that are not needed if
specific knowledge acquired as a result of experience. The the expert system is to be used to provide expert
search space for a particular problem can be rapidly
judgements in consultation. If the system is to be used to
narrowed down by this sort of knowledge. screen input and select cases that need detailed attention,
In some cases the search space is not provided, but its knowledge base should be broad, but without great
generated as the system proceeds and evaluated shortly depth. On the other hand, if the system is to function as
thereafter. This is known as ’generate/test’. an expert in complicated cases, the knowledge base should
be large but limited to a defined field of expertise.
Forward chaining is testing preconditions for the truth of
a given hypothesis for given data, rather like pruning a One of the most important features of an expert system
decision tree. Forward chaining is useful when the number is that it can be improved by the addition of further
of possible outcomes is limited. information from users. Provision should be made for
maintaining the knowledge base. Because of rule overlaps
Backward chaining, with the system seeking data to justify and the complex interrelationships between rules, main-
the selected hypothesis, is the reverse and is more taining the knowledge base can become complex. Tools
appropriate when there are a large number of possible exist for knowledge base validation, test and maintenance.
outcomes. Should a hypothesis turn out to be false, the Guidelines and references are given by Pau [14].
system can undo conclusions that led to or followed from
the false hypothesis. A further question is one of acceptance and compliance.
Will the intended users enter the (correct) information
Some sets of features that occur during problem solving requested by the expert system and will they accept the
require specific actions. These are solved usually by IF advice offered by the expert system? Many medical
(features pattern) THEN (take action) rules. This type diagnostic expert systems seem to fail to acquire common
of inference mechanism is called ’recognize/act’. use because they do not address these sorts of problems.
In a constraint-directed search the main skeleton of what
is acceptable is known, but not exactly what it is.
Programming languages and hardware
Metarules are part of the procedural knowledge of expert
systems. They are used to evoke programs and to guide The first hardware architecture widely used in computers
the inference process and thus define how to use the rules consisted of a central processor connected by a single data
in the system. They are basic blocks of (usually) bus to memory. The processor, acting on instructions from
IF_THEN rules for exhaustive mapping. the software program stored in its memory, dictates what
is to be done at any given time, sends data to and receives
The inference engine may make use of one or a number data from memory and performs the bit operations that
of these problem-solving mechanisms. are the basis of computations and logic operations at the
machine (semi-conductor) level. As well as holding the
stored program, the memory stores initial data and
Problems with expert systems intermediate and final results of computations. Even very
complex computers still build on this simple (von
Since the expert system is built using knowledge from the Neumann) architectural unit.
expert, it may perform more or less like an expert.
However, as an expert system does not get tired, it will As the system becomes more complex, external memories
give more consistent results than its human counterparts. are added, together with input/output units to allow the
A good expert system may even give better results than system to communicate with the outside world. This
human experts performing at their best. requires an increase in the number of pathways for
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
Table 5. Computer languages. design is the transputer chip designed to run the software
language Occam [17]. The transputer is a single chip
Generation Characteristics Examples computer with a processor, local memory, reset, clock and
the necessary interfaces for communication. Such a chip
Object-oriented Smalltalk, C + + avoids most of the communication problems of conven-
Variable unification Prolog, LISP tional computers and can be built into arrays and
backtracking
Recursion Pascal, Modula, ADA, C networks that resemble more closely the sort of structure
Parameter procedures Fortran, Cobol found in the human brain. The Occam language has an
Macros, ’go to’ Basic architecture that corresponds to the transputer so that
Machine code Assembler hardware and software is compatible in a structural way.
Theoretically, at least, this makes the system more robust
and would allow it to continue to perform even if a part
is removed.
communication. Digital signals representing memory
locations, control instructions and data are passed along
such buses. An internal clock is required to pace the
Neural networks
operations and to ensure that all the parts of the computer
stay synchronized. In fact software control and syn-
chronization (housekeeping functions) become a major
The human brain does not solve problems by using a
well-defined list of computations or operations. Decisions
concern as complexity increases,, mainly because the
are based almost exclusively on what has been learned
computer operates in a series of sequential steps and each
operation must wait for its predecessors to be completed. by experience. The concept of modelling a system on the
structure ofthe human brain was introduced by McCulloch
Many computer languages (table 5) have been developed and Pitts as the neural model in 1943 [18]. In neural
over the years to deal with this increasing complexity. networks (figure 6), individual processing elements
They range from the machine-code languages, like communicate via a rich set of interconnections with
Assembler, to object-oriented languages, such as Smalltalk. variable weights. Just like biological neurones, processing
Most of these computer languages have been largely elements exist in a variety of types that operate in different
constrained by the von Neumann computer architecture ways. One or more inputs are regulated by the connection
and have been written for serial processing. weights to change the stimulation level within the
The increase in computer capacity building on the von processing element. The output of the processing element
Neumann architecture has been made possible by the
development of surface (quasi two-dimensional) technology Input
in the 1970s and 1980s. As a result, computing capacity
per dollar has been able to double every three years since
the early 1950s [-15]. At first it looked as though the
increase in hardware capacity would meet the increased
demands for power and speed that would be needed to
solve real problems. However, in 1973, Sir James Lighthill
suggested that real problems would require too great an
increase in the power and speed of conventional computer
hardware to be feasible [16]. He called this the ’combina-
torial explosion’. Building intelligence on the base of
slavish searching through data by serial processors was
just not practical--especially not for problems like pattern
recognition that characterize intelligence.
The human brain does not succumb to this combinatorial
explosion in order to identify, for example, a face. In fact,
we find it relatively easy to do this, while artificial image
recognition systems find this task particularly difficult.
Some capacity problems can be solved by parallel
processing. Human intelligence is of a concurrent nature
and therefore many applications of artificial intelligence
concern processes that occur concurrently and are best
solved by processing in parallel. Actually, it is possible.
and not unusual to emulate parallel processing using serial
processors, but this requires extremely complex control
mechanisms to interrupt and synchronize the concurrently
running processes.
Attempts have been and are being made to design systems
more suited to solving the mixed sequential and concurrent
Output
nature of real world problems, in which the software and
hardware architectures are superimposable. One such Figure 6. Elements of a neural network.
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
is related to its activation level and this output may be Such physical constraints of structure have led researchers
non-linear or discontinuous. The network is taught rather to look for alternatives to the semiconductor approach to
than programmed and teaching results in an adjustment hardware. Optical signals have several advantages over
of interconnect weights, depending on the transtr electrical signals as a basis for computing. For example,
function of the elements, the details of the interconnect they may allow speeds three orders of magnitude faster.
structure and the rules of earning that the system follows. Light beams do not need carriers and can even cross
Thus memories are stored in the interconnection network without interacting and are thus more amenable to three-
as a pattern of weights. Information is processed as a dimensional structures. The main limitations to this
spreading changing pattern of activity distributed across approach are related to the materials used. Some progress
many elements. A short review of neural networks and has been made, for example using holographic images as
their potential applications in biotechnology is given by the connectors between opto-electronic neurones in
Collins [19]. This includes a table of software tools that photoreactive crystals [20-], still optical computing is
are available for neural net development. probably more science fiction than fact. Attempts are also
being made to develop hardware on a biological or
The neural network is thus a third type of information- molecular basis.
processing technology (the other two being conventional
software languages and expert systems). The .neural
network is good at doing the sort of things the human A model of human intelligence?
brain is good at, in particular pattern classification and
functional synthesis. These will be described in the section While the expert system can justify its reasoning strategy,
on applications. The neural network acts as an associative
declaring which search space was used, explaining
memory, it is good at generalizing from specific examples, decisions taken and the certainties involved, this is not
it can tolerate faults in the network and it can be possible with a neural network. As indicated above, neural
self-organizing. Differences between neural networks and networks process time-dependent spatial patterns and
expert systems..are given in table 6. have to be trained rather than programmed.
In theory, part of the intelligent performance of neural just as we have no difficulty recognizing a face, but a
networks relies on the physical structure of the network. great deal of difficulty trying to explain how or why we
In practice, neural networks are usually simulated in recognized the face, a trained neural network can
computers using conventional hardware, so that the recognize the solution to a problem but cannot really
software structure and the hardware structure are not explain why. A recent paper by Pau and G/itzsche [21]
physically related. addresses this question. Future AI systems may come to
rely on a combination of the experienced neural network
Constructing networks in hardware is a problem. The to recognize the solution to a problem and an expert
reason is that semiconductor technology is essentially two system to rationalize the solution given. This combination
dimensional, whereas the physical embodiment of neural is termed a cognitive expert system and provides an
networks is basically three dimensional. All integrated
circuits become problematic when they are miniaturized.
interesting model of human intelligence.
Connections act as antennas over minute distances leading
to cross talk; electrical signals are slowed by inductance
APPLICATIONS
and capacitance; the. dwell’ time of electrons passing
rapidly may not be sufficient to effect a change in the Introduction
circuit; there may be problems with the random distribu-
tion of circuits on the microscopic scale; electrons may
In the clinical laboratory, improvements in instrumental
even cause breaks by electromigration of atoms. Apart
systems as a result of the intensive use of microprocessors
from this there are significant problems with the slow and better sensors have increased dramatically the rate
speed of off-chip connections and physical limits to the at which measurement data (both quantitative and
number of connections that can be made to a chip.
qualitative) can be obtained. This has been termed ’data
affluence’. However, such complete and complex data do
Table 6. Expert systems and neural networks compared. not represent information in a really useful form.
Several tools, including expert systems, are available for
Expert system Neural network
the interpretation of data or basic information into useful
Software defined Hardware defined information, using either conventional data processing or
Knowledge base Structure Processing elements neural networking systems that are adequately trained.
Inference engine Interconnections A major distinction can be made between, on the one
Standard computer Memory Interconnect weight
hand, stand-alone systems (examples in table 7) that interact
(transputer) pattern
with the user, the main purpose of which is to replace
Programmed Learning Trained
Can generalize human experts, and embedded systems that are integrated
Depends on knowledge Action mode Processes time- into large electrical or mechanical systems and function
engineer dependent spatial autonomously.
patterns The classical illustration of this is two different chess-
Self organizing
Small tolerance Faults Very high tolerance playing systems. The first is a computer program with the
potential to beat experts: players communicate with the
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
system via a keyboard terminal and text display. The it is relatively simple to reuse the system shell for a different
second plays poor chess, but it inspects the board via a knowledge base. Despite the development of AI languages
TV camera, makes its own moves for itselfwith a computer that are more efficient in declaring goals and acquiring
driven hand, explains its moves in passable English, knowledge, most expert systems are still written in
improves its play with practice and can accept strategic conventional languages (for example C and Fortran).
hints and advice from a tutor. Both of these systems
represent fairly advanced projects and both would
Many expert system development tools are therefore
derived directly from existing expert systems simply by
probably be regarded as intelligent by most people. The
stand-alone system is intelligent in a theoretical sense, removing the knowledge base. The classic example is the
while the second instrument is an example of what can
EMYCIN tool developed at Stanford University (Van
be called an integrated system with embedded AI. Melle et al. [25]) from the MYCIN expert system. The
newer tools for development applications are better than
Developing a stand-alone computer expert system that is EMYCIN. Using such development tools, rules and
capable of beating international grand masters would be decisions can be changed with a minimum of inconvenience
a major task for the knowledge engineer. However, chess
and the system iterated until it performs at least as well
programs operating at a popular level are now fairly as the human expert would.
widespread. The chess-playing machine would be more
challenging in many areas such as image processing, Some examples of these development tools are given in
mechanical manipulation, and natural language inter- table 8. In some tools, the knowledge base is entered in
facing. the form of production rules. In others, the knowledge is
entered either as a matrix of attributes and outcomes from countries) under the European Community’s AIM
which the system derives rules, as the rules themselves, or Research and Development Programme is to integrate
as networks of interactions. Purely rule-based systems have knowledge-based systems and telematics with laboratory
the limitation that the structural and strategic concepts information systems and equipment.
must be incorporated in the rules on entry.
Trendelenburg [39] makes a clear distinction between
The evaluation of tools for developing expert systems is internal medicine, with its large knowledge domains, where
discussed by Rothenberg et al. [26]. Predictably, the there is little computer-supported data flow and not much
choice of tool depends on the domain, the problems to be familiarity with medical computer applications; and
solved and the expertise available. The target environment laboratory medicine, where the domains are clear cut, staff
may impose restraints, for example, the need to integrate are familiar with medical computer applications and the
the shell into existing hardware and with existing software. data flow is computerized. More than 50 European
Clearly the user interface must be tailored to the user. clinical laboratories are applying or developing knowledge-
Such development tools allow the domain expert, together based systems for generating reports on special findings
with the knowledge engineer, to create and debug a in an instructive and interpretative way. Trendelenburg
knowledge base and integrate it into an expert-system emphasizes that discussions about the computer-based
framework that provides the necessary interference engine nature of the reports produced are best avoided, and the
and user interface. responsibility for using the report must be with the user.
Software tools for neural netdevelopment are also readily Comparisons have been made of the performance of
available. Collins [19] lists 15 such tools, some of which standard computing programs, expert systems and neural
(for .example BrainMaker from California Scientific networks in medical diagnostic situations. For example,
Software) can be implemented on a standard personal Wied et al. [40] discuss the ability of discriminate function
computer. analysis and neural networks to classify DNA ploidy
spectra. Dawson et al. [41-] discuss the use of nuclear grade
as a prognostic indicator for breast carcinoma. Advances
Medical diagnosis in image analysis can reduce the effect of inter-observer
There is a clear need to be able to deal optimally with variability. Both Bayesian analysis and neural networking
the affluence of data and explosion of knowledge still agreed with the human observer for low-grade lesions,
occurring in the medical field. Computer-supported whereas nuclear heterogeneity in high-grade tumours
medical decision making (CMD) systems may be the resulted in poor agreement (only 20 successti21 classifica-
solution and there is evidence that clinical performance tion). Wolberg and Mangasarian [42] discuss a more
may be improved by the use of CMD (see Adams et al. successful comparison using 57 benign and 13 malignant
[27]). samples, in which a multi-surface pattern recognition
system scored one false negative, a neural network scored
Perry [28] reviewed knowledge bases in medicine, looking two false positives, and a decision-tree approach scored
at three basic models. MYCIN and its domain-inde- three false positives. Other examples of neural networks
pendent version EMYCIN are rule-based systems and are used in medical diagnosis are given in table 9.
the basis of expert systems, such as PUFF for lung disease
(Buchanon and Shortliffe [29]) and ONCOCIN for A successful neural network will show a balance between
cancer therapy (used by Stanford University). Causal the error that it exhibits with its training set and its power
models such as CASNET (Weiss et al. [30-]), that was to generalize based on the training set. Reduction of
generalized and extended to form the shell system training error should thus be treated with caution as
EXPERT, support expert systems like AI/RHEUM overtraining can seriously impair generalization power
(Porter et al. [31]) and AI/COAG (Lindberg et al. [32]). (Astion et al. [43]). It is known that the number and size
Present Illness Program, PiP (Pauker et al. [33]), of hidden neuron layers influences the representation
Acid-Base Electrolytes, ABEL (Paril et al. [34]), and capabilities and the generalization power of the neural
INTERNIST-1 (Miller et al. [35]) are hypothesis-based network, yet these parameters are often chosen quite
systems (Bouckaert [36]). arbitrarily.
Many medical problems, especially those related only to Expert systems are used widely in clinical laboratories for
laboratory data, can be solved using conventional the validation of biochemical data. For example, Valdigui
algorithms as they contain data of known imprecision. It et al.[44] have used VALAB, an expert system, incor-
has been suggested that only solutions requiring between porating 4500 production rules and the forward chaining
10 min and 3 h of clinician time are suitable for expert inference engine KHEOPS, to validate more than 50 000
systems (Frenzel [37]). The upper limit may be the
boundary for knowledge representation.
Table 9. Examples of neural networks used in medical diagnosis.
In a review by Winkel [38], 13 expert systems that
emphasize the use of laboratory data were listed. Winkel Domain Reference
points out that for practical reasons (for example entering.
data), it may not be meaningful to introduce an expert Myocardial infarction Furlong et al. [80]
system into the laboratory or elsewhere unless it is ECG Dassen et al. [81-]
Neonatal chest radiographs Gross et al. [82]
integrated into the .hospital or laboratory information Dementia Mulsant [83]
system, and this may be a difficult task. The main purpose IR spectroscopy Wythoff et al. [84]
of the OpenLabs project (with 28 participants from 12
10
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
laboratory reports. The system runs as a silent partner in minimizing the time taken, effective
not detecting failures,
the laboratory organization and is able to dramatically integration of knowledge, ease of maintenance and ex-
reduce the work involved in the routine validation pandability, broad application, the ability to handle
of data. Neural networks are also being applied to clinical uncertainties and a large explanation capability.
laboratory data. Reibnegger et al. [45] show that neural
networks are capable of extracting hidden features, but A major task is to devise schemes of knowledge represen-
tation to allow failure events to be analysed by the merging
emphasize that validation is even more important when
neural networks are used for medical diagnosis based on of very diverse information sources, for example analogue/
laboratory information. digital signals, logical variables and test outcomes, texts
from verbal reports and inspection images. Embedding
the expert systems in a distributed reasoning structure
Embedded systems shortens execution times and allows for easy updating.
Groth and Modn [47] describe a system providing
Embedded knowledge-based systems (i.e. AI systems with real-time quality control for multitest analysers using a
no human interaction) are used in laboratory instrumen- relational data-base management system. This includes a
tation at a variety of levels (see table 10). Unfortunately knowledge-based troubleshooter and advisor and was
they are not as commonly published as stand-alone designed for the PRISMA mutichannel analysis system.
systems for medical diagnosis. Another example is Woodbury [48] who developed an
The following is a brief description of six different expert system for troubleshooting an Hitachi analyser.
applications of embedded systems that illustrate increasing
degrees of compatibility with neural networks; these are: Image understanding vision system
failure detection, user interfacing, image processing,
natural language processing, robotics and machine Bartels et al. [6] describe an image understanding machine
learning. vision system for histology which is based on three expert
systems. A scene segmentation expert system extracts
information from the image. This is an important function
Applications to failure detection, testing and maintenance and it has been shown [49] that machine vision systems
Failure detection, testing and maintenance are knowledge- are able to detect subtle differences in morphologic
intensive tasks that rely on experience [46]). Apart from architecture that the human observer is generally unable
using test procedures, skilled maintenance staff use to discriminate beyond a certain lower limit. The knowl-
heuristics and an understanding of how the system works edge representation is in the form of a semantic net with
to solve a problem. The high level of performance of associated frames and the knowledge is written in declar-
maintenance experts suggests that they use powerful ative statements. In the case of gland imaging for example,
hierarchical structures in problem solving. The goal of the system might look for a lumen (that is defined in terms
expert systems in this area is to exploit this procedural of an ellipse ratio of more than 1"50 and an area less than
knowledge in conjunction with failure detection to 650 pixels) with glandular nuclei grouped within a certain
generate self-testing procedures and to ease the operator distance around the periphery (that are defined under
workload in the man-machine interface. object groups) using a statement such as:
The requirements for an ideal troubleshooting system Gland IS A SINGLE Lumen WITH
include minimizing the probability of false alarms and of a SET OF Object Groups
The different features found by this system are listed and
Table 10. Typical applications of embedded expert systems. called by the second interpretative expert system that
relates diagnostic conceptual knowledge to them. The
Instrument self testing and diagnosis, including ’degree of cibriformity’ in prostate glands is, for example,
calibration of sensors and displays, trouble shooting measured by tracing the circumferences of the lumina and
of instrument.
Handling signal acquisition. Transduction of signals
relating this to total area to obtain form factor ratio. The
from sensors and fast storage of measurement data. structure of the interpretative expert system is very similar
Treatment of measurement data from signal acquisition to the scene segmentation system.
and extraction of features for further interpretation. The third diagnostic expert system is in the form of a
Conversion of data to primary information.
Image processing in microscopy, for the sorting of spreadsheet, with diagnostic categories, clues, clue values
biological structures by size, absorption, colour. and certainty values. This expert system can run auton-
Timing and sequencing of mechanical functions omously as an embedded system or can be run inter-
(especially for multichannel analysers). For example actively as an expert system.
keeping track of position sensors, clock timing,
parallel movements, and prioritization by queuing
theory. User interfaces
Storage and retrieval of information for statistics, The success of knowledge-based instrumentation systems
record comparisons, calibration, standardization.
depends to a great extent on the man-machine interface,
Interpretation and presentation of results. Layout of both during development when the domain expert and
man-machine interface screens, generation and
presentation of reports and explanations in customized knowledge engineer provide the system with a represen-
formats. tation of real knowledge and human reasoning, and later
when the user tries to implement the system.
11
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
Table 11. Generic functions for man-machine interfaces (based not achieve greater accuracy is due to the many strange
on Pau [50]). rules of languages for example co-articulation of syllables.
Japanese and Finnish may actually be rather more logical
Data handling and presentation (acquisition, coding, in this respect than other languages, for example English.
compression, presentation, recording)
Extractingfeatures from measurements for interpretation In what might be an imitation of nature, Kurogi [53]
Control of instrument functions and utilities (trouble- used the anatomical and physiological findings on the
shooting and repair are excluded from the man- afferent pathway from the ear to the cortex to develop a
machine interface) neural network that recognizes spoken words, allowing
Calibration of sensors and displays for changes in pitch, timing etc.
Development of procedures and experiments (for
example sequencing experimental steps)
Generation of reports and explanations Robotics
Retrieval of user information/documentation (for
example about the experimental set-up). The mechanical functions of a robot are like a human is,
but remain relatively unintelligent if the robot does not
include sensor or recognition systems that can allow it to
know its environment. Robots are particularly useful for
Embedded expert systems may not require a user inter- control and handling of samples in sterile or hazardous
face. Many embedded systems are controlled by a com- environments, in situations where a human operator
puter system, or are designed, to communicate with other either cannot or should not be present [3].
expert systems for example through what is called a
’blackboard’ mechanism. Robotic arms are usually programmed to perform one
specific task ata time. The intelligent robotic arm would
Pau [50] has described generic functions for man-machine be able to move across a variety of pathways carrying a
interfaces (see table 11). These functions can be backed variety of loads. The adaptive robotic arm is a typical
up by knowledge-based techniques to render them in- application area for expert systems and neural networks.
telligent. Apart from the knowledge bases themselves, For example, Kawato and coworkers in Osaka have
such techniques would incorporate search and inference developed an adaptive robot arm controller based on a
to suggest feasible actions to the user, and rule-based model of human motor control and neural networks,
spreadsheets or other presentation systems for display, having three degrees of freedom.
control and explanation.
The potential of user interfaces goes far beyond simple Machine learning
screen interaction. Tactile and vocal interaction can be
added to allow freedom of movement, with a possibility As early as 1950, Turing [54] foresaw the potential of
of communication and delegation. This type of system systems capable of autonomous learning and admitted
that he would regard a machine that could program itself
(known as virtual reality) is being developed for com-
mercial purposes, for example in the entertainment in- as intelligent.
dustry. It is possible to imagine how such systems could Bartels et al. [55] discuss the issues involved in the design
be used for example for training surgeons and there is a of autonomous-learning expert systems in histopathology.
challenge now for architects and designers to invent new The autonomous learning process must have a built-in
forms of knowledge representation based on these new control function to keep the process goal-oriented (and
technologies that have become available. by implication the process must have a goal and criteria
defined by which progress towards the goal can be
Natural language processing assessed).
Landau et al. [51] argue that computer-aided medical
A decisive element in expert systems is the structuring of
diagnosis will not attain widespread acceptance among knowledge and the interrelationships between knowledge
physicians until it can include automatic speech recogni- components. In histopathology, facts are obtained from
the human observer or from the image processing and
tion and proposed a prototype natural language interface.
Speech recognition is not just a matter of words in a series. image-extraction programs. One type of learning consists
Not surprisingly, intelligent use of language depends on of updating a knowledge base with new blocks of data
an understanding of the subject. Context, guessed mean-
that can fine tune or stabilize decision rules. However,
real autonomy is first reached in systems that can augment
ings and memories of speech fragments have a lot to do their own rule sets or, like neural networks, are self-
with being able to hear what is being said. This is a
pattern-recognition problem and typical of the human organizing.
brain.
Kohonen [52] developed a phonetic typewriter using a CONCLUSIONS
competitive filter neural network. This system achieves
92 to 97 accuracy, with continuous speech and Introduction
unlimited vocabulary for Finnish and Japanese languages.
One hundred words of each new speaker are taken as This review has looked at the development of pro-
training samples. The network automatically classifies grammed software, the flexibility this has allowed in
speech sounds and stores related sounds spatially near system operation and control, and how this has led to
each other in the network. The fact that the system does intelligent clinical laboratory systems.
12
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
The knowledge base of expert systems contains the col- Berne Convention for the Protection of Literary and
lective expertise of domain experts (in the clinical labor- Artistic Works.
atory and related medical domains) and the knowledge
engineers who represent this expertise. An unresolved
question concerns the ownership and exploitation rights Evaluation of expert systems
of such knowledge bases.
Miller [62] distinguishes three levels of evaluation of
Since this expertise will be used to guide medical decision
expert systems; evaluation of research contribution of a
making, it will be subject to regulation by the appropriate
authorities. Just like any other medical device, expert development prototype, validation of knowledge and
systems need to be evaluated and documented. Re-
performance and evaluation of the clinical efficacy of an
sponsibility for the consequences of any inappropriate use operational system.
of knowledge bases and AI rests with the professional user Evaluation of developmental prototypes is an iterative
of such systems. process and the requirements of a particular system will
change as the system matures towards the final operational
system. One of the most interesting questions is how to
Regulatory affairs evaluate and limit a neural network that can continue to
Kahan [56] has reviewed US Food and Drug Adminis- learn.
tration (FDA) regulation of medical device software. In
the US, the FDA regulates medical device software via a Knowledge should be assessed in terms of accuracy,
draft GMP Guidance for software [57] and a draft completeness and the consistency and performance of the
reviewer guidance [58]. This is supported by a technical system matched with that of experts. A critical aspect is
report [59] used by FDA investigators. the capacity of the system to know its own limitations
and, for example, to fail to make a diagnosis when
The 1987 Draft Policy on Software [60] is applied to presented with an unfamiliar case. At best expert systems
computer products that are intended to affect the diag- used to perform just as well as the experts themselves (Yu
nosis and treatment of patients. It was intended to be et al. [63], Quaglini and Stefanelli [64]). This situation
primarily applicable to stand-alone medical devices, rather is changing as validation, verification and maintenance
than to software that is a component of a medical device tools are introduced (Pau [14]).
(embedded). It does apply to some stand-alone AI prod-
ucts now entering the US market. For example, in 1988 Evaluation of an AI system in operation depends on the
the FDA decided to include stand-alone software for blood domain and the clinical role it is expected to play. There
banks under regulation (these were previously exempt). are a number of issues of interest that are very difficult
The 1987 Draft Policy will probably not be finalized and to assess, for example the impact of the system on the
stand-alone software will probably be treated as exempt quality of health care and the user’s subjective reaction
from premarket approval except when a risk is perceived. to the system. Mainly because of this there are few
evaluations of operational systems.
Embedded systems are a more difficult question. In
November 1990 a Canadian company issued a safety
bulletin because their embddded software to calculate The need for transparency
patient dose in radiation .therapy was flawed. This
sharpened FDA interest. The draft GMP guidance for We would like greater transparency in the application of
software is specific about the analysis of software and for artificial intelligence in the clinical laboratory. It is
example, stipulates that software functions used in a device particularly difficult in the clinical laboratory environ-
must perform as intended and testing should verify this. ment to evaluate embedded expert system as they gener-
Such software must be tested separately by simulation ally appear as black boxes to the user. Wulkan [7] asks
testing in the environment in which the software will be the question who is to be held responsible if the system
used. contributes with faulty information that could result in
a decision that ultimately proves fatal to the patient?
The Health Industry Manufacturers’ Association (HIMA)
has expressed concern about the FDA Guidance--for Hoffmann [65] suggests that responsibility should lie with
example arguing that source codes should not be copied those who apply such systems to provide information for
during inspection by FDA and confidential algorithm the clinic.
specifications should not be revealed. There is a danger that expert systems may lend authority
In Europe, a recent EC Medical Device Directive to and even prejudices to shallow decisions by not disclosing
Trade Associations includes software in the definition of the exact nature of the inference mechanisms, not iden-
medical devices for the first time. Previously the definition tifying the human experts who have provided the expertise
or the knowledge engineers who have interpreted this
only included software that was incorporated into a
device, but now the definition includes software itself expertise for the expert system, or not disclosing the
(which presumably means stand-alone systems). This has learning set used to train a neural network. Neural
caused some confusion among European manufacturers. networks in particular are less amenable to logical ex-
planation, although they can be evaluated in terms of the
In May 1991 the EC issued a Directive [61] on the legal training data and the network performance on test data
protection of computer programs, requiring member (Hart and Wyatt [66]). There are positive signs that this
states to protect such programs by copyright under the situation is beginning to change, with the introduction of
13
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
explanation facilities that are also implemented on neural H., BIBBI, M., GRAHAM, A., PAPLANUS, S., SHOEMAKER,
10. BARTELS, P.
networks (Pau [14]). R. L. and THOMPSON, D., Analytical Cellular Pathology, (1989),
195.
It would seem ironic if the mechanisms of artificial 11. Du Bos, P., HERMANUS, J. P. and CANTRAINE, F., Medical Informatics,
intelligence were applied ineffectively, indeed unintelli- 14 (1989), 1.
gently, to clinical laboratory information. Concerns have 12. SIELAFF, B. H., CONNELLY, D. P. and SCOTT, E. P., IEEE Transac-
been expressed that applications of artificial intelligence tions on Biomedical Engineering, 36 (1989), 541.
13. COMPTON, P., Clinical Chemistry, 36 (1990), 924.
could lead to a false uniformity and discourage dissenting
14. PA:, L. F., Guidelines for Prototyping, Validation and Maintenance of
opinion by crystallizing the majority opinion and current Knowledge Based Systems Software (Electromagnetics Institute, Tech-
state of the art in a more permanent form. Again the nical University of Denmark, 1990).
situation seems to be improving and it now seems more 15. TESLER, L. G., Scientific American, 265 1991 ), 54.
likely that minority opinions can be included in the 16. ALEKSANDER, I., In M. Reeve and S. E. Zenith (Eds.), Parallel
accumulation of incremental knowledge as the knowledge Processing and Artifical Intelligence (John Wiley & Sons, Chichester
base is built up. 1989).
17. FORNILX, S. L., Research and Development (Cahners Publishing,
We believe that it would be usethl to look at the Hoofddorp, 1989).
knowledge base in an AI system as a form of’collective 18. McCULLOCH, W. S. and PITTS, W., Bulletin of Mathematical Biophysics,
intellectual property’ and to elaborate a set of recom- 5 (1943), 115.
mendations for its exploitation. 19. COLLINS, M., Bio/technology, 11 (1993), 163.
20. PSALTIS, D., BRADY, D., Gu, X. G. and LIN, S., Nature, 343 (1990),
325.
21. PAu, L. F. and GSTZSCHE, T., Journal of Intelligent and Robotic Systems
Items for future debate (April 1992).
22. LEVY, M., FERRAND, P. and CmRAT, V., Computers in Biomedical
This review could be supplemented with a survey, Research, 22 (1989), 442.
in the form of a.questionnaire, concerning applications of 23. PEYRET, M., FLANDROIS, J. P., CARRET, G. and PICHAT, C.,
embedded systems in laboratory instrumentation. Pathologic Biologie, 37 (1989), 624.
24. BRONZINO, J. D., MORELLI, R. A. and GOETHE, J. W., IEEE
We recommend that a Working Group should be set up Transactions on Biomedical Engineering, 36 (1989), 533.
to draft guidelines for applications of artificial intelligence 25. VAN MELLE, W., SHORTLIFFE, E. H. and BUCHANAN, B. G., Machine
Intelligence, Infotech State of the Art Report, Series 9 (1989), No. 3.
in the clinical laboratory. For expert systems, recom-
26. ROTHENBERG, J., PAUL, J., KAMENY, I., KIPPS, J. R. and SWENSSON,
mendations should be considered for the identification of M. Evaluating Expert System Tools. A Framework and Methodology (The
the domain experts who have contributed to the expert RAND Corporation, Santa Monica, 1989).
system, the representation used (and its authorship) and 27. ADAMS, I. D., CHAN, M. and CLIFFORD, P. C., British Medical Journal,
the inference strategy used. For neural networks, we 293 (1986), 800.
believe that the Working Group should consider recom- 28. PERRY, C. A., Bulletin of the Medical Libraries Association, 78 (1990),
mendations for the training data and network perform- 271.
ance on test data. 29. BUCHANON, B. G. and SHORTLIFFE, E. H., (Eds.) Rule Based Expert
Systems: The MYCIN Experiments of the Stanford Heuristic Programming
In addition, we would like to see the Working Group Project (Addison-Wesley, Stanford, 1984).
debate the question of intellectual property rights of 30. WEISS, S. M., KULIKOWSKI, C. A., AMAREL, S. and SAFIR, A., Artificial
authors of expert systems and additional knowledge that Intelligence, 11 (1978), 145.
31. PORTER, J. F. KINGSLAND, L. C., LINDBERG, D. A. B. and SHAH,
may be added to an existing knowledge base. Finally, the I., Arthritis and Rheumatism, 31 (1988), 219.
Working Group might debate the question of copy. 32. LINDBERG, D. A. B., GASTON, L. W., KINGSLAND, L. C. and
protection of commercial systems in relation to publica- VANIER, A. D., In Proceedings of the Fifth Annual Conference on
tion and the need for greater transparency. Computer Applications in Medical Care (November 1981).
33. PAUKER, S. G., GARRY, G. A., KASSIRER, J. P. and SCHWARZ, W. B.,
References American Journal of Medicine, 60 (1976), 981.
34. PARIL, R. S., SZOLOVITS, P. and SCHWARTZ, W. B., In Szolovits, P.
1. B/STTNER, J. and HABRICH, C., XIII International Congress of Clinical (Ed.) Artifical Intelligence in Medicine (Westview Press, Boulder, 1982).
35. MILLER, R. A., POPLE, H. E. and MYERS, J. D., New England
Chemistry. (GIT Verlag, Darmstadt, 1987).
2. SKEOGS, L. T., Analytical Chemistry, 38 (1966), 31A. Journal of Medicine, 307 (1982), 468.
3. OZAWA, L., SCHNIPELSKY, P., PARDUE, H. L., PLACE, J. F. and 36. BOtJCKAERT, A., International Journal of Biomedical Computing, 20
TRUCHAUD, A., Annales de Biochimie Clinique, 49 (1991), 528. (1982), 123.
4. PARDUE, H. L., OzAwA, K., PLACE, J. F., SCHNIPELSKY, P. and 37. FREZEL, L. E., Understanding Expert Systems (Howard W. Sams &
TRUCHAUD, A., Systematic top-down approach to clinical chemistry Co., Indianapolis, 1987).
Annales de Biologie Clinique 52 (1994) 311-320. 38. WNKEL, P., Clinical Chemistry, 35 (1989), 1595.
5. CAU)ILL, M. and BUTLER, C., Naturally Intelligent Systems. (MIT 39. TRENDELENBURe, C. and POHL, B., Anales de Biologie Clinique, 51
Press, Boston, 1990). (1993), 226.
6. BARTELS, P. H. and WEBER, J. E., Analytical and Quantitative Cytology 40. WIED, G. L., BARTELS, P. H., BIBBO, M. and DYTCH, H. E., Human
and Histology, 11 (1989). Pathology, 20 (1989), 549.
7. WULKAN, R. W., Thesis: Expert systems and multivariate analysis 41. DAWSON, A. E., AUSTIN, R. E. and WENBER, D. S., American
in clinical chemistry (Erasmus University, Rotterdam 1992). Journal of Clinical Pathology, 95 (1991), 29.
8. SHORTLIFFE, E. H., Computer-Based Medical Consultations: MYCIN 42. WOLBERe, W. H. and MANGASARIAN, 0. L., Analytical and Quanti-
(Elsevier, New York, 1976). tative Cytology and Histology, 12 (1990), 314.
9. O’CONNOR, M. L. and McKNNEY, T., Archives of Pathology and 43. ASTION, M. L., WENER, M. H., THOMAS, R. G., HtNDER, G. G. and
Laboratory Medicine, 113 (1989), 985. BLOCH, D. A., Clinical Chemistry, 39 (1993), 1998.
14
j. F. Place et al. Use of artificial intelligence in analytical systems for the clinical laboratory
44. VALDIGUIt, P. M., ROGARI, E. and PHILIPPE, H., Clinical Chemistry, 66. HART, A. and WYATT, J., Medical Informatics, 15 (1990), 229.
67. WEBER,J. E., BARTELS, P. H., GRISWOLD, W., KUHN, W., PAPLANUS,
45. REIBNEGGER, G., WEISS, G., WERNER-FELMAYER, G., JUDMAIER, G. S. H. and GRAHAM, A. R., Analytical and Quantitative Cytology and
and WACHTER, H., Proceedings of the National Academcy of Sciences, 88 Histology, 10 (1988), 150.
(1991), 11426. 68. WEBER, J. E. and BARTELS, P. H., Analytical and Quantitative Cytology
46. PAU, L. F., Expert Systems, 3 (1986), 100. and Histology, 11 (1989), 249.
47. GROTH, T. and MODIN, H., Computer Methods and Programs in 69. LENG, Y. W., PAu, L. F., SIGAARD ANDERSEN, O. and GOTHGEN,
Biomedicine, 34 (1991), 175. I. H. Proceedings A.A.A.I. Symposium, AI in Medicine, (Stanford, CA,
48. WOODBURY, W. F., Laboratory Medicine, 20 (1989), 176. March 1990).
49. PAPLANUS, S. H., GRAHAM, A. R., LAYTON, J. M. and BARTELS, 70. WIENER, F., DE VERDIER, C.-H. and GROTH, T., Scandinavian ffournal
P. H., Analytical and Quantitative Cytology and Histology, 7 (1985), 32. of Clinical Laboratory Investigation, 50 (1990), 247.
50. PAU, L. F., Jinko Chino Gakkaishi, 5 (1990), 106. 71. PFURT$CHELLER, G., SCHWARZ, G., MOIK, H. and HAASE, V.,
51. LANDAU, J. A., NORWICH, K. H. and EVANS, S. J., International Biomed. Tech. Biomedicinische Technik, 34 (1989), 3.
Journal of Bromedical Computing, 24 (1989), 117. 72. BONADONNA, F., Medical Informatics, 15 (1990), 105.
52. KOHONEN, T., In I. Aleksander (Ed.), Neural Computing Architectures 73. ALONSO-BETANZOS, A., MORET-BONILLO, V. and HERNADEZ-SANDE,
(MIT Press, Harvard, 1989). C., IEEE Transactions on Biomedical Engineering, 38 (1991), 199.
53. KUROGI, S., Biol. Cybern., 64 (1991), 243. 74. WEiss, S. M. and KULIKOWSKI, C. A., In Proceedings Sixth International
54. TURING, A. M., Mind, 59 (1950), 433. Conference on Artifical Intelligence (Information Processing Society of
55. BARTELS, P. H., WEBER, J. E. and DUCKSTEIN, L., Analytical and Japan, Tokyo, 1979).
Quantitative Cytology and Histology, 10 (1988), 299. 75. GILL, H., LUDWIGS, U., MATELL, G., RUDOWSKI, R., SHAHSAVAR,
56. KAHAN, J. S., Clinica, 463 (1990), 14. R., STROM, C. and WIGERTZ, O., Int. J. Clin. Monit. Comput., 7
57. Medical Device GMP Guidance for Investigators: Application of Medical (1990), 1.
Device GMPs to Computerized Devices and Manufacturing Processes (US 76. MARCHEBSKY, A. M., Analytical and Quantitative Cytology and Histology,
FDA Draft, November 1990). 13 (1991), 89.
58. Reviewer Guidance for Computer Controlled Medical Devices undergoing 77. MERRY, M., GEC Journal of Research (1983), 39.
510(k) Review (US FDA, Second Draft, February 1991). 78. DE SWAAN, A. H., GROEN, A., HOVLAND, A. G. and STOOP, J. C.,
59. Software Development Activities--Reference Materials and Training Aids Delft Progress Report, 9 (1984), 63.
for Investigators (Office of Regulatory Affairs, US FDA;July 1987). 79. POHL, B. and TRENDELENBURG, C., Methods of Information in Medicine,
60. Draft Policy for the Regulation of Computer Programs (US FDA, reissued 27 (1983), 111.
November, 1989). 80. FURLONG, J. W., DuPtY, M. E. and HEINSlMER, J. A., American
61. European Council Directive
91/250/EEC (May 1991 ). Journal of Clinical Pathology, 96 1991 ), 134.
62. MILLER, P. L., Computer Methods and Programs in Biomedicine, 22 81. DASSEN, W. R., MULLENEERS, R., SMELTS, J., DEN DtLI, K. and
(1986), 5. CRUZ, F., Journal of Electro-cardiology 23 (1990), 200.
63. Yv, V. L., FAGAN, L. M. and WRAITH, S. M., Journal of the American 82. GRoss, G. W., BOONE, J. M., GRECO-HUNT, V. and GREENBERG, B.,
Medical Association, 242 (1979), 1279. Invest. Radiol., 25 (1990), 1017.
64. QtJAGLINI, S. and STEFANELLI, M., Computers in Biomedical Research, 83. MULSANT, B. H., MD Comput., 7 (1990), 25.
21 (1988), 307. 84. WYTHOFF, B. J., LEVINE, S. P. and TOMELLINI, S. A., Analytical
65. Hoffmann Arbeitsgruppe Ger/ite-evaluation: Interaktion zwischen Chemistry, 62 (1990), 2702.
mechanisierten Analyseger/iten und wissensfor-arbeitenden Sys-
temen. Mitteiling Deutsche Gesellschaftfr Klinische Chemie, (1990), 25.
15