NORTH- HOLLAND

Innovation Forecasting
R O B E R T J. WATTS and A L A N L. P O R T E R

ABSTRACT
Technological forecasting is premised on a certain orderliness of the innovation process. Myriad studies
of technological substitution, diffusion, and transfer processes have yielded conceptual models of what matters
for successful innovation, but most technological forecasts key on limited empirical measures quite divorced
from those innovation process models. We glean a number of concepts from various innovation models, then
present an array of bibliometric measures that offer the promise of operationalizing these concepts. Judicious
combination of such bibhometrics with other forms of evidence offers an enriched form of technological
forecasting we call "innovation forecasting." This provides a good means to combine technological trends,
mapping of technological interdependeneies, and competitive intelligence to produce a viable forecast. We
illustrate by assessing prospects for ceramic engine technologies. 1997 Elsevier Science Inc.

Introduction

Technological forecasting purports to provide timely insight into the prospects for
significant technological change. Such information should help management make better
decisions with regard to strategic corporate planning, R&D management, product development, investment in new process technology, production and marketing, purchasing
of new technology, and so forth. Technological forecasting encompasses varied objectives, time horizons, and approaches (c.f. [1-3]). Table 1 offers one listing of forecasting
techniques as categorized by Vanston [4].
The conceptual foundation upon which technological forecasting rests is a degree
of orderliness in the innovation process. Emergence of new or improved technologies
depends on successful completion of the innovation process--"any system of organized
activities that transforms a technology from an idea to commercialization" [5].
We draw upon various innovation and technological change models to generate a
set of concepts pertinent to gauging the prospects of particular technologies becoming
successes. Various technological forecasting approaches provide context for our "innovation forecasting." This draws particularly upon bibliometrics, which we briefly overview.
We propose candidate bibliometric measures to operationalize a number of innovation
ROBERT WAITS is on the senior staff of the U.S. Army Tank-automotive and Command Armaments
(TACOM), Warren, MI.
ALAN PORTER directs the Technology Policy and Assessment Center and is Professor of Industrial
and Systems Engineering, and Public Policy, at Georgia Tech.
Address reprint requests to Robert Watts, 22430 St. Clair Drive, St. Clair Shores, MI 48081. E-mail:
wattsb@cc.tacom.army.mil
For information about Technology Opportunities Analysis, contact Alan Porter. E-mail: alan.porter@isye.
gatech.edu
Technological Forecasting and Social Change 56, 25-47 (1997)
1997 Elsevier Science Inc.
655 Avenue of the Americas, New York, NY 10010

0040-1625/97/$17.00
PII S0040-1625(97)00050-4

26

R.J. WATFS AND A. L. PORTER

TABLE 1
ForecastingTechniques

Extrapolator
Goal analyst
Intuitor
Delphi surveysb
Technical trend analysis~
Implication analysis
Content analysisb
Nominal group conferenceb
Substitution analysisa
Structured/unstructured interviewsb
Growth limit analysis'
Stakeholder analysis
Patent analysisb
Learning curvesa
Comprehensive opportunity analysisb
Pattern analyst
Counterpuncher
Analogiesb
Scanning
Monitoringb
Precursor trend#
Morphological analysis
Alternate scenario planningb
Feedback models~
Monte Carlo models'
Source: Vanston, John H., Technology Futures Inc., Austin, Texas [4]. [See also Note Added in Proof
appearing before the reference list.]
aHighly quantitative.
bSemi-quantitative.
concepts. The second part of this article illustrates innovation forecasting by showing
how selected measures can be combined with other information to assess the prospects
for ceramic engine innovation.
Toward Innovation Forecasting
A t t e m p t e d technological innovation may or may not be successful. Successful innovation relies on many variables, including the technology's characteristics, the fit between
the innovating firm and the technology, familiarity of the firm with the market and
associated infrastructure, market forces, the economic climate and resource commitments, other socioeconomic factors, and institutional actions or interactions (c.f. [3, 5,
6]). Underlying each of these variables there exist organizational elements that view a
new technology from different perspectives and act to influence its development toward
their own interests [7]. For example, in assessing national strategic value, a country
might well appraise a new technology's trade balance implications, the technology's
indispensability (i.e., viability of substitutes and associated economic impacts, if displaced), the development capacity and pervasiveness (e.g., diffusion capacity to other
industries), the available work-force skills and know-how, the versatility and flexibility
of the resulting industry, the exploitation of resources, and the attendant environmental
impacts [6, 8].
Much research on technological innovation documents the factors that either promote or inhibit successful product development. Many researchers have performed
p o s t m o r t e m assessments of technology transfer activities, technology diffusion, and
technology substitution processes to characterize significant factors and recommend
managerial practices that promote success in new product technology innovation (c.f.
[9, 10]. We have scavenged "innovation success" concepts from various sources. In particular:
Michael Porter's four-factors framework highlighting the importance of various
competitive forces [6]
William Souder's identification of statistically significant organizational factors
relating to either the technical or commercial success of an innovation project [5]
Steven Dunphy et al.'s juxtaposition of the factors of an "innovation funnel" [11]
Clayton Smith's specification of levels and forms of substitution [12]
Ted Modis' observations on compatibility with infrastructure and complementary
products [13]

INNOVATION FORECASTING

27

Anderson and Tushman's evidence on the interplay of industry participants [14]

Isreal Dror's use of patent information to infer design standards [15]
Metcalfe's technology diffusion considerations [16, 17]
Cetron, Cohen, and Rogers each identifying sets of factors contributing to technology transfer [18-20]
Souder et al. on roles of sponsor and adopter [21]

We will return to the innovation success concepts in the next section.

The technological literature is immense. To capture some of the information inherent in the content and patterning of the literature, a field called "bibliometrics" has
emerged. Bibliometrics uses counts of publications, patents, or citations to measure and
interpret technological advances. Such analyses assume that counts of papers or patents
provide useful indications of R&D activity and of innovation, depending on the sources
examined. Another key tenet is that one can ascertain important links by analyzing
which topics occur together, which organizations produce what papers and patents, and
who cites what [22]. Bibliometric applications range from the strategic (e.g., classifying
British science [23]) through the tactical (e.g., providing competitive intelligence on
who is doing what on a particular technology). Various forms of bibliometric analysis
have emerged. Citation analysis (c.f., [24]) examines referencing patterns among papers
and/or patents to detect seminal contributions and interaction patterns, and even to
forecast emerging research areas. Patent analysis relates patenting activity to profile
company interests and trends. Publication analyses take articles and such as telling
indicators of R&D activities.
Linkage is a particular interest in bibliometrics, leading to the development of
several analytical approaches based on entities appearing together--co-occurrences [25].
Co-citation analysis identifies pairings of articles jointly cited by later articles. From
these, cognitive structure may be inferred (c.f. [26, 27]). Co-word analysis, dating mainly
from the 1980s in Europe, looks for words appearing together [28]. Some focus on
keywords (index terms); Kostoff has extended these analyses to whole texts (c.f. [29]).
Mapping is particularly useful in facilitating interpretation of bibliometric findings
[30, 311.
Bibliometric limitations need to be noted. Counts do not distinguish quality, and
much technological development work is not reflected in publications or patents, at
least not in a timely manner. Publishing and patenting practices vary considerably across
fields and by institutions (e.g., one company may publish heavily; another, not at all).
Nonetheless there is a wealth of information to be mined using these approaches. Such
information should be combined with other measures and expert opinion to develop a
balanced assessment [32].
Forecasting--including technology or technological ("tech") forecasting---depends
on theory, data, and methods [2]. The theory behind tech forecasting consists of the
conceptualizations of the innovation process in its various guises. The linkage between
that theory and tech forecasting practice is weak. Innovation forecasting seeks to tighten
this linkage to take better advantage of lessons learned in efforts to model innovation
processes. Data for tech forecasting are usually weak. Emerging technologies offer only
short time series potential to begin, but this is typically weakened because a government
has not emphasized collection of tech indicators 1 and industry often seeks to protect
Of course there are many notable sources of data technology; for instance, the U.S. National Science
Foundation's biennial Science and Engineering Indicators, UN Statistical Office data on exports by SITC
categories, Elsevier's Electronics.

28

R.J. WATI'S AND A. L. PORTER

proprietary information. Expert opinion becomes a vital complement to statistical measures. The bibliometric measures advocated herein provide an interesting alternative
data source of both quantitative counts of evidence of R&D activity and interesting
text materials to be exploited. Tech forecasting methods (Table 1) do the best job possible
with limited theory and data. The cornerstone for innovation forecasting is monitoring.
Monitoring is vital in its own right to comprehend "who is doing what now?"
with respect to a technology under scrutiny. It underlies forecasting in two critical
ways--forthcoming tech change is foreshadowed by current developments and will be
influenced by changes in related technologies and socioeconomic influences. Monitoring
of the target technology, related technologies, and the relevant contextual influences
is the most essential ingredient in effective tech forecasting [2].
Innovation forecasting extends traditional monitoring (e.g., literature review) by
tapping the newfound electronic information resources. Information on a given technology often abounds on the Internet and in electronic databases. We emphasize the latter
because the data have been screened and structured. The worldwide web, however, is
a rich "icing on the cake" in providing access to a wealth of more current and more
varied sources. As intelligent search and retrieval tools improve, the web will become
an increasingly valuable source for tech monitoring.
Since 1990, the Technology Policy and Assessment Center (TPAC) at Georgia
Tech has been developing a bibliometrically based approach to technology monitoring,
forecasting, and assessment. Since 1993 this has centered upon development of proprietary software to facilitate exploitation of bibliographic (text) sources--the Technology
Opportunities Analysis TM Knowbot (TOAK). TOAK has enabled us to collect a range
of measures from electronic search results [33]. Those searches take place in large,
publicly accessible databases such as The Engineering Index (ENGI), 1NSPEC, and
U.S. Patents. TOAK capabilities have advanced in an iterative, empirical fashion--the
software enables a tabulation; that leads TOA analysis to request a refinement; the
programmers provide that capability; the analysts working with particular users then
come up with additional desires; and so forth. The result has been a nice growth in
empirical capabilities to identify technology opportunities.
We now turn to using the empirical capabilities of TOA (bibliometrics) to operationalize the innovation concepts compiled from the technological innovation, diffusion,
and transfer literatures. The premises are that those concepts provide important clues
to the potential success of nascent innovations and that those concepts can be measured.
Bibliometrics provide a nicely accessible and cost-effective means to obtain critical
innovation measures in a timely fashion for midterm forecasting (i.e., 3- to 10-year
horizon). The resulting sets of conceptually linked measures, when combined with other
information such as expert opinion, can provide a better basis to forecast the prospects
for successful technological innovation.

Innovation Forecasting
WHAT
Innovation forecasting seeks to garner information on:
1. Technology life cycle status
2. Innovation context receptivity
3. Product value chain and market prospects
Technology life cycle information keys on determining how far along the development

INNOVATION FORECASTING

29

TABLE 2
Technology Life Cycle Indicators
Factor
R&D profile
Fundamental research
Applied research
Development
Application
Societal impacts
Growth rate
Technological issues
Maturation
Offshoots

Indicator
Number of items in databases such as Science Citation Index
Number of items in databases such as Engineering Index
Number of items in databases such as U.S. Patents
Number of items in databases such as Newspaper Abstracts Daily
Issues raised in the Business and Popular Press abstracts
Trends over time in number of items
Technological needs noted
Types of topics receiving attention
Spin-off technologies linked

pathway the technology has advanced, its growth rate, and the status of technologies
upon which it is dependent. Contextual factors include economic and other influences on
development of the target technology. Product value chain issues concern the potential
payoffs and requirements to enable them to be fulfilled. These influences interlink in
complex ways so that our separation is somewhat arbitrary.
Tables 2, 3, and 4 offer our distillation of technological innovation process concepts
for which we believe bibliometric measures can be obtained. Table 5, discussed in the
next subsection, indicates steps to be taken to gather and interpret such measures. To
set the context, our approach calls for downloading a set of bibliographic abstracts (e.g.,
perhaps 100 to 10,000) gathered on the topical technology (or function or product) of
interest. One then tabulates and analyzes that information in various ways to get at the
innovation success indicators.
The technology life cycle indicators begin by locating the focal technology on a
putative life cycle curve. The simplest metric is to count the number of hits from
searching on the technology in various databases that emphasize different stages along
an R&D profile (Table 2). The precise databases to be explored depend on availability2
and the nature of the target technology. For instance, were one probing the status of a
new chemical, Chem Abstracts would be preferable to the general science and technology
databases indicated.
Growth rate can be ascertained by partitioning the item counts, either for the
general technology or for specific contributing technologies, over time. This can combine
nicely with the R&D profile by plotting hits/year in each database studied. In the "clean"
case, one would expect to see the topic first rise, then decline, in fundamental research;
with a similar but lagged pattern in a more applied research database; followed in turn
by evidence of development, application, and possibly impact.
Trend models can be fit to bibliometric time series data. Examination of raw
frequencies can be informative. Moreover, fitting of logistic growth curves (c.f., [2]) to
cumulative frequencies can help one perceive the life cycle with respect to the underlying
aspect being tracked. Evidence of a fast life cycle has significant implications for other
innovation factors too, implying heightened sensitivity to complementary technologies
and the innovator's market experience.
2The TOA approach relies on accessing sets of abstracts in electronic form. To keep costs reasonable,
this implies that one must license access to the databases of prime interest rather than paying for each
abstract downloaded. This is supported by database providers offering CD-ROM and unlimited access dialin subscriptions to their databases.

30

R. J. W A T F S A N D A. L. P O R T E R
TABLE 3
Innovation Context Indicators

Factor
Supporting technologies
Identification
Status
Players
Technology accessibility
Requirements for success
Constraints (regulations, etc.)
Competition
Alternative technologies
Institutional interests
Issues

Indicator
Technologies mentioned in articles on target technology
Technology decomposition
Individuals, institutional affiliations
Patent concentration profiles
Status of standards, government backing, private backing
Regulations
Functional equivalency identification
Profiling competitor interests
Tabulation of issues posed (i.e,, in business press)

Several databases provide technology class codes--for instance, INSPEC, Derwent's

Worldwide Patents. Occurrence of secondary codes in conjunction with a target technology provides an indicator of technology diffusion.
Software such as the Georgia Tech TOAK can facilitate quick tabulation of keywords, title words, abstract words, and abstract phrases. Jumping ahead a bit to our
ceramic engine example, Table 9 notes certain materials, including "superalloys." By
pulling up the abstracts in which the term is prominent, one can see what needs and
issues are being addressed (e.g., Table 9 shows that one abstract linked "superalloys"
with "production"). Depending on one's initial familiarity with the focal technology,
selectivity examining search results in this manner can be an effective way to delimit
many of the vital issues. (We emphasize that it is highly desirable to validate such
observations by experts in the technology.)
Maturation can be gauged by recognition of the sorts of issues linked to the technology and the degree of detail (this will be illustrated in depth in the following ceramic
engine example).
Offshoots suggest additional variants of the technology that may have potential in
their own right. These could include incorporation of the technology, or one or more
of its major components, with other technologies to form functional systems. Working
backwards in the ceramics case, we found that most of the R&D was being done with
semiconductor interests. From the standpoint of those developing that technology,
ceramic bonding of engine parts would likely be an offshoot technology, as well as a
different application domain.
Table 2 emphasizes indicators deriving from publication and patent abstracts databases. One could extend the list to other bibliometrics. For instance, examination of
the most-cited authors is a strong indicator of leaders in the field. Profiling of citation
TABLE 4
Product Value Chain Indicators
Factor
Gap analysis
Know-how availability
Applications
Economic dispersion
Geographical dispersion

Indicator
Self-profile over component technologies
Extent and identification of sources of trained personnel
Range of possible applications noted
Sectoral activity concentration
Location of activity

INNOVATION FORECASTING

31

TABLE 5
Steps of an Innovation Forecasting Process

1. Search on the basic topical term(s) in multiple databases.

2. Download electronic abstracts from a prime, available database; examine cumulated keywords, etc., to
refine topic understanding to generate a good search algorithm.
3. Redo search in most advantageous database(s); download abstracts.
4. Examine keywords, title words, and abstract words and phrases; read abstracts to gain fluency with related
activities, applications, key players, dispersion.
5. Plot trends in overall activity, topic-specific activity, institution-specific activity, etc.
6. Consider activity patterns by type (academic, government, industry) or other delimiters of interest.
7. Model the technology life cycle.
8. Cluster technological or other activity associated with the target.
9. Map key supporting technologies, institutional interests, etc.
10. Depict maps at different time slices.
11. Map likely future technological or competitive profiles, if appropriate.
12. Develop a technology decomposition tree, including tagging players; breakout for key contributing technologies.
13. Perform analyses on special areas (e.g., gap analysis).

patterns across fields or sectors can point to likely progression (e.g., interest in the use
of scanning tunneling microscopy rapidly spread across multiple fields enabling molecular level R&D never before possible).
Table 3 outlines innovation context indicators. The TOAK software facilitates tabulation of which other technologies, features, and issues are prominent in the search set
results (e.g., in the 214 abstracts relating to ceramic turbines analyzed later). These can
then be grouped (e.g., "production" issues, "automotive" applications, components
relating to "injectors") to develop a map of the related technologies [33]. Another
approach is to develop a "tree" showing a system branching into its component functions,
with particular technologies contributing to attainment of each function shown as another branching layer.
Such analyses can identify alternative technologies to the target technology or
alternative technologies for component technologies. In terms of innovation prospects
for the target technology, alternative technologies competing with it for potential market
are a threat; they may warrant separate examination of their own innovation prospects.
Conversely, identification of alternative component technologies to fulfill a need of the
parent target technology are a boon. If one were an automotive manufacturer considering
commitment to some aspect of ceramic engines, having several alternative technologies
competing for your favor would enhance your prospects of finding a successful and
cost-effective component for that need. 3 In this situation, one would likely probe to
ascertain the status of each of those technologies (analogously to how we describe
determining the status of the original target technology). In addition, one could benefit
by identifying the players to seek potential contacts. In the ceramic engine case, the
Army Tank Automotive Research Development and Engineering Center (TARDEC)
managers pursued technology decomposition to identify the tree of contributing technologies, then identified the status of development and who was pursuing those with the
intent of leveraging that external R&D. Indeed, one result was the establishment of
T A R D E C programs to adapt ceramic technologies under development elsewhere to
tank needs.
3 Conversely, if one represented the developer of the technology, the absence of viable alternatives could
make one's technology "Indispensable," enhancing Product Value Chain prospects.

32

R.J. WAJ~FS AND A. L. PORTER

Several socioeconomic factors extend beyond the technological context just noted.

Technology accessibility may be constrained by other companies' or agencies' patenting.

Patent concentration profiles can be developed by combining search on the technology
term per se, pertinent patent subclasses, and closely linked technology terms or specific
applications. Given the sometimes wily ways of patent attorneys, a combination of
measures may be needed to comprehend the situation. It may be particularly helpful
also to search on the key companies interested in the technology to determine if they,
in fact, have patent concentrations spreading around the technology per se. Profiling
competitor activities in terms of both patents and publications can provide valuable
competitive intelligence.
We have found that a set of electronic abstracts can provide a gratifying sweep of
information. While perhaps not decisive, a search on a target technology is likely to
indicate if action is being directed to standards, regulations, or other legal obstacles,
and other critical supporting or impeding factors. If one locates significant activity, time
slices may help show whether the topic is heating up and how it is spreading. Finally,
we note that topical activity in the popular press or the policy literature can show trends
in public interest (e.g., any indications of opposition to particular applications of the
target technology on environmental or other grounds?). This can be augmented by
search and retrieval wherein certain key phrases, such as "pollution" or "ban" are linked
to the target technology.
Product value chain indicators (Table 4) seek to evaluate the market potential for
the technology, possibly from the perspective of a particular enterprise (e.g., its developer). Gap analysis begins by laying out the set of enabling technologies required to
take the technology to market. The analyst then steps through each of those to ascertain
whether the enterprise has the requisite capabilities in house. If not, TOA can help
identify outside sources that might provide those capabilities (e.g., through partnering,
recruiting persons with critical skills, licensing critical enabling technologies). Knowhow availability can be suggested by noting what institutions and which individuals are
active in those technologies. For instance (hint?), were these academics, this might
indicate opportunities to obtain skilled consultants or hire students trained on those technologies.
Application profiles can be sketched by cumulating keywords or other terms appearing in publication or patent abstracts. This is a good way to initiate contact outside
one's own domain (e.g., ceramic semiconductor vs. automotive interests). As for many
of the other innovation success indicators, it can be worthwhile to cross-validate. Suppose
we turned up some hints of ceramic prosthesis application potential in Engineering
Index. We might now search in a biomedical database to assess the prominence of
ceramics in this arena.
An extension of the application indicators is to ascertain economic dispersion of
application activity by sector. Many databases provide classification codes (e.g., standard
industry coes [SIC] codes) that can be cross-searched to determine the extent of involvement with the target technology. That is, one performs a new search on Code X to
assess the relative preponderance of the target technology.
Geographic dispersion is easily gauged by tabulating extent of activity by country
or state codes. For instance, in a 1994 TOA analysis for the Critical Technologies
Institute, we were able to show declining interest in the U.S. and Germany in metal
casting R&D but an increasing interest in China and Russia. In addition, the distribution
of U.S. R&D by state was of interest to national policymakers. That study also benefited
from breakout of the R&D activity by performer type--whereas U.S. industrial metal

INNOVATION FORECASTING

33

casting R & D showed a precipitous decline in the early 1990s, academic activity was increasing.
In sum, Tables 2-4 present a set of bibliometric measures to help the innovation
forecaster assess the current status of a technology in terms of its life cycle prospects,
significant contextual influences on the technology's development, and its product value
chain potentials.
HOW

Table 5 lays out one way to gather electronic information, process it to generate
indicators, and interpret these to generate an innovation forecast. This is offered as an
example process, not the way. Our presentation of innovation forecasting aims to get
others to consider bibliometric measures of various sorts and to employ them in various
ways to enrich tech forecasting practices. This section offers a quick run through the
steps, some of which are elaborated in the example T O A in the next section.
Step 1 (Table 5) initiates the search process (e.g., on ceramic engines). This requires
facility with one's search engine to target well. In particular, Boolean adjacency operations contribute greatly. A search on ceramic "and" engine would generate horrible
noise; a search on ceramic "adjacent to" engine might leave out a lot. We experiment
with various searches, such as ceramic "within 3 words of" engine to get on target.
Incorporation of additional terms such as "trend," "forecast," "delphi," "assessment,"
and so forth may call up other forecasts and assessments relating to the topic technology.
Where one searches depends on the focal interests. For a technology broaching commercial introduction, one might concentrate on diffusion issues, thus tapping economic and
market databases. For a technology still in the laboratory, one would likely concentrate
on research databases.
Step 2 examines the preliminary search results of Step 1 to refine understanding
of the technology and related factors and issues to depict the "technology space" of
interest and refine the search algorithms for Step 3.
Step 4 repeats Step 2 but in depth on the actual search set of abstracts generated
in Step 3. This is an excellent step in which to engage subject-matter experts to ensure
the analysis is on target. Significant "indicators" information can be generated by reviewing lists (e.g., keywords, affiliations) to sort for leading issues and players in conjunction with the focii of Tables 2-4. T O A K generates abstract phrases that allow one to
display the noun phrases containing a particular t e r m - - a useful way to gain perspective
on its context.
Step 5 involves plotting trends. These can depict technology growth rate or other
factors for which temporal patterns are of interest (e.g., emergence of an issue, extent
of a competitor's interest in a related technology). Fitting trend models, logistic or
otherwise, can be informative. However, one should perform sensitivity analyses quite
thoroughly. Bibliographic time series are vulnerable to shifts in terminology over time,
noisy data, and lagging data (e.g., it takes time for articles to get published and more
time to get incorporated into databases that often show considerable delay in completing
a year's data entries). Smoothing may be in order to reduce year-to-year variability [2].
For many purposes, it will be advantageous to group several years together to compare
with earlier or later time periods to ascertain changes.
Step 6 entails grouping items by type. T O A K automatically groups ~ academic,
governmental, and business affiliations. Other groups can be tailored to meet casespecific interests (e.g., "materials" seen in Table 9).
TOAK generates cumulative lists across the records in the dataset being analyzed. Classification is based
on a combination of thesaurus (look up), fuzzy rules, and syntactic and semantic algorithms. TOAK "learns"
with repeated use as the thesaurus grows.

34

R . J . WATTS A N D A. L. P O R T E R

Step 7 combines results of Step 5 with broader understanding (exemplified later

for ceramic engines).
Steps 8, 9, 10, and 11 cluster like items and depict these as "maps" of various
forms [33].5
Step 12 addresses one major framework. TARDEC has demonstrated utility in
"technology decomposition" wherein functions are linked to alternative technologies
that can achieve them, and alternative technologies are linked to competent technologies.
At any node of interest, one can break out key players, issues, and the like.6
Step 13 is an open-ended invitation to mine the abstracts for information to key
on particular innovation success indicators of interest. We are still exploring.

Innovation Forecast for Ceramic Engines

This inquiry addressed a possible technological substitution for the U.S. Army--use
of ceramics in place of steel in tank or automotive engine components [35]. The U.S.
Army has considerable interest in advances made in ceramic engine technology. Budgetary constraints may require that the existing military fleet be maintained in inventory
beyond the year 2010. Rapid global technology advancements and foreign military
R&D investments heighten military needs and threaten U.S. superiority on the future
battlefield. Army R&D investments, therefore, must strive to maximize functional performance improvements while retaining systems configurations compatibility (i.e., technology insertion through form and subsystem/component interface). One way to achieve
this is through advanced materials engine technology insertion programs. Ceramic engine
components enable lower wear rates and permit higher operational temperatures, along
with the associated combustion benefits of reduced exhaust emissions and increased
engine performance (i.e., greater on-demand horsepower or extended vehicle range,
enabled further by complementary ceramic component weight reductions).
Ceramic engine technology has not gone unnoticed by the commercial automotive
industry. Though unconcerned about long-term form and interface technology compatibility, the automotive engine manufacturers have recognized ceramic technology as
industry capability enhancing, rather than destroying. Unlike the competing vehicle
power-source technologies, ceramic engines would promote automotive manufacturer
maximization of existing investments in manufacturing and assembly while meeting
ever-more stringent exhaust emission standards. Ceramic engine technology utilization,
therefore, offers a medium for the orderly transition between present and future engine technologies.
A preliminary search (Step 1, Table 5) located prior forecasts, in particular, a
Delphi study [36]. The Delphi respondents had identified enabling technologies and
application barriers that existed in the mid-1980s. These provided good leads for further
bibliometric searches on both the enabling and primary technologies from Engineering
Index and U.S. Patents (Step 3). The main search addressed 1985 to 1995 on "ceramic"
TOAK applies a variety of matrix operations over terms by records to cluster similar terms or similar
records. One can address normalized or raw data, rotated or unrotated factors, independent or linked rotations,
and so forth to help identify clusters of interest. The key matrix manipulation applies singular valued decomposition to generate factors on which terms or records load. This shares somewhat with factor analysis and with
latent semantic indexing [34].
6 With Defense Advanced Research Projects Agency support, we are currently working to improve the
TOAK software to facilitate analyst discovery processes. Our vision includes automatically marking links and
applying relevance scores to generate "MASTs" (automated abstracting of abstracts) on chosen subtopics,
capability to view abstracts most relevant to such a subtopic, or to call up related issues or players.

INNOVATION FORECASTING

35

TABLE 6
Ceramic Engine Publications (ENGI)

Universities

Labs

Firms

Labs

Firms

Year

NT

NT

NT

T,C

T,C

1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996

1
5
27
5
2
4
2
7
14
1
3
1

1
6
33
38
40
44
46
53
67
68
71
72

7
3
18
3
3
1
1
0
2
1
3

7
10
28
31
34
35
36
36
38
39
42
42

18
20
59
7
2
7
4
6
5
0
5

18
38
97
104
106
113
117
123
128
128
133
133

13
4
5
2
4
4
1
2
8
3
3
1

13
17
22
24
28
32
33
35
43
46
49
50

13
15
15
6
9
6
20
1
11
7
6

13
28
43
49
58
64
84
85
96
103
109
109

Abbreviations: NT = nonturbine, C = cumulative, T = turbine.

within 6 w o r d s of " e n g i n e . " T h e resulting search r e c o r d s w e r e d o w n l o a d e d in e l e c t r o n i c

f o r m a n d s u b d i v i d e d into two f i l e s - - t u r b i n e and other. T u r b i n e s (file of 214 r e c o r d s )
p r o v i d e a possible l e a d t e c h n o l o g y indicator. F o r s o m e p u r p o s e s , the files w e r e f u r t h e r
p r u n e d to i n c l u d e only r e c o r d s f r o m the top 100 i n s t i t u t i o n s - - u n i v e r s i t i e s , g o v e r n m e n t
labs, and c o m m e r c i a l f i l m s - - p u b l i s h i n g on c e r a m i c engines.
T a b l e 6 p r o v i d e s the c h r o n o l o g y of the publications for the t h r e e s o u r c e g r o u p i n g s
for the two c a t e g o r i e s of c e r a m i c e n g i n e publications. ( N o t e only five plots are p r o v i d e d
in the T a b l e 6 - a s s e m b l e d d a t a b a s e s h o w n in F i g u r e 1 - - t h e r e w e r e only t h r e e t u r b i n e
abstracts f r o m universities.) T a b l e 7 shows the c o - o c c u r r e n c e m a t r i x for the g o v e r n m e n t
l a b o r a t o r y o r g a n i z a t i o n s that p r o d u c e d the m o s t n o n t u r b i n e publications and t h e n u m b e r of m a t c h e s o f t h e m o s t f r e q u e n t l y used k e y w o r d s . Similar tables w e r e c o m p i l e d for
a c a d e m i a a n d industry. T h e s e tables identify t h o s e w h o are m o s t active in p u b l i c a t i o n
140
120
100
80
60
40
20

0(!,~
1965

1986

1987

1988

Univ., Cumulative
x

1989

Labs,T, Cumulative ~

1990

1991

1992

Labs, Cumulative

1993

~Firms,

1994

1995

Cumulative ]

Firms,T, Cumulative

Fig. 1. Ceramic engine publication. Abbreviation: T = turbine.

1996

t~

o"

~'~

~.

%
Ceramic materials
Diesel engines
Tribology
Mechanical properties
Heat engines

Thermomechanical ceramic
Physical properties
Ceramic products
Silicon nitride
Superchargers and supercharging
Wear of materials
Materials testing
Impact damage
Impact testing
Grinding (machining)
Microcrack
Durability
Cracks
Protective coatings
Scanning electron microscopy
Air engines
Surface properties
Degradation
Rocket engines
Total

t~

37

INNOVATION FORECASTING
TABLE 8

Ceramic Coating and Bonding Patents

Year

Patents

Cumulative

1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995

13
6
6
3
4
1
16
16
11
11
20
22
29
33
27
16

13
19
25
28
32
33
49
65
76
87
107
129
158
191
218
234

of ceramic engine R&D. The co-occurring keywords begin to define the areas of concentration. One can note the balance of development cycle participants, with industry
taking a strong lead in applied research and development. For less mature technologies
(i.e., electrorheology or artificial intelligence), a greater proportion of activity by basic
research institutions and lower activity from influential sponsors might be observed.
The abstracts also revealed a balance of R&D activity across the industry infrastructure
(i.e., components, engine, and vehicle manufacturers).
Table 6 reveals a surge in publication activity in 1987 from all three source groupings
for nonturbine topics. The Delphi study conducted during this period provided expert
opinions on the benefits and barriers, and rate of progress in overcoming the barriers,
for utilization of ceramic engine components [36]. Current literature points to the
following enabling issues: lower cost of raw materials; more efficient and lower cost
manufacturing processes; component materials consistency and end-product structural
verification (i.e., nondestructive testing); and ceramic coating, bonding, and joining
technologies. Production cost reduction is important to the use of ceramics; advantages
to applying ceramics to wear-resistant parts have been confirmed [37]. This general theme
of acknowledged potential rings through the referenced material with the exception of
Razim and Kaniut, who elaborate on the hurdles confronting ceramic engine component
adoption ]38]. In addition to the material weaknesses at which research has been
directed, these authors pointed out the salient issues of limited materials, design experience, materials properties verification, and (most important) different performance
standards expected of the new material (i.e., higher operational temperatures and speeds
and lower use of ancillary cooling and lubrication support systems).
The literature conveys that the advantages of ceramic components have begun to
be proven; the technology is maturing. The three most-cited barriers include cost,
material properties verification, and coating and bonding technologies--three candidates
to explain the 1987 surge in publication activity. Using the terms "ceramic" adjacent
to "coating" or "ceramic" adjacent to "bonding" yielded 234 related patents during the
1980 to 1995 period. Table 8 and Figure 2 depict the chronology of patents issued and
the cumulative patent growth in the ceramic coating and bonding field. The significant

38

R.J. WATTS AND A. L. PORTER

rise in number of patents issued in 1986 and 1987 may provide an explanation for the
industrial publication surge in 1987; with proprietary confirmation in hand, a technical
capabilities announcement through these publications could follow. The anticipated
precadence of applied research publications ahead of patents appears to be violated
here. The drop in patent activity in 1994 might represent the passage of an inflection
point on the technology growth curve. Knowledge growth and engineering productivity
in a given technology have often been likened to a logistics function, as might be
predicted by the Fisher-Pry equations. The cumulative ceramic coating and bonding
patents were modeled (Step 7, Table 5) by three Fisher-Pry equations, each with a
different technology growth limit (i.e., 350, 450, and 550 patents). Each of the three
equations provided coefficients of determination greater than .99. The growth limits
were selected because limits below 350 patents and above 600 patents provided lower
coefficients of determination. These equations were then used to generate patent forecasts through the year 2005, as shown in Figure 3. The upper and lower growth limits
provide a visual sensitivity analysis for the models and serve as surrogate confidence
intervals for the future growth in this technology, assuming that 450 patents represent
the actual anticipated growth. In assessing these growth curves in respect to technology
maturity, we suggest that, although the capabilities have begun maturing and new entry
into the field would be most difficult because of the pace being set by the current
participants, there still will be significant technology growth in the next 9 years.
To extend the maturity analysis from enabling technologies (i.e., ceramic coating
and bonding) to ceramic engine technology more generally, two bibliometric approaches
were applied. The 100 most-used keywords from the 426 nonturbine ceramic engine
abstracts were subdivided into two groups: material types and a combined group of
material properties and applications. We then generated a co-occurrence matrix-materials versus properties and applications (Table 9). Two observations from this table,
in regard to ceramic engine technology, include the apparent emergence of silicon
nitride as the ceramic material of choice and the presence of competing materials (e.g.,
aluminum compounds, metal matrix composites, metals and alloys, superalloys). For
silicon nitride (Row 1, Table 9), note the considerable level of use of application,
process, and property verification terms--an indication of technological maturation.
To obtain a temporal perspective on the types and usage of keywords related to
ceramic engine technology, the nonturbine ceramic engine abstract file was subcategorized into five 2-year periods of publication abstracts. Co-occurrence matrices of sources
versus keywords were generated. Table 10 summarizes the co-occurrence matrices by
defining the level of activity (e.g., the number of discrete publication sources and
associated number of publications) and the level of focus of the documented research
(e.g., the number of discretely different keywords). The evolution of a technology can
be observed in Figure 4, which depicts Table 10 data, and considered in terms of the
Utterback and Abernathy model on product and process innovations [39]. That model
prescribes that early research is product focused and attracts many industry participants.
Once a dominant design emerges, research shifts towards process technology, and the
number of industry participants declines. In the 1987-1988 period, the level of interest
in the technology peaked as indicated by the numbers of publications (207) and participating organizations (120). The areas of R&D, however, were quite focused, as indicated
by the number of different keywords used (29). Contrast this profile with that for
1993-1995: far fewer participating organizations (42), a proportional reduction in number

INNOVATION FORECASTING

39

250

200

150

100

5O

Fig. 2. Cumulative ceramic coating and bonding patents.

of publications, but tremendous expansion of the detail and issues addressed (201
different keywords used)?
To see the evolution of the types of technological activities addressed over the
time periods, the common keywords across periods were eliminated. Table 11 presents
the chronology of the use of the remaining words. Innovation sequences often start
with an invention (e.g., technology application such as the invention of the internal
combustion engine), followed by the emergence of related sciences (e.g., tribology,
combustion, etc.). As observed in Table 11, the ceramic engine technology terms have
500,00
450.00
400.00
350.00
i

300.00

~> 250.00
i

200.00
150.00
100.00
50.00

1980 to 2005

- e - Cure. Patents - I - Fisher-Pry(350) --*-- Fisher-Pry(450) ~

Fisher-Pry($50)]

Fig. 3. Ceramic coating and bonding patent projection.

~_ =

~o

~ ~ - ~

Z~,~

~'

#-a

g
Properties
Strength
Applications

Application
Performance
Process
Structural
Piston
Parts
Coatings
Processing
Production
Valve
Reliability
Coating
Characteristics
Problems
Automotive
Cost
Economy
Quality
Processes
Durability
Fatigue
Manufacturing

Liner

Pistons

Turbocharger

Valves

.=

INNOVATION FORECASTING

41
TABLE 10
Co-occurrence Matrices Summary

Years
1985-1986
1987-1988
1989-1990
1991-1992
1993-1996

Discrete
sources
44
120
35
29
42

Number of
publications
79
207
44
36
60

Number of
keywords
17
29
17
15
201

evolved toward analytic sciences in addition to expanding to processes, material properties verification, and application fields. This supports the notion of a maturing technology
poised to assume niche positions in specialty material growth markets.
The other two application barrier issues (cost and manufactured material property
verification) support the coming of age of ceramic engine technology in a different
manner--through the absence of publicly available information. Component cost data
were sought through both literature review and phone contacts with material journal
publishers and ceramic engine component manufacturers. These efforts uncovered the
fact that ceramic component cost data represent confidential information between component suppliers and end-item manufacturers (e.g., automotive and engine). The engine
manufacturers have begun using ceramic components (Table 12) and must perceive
that their actions provide a competitive advantage. One can assume that until an aftermarket emerges for replacement ceramic engine components, cost information will most
probably be closely guarded. Takao et al. [37] noted that once a component probability
of failure on the order of 10 -6 has been achieved, the material weight used in automotive
systems is inversely related to the square of the component cost-to-weight ratio (WP-2).

250

20(

10
lumber Publications
te Sources
wo~s
VW

V ~

g3-g5
Fig. 4. Technology maturity and keyword diffusion.

Generic material

Generic application

Sintering. braiding process

1993-1995

Specific approach/process

Gas engines

Air engines, sensors, superchargers and supercharging

Machine components, superchargers and supercharging

Castings

Powders, automotive engineering,

powder metallurgy

Generic approach

Metals and alloys, nonmetallic materials

Porous materials, aluminum compounds,
amorphous materials, metallic matrix
composites, composite materials

Materials, metallic silicon alloys.

refractory materials
Materials, metals and alloys, refractory
materials, superalloys, composite
materials
Composite materials

1989-1990
1991-1992

1985-1986
1987-1988

1991-1992
1993-1995

1989-1990

1987-1988

1985-1986

Report
period

Microstructure. strength of
materials, material testing.
physical properties, volume
fraction, high temperature
properties, mechanical
properties, fatigue testing,
reliability, wear of
materials, durability,
axial/circumferential
strength, creep, defects

Hydrogen fuels,
methanol, diesel
fuels (alternative
fuels)
Aromatic polyphenyl
ether type oil

Enabled
technology

Material characteristic
verification

Microscopic examination

Zirconia
Silicon nitride,
ceramic fibers,
aluminum titanate

Mathematical models,
tribology, finite element method, computer simulation,
computational
geometry

Analytical science

Domes and shells, valves and

valve gear
Pistons
Seals, ceramic heat-insulated
engine, braided ceramic fiber
seals, adiabatic engines

Silicon nitride

Friction materials, heat

transfer
Thermal insulating materials.
lubricants, thermomechanical ceramic

Combustors, domes and shells

Specific application

Silicon nitride,
silicon carbide

Specific material

Protective coatings, waste

heat utilization

Lubricants

Needs/function

TABLE 11
Technology Maturity versus Keyword Usage

43

INNOVATION FORECASTING
TABLE 12
Ceramics Automotive Applications

Ceramic components

Material

Intake and exhaust valves

Silicon nitride

Exhaust portliner

Aluminum titanate

Brake engine retarder

master piston wear pad
Cam roller follower
Ceramic tappet
Ceramic coatings

Silicon nitride

Ceramic coatings

Zirconia coating with

strain isolator

Silicon nitride
Sintered silicon nitride

Supplier

User

Ceramtec Div. of
Hoechst A.G.
Ceramtec Div. of
Hoechst A.G.
ENCERATEC, Inc.

Daimler Benz

Kyocera Corp.
NKG Spark Plug Co,
Ceramics Corp. of
America (Cercoa)
Technetic's Corp.

Detroit Diesel (Series 50)

Nissan Diesel Motor Co.

Porsche A.G.
Cummins N14 Engine

This material cost-to-weight relationship remains true except when government requirements (i.e., exhaust emissions) mandate a materials usage, as in catalytic converters. Since
ceramic component usage has begun, one might assume that the just-stated probability
of failure level has been achieved and verification procedures developed. However,
component material property verification both reflects and embodies one competence of
the manufacturing process. Management of technology principles stress that distinctions
must be made between technologies and technical competencies. Competencies represent the essence of competitive advantage and must be more closely protected than
technologies, which can be imitated and designed around. Manufacturing competence
involves a complex mixture of employee training and involvement, supplier integration,
statistical process control and value engineering, as well as design for manufacture and
end-product verification [40]. A search of U.S. Patents using the terms "ceramic material
quality," "ceramic non-destructive test," and "ceramic property test," uncovered only
four relevant patents. The fruitless component cost and patent searches, along with
commercialization announcements, support one conclusion: the manufacturing costs
and process verification techniques are being held secret to obtain and maintain competitive advantage.

Interpretation
This assessment concentrates on issues considered most relevant to Army policy
decisions related to this technology. Institutional forces impact ceramic engine developments. As affirmed by the bibliometrics, ceramic engine R&D emanates from government laboratories and the automotive industry infrastructure, not from the ceramics
industry (that emphasizes semiconductors in particular). This fact warrants that the
ceramics industry R&D be monitored by the automotive sector for potential offshoots
(desired ceramic functions) to speed recognition and diffusion of new technology discoveries to automotive applications.
The automotive industry "need" for specialized engine materials, including ceramic
components, has resulted primarily from government mandates on exhaust emissions
and fleet fuel economy standards. This imposed need creates a delicate balance between
the degree of legislated stringency and the allocation of commercial R&D resources.
Too tight or too loose legislative mandates would significantly reduce the commercial
R&D available to leverage. This suggests ongoing monitoring of regulatory developments.

44

R.J. WATTS AND A. L. PORTER

Zero-based exhaust emission requirements could force premature adoption of electric vehicle technology and pull scarce R&D resources away from ceramic engine
development. Less stringent requirements could reduce the demand for more efficient,
hotter burning, and lighter engines and could drain commercial ceramic engine R&D.
Gradual and ever-tightening requirements are likely to promote specialized component
development and extend the evolution of the internal combustion engine (ICE). This
scenario advances automotive manufacturing capital investments stability and a supporting infrastructure that remains familiar to a large proportion of the participants, both
businesses and consumers alike, thereby promoting the probability of technology acceptance.
Another oil embargo or an extended Middle East war could drive oil and fuel
prices higher and make alternative, perhaps methanol, fuels more competitive in respect
to cost. However, current ICE incompatibilities preclude the use of fuels such as methanol without experiencing high engine wear rates and increased oil consumption [41].
An accelerated development pace for specialized materials--particularly to modify
engine combustion chambers with high-temperature, corrosion-, and wear-resistant materials-would better prepare the country for such a crisis. Implementation, obviously,
would still be subject to a mortality substitution rate, one which under normal attrition
would require 10 to 20 years to transition. Again, these contextual influences on ceramic
engine innovation merit continued monitoring.
The most significant impact of ceramic engine technology adoption and commercial
diffusion will be on automotive component suppliers. Ceramic engine technology represents the early stage of a materials revolution, one where material properties will be
designed and developed for specific applications. Suppliers lacking material design
capabilities, as well as the emerging specialized material manufacturing and component
properties certification competencies, will be supplanted by larger more affluent companies that can and will develop the needed skills. Ceramic Division of Hoechst A.G.,
Kyocera Corporation, NGK Spark Plug Company, and Enceratec Incorporated (Table
12) represent a sample of the firms that have been strategically positioning themselves
to be the next generation automotive component suppliers. One can speculate that the
automobile industry involvement in the ceramic engine and specialized material R&D
activities strives to develop suppliers' certification capabilities and, more important, to
have proprietary interest in the new technology to create licensing revenues and avoid
limited source situations. This automotive industry supplier issue could force costs higher
due to inadequate competition. These higher costs would impact the Army fleet due
to increases in both initial acquisition and operational and support component expenses.
Condusions
Bibliometrics are limited by the secrecy of some R&D and variations in publication
practices among organizations. This was demonstrated by conspicuous absences of firms
such as General Motors and Chrysler from the publications and patents on ceramic
engine technology. It would be naive to believe these companies' R&D programs do
not include ceramic engine technology. Such variations in publication practices create
caveats against simplistic literature source analyses. More important, they justify the
more sophisticated bibliometric process analyses and measures proposed in this article.
Time lags between R&D performance and subsequent documentation also limit bibliometrics. To confirm conclusions from our innovation forecasting, expert opinions were
obtained from T A R D E C propulsion personnel and the Ceramic Information and Analysis Center at Purdue University. Such expert opinion usage should be standard practice.

INNOVATION FORECASTING

45

As demonstrated with the ceramic engine case assessment, bibliometric limitations can
be minimized by searching for general trends rather than specific events. Innovation
forecasting processes and models (see Tables 2--4) can also provide corroborating analyses for traditional forecasting techniques (Table 1).
This case analysis demonstrates that innovation success factors can be gauged by
using bibliometric measures. These, in turn, serve to assess prospects for next-generation
technologies. Many innovation concepts were applied during the sample ceramic engine
forecast. In particular, we point to the effort to operationalize a number of the innovation
success indicators offered in Tables 2-4. A key to the conclusions drawn, resulting in
initiation of two major T A R D E C ceramic engine programs, was the evidence that this
family of technologies is really maturing. 7 The evolution of keyword usage and the
empirical evidence of movement toward process technology development proved especially compelling.
Our development of innovation forecasting continues. Monitoring programs promote technology awareness and diffusion to operational programs. This goal is being
pursued through a joint Defense Advanced Research Projects Agency (DARPA) and
Tank-automotive and Armaments Command (TACOM) Small Business Technology
Transfer (STTR) program. Under this combined program, the TPAC T O A K will be
modified to expand its analysis capabilities and to implement a menu-driven operator
interface. The enhanced T O A K will facilitate the development of a database containing
military vehicle technologies hierarchy breakout ("technology decomposition"). Related
"technology space" information (i.e., the who, what, when, where, and how) can be
updated as needed. One of the efficiencies of the innovation forecasting approach is
its use of established databases. Sources such as Engineering Index and U.S. Patents
are orders of magnitude richer than one's own database could ever be. Through tools
like TOAK, we are able to tap such resources quickly and effectively.
We invite others to consider the use of bibliometric indicators as a major asset in
forecasting technology. The framework proposed in Tables 2-4 is a start toward truly
more effective innovation forecasting.

Note Added in Proof: Additional sources regarding forecasting techniques (such as

those in this article's Table 1) have been suggested by John H. Vanston which include the
article "Technology Forecasting: A Practical Tool for Rationalizing the R&D Process"
published in the New Telecom Quarterly, Vol. 4, Issue 1 (First Quarter), 1996, and
Technology Futures, Inc. website under "Who We Are" and "Tools and Techniques"
(http://www.tti.com).
References
l. Martino, J, P.: Technological Forecasting for Decision Making, McGraw-Hill, New York, 1993.
2. Porter, A. L., Roper, A. T., Mason, T. W., Rossini, F. A., and Banks, J.: Forecasting and Management of
Technology. Wiley, New York, 1991.
3. Twiss, B. C.: Forecasting for Technologists and Engineers: A Practical Guide for Better Decisions. Peter
Peringrinus, London, 1992.
4. Vanston, J. H.: Technology Forecasting: An Aid to Effective Technology Management. Technology Futures,
Inc., Austin, TX, 1995.
5. Souder, W. E.: Managing New Product Innovations. Lexington Books, New York, 1987.

7This was a true challengebecausethe Armyhad been "burned" before (investingin ceramictechnologies
into the 1980swithout notable payoff),so considerable initial skepticismhad to be overcome.The innovation
success indicators accomplishedthis.

46

R . J . WATTS A N D A. L. P O R T E R

6. Porter, M. E.: Competitive Advantage: Creating and Sustaining Superior Performance. Free Press, New
York, 985.
7. Linstone, H. A.: Multiple Perspectives for Decision Making: Bridging the Gap Between Analysis and Action,
Prentice Hall, Englewood Cliffs, NJ, 1984.
8. Barbiroli, G.: Towards a Definition and a Dynamic Measure of Strategic Technology, Technovation 12
(5), 285-296 (1992).
9. Millson, M. R., Raj, S. P., and Wilemon, D.: A Survey of Major Approaches for Accelerating New Product
Development, Journal of Product Innovation Management 9, 53~59 (1992).
10. Mahajan, V., and Muller, E.: Timing, Diffusion, and Substitution of Successive Generations of Technological
Innovations: The IBM Mainframe Case, Technological Forecasting & Social Change 51, 109-132 (1996).
11. Dunphy, S. M., Herbig, P. R., and Howes, M. E.: The Innovation Funnel, Technological Forecasting &
Social Change 53, 279-292 (1996).
12. Smith, C. G.: Understanding Technology Substitution: Generic Types, Substitution Dynamics and Influence
Strategies, Journal of Engineering~Technology Management 9, 279-302 (1992).
13. Modis, T.: Technology Substitutions in the Computer Industry, TechnologicalForecasting & Social Change
43, 157-167 (1993).
14. Anderson, P.. and Tushman, M. L.: Technological Discontinuities and Dominant Designs: A Cyclical
Model of Technological Change, Administrative Science Quarterly 35, 604--633 (1990).
15. Dror, I.: Technology Innovation Indicators, R&D Management 19, 243-249 (1989).
16. Metcalfe, J. S.: The Diffusion of Innovations: An Interpretive Survey, in Technical Change and Economic
Theory. G. Dorsi, C. Freeman, R. Nelson, G. Silverberg, and L. Soete, eds., Printer, London, 1988,
pp. 560-589.
17. Metcalfe, J. S.: Impulse and Diffusion in the Study of Technological Change, Futures 13, 347-357 (1981).
18. Cetron, M. J.: Technology Transfer: Where We Stand Today, 21st Joint Engineering Congress, 11-28 (1973).
19. Rogers, E. M.: Diffusion of Innovation. 3rd ed. Free Press, New York, 1983.
20. Cohen, H. S., Keller, S., and Streeter, D.: The Transfer of Technology from Research to Development,
Research Management 22(3), 11-17 (1979).
21. Souder, W. E., Nashar, A. S., and Padmanabhan, V.: A Guide to the Best Technology Transfer Practices,
Technology Transfer 5-16 (Winter-Spring, 1990).
22. Narin, F., Olivastro, D., and Stevens, K. A.: Bibliometrics---Theory, Practice and Problems, Evaluation
Review 18(1), 65-76 (1994).
23. Cunningham, S. W. The Content Evaluation of British Scientific Research. Ph.D. thesis. Science Policy
Research Unit, University of Sussex, Brighton, UK, 1997.
24. Garfield, E., Malin, M. V., and Small, H.: Citation Data as Science Indicators, in The Metric of Science:
The Advent of Science Indicators. Y. Elkana et al., eds. Wiley, New York, 1978.
25. Kostoff, R. N.: Research Impact Assessment: Problems, Progress, Promise, in Fourth International Conference on Management of Technology. Miami, FL, 1994, pp. 423-433.
26. Small, H., and Griffith, B.: The Structure of Scientific Literatures, Science Studies 4, 17-40 (1974).
27. Franklin, J. J., and Johnston, R.: Co-citation Bibliometric Modeling as a Tool for S&T Policy and R&D
Management: Issues, Applications, and Developments, in Handbook of Quantitative Studies of Science
and Technology. A. F. J. van Raan, ed. North-Holland, New York, 1988.
28. Kostoff, R. N.: Co-word Analysis, in Evaluating R&D Impacts: Methods and Practice. B. Bozeman and J.
Melkers, eds. Kluwer, Boston, 1993, pp. 63-78.
29. Kostoff, R. N.: Database Tomography: Origins and Applications, Competitive Intelligence Review 5(1),
48-55 (1994).
30. Tijssen, R. J. W., and van Raan, A. F. J.: Mapping Changes in Science and Technology, Evaluation Review
18(l), 98--113 (1994).
31. Rip, A.: Mapping of Science: Possibilities and Limitations, in Handbook of Quantitative Studies of Science
and Technology. A. F. J. van Raan, ed., North-Holland, New York, 1988.
32. Kostoff, R. N.: Research Impact Quantification, R&D Management 24(3), 207-218 (1994).
33. Porter, A. L., and Detampel, M. J.: Technology Opportunities Analysis, Technological Forecasting and
Social Change 49, 237-255 (1995).
34. Deerwester, S., Dumals, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R.: Indexing by Latent
Semantic Analysis, Journal of the American Society for Information Science 41(6), 391-407 (1990).
35. Watts, R. J.: Defining Next Generation Technologies, Management of Technology Thesis. National Technological University, Fort Collins, CO, 1996.
36. Larsen, R. P., and Vyas, A. D.: The Outlook for Ceramics in Heat Engines, 1990-2010: Results of a
Worldwide Delphi Survey, Society of Automotive Engineers, International Congress and Exposition, pp.
4--6, 9, 1988.

INNOVATION FORECASTING

47

37. Takao, H., Okada, A., Akimune, Y., and Hirosaki, N.: Ceramics for Reciprocating Engines: An Application
Review, Proceedings of the International Symposium on Ceramic Materials"and Components for Engines.
4th. Goeteborg, Elsevier, London, 1991, p. 130.
38. Razim, C., and Kaniut, C.: Ceramics in the Automotive Industry, Mercedes-Benz AG. 1991, pp. 277-278.
39. Burgelman, R., Maidique, M., and Wheelwright, S., Strategic Management of Technology and Innovation.
2nd ed. Irwin, Homewood, IL, 1996, p. 159.
40. Hamel, G., Doz, Y., Prahalad, C.: Collaborate with Your Competitors and Win, in Strategic Management
of Technology and Innovation. 2nd ed., R. Burgelman, M. Maidique, and S. Wheelwright, eds., Irwin,
Homewood, IL, 1995, p. 596.
41. Funatani, K., Kurosawa, K., Fabiyi, P. A , and Puz, M. F.: Engine Performance Improvements, Automotive
Engineering 15-16 (January, 1995).

Received 14 October 1996; accepted 11 April 1997