TFC2005_Problems_of_Forecast
TFC2005_Problems_of_Forecast
PROBLEMS OF FORECAST1
Dmitry KUCHARAVY
dmitry.kucharavy@insa-strasbourg.fr
Roland DE GUIO
roland.deguio@insa-strasbourg.fr
Abstract
The ability to foresee future technology is a key task of Innovative Design. The paper focuses on the obstacles
to reliable prediction of technological evolution for the purpose of Innovative Design.
First, a brief analysis of problems for existing forecasting methods is presented. The causes for the complexity
of technology prediction are discussed in the context of reduction of the forecast errors. Second, using a
contradiction analysis, a set of problems related to technology forecast is proposed. Third, the paper examines
how some knowledge from the Theory of Inventive Problem Solving (TRIZ) can be applied in response to
hal-00282761, version 1 - 28 May 2008
forecasting problems. Fourth, conclusions are drawn about the perspectives for reliable technology forecasting.
All of these reflections are supported by the research experience gained in the project with the European
Institute for Energy Research (EIfER, Karlsruhe).
1. Introduction
Let us start with an idea that "…Any system which depends on human reliability is
unreliable2…" Why is reliable technology forecast so urgent today? The Problem might be
formulated in the following way: If technology forecasting practice remains at the present
level, it is necessary to significantly improve the efficiency of design to make it possible to
develop adequate technologies in a short period in response to new demands (like Green
House Gases - GHG Effect reduction or covering exploded nuclear reactor); If technology
forecasting is more efficient, it is possible to develop an adequate technology for coping with
the new requirements without being in a hurry (if GHG Effect could have been predicted 50
years before it appeared). Thus, in order to survive in a highly technological environment, an
efficient engineering design and production are not enough. The absence of efficient and
reliable forecast limits decision makers and the pace of changes.
What if we look at technology forecast from the problem-solving viewpoint? Let us assume
that a problem situation can be defined as a difference between the actual state and the target
situation (objectives). In other words, Problem = Target situation – Actual state (or Target
situation = Actual state + Changes, where Changes include Typical solutions and Problems).
Where do objectives come from? They come from the vision of future. When a certain
technical problem is discussed, the vision of future appears as a result of a technology
forecast. Even if the formal forecasting was not performed the intuitive one was done.
What is a problem origin in such reasoning? The target situation seems unreachable.
However, what if the vision about future is wrong? The Problem could be defined in the
1
Article was prepared for ETRIA TRIZ Future 2005 (Graz, November 16-18, Austria)
conference. Last update: November 28, 2005
2
Merphy's laws (Gilb's Laws of Unreliability)
1
wrong way. It is a common place today that a solution fails more often because a wrong
problem has been chosen rather than because a wrong path to the right problem was followed.
"…We suffer more from the good solutions for wrong problems, but not from weak solutions
for right problems…" [Russell L. Ackoff – an American scientist in operations research and
systems theory; iconoclastic management authority, advocates a ‘‘systemic’’ approach to
innovation]
Why do we need a reliable technology forecast? From the technological viewpoint, it is
required for setting priorities for research and development; and definition of the right
problems instead of approaching whole of believable. From the social standpoint it is essential
for assessing and managing threats of emergent technologies and new scientific discoveries in
advance as well as an for adaption of social institutions, such as educational system, for
coming changes. From the business environment context it is substantial for efficient
management of intellectual property and for extension of technological competitiveness of
products, processes, and services.
This paper seeks to discuss a set of problems related to a reliable technology forecast.
Event is a physical effect (changes), that occurs in a particular place of space, at a particular
point in time. An Event can be defined by several features, such as physical content of
changes, location in space and time (What? Where (How large)? When (How long)?). In
order to unify the applied descriptions, the ENV (Element-Name of Feature-Value of Feature)
model is used to describe Events. For details about the ENV model, see Khomenko [1].
According to the ENV model physical contents, place and time are considered and described
as features.
Time is an artificial concept (method) to put in order the observed events and to measure
relative duration of processes. Discussion of the epistemology of time is beyond the scope of
the given paper.
Present is a set of events that occurs in a particular place of space.
Future is a set of events that may be affected by some events in Present. One of the crucial
questions of any forecast can be formulated as follows: "How to recognise which events in
present will influence the future and which ones will not affect it?" Another critical question
is: "How to foresee uncertainties that will affect future as well?"
Past is a set of events that may influence the events in Present.
1.2 Technology forecast
The difference between forecasting methods and forecast results (model description) is
presented in Table 1.
2
Table 1
Features of forecasting (process) and forecast (results)
IF
IF
IF
D
C
B
A
IF
A B C D E F G H
3
In order to be precise with a technology forecast the system approach is inevitable [2, 12].
Using a Multi-screen analysis different levels of details can be recognized for any technology
forecast:
1. sub-micro (technology, product, process)
2. micro (company, group of companies)
3. industry (competitors by the same product, competitors by the same function)
4. macro (financial, economic community, country)
5. super-macro (environment, demographic, etc.)
Such a list can be expanded in both sides. What complicates a technology forecast is
interconnections between all these levels. It is evident that an efficient forecasting process has
to provide a means for a simultaneous work on several levels.
We propose to define the efficiency of technology forecast as a ratio between a reliable
forecast and aggregated expenses necessary to perform forecasting, where a reliable forecast
is characterized by accuracy of forecast and transparency of the forecasting process while
aggregated expenses include resources spent on development and communication of forecast
results. This working model of technology forecast efficiency is introduced in order to make it
possible to measure the difference between "good" and "not so good" forecasts while
comparing two forecasts about the same question.
hal-00282761, version 1 - 28 May 2008
4
Method should be short and simple, in order to be easy-to-learn, easy-to-apply, easy-to-
integrate into super-systems (e.g. planning and decision making processed for forecasting);
BUT the Method should be redundant and complex, in order to be reliable, to have one
universal method instead of multiple, in order to decrease the probability of application
mistakes; and to secure achievement of the initial goals.
From the TRIZ viewpoint, one of the typical solutions for a conflict "complex vs. easy" is
performing segmentation and then building a new system from the segmented parts in
accordance with the requirements of a specific situation. Thus, it seems obvious that at
present combinations of several methods are preferred to a single one.
It is not evident however, how a combination of methods influences the technology forecast
errors. Normally, methods are combined in order to reinforce the weakness of one method by
the strength of another. However, when several methods are combined, new forecast errors
can appear as a result of the synergy effects. At present, the question of measurability of
forecast errors arising as a result of an integration of several methods remains unanswered.
In order to review the existing technology forecast methods in a holistic and systemic way
we propose to classify all forecasting methods from the point of view of a problem solving
approach by indicating which problem or a set of problems a given method addresses and
hal-00282761, version 1 - 28 May 2008
aims to resolve.
In order to illustrate our point better, let us consider a method of Role-playing [4, p.p.15-
30] that is used to forecast decisions. Using the same data, depending on a given role ("cost
analyst" or "feasibility analyst"), a forecaster can obtain different forecast results. Moreover,
Role-playing works on simulation of interactions between roles A and B, when several
participants try to simulate decision making in order to forecast a decision.
Which problems are solved by this method? It is necessary to note, that there are many
problems that remain unsolved and eventually influence the accuracy of such a forecast.
Nevertheless, the evident advantage of Role-playing is that it applies a mechanism decreasing
subjective biases of forecasters: Forecaster should have personal inclinations, because he has
subjective opinions; BUT Forecaster should not have personal inclinations, because they
decrease the accuracy of a forecast.
Another interesting point is that Role-playing works with interactions, rather than with
subjects. In accordance with a systemic approach (a Multi-screen scheme of thinking, or the
so-called System operator) a forecast will be more efficient when interactions rather than just
separate objects are taken into consideration. It is quite obvious that in order to forecast a
future of some transportation technology, it is inevitable to analyze interactions of such
technology with present and future social, environmental, and economic contexts. Role
playing offers no formal mechanisms for performing systemic thinking; nevertheless, its
primary focus is on interactions: forecaster has to analyze interactions between objects, in
order to improve the accuracy of a technology forecast; BUT forecaster has to analyze
separated objects, because interactions appear when objects start to act and react.
Thus, Role-playing offers a concept solution to the two problems in a network of problems
dealing with a technology forecast. Further on, we will describe this network and categorize
the existing forecast methods according to the problems from the network.
2.2 Technology forecast errors
In order to be confident about a technology forecast, one crucial question should be
answered with certainty, "How can one distinguish between valid and invalid forecasts?" Or,
in other words, "How to measure errors and accuracy of forecasting results?"
When reformulating the above question in the form of conflicting requirements, we arrive
at the following contradiction: the Difference (between valid and invalid forecasts) should be
5
measurable, in order to evaluate the technology forecast with minimum bias; BUT the
Difference cannot be measurable, because forecasted events will happen in future.
One of well-known definitions [4] proposes to identify a forecast error as the difference
between the forecasted value (F) and the actual value (A). The accuracy of forecast is used as
an optimist's term for forecast errors. Regarding the definition of forecast given above
(section 2.1, 2.2), the value of technology forecast is presented through event, time, and place
characteristics. Thus, an invalid forecast may be a result of a mistake in terms of event (what
happened?), time (when?), and place (where?). In a general sense, in order to judge a
technology forecast it is necessary to be able to measure both a forecasted value and a real
value. The difficulty arises only because these two values are separated in time.
What typical solutions exist for improving accuracy of technology forecasting? In case of a
short term forecast, when qualitative methods are used, recalculation can be performed
several times and the resulting model can be tuned in an interactive mode to better fit the
reality. The interactive techniques for tuning forecast equations are well known in data mining
area.
In case of a long-term technology forecast, it is interesting to consider the experience of the
Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) [9].
Starting from the early 1970s, MEXT has been conducting a regular Delphi survey every 5
hal-00282761, version 1 - 28 May 2008
years. Surveys have involved more than thousand experts (last two survey involved more than
three thousand) and aimed at forecasting long-term trends in various fields of technology with
forecast time horizon of thirty years. Thus, the technology forecast is updated regularly (every
5years) to decrease errors. It is necessary to note that the applied methods are regularly
modified as well in order to reduce errors of technology forecast.
Both examples we consider use a similar solution concept: to forecast, to wait for the
forecast time to come, and to compare the forecasted value (F) with the actual value (A).
The second example shows us a much more expensive way for improving forecast methods
with the purpose of evaluation of the validity of a technology forecast. Such an expensive way
is not acceptable for most of the technology forecast demands. Thus, the question about the
difference between valid and invalid forecasts needs to be answered mostly for medium and
long-term technology forecasts.
There are many qualitative ways for performing forecast evaluation. For instance, it was
proposed [6] to use a set of characteristics such as: market impact, market timing, predicted
use, and social impact. The market impact can be evaluated as strong, moderate, weak, or
virtually none; the market timing can be on target or optimistic; while predicted use can be
characterised as mostly right, mixed, right, mostly wrong, wrong; etc.
Such quantitative evaluations appear easy-to-use, however they lead to the same problem:
forecasted events should happen in order to perform a definite evaluation of a forecast.
2.3 Problems of technology forecasts
In order to build a network of problems, the initial set of problems should be collected. For
the purpose of the present research, it was decided to group the problems according to four
stages of a technology forecast life cycle: Design stage; Developing stage; Application of a
forecast stage and Retirement-and-Update forecast models stage.
In order to focus on the most difficult technology forecast problems the literature review
and practical case study were performed. According to literature [for example 2, 10] and the
practice of forecasting a technological future of a new product, most mistakes are made in
medium and long-term forecasting dealing with pioneering and new-to-the-world
technologies.
Let us try to define what is meant under "a new product"? Crawford's list [11] distinguishes
six types of new products:
- cost improvement (reduced cost versions of a product for the existing market);
6
- product improvement (new versions of existing products, targeted at the existing
market);
- line extension (innovations added to the existing product lines and targeted at the actual
market);
- market extension (taking existing products to new markets);
- new category entry (new-to-the-company product and new-to-the-company market, but
not new to the general market);
- new-to-the-world (radically different products).
For practical use of a technology forecast it is proposed to distinguish four types of new
product/technologies:
- new version of a product (e.g. a new bicycle without a chain, with a new lighting
system, with a new break system; high capacity small batteries; low price small
batteries);
- new market penetration (e.g. German bikes to the Russian market; high capacity
batteries for toys);
- new family of products (e.g. mountain bikes; fast rechargeable batteries; noise-
cancelling headphones);
- new-to-the-world (e.g. a new personal transport instead of bicycle; a small stationary
hal-00282761, version 1 - 28 May 2008
One should be aware that the problem of signal-noise often hides behind the so-called lack
of data. If we assume that all data are available, the time necessary for data processing will
increase dramatically. Thus, we face the following problem: How to recognize events in
present and past that will influence the forecasted future? In other words, how to collect
"signal-data" and avoid spending time and resources on "noise-data"?
At the step of an appropriate method selection the following questions may appear:
- How to know about most efficient methods for a particular situation?
- How to select right method(s) out of a set of methods?
- How to evaluate the selected methods or how to be sure about the chosen techniques?
- How to apply the selected methods in the most effective way?
One of the contradictions that appears at this stage relates to specific competences: in order
to make the right choice of a method(s) it is necessary to be a specialist in all known
forecasting methodologies, BUT it is necessary to be a specialist in a particular domain of
knowledge, in order to be able to perform forecasting and to valuate the obtained results.
The stage of Forecast Model development can be presented through three basic steps:
method(s) implementation including evolvement of the forecast models, evaluation of the
applied methods, and evaluation of the forecast models.
Most of forecast problems are concentrated at the step of methods implementation. Some
problems are generic and appear when facing such questions: How to assess uncertainties?
How to keep forecasting methods simple? How to perform a realistic representation and link
obtained results with contexts (super-systems)? How to reach an agreement in a working team
that consists of specialists with various competences and different visions? Which reasoning,
causal or naive3 will be appropriate for a particular forecast?
Another group of problems is linked with the nature of selected methods. For qualitative
methods most critical queries to satisfy are the following: How to measure (to be objective)
with qualitative criteria and expectations of different experts? How to frame questions to
experts in order to avoid misunderstandings and biases? How to combine diverse quantitative
criteria and forecasts? How to scale in time quantitative proposals of forecasting? It is
necessary to note that judgmental methods are usually limited by what people perceive as
feasible that depends in fact on their shared beliefs and their restricted imagination.
3
Naïve model – a model that assumes things will behave as they have in the past. [4]
8
For quantitative approaches many problems are generated by the following questions: How
to match models with underlying phenomena? How many parameters will be enough to
provide a useful result for a certain time horizon? How to present new quality results
(unknown novelty) using quantitative output of models? Limitations of quantitative
approaches come indirect way through boundaries of applied models and are similar to
limitations of statistical methods.
Combination of quantitative methods with qualitative ones, as well as performing
qualitative forecasting with a support of quantitative approaches helps to improve the
accuracy of forecasting [3, 4, 9]. At the same time, it generates another complex bunch of
problems and consumes a lot of resources. One of practical ways to improve the reliability of
a forecast and to avoid large errors is a combination of forecast results obtained by various
methods (combination at the level of results). However, such a combination requires extra
time and decrease prediction probability of uncertainties. A valuable forecast presents an
event that seemed improbable at the time of a forecast, but was proven by reality.
- repeatability
High
(judgment)
+ accuracy
Expenses
to provide Forecast
+ repeatability
hal-00282761, version 1 - 28 May 2008
Low
(formal)
- accuracy
- expenses, time
A lot
+ accuracy
Relevant data-
information
+ expenses, time
A few
- accuracy
- expenses, time
several
+ accuracy
Forecasting
method
+ expenses, time
one
- accuracy
- accuracy
long + resources to
resolve
Forecast time forecasted risks
horizon
+ accuracy
short - resources to
resolve
forecasted risks
9
3. Field experiments
3.1 TRIZ and forecast
One of the authors of this paper has dealt with TRIZ and OTSM-TRIZ (General Theory of
Advanced Thinking) for more than fifteen years. It seems evident that first forecast practice
was based on the ideas of G.S. Altshuller [12, 13] and other researches from the TRIZ society
[14, 15, 16]. Some of those ideas appeared in the English language publications as
translations or rediscoveries after 1991 and they continue to be under development in the
international TRIZ society.
It is evident that a new innovative solution is normally developed as a result of problem
solving. Such a solution requires time and effort to be implemented in practice. As a result,
solving an inventive problem causes technology evolution. As soon TRIZ and its posterior
generations are based on the objective laws of technology evolution, these laws can be
purposefully applied to foresee the technological future. (Discussion of the laws of evolution
discovered in the context of TRIZ research remains beyond the scope of this paper.)
Most of the observed publications about TRIZ technology forecasting speculate about the
S-curve of system evolution, ideality, multiple technology trends (patterns) and laws of
technical systems evolution. On a closer examination, the law of increasing ideality of
hal-00282761, version 1 - 28 May 2008
technical systems is nothing more than a sequent of analysis of several S-curves for several
generations of a system (to measure time, to move load – transportation; to deliver
information, etc.).
Main indices of the system
3. Physical limits γ
of resources in
super-system β
Main indices of the system
2. Limits of
available
resources 5
1. Limits of
system’s 4
resources A B
α Time Time
e
d
a
c
3
1
2
The inherent meaning of the S-curve becomes evident when the reason for such a shape of
system evolution is clarified. Limitations of resources drive system evolution. As soon as
resources expire, a given system has to be replaced by a new generation of system. This next
generation will spend fewer resources of space, energy and substance for satisfying the needs.
When applying the S-curve for technology forecast, one comes across a number of
questions. Let us consider some of them: How to define the main index of an analyzed system
in a computable way? How to scale the S-curve in time? How to accurately define the
position of the analyzed system on the S-curve? It is interesting to notice that some of these
questions considered in the context of the research dealing with genetic algorithm
development [17].
One of the authors of this paper successfully applied the S-curve analysis in combination
with resources and contradiction analysis for forecasting the future of a technical system at
10
the end of 1999. As was proven later that the results of the forecast with a time horizon of
three years corresponded to reality with quarter accuracy and forecast was right for three
countries out of five forecasted. This experience gave an over optimistic viewpoint for the
applied approach. Several express forecasts performed in the period from 2000 to 2003 just
reinforced this optimistic viewpoint.
3.2 New product technology forecast
An improved approach was tested in 2004 for forecasting the future of Stationary Fuel Cell
technology in framework of a project with European Institute for Energy Research (EIfER,
Karlsruhe, Germany). The results of first phase of the project were recognized as unsatisfied.
Among other reasons, an analysis of causes for failure showed drawbacks of the applied
forecasting methodology. It was concluded that the methodology which was successfully
applied for technologies in the second or third phases cannot be applied to a technology in the
first phase of evolution (before point α on the S-curve). In addition, it was required to improve
the transparency of the forecasting methodology.
The project dealing with Forecasting of Stationary Fuel Cell technology was redesigned
and restarted. Using the results of the first phase of the project, the initial forecast problem
was reformulated and divided into three questions:
1. What will be the Evolution of the market penetration for Stationary Fuel Cell
hal-00282761, version 1 - 28 May 2008
(SFC)?
2. What will be the evolution of the SFC in comparing to competitive technologies?
3. What is the best path towards the ideal system of SFC?
80%
60%
0%
Figure 3.2.1. Example of results for critical features analysis for Proton Exchange Membrane Fuel Cell
(PEMFC)
It was assumed that all questions would be answered for a fixed market (countries and
regions) (Where?); for three power ranges (What?); for a time horizon: up to 2050 (with a
nonlinear scale of time). (When?)
At the next step, a set of critical features was formulated in accordance with the rules of
ENV modeling [1]. In order to estimate a gap between the required values of formulated
features and the actual ones, an analysis of best samples was performed. The results of the
analysis were presented in the shape of radar diagrams (see Fig. 3.2.1) for alternative
technologies vs. required values of critical features.
In order to estimate limitations of resources a network of contradictions [18] (Map of
Problems) was developed in accordance with required values of critical parameters. An
analysis of the network (see Fig 3.2.2) in accordance with resources limitation support timing
of technology forecast and answering for some of the formulated questions. On the other
hand, such a map of problems supports the monitoring of new technologies impacts to the
market capacity of small SFC. Every new patent of the project result can be associated with a
problem from the map and the impact of such novelty can be examined in a systemic way.
11
2. Durability Uniform
Uniform
low Non-uniform
present
missed
5.1.3. Humidity rate and
Temperature variation
inside stack
5.1.1.2. unstable
FUEL flow high
rate high
low
low
3. Energy Efficiency in
use (including load
variation) 5. Maintenance
high intervals
hal-00282761, version 1 - 28 May 2008
Low High
high
Figure 3.2.2. A fragment of the contradiction network for a small SFC (PEMFC)
Results of the second phase of the project were reported in June 2005 with a positive
feedback. It was decided to continue the project in order to build comprehensive answers to
the formulated questions.
4. Preliminary results
The results of the present study should be considered preliminary. These results offer the
following:
1. Despite many existing methods, medium and long-term technology forecast of a new
family of products and new-to-the-world technologies is not accurate enough to
validate expenses for forecasting. The accuracy (mostly timing and social impact) of
obtained technology forecasts is usually far from required.
2. Most currently applied methods aimed to satisfy the needs of a long-term technology
forecast are modifications of Delphi surveys and scenarios building. Often they are
applied in combination.
3. Knowledge extracted from the Theory of inventive problem solving (TRIZ) and its
posterior generations can contribute to the accuracy of technology forecasting, but it
should be done carefully taking into account the peculiarity of a forecast problem.
4. Complex forecasting methods do not necessarily provide more accurate forecast
results than simple methods. Simple methods are less subjected to data inaccuracy
than complex ones, while implementation expenses for simple methods are lower. As
a result, the efficiency of simple forecast methods is higher.
5. The choice of a forecast method(s) mostly depends on data availability. A formal
method(s) is reproducible, however it does not work well with qualitative parameters.
6. The efficiency of an existing forecasting method depends upon a forecasting horizon.
Increasing the time horizon of a forecast dramatically decrease the efficiency of
methods.
12
7. The efficiency of forecasting depends not only on applied method(s) but also the
management of the whole forecast process. Effective method(s) can lead to poor
results if project management is careless.
5. Concluding remarks
This paper aimed to demonstrate a set of problems that need to be solved on the way to a
reliable technology forecasting. The collected set of problems should be structured in the
shape of a map of problems and this map should be validated through further research.
It is also necessary to improve the transparency of a forecasting technique(s) in order to
improve the applicability of proposed ideas and approaches.
The efficiency of next generation forecasting methods should suffer less from increased
time horizons. It is necessary to propose a mode for description of uncertainties in a
comprehensible way in order to communicate them and to be prepared for real threats rather
than only for probable or believable ones.
6. Acknowledgements
We wish to acknowledge the European Institute for Energy Research (EIfER), Karlsruhe
for support of the research on Technology forecasting. We also would like to thank our
hal-00282761, version 1 - 28 May 2008
colleagues from LICIA team of the Laboratory of Engineering Design (LGéCo, INSA
Strasbourg) for their questions and discussions that helped to clarify many points presented in
this paper. Alexander Sokol provided a lot of corrections and advices about language and
readability of this version of paper.
7. References
1. Khomenko, N., Materials for seminars: OTSM-TRIZ: Main technologies of
problem solving, in “Jonathan Livingston” Project. 1997-2002: Minsk-Pyungtaek-
Toronto.
2. Schnaars, S.P., Megamistakes: forecasting and the myth of rapid technological
change. 1989, New York: The Free Press. 202.
3. Makridakis, S., S.C. Wheelwright, and R.J. Hyndman, FORECASTING: methods
and applications. 3rd ed. 1998: John Wiley & Sons, Inc. 642.
4. PRINCIPLES OF FORECASTING: A Handbook for Researchers and Practitioner.
1st ed, ed. J.S. Armstrong. 2002, Boston / Dordrecht / London: Kluwer Academic
Publishers. 849. http://hops.wharton.upenn.edu/forecast/
5. Porter, A.L. et al., Technology futures analysis: Toward integration of the field and
new methods. Technological Forecasting and Social Change, 2004. 71: p. 287-303.
6. Schnaars, S.P., S.L. Chia, and C.M. Maloles III, Five Modern Lessons from a 55-
Year-Old Technological Forecast. Product innovation management, 1993. 10: p. 66-
74.
7. Zlotin, B. and A. Zusman, Directed Evolution. Philosophy, Theory and Practice, ed.
V. Roza. 2001: Ideation International Inc. 104.
8. Vanston, J.H., Better forecast, better plans, better results. Research Technology
Management, 2003: p. 47-58. http://www.tfi.com/rescon/five_views.html
9. Kameoka, A., Y. Yokoo, and T. Kuwahara, A challenge of integrating technology
foresight and assessment in industrial strategy development and policymaking.
Technological Forecasting and Social Change, 2004. 71: p. 579-598.
10. Kahn, K.B., An exploratory Investigation of new product forecasting practices.
Journal of Product Innovation Management, 2002. 19: p. 133-143.
11. Crawford, M. and A.D. Benedetto, New product management. 7th ed. 2002, Irwin:
McGraw-Hill. 588.
13
12. Altshuller, G.S., About forecasting of technical systems development., in Seminars
materials. 1975: Baku. (in Russian)
13. Altshuller, G.S., CREATIVITY AS AN EXACT SCIENCE. Cybernetics. 1979,
Moscow: Sovietskoe radio Publishing House. 184. (in Russian)
14. Altshuller, G.S., et al., SEARCH FOR NEW IDEAS: from insight to technology
(theory and practise of inventive problem solving). 1989, Kishinev: Kartya
Moldovenyaske Publishing House. 381. (in Russian)
15. Zlotin, B.L. and A.V. Zusman, Laws of Evolution and Forecasting for Technical
Systems., in Methodical recommendations. 1989, STC Progress in association with
Kartya Moldovenyaska: Kishinev. p. 114. (in Russian)
16. Salamatov, Y.P., System of The Laws of Technical Systems Evolution, in Chance
to Adventure. 1991, Karelia Publishing House: Petrozavodsk. p. 7-174. (in
Russian).
17. Goldberg, D.E., The design of innovation: lessons from and for competent genetic
algorithms. Genetic Algorithms and Evolutionary Computation. 2002: Kluwer
Academic Publishers. 248.
18. CAVALLUCCI, D., N. KHOMENKO, and C. MOREL. Towards inventive design
through management of contradictions. in 15th International CIRP Design Seminar.
hal-00282761, version 1 - 28 May 2008
14