Available online at www.sciencedirect.

com

ScienceDirect
JOURNAL OF IRON AND STEEL RESEARCH, INTERNATIONAL. 2012, 19(3): 25-30

A Novel Method for Flatness Pattern Recognition via

Least Squares Support Vector Regression
ZHANG Xiu-ling , ZHANG Shao-yu, T A N Guang-zhong, ZHAO Wen-bao
(Key Laboratory of Industrial Computer Control Engineering of Hebei Province, National Engineering Research Center
for Equipment and Technology of Cold Strip Rolling, Yanshan University, Qinhuangdao 066004, Hebei, China)

Abstract: To adapt to the new requirement of the developing flatness control theory and technology, cubic patterns
were introduced on the basis of the traditional linear, quadratic and quartic flatness basic patterns. Linear, quadratic,
cubic and quartic Legendre orthogonal polynomials were adopted to express the flatness basic patterns. In order to over-
come the defects live in the existent recognition methods based on fuzzy, neural network and support vector regres-
sion (SVR) theory, a novel flatness pattern recognition method based on least squares support vector regression (LSSVR)
was proposed. On this basis, for the purpose of determining the hyper-parameters of L S S V R effectively and enhan-
cing the recognition accuracy and generalization performance of the model, particle swarm optimization algorithm
with leave-one-out (LOO) error as fitness function was adopted. To overcome the disadvantage of high computational
complexity of naive cross-validation algorithm, a novel fast cross-validation algorithm was introduced to calculate the
LOO error of LSSVR. Results of experiments on flatness data calculated by theory and a 900HC cold-rolling mill
practically measured flatness signals demonstrate that the proposed approach can distinguish the types and define the
magnitudes of the flatness defects effectively with high accuracy, high speed and strong generalization ability.
Key words: flatness; pattern recognition; least squares support vector regression; cross-validation

With the higher requirement of flatness quality the change of any adjusting method has influence to
following the development of industry, controlling all defects has been proved and precise mathematical
technique of flatness has become a hot topic of strip relationships between defects and controlling meth-
roll. Flatness pattern recognition is an important ods have been built. This discovery broke the tradi-
constituent of flatness controlling system, its results tional understanding to flatness control. Develop-
have direct effect on flatness control. T h e task of ment of controlling theory and technique put for-
flatness pattern recognition is t o convert the practi- ward new requirement to pattern recognition, tradi-
cal flatness signals into several characteristic param- tional flatness basic patterns which include linear,
eters through mathematical methods"' , confirm the quadratic and quartic patterns do not conform to the
types and magnitudes of flatness defects to provide demand of flatness controlling. In Ref. [ 3 1 , cubic
basis for flatness controlling strategy. patterns were introduced. This strategy details the
In general, there are linear, quadratic and quar- target of control, which is in favor of the enhance-
tic defects in flatness controlling field. Correspond- ment of controlling accuracy.
ingly, the controlling method is roll displacing, T h e recognition methods of fuzzy and neural
work roll bending and intermediate roll shifting re- network have characters themselves. T h e algorithm
spectively. Along with the development of rolling e- of fuzzy recognition approach is easy and stable, but
quipment, controlling methods display diverse, new it is intractable to obtain the fuzzy rules and the ac-
methods such as asymmetric roll bending and roll shift- curacy is low"'. Neural network recognition model
ing have been proposed successively. In Ref. C23, influ- improves the recognition precision, however, it has
ence matrix theory has been proposed, the view that the disadvantages of poor generalization ability, lo-

Foundation Item: Item Sponsored by National Natural Science Foundation of China (50675186)
Bi0graphy:ZHANG Xiu-ling(1968-), Female, Doctor, Professors E-mail: zxlysu@ysu. edu. cni Received Date: January 1 2 , 2011
 26 Journal of Iron and Steel Research, International VOl. 19

cal minimum and so ~ n ~ Though ~ - ~the~ method

. based presses the flatness with eight basic patterns. Their
on SVR can overcome the flaws of the methods men- normalized equations are as follows.
tioned above and has high improvement in both accuracy Normalized equation of left waves:
and generalization abilitf] , the computational complex- Yl =p1 ( y )= y (1)
ity is high, thus poor efficient. In this paper, a novel Normalized equation of right waves :
recognition method of fast cross-validation L S S V R Yz = -p1 ( y )= - y (2)
(least squares support vector regression) was pro- Normalized equation of center waves :
posed. Ensuring of accuracy and generalization ability, 3 1
Y , = p , ( y ) =-y2 -- (3)
it also has higher computing speed, therefore, it is 2 2
an effective intelligent recognition method. Normalized equation of double-edge waves :

1 Basic Patterns of Flatness [;

y4= -p z (y ) = - -yz -- (4)
Normalized equation of right-one-third waves :
Judging from the procedure of roll, the curves 1
of flatness are the transverse distribution (width ori- Y5=p3(y)=-(5y3-3y)
2
(5)
entation) of residual stress. T h e residual stress Normalized equation of left-one-third waves :
meets self-canceling qualification that the integral 1
value by width equals zero. Legendre polynomials Y , = -p3 ( y )= - 7(5y3 - 3 y ) (6)
are orthogonal and meet self-canceling qualification, Normalized equation of quarter waves :
thus it is accurate to express flatness basic patterns
Y7 =p4 ( y ) = - (813 5 y 4 -30y2+3) (7)
with Legendre orthogonal polynomialsc8'.
Traditionally, there are six kinds of flatness basic Normalized equation of edge-center waves :
patterns in industry: left waves, right waves, center 1
Y8=-$4 ( ~ ) = - j ~ ( 3 5 ~ " ~ 0 ~ ~ + 3 (8)
)
waves, double-edge waves, quarter waves and edge-
center waves. On account of the diversification of Setting original point at the center of strip, ex-
flatness controlling methods and the demand of ac- pressing the positions of the measuring centers with
curate control, right-one-third waves and left-one- x-axis, the residual stress with y-axis, normalizing
third waves were introducedc3', making the flatness the stress and positions, the curves of basic patterns
basic patterns more self-contained. This paper ex- are shown in Fig. 1.

-1.0 -0.6 0 0.6 1.0 -1.0 -0.6 0 0.6 1.0

Y

Fig. 1 Curves of basic flatness pattern

Issue 3 A Novel Method for Flatness Pattern Recognition via Least Squares Support Vector Regression 27

-
-

The recognition procedure extracts flatness charac

where l=Cl,.-.,I]', a = [ a l ,a2 , - * - ,allT, Y = [yl,
teristic parameters a&( k = l , 3 , 5 , 7) through mathe-
matical method. T h e flatness after roll can be for-
yz ,*. ,yllT, n, =$(X,>TI(X,) =K(Xk,xj ) ( k ,j=
1,2 , , I ) , K ( XI, Xj) is kernel function which
mulated as the linear combination of basic patterns:
meets Mercer qualification.
f(y)=alpl(y)+a3pg(y)+aSp5(y)+a?p?(y)(9)
T h e L S S V R is formulated as:
where a l , a 3 , a5 and a7 are the linear, quadratic, 1
cubic and quartic defect respectively; and ah is the f<X>
= 2 aiK (X,Xi)
i=l
+b (13)
content of corresponding flatness defect. T h e sym- where a , b are the solutions of Eqn. (12).
bols of them reflect the types of flatness. When the
values of 4) are positive, they represent left waves, 2. 2 Hyperparameters selection based on particle
center waves, right-one-third waves and quarter swarm optimization
waves, otherwise, the right waves, double-edge In face of actual problem, suitable L S S V R
waves, left-one-third waves and edge-center waves. model which is attributed to the choice of hyper-pa-
2 Fast Cross-Validation Least Squares Sup- rameters such as kernel parameter and regularization
port Vector Regressions parameter that govern the generalization perform-
ance of the model has decisive influence to the solve
2.1 Feature of LSSVR of the problem. T h e popular way to obtain the opti-
Support vector machine ( S V M ) , proposed by mal hyper-parameters is grid searchc"] , however,
Vapnik, is a novel learning method based on statisti- one should be sure of the scope and choose appropri-
cal learning theory. In LSSVM designed by SuykensCgl, ate sampling step in advance, so it is intractable to
inequality constraints are replaced by equality con- use this method when there are too many hyper-pa-
straints, thus, transform the quadratic program- rameters.
ming problem into a linear problem. This transition Here, particle swarm optimization algorithm
reduces the difficulty of solving, therefore, raises was introduced to determine the optimal hyper-pa-
the efficiency. T h e algorithm of L S S V R is de- rameters, and it is conducted through the MATLAB
scribed as followsc103: particle swarm optimization toolbox In the toolbox, the
Given a training sample set D = { ( Xl, y11,
order for solving the optimal solution of the function is
(XZ,yz), -*, (X,,y,)), X,ERd, y,ER.T h e non- Y = pos-Trelea-vectorized(f,k) (14)
linear mapping function $ : P - H maps the data into a where, f is fitness function that evaluate the per-
high-dimensional feature space H , then construct the
best regression function f( X)= <w +
, $(X)> b in
formance of particle; k is the dimension of particle;
and Y is a column vector of ( k + l ) dimension which
H , where W E H is weight vector, X is input vector show the position of the optimal particle with the
and b is the bias. In this way, the non-linear regres-
top k dimension and the optimal function solution
sion in the original space is achieved.
with the last dimension. T h e other parameters of
T h e algorithm of L S S V R is to solve the follow-
particle swarm optimization algorithm is default in
ing optimization problem:
the toolbox, they can also be set according to the
minimize - 1
2 11 W 11+z Y'
2 ef
,=1
user's requirements.
T h e fitness function is the LOO (leave-one-out)
subject to yt=WT$(X,) +b+e, ( i = 1 , 2 , ,I)
error which is used to estimate the generalization per-
(10)
formance of LSSVR under certain hyper-parameters.
where, y is the regularization parameter; and e, is
the error variable.
2.3 Fast cross-validation algorithms
Introducing the Lagrange multipliers a, , the
Lagrangian of the preceding problem is:
A popular way to estimate the generalization
performance of a model in the procedure of hyper-
parameters choosing is k-fold cross-validation. In k-
b+ei -yi] (11) fold cross-validation, one divides the data into k
According to the conditions for optimality to non-intersecting subsets s, , sz, - * * , s k , then,
Eqn. (11), the following matrix form can be obtained: trains and tests k times with the ith conducted as
-T follows: leaving out Si as testing subset, then resol-
(12) ving the regression function on the basis of the sum-
mation of the other subsets, computing the testing
 28 - Journal of Iron and Steel Research, International VOl. 19

error of Si. Take the average error of k times as the between the sample to be recognized and the basic
error estimation of the model. When k = Z , there is patterns, were introduced as inputs of the flatness pat-
only one sample left out as testing sample for each tern recognition model. This strategy fixes the dimen-
time, this is known as LOO, with its average error sion of input at six no matter how the width of the strip
called as LOO errorC1''. However, k-fold cross-vali- changes. In Ref. [14], differences of Euclidean distance
dation has a quite high computational complexity for were used as inputs, the structure of recognition model
it has to train the regression function for k times. In was simplified further. Taking example of the method
this paper, a novel fast cross-validation algorithm mentioned above, differences of Euclidean distances
was introduced, it is described as followsc131. were used as inputs of the model in this paper.
Given the dimension of Si ( i = l , 2 , k) is a * * , Suppose the normalized sample to be recognized
k
i s Y = [ d l ) , CT(2), " ' 9 d m ) ] , Yk=[@k(1)9 CTk(2)9
n i , C n i = I , correspond the divide of the sample set,
i=l *.., f&(m>] ( k = l , 2 , ---, 8) represent the basic
Y and a are split as follows: Y'CY1, Yz YklT9 patterns where m is the amount of measuring sec-
...
, , ( I k l T , Y i=Cyi,l yi,z
a=Cal , ( I ~ , Y;.,,~IT,a;= tion. T h e Euclidean distances are as follows:
C(1i.1 9 ai.2 9 ..* 9 1'.
f
i=l

In Eqn (121, define A = f l + y - ' I ,

-T -
A,=
[p:'] ( k = 1, 2 , * - - ,8) ( 1 9 )
Considering the fact that the eight basic pat-
terns are four pairs of opposite patterns and the op-
denote d = - 1 A-' 1 , the inverse matrix of A, is:
posite patterns can not exist in sample meanwhile,
-_1 iTA4 differences f?k ( k = 1, 3 , 5 , 7) of Euclidean distances
d
could reflect the similar degree between the sample
to be recognized and the basic patterns, They are
denoted as follows:
ek =Dk+i -Dk (k=1,3,5,7) (20)
T h e membership degrees uk between the sample
to be recognized and the basic patterns were used as
'C11 c12 ... clk the outputs of the recognition model. T h e relation-

. ..'
ships between ah and uk are as follows:
c= CTZ
. c22
... .. CZk
(16)
ak=ACTmax ' uk (21)
CTk CTk '*' ck&
where ACT,,,.,is the maximum of the residual stress.
T h e flatness pattern recognition system has
many outputs. In order to meet the requirement of
recognition, a four-output system was constructed
according to the working mechanism of L S S V R and
used in flatness pattern recognition. Gaussian func-
tion was chosen as the kernel function of the model,

1 11 1
In implementation of k-fold cross-validation, just like follows:
the computational complexity of the naive algorithm K ( X j, X , > = e x p xi:kxk 11 (22)
is about (k-1)3z3 9 and the fast algorithm is about T h e flatness pattern recognition model based on
3k2
fast cross-validation LS-SVR is shown in Fig. 2.
times as efficient as
the naive one. T o LOO, the efficient is enhanced by
1 time. According to the above method, one can cal-
culate the generalization error of the model and as-
sess the performance of it conveniently.
3 Structure of Flatness Pattern Recognition
Model
In Ref. [81, Euclidey distances, the distances Fig. 2 Model of flatness pattern recognition based on LSSVR
Issue 3 A Novel Method for Flatness Pattern Recognition via Least Squares Support Vector Regression 29

sets in total were used to train the model. And 16 sets

4 Experimental Results of samples were chosen to test the model. A part of
the testing results are shown in Table 1. As shown,
To assess the recognition capacity of the LSSVR the recognition precision of the testing samples is
model, eight basic patterns and their combination 40 high, the model appears strong generalization ability.

Table 1 Results of flatness pattern recognition

Test sampltj Standard output Test output SSE

u1 =o ~3=-0.5 ~i=-O.O116 U3=-0.4839

8.0 X 10W4
1 ,
us=o U7=-0.5 us=0.0130 u,=-O. 4862
u1 =o. 2 u3 =O. 2 ui=O.1776 ~3~0.2165
3 Y=O. 2Y140. 2 y 3 i - O . 6Y7 1. ox 10-3
u5=0 u7 =O. 6 u~=0.0010 u7=O.6152

To show the recognition ability of the model lively, bic and quartic flatness, the test outputs are very
taking the sixth test samples for example, the curves of close to the standard, recognition accuracy is high.
its standard outputs and the test outputs are drawn To compare the recognition capacity of LSSVR
in the same graph, as shown in Fig. 3. Fig. 3 ( a ) is model with eight basic patterns (four outputs LSSVR) ,
the synthesize curves and Fig. 3 ( b ) to Fig. 3 (d) re- SVR model with eight basic patterns (four outputs
present the linear, quadratic, cubic and quartic flat- SVR) and SVR model with six basic patterns (three
ness component respectively. No matter the synthe- outputs SVR) , a group of practically measured flat-
size curves or the curves of the linear, quadratic, cu- ness signals of a 900HC roll with specification is as

1.0 (a> -Standard results

x Testresults

I 0.2

"
-1.0 -0.6 -0.2 0 0.2 0.6 1.0

1)2
I -0.2
0.10O

g 0.06

0
-0.06
-1.0 -0.6 0 0.5 1.0 -1.0 -0.6 0 0.6 1.0
Y

Fig. 3 Recognition results of sixth test sample

 30 Journal of Iron and Steel Research, International VOl. 19

follows: 2. 2 X 742-0. 245 X 742, material is S P H C validation algorithm is introduced to calculate the
and 15 measuring sections in measuring roll, are rec- LOO error. This method can not only determine the
ognized. T h e stress of one is u= [10, 6 , 6 , 6 , 3 , 1, optimal hyper-parameters effectively but also has a
15, 27, 22, 48, 38, 74, 73, 30, 361 and the nor- high speed, thus, meets the requirement of online
malized result of its flatness deviation is Au= [-0.342 7 , real-time training.
-0.4266, -0.4266, -0.4266, -0.4895, -0.5315, 3) T o overcome the defects occurring in the in-
-0.2378, 0.0140, -0.0909, 0.4545, 0.2448, telligent recognition methods and the SVR recogni-
1.0000, 0. 9790, 0.0769, 0.20281. T h e recogni- tion method, a new recognition approach based on
tion results of the three models and the practically promoted L S S V R is proposed. T h e comparison
measured curve are shown in Fig. 4. with the four outputs SVR and three outputs SVR
demonstrates that this approach can recognize the
characteristic parameters which represent both the
types and the magnitudes of the flatness defects ef-
fectively, therefore, satisfies the demand of high
precise flatness automatic control.

I -0.5 I.

References:
d
2 -1.0 - II ZHANG Xiu-ling, LIU Hong-min. GA-BP Model of Flatness
-1.6 I Pattern Recognition and Improved Least Squares Method [J].
Iron and Steel, 2003, 38(10) I 29 (in Chinese).
-1.0 -0.6 -0.2 0 0.2 0.6 1.0
.V LIU Hong-min, ZHANG Xiu-ling. Transfer Matrix Method of
Flatness Control for Strip Mills [J]. Journal of Materials Pro-
Fig. 4 Flatness recognition results of cessing Technology, 2005, 15(166): 237.
practically measured data SHAN Xiu-ying, LIU Hong-min. A Recognition Method of
New Flatness Pattern Containing the Cubic Flatness [J]. Iron
T h e recognition results of the models with cubic and Steel, 2010, 45(8) I 56 (in Chinese).
patterns coincide with the practically measured curve LIU Jian, WANG Yi-qun. Fuzzy Pattern Recognition Method
of Flatness Based on Particle Swarm Theory CJ]. Chinese Journal
better than that without cubic patterns, thus, the
of Mechanical Engineering, 2008, 44(1) I 173 (in Chinese).
recognition method containing cubic patterns has ZHANG Xiu-ling, PANG Zong-peng. Flatness Pattern Recog-
higher recognition accuracy. The recognition capacity of nition Based on Adaptive Neuro-Fuzzy Inference System CJI.
LS-SVR model with four outputs and SVR model Journal Iron and Steel Research, 2009, 21(9) I 59 (in Chinese).
JIA Chun-yu, SHAN Xiu-ying. Fuzzy Neural Model for Flat-
with four outputs are comparable. However, com-
ness Pattern Recognition [J]. Iron and Steel Research, Inter-
pile the program using the MATLAB language on national, 2008, 15(6): 30.
the same computer, calculate the regression function ZHANG Xu-gong. Introduction to Statistical Learning Theory
and test 16 sets of testing samples when the hyper- and Support Vector Machines [ J]. Acta Automatic Sinica,
parameters are definite, the calculating time of the 2000, 26(1): 32 (in Chinese).
ZHANG Xiu-ling. Research on Intelligent Control and Recog-
regression function is tl =59. 3870 s and the testing nition of Flatness for Cold Strip Mill [D]. Qinhuangdao: Yans-
time is t z =O. 002 2 s to SVR model, but the calcu- han University, 2002 (in Chinese).
lating time is only tl =O. 009 1 s and the testing time Suykens J A K , Vandewalle J. Least Squares Support Vector Ma-
is t z =O. 001 9 s to LS-SVR model. LS-SVR model is chine Classifiers [J]. Neural Processing Letter. 1999, 9(3) : 293.
Suykens J A K , Van Gestel T, De Brabanter J , et al. Least
more efficient than SVR model obviously.
Squares Support Vector Machines [ M I . Singapore: World
5 Conclusions Scientific Publishing Co Pte Lte, 2002.
Van Gestel T, Suykens J A K. Benchmarking Least Squares
1) Two basic patterns are added on the basis of Support Vector Machine Classifiers [J]. Machine Learning,
the traditional six kinds of basic patterns, this strat- 2004, 54(1)1 5.
DENG Nai-yang, TIAN Ying-jie. A New Method of Data
egy makes the basic patterns more self-contained and Mining-Support Vector Machines [MI, Beijing t Science
meets the demand of the developing flatness control- Press, 2004 (in Chinese).
ling theory and technique. It is in favor of the im- Senjian An, Wanquan Liu, Svetha Venkatesh. Fast Cross-
provement of controlling accuracy. Validation Algorithms for Least Squares Support Vector Ma-
chine and Kernel Ridge Regression [J]. Pattern Recognition,
2 ) T o overcome the difficulty of selecting hy-
2007, (40): 2154.
per-parameters, MATLAB particle swarm optimiza- H E Hai-tao, LI Nan. The Improved RBF Network Approach
tion toolbox is used to determine the optimal hyper- to Flatness Pattern Recognition Based on SVM [J]. Process
parameters. Furthermore, a simple and fast cross- Automation Instrumentation, 2007, 28(5) : 1 (in Chinese).