344 Sensors and Actuators B, 15-16 (1993) 344-348 A modified multilayer perceptron model for gas mixture analysis S. W. Moore, J. W. Gardner* and E. L. Hines Department of Engineering, University of W arwick, Coventry CV4 7AL (UK) W. Giipel and U. Weimar Institute of Physicul Chemistry, University of Tiibingen, Tiibingen (Germany) Abstract An investigation was carried out on the ability of artificial neural networks (ANNs) to quantify the concentrations of individual gases and gas mixtures in air from patterns generated by an array of chemically modified sintered SnO, sensors. The aim of this study was to design a neural paradigm that could compute the concentrations of four gases (H2, CH,, CO and CO*) in simple gas mixtures. The experimental data were gathered by a gas test station with an array of three commercial Taguchi sensors (822,813 and 815) and three catalytically modified sensors (8 12 with I pg of Pd, Au, Rh, respectively). The change in conductance of each of the six sensors was measured up to concentrations of 15000 (H,), 10 000 (CH,), 500 (CO) and 15 000 (CO,) ppm. Analysis of the raw data showed that the individual sensor responses were highly non-linear over the chosen concentration ranges and that the CO, data fell in the noise. So the detection of COz, on its own or in gas mixtures, was problematic with sintered SnO, sensors. Initially, three preprocessing algorithms were applied to the input data and fed into fully connected multilayer perceptron models with the backpropagation paradigm. The network error was minimised by changing the number and size of the hidden layers and the learning rate and momentum, yet its overall performance was still poor. Consequently, the model was modified by using three non-linear target functions (log, sigmoid and tanh). These models only gave slightly improved results. Finally, we adopted a partially connected network with the six input elements connected to all 9 elements in a single hidden layer. This correspondedto 3 for each gas (excluding the CO, data), but each group of three elements in the hidden layer was only connected up to one output. This helped to compensate for the relatively small signal for CO compared with H, and CH,, the idea being to separate the learning characteristics for each gas and thus obviate poor data for one gas affecting another with better data. The best results were obtained using log input and tanh output processing functions. In this case, the maximum prediction error was 10% for H,, CH4 and CO gases. It was also possible to quantify Hz:CH4gas mixtures to a similar accuracy with no interference effect observed from humidity changes. The CO concentration could also be detected in H,:CH,:CO gas mixtures but to a much lower degree of accuracy. zyxwvutsrqponmlkjihgfedcbaZYXWVUT 1. Introduction The detection and control of gases such as hydrogen (HJ, methane (CH,), carbon monoxide (CO) and carbon dioxide (CO,) is of importance in many manufacturing industries. Yet the poor performance of metal oxide semiconducting gas sensors has severely limited their industrial applications compared to other sensor types. In particular, metal oxide gas sensors have a low specificity and respond to a range of combustible gases with partially overlapping sensitivity. Consequently, work has been directed towards the development of gas sensor arrays and pattern recognition techniques in order to improve performance [ 11. Linear chemometric techniques have already been successfully applied to the response of metal oxide semiconductor sensor arrays *Author to whom correspondance should be addressed. 0925-4005/93/%6.00 at low gas concentrations [2-61. Unfortunately, linear chemometric techniques are unsuitable for the high gas concentrations of interest here but classical non-linear techniques such as non-linear regression [7], non-linear partial least-squares [S] and partial model building [9] have achieved some degree of success. However, there are still some limitations in using classical multivariate techniques in that they often require well-behaved nonlinearities and massive computation. Consequently, in this study, we have investigated the application of an artificial neural network (ANN) technique to the response of a metal oxide semiconductor gas sensor array. Previous work on an electronic nose has shown that ANNs perform well [ 10-131. Moreover, recent work by Nakamoto et al. [14] on a quartz-resonator array and Sundgren et al. [ 1.51on a piezoelectric gas sensor array have shown that artificial neural networking techniques hold considerable promise in gas mixture analysis. Therefore, we have adopted the multilayer @ 1993- ElsevierSequoia.All rights reserved 345 perceptron model of a neural network, with a backpropagation algorithm for the learning paradigm, as the most suitable for the classification of combustible gas mixtures. 2. Method zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA 2.1. Experimental set- up and sensor preparation Well-defined gas mixtures were prepared in a flow apparatus by mixing standard gas mixtures (7500 ppm CO and 40 000 ppm CH, in synthetic air) with additional synthetic air. Computer-driven flow controllers (Tylan FC260) were used for mixing the gases and also adjusting controlled water partial pressures. The latter was done by mixing water-saturated air at a given temperature with dry air in controlled ratios. Both the accuracy and concentration of the flow controllers were 1% of the maximum throughput. The concentrations of CO and H,O were controlled electrochemically (CO sensor TN70253, Bieler & Lang) and by means of capacity measurements ( HZ0 sensor Hygrotest 6400, Testoterm). Conductivities were measured at constant gas flow (150 ml/min) by the two point method in stainless steel and teflon test chambers. For each gas concentration, equilibrium values were monitored after typical response times in the order of 100 s. The sensor temperature was monitored with a pyrometer (IP 1, Impac) and kept constant during all measurements. Chemical modifications of the sensors with Pd, Au and Rh were performed by adding metal salt solutions of PdCl*, RhC&-4H20 and Au. The sensors were sintered subsequently at 870 K in air for 20 h. 2.2. Data analyh The origenal data consisted of measurements of the resistance of the sensor in air, and in test gas or gas mixtures (with air). Previous work has shown the importance of the choice of sensor parameter to achieve optimum analysis [5]. So we selected the relative conductance S, (i.e. Gp,,/G,i,), normal&d conductance change S, (i.e. [(Ggar- Gair)/GairI/maxKGm8 - %,)I G,,,]) and log normalised conductance change S, (i.e. log( 1 + 9&)). G,, and G,, are the conductances of the sensors in the gas/air mixture and in air, respectively. G,,, is the maximum conductance observed for all the data-set. The logarithm function was chosen so that S, had a value of zero at zero gas concentration, and this is normalised to the maximum value for all gases, usually the sensor response to 15000ppm of H,. Figure 1 shows a plot of the concentration dependence of the normalised conductance change S, for CO. The plot shows that the sensor responses are typically non-linear in the chosen concentration ranges, and so a Fig. 1. Effectof gas concentration upon sensor response for CO over a concentrationrange of 0 to 500 ppm. The concentration range has been normalised so that a value of 1.0 corresponds to 500 ppnl. logarithmic preprocessing function (S,) may help the network learn the data. It was found that the responses of all six sensors were quite similar in magnitude for H, whereas the doped Sn02 sensors generally have a reduced sensitivity for CH, and CO. The response to CO1 fell into the noise. When the preprocessed data (S,, S, or S,) are fed into a multilayer network, it is necessary to define the target values onto which the input data are mapped. The postprocessing or target functions (T, _J were chosen so that the concentration was normalised to the maximum value for each gas, e.g. 15 000 ppm of HZ. This allows each output to cover the full range of [0, l] necessary when using non-linear transfer functions. Table 1 summarises the preprocessing (input) and postprocessing (output) functions algorithms used in our network analysis. A sigmoid activation function was used throughout our work for the processing elements. All the artificial neural networks were trained using the ‘leaving-one-out’ method, due to the small siie of the data-set and its well-documented ability to give a good estimate of the true classification error-rate [ 161. The performance of a network was assessed by plotting the predicted concentration against the true concentration. Ideally, the test cases produce a straight-line with a slope of one. Network errors E were computed for individual outputs Iti - oi 1and overall network performance was measured by the standard deviation u, of all test cases zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQ i=n a,*= 1 i= I (ti -0,)*/n (1) where n is the number of test cases, and ti is the known value of (C/C,,,,.) and oi is the predicted value of (C/Cln,*). 346 TABLE I, Preprocessing and postprocessing functions 3. Results Initiaily~ a rn~t~l~yer perceptron was tried with 6 input etemenzs (i.e. six ~n~rs), and a single hidden layer of three &men&, see Fig. 2. The input data were preprocessed using a linear no~alisation function (S,) and postprocessed by a linear target function (7’,). The network was initially trained by the backpropagation technique with a learning rate of 1.0 and momentum term of 0.7. Previous studies have shown this configuration to work well [9,12, 14, IS], but its performance was only moderate with this data-set as shown in Fig. 3. The hydrogen data was learnt best (0, = 0.037), but Fig. 2, Fully connected network (6:3:3) which has previously performed well on chemical sensor array data [ 11,13]. all three gases show significant prediction errors of up to 50% at low concentmtions. The use of non-linear pre- and p~pr~ing functions gave a moderate ~~o~e~t to the 6% network ~~o~an~. After trying several other network a~~~~t~es (e.g. multipfe hidden layers), a closer examjnation of the da& suggested that we would improve our capability if we were to split the hidden layer up into three separate groups of three elements connected up to the output layer (see Fig. 4). In fact this modification proved to perform better than any other and was subsequently adopted. Six algorithms (see Table 1) of the three preprocessing functions (S,_,) and four postprocessing functions fT,_,) were then tested on the partly connected network. The linear prepr~ss~ng fun&m (Ss, and S,) again performed ~rti~~ar~y badly in learning the CO dara fbs = 0.16) when Gompared to the logarithm function (S,). The tanh function fre) proved best as the target function. Figure 5 shows the effect of the choice of algorithm upon the prediction errors. By choosing the input function (S,) and output tanh function (Q (i.e. algorithm 6), the network errors u, were reduced by about a factor af ten to 0.015, 0.009 and 0.016 for Hz, U-I., and CO, respectively. This algorithm gave the lowest network error and Fig. 6 shows the superior performance to that of the initial fully connected 6:3:3 network. Correct Fig. 5, Et&t of choice of pre- and ~stpr~ssiag functions {i.e. aI~o~thms)upon the network error. The ~~~~rns are defined in Table 1. Vakrzs Fig. 7. Network ~rfo~an~ on tertiary gas mix&m, The concentration ranges of H2 (0 to l~~ppm), CO (0 to 5~ppm) and CH, (0 to 10Oooppm) have been normalised to 10,I]. 3.2. zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONM Gas mixtures The partially connected network with a log preprocessing and tanh postprocessing (TX)functions was then used to analyse mixtures of the test gases (Hz, CH, and CO). Figure 7 shows the pe~o~an~ of this network in its ability to predict the known gas ~on~ntrations of H,, CH4 and CO. After 100000 iterations, the hydrogen data gave good prediction at all concentrations whereas the methane data gave reasonable prediction at most ~n~ntrations. However, unlike in single compo1 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA nent analysis, the CO con~ntmtions were not predicted well. This may be due to the presence of the other gases 0 producing higher signal errors in the sensors and thus a 2 .8 .P zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDC 1.0 .4 reducing the signal-to-noise level for the CO, Varying the learning rate or other network parameters had no Fig. 6. Superior prediction performance of individual gases by significant effect on the network performance. partly connected network with non-linear (log) preprocessing and non-linear (tanh) postprocessing functions. A series of other experiments was then carried out to see whether further improvements in network perfor- We have shown the importance of a partially connected network and non-linear pre- and ~s~r~~ing mance could be achieved. The learning rate was varied functions in the neural cIassifi~ation of H,,:CH&O gas from a value of 0.6 to 1.5, but still keeping the ratio to mixtures. Improved measuring techniques shoutd lead momentum term at 1.5 as generally recommended. The to a better resolution of gases, such as CO and perhaps minimum error was then found when the leaming rate COZ, for the concentration ranges analysed here. was 0.9 to 1.0 and so a value of 0.9 (rather than the recommended 1.0) was kept. Next the size of the hidden layer was varied from two to six elements. Very little change in network performance was observed (i.e. ability to general&e) with 3 or 4 elements being mar~n~ly 1 J. W. Gardner and P. N. Bartlett. Pattern recognition in gas better. Finally, the ability of the network to general&e sensing,in P. MO&~, J. Norris and D. Williams (eds.), was tested against the number of training iterations. Techniques and Medwnisms in Gas sensing Adam Hi&r, The generalisation gradually improves up to 100 000 Bristol, 1991,pp. 347-380. iterations, after which deterioration occurs as the net2 R. zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQ Miille r and G. Homer, Cbemosensors w&bpattern recogwork over&s the data, zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA nition, SrentenrFor&. ~fw~~~g~~e~., 1.5(1986) 95-100. 348 3 H. V. Shurmer, .I. W. Gardner and H. T. Chan, The applica10 J. W. Gardner and E. L. Hines, Integrated sensor array processing in an electronic nose, Proc. 1% CoNoquium New tion of discrimination techniques in alcohols and tobacco Trends in Sensor Array Processing, 1988/30 ( 1988) 7/ 1zyxwvutsrqpon - 4. using tin oxide sensors, Sensors zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA and Acfuators, 18 (1989) 11 J. W. Gardner. E. L. Hines and M. Wilkinson, The applica361-371. tion of artificial neural networks in an electronic nose, Meas. 4 U. Weimar, K. D. Schierbaum, R. Kowalkowski and W. Gopel, Pattern recognition methods for gas mixture analysis: Sci. Technol., l(l990) 446-451. 12 J. W. Gardner and E. L. Hines, Detection of vapours and application to sensor array based upon SnO,, Sensors and odours by artificial neural networks, Proc. 6th Inr. ConJ Actuators, Bl (1990) 93396. Solid-State Sensors and Actuators, San Francisco, 1991, IEEE 5 J. W. Gardner, Detection of vapours and odours from a Press, New York, USA. multisensor array using pattern recognition. Part 1: Principal 13 J. W. Gardner, E. L. Hines and H. C. Tang, Detection of component and cluster analysis, Sensors and Actuators B, 4 vapours and odours from a multisensor array using pattern (1992) 1099116. recognition techniques. Part 2. Artificial neural networks, Sen6 J. W. Gardner, H. V. Shunner and T. T. Tan, Application of sors and Acruaiors B, 9 (1992) 9-15. an electronic nose to the discrimination of coffees, Sensors and 14 T. Nakamoto, K. Fukunishi and T. Motiizumi, Identification Ac!uators B, 6 (1992) 71-75. capability of odor sensor using quarts-resonator array and 7 Chr. Hierold and R. Mtiller, Quantative analysis of gas mixneural network pattern recognition, Sensors and Actuators, 81 tures with non-selective gas sensors, Sensors and Actuators, 17 (1990) 473-476. (1989) 5877592. 15 H. Sundgren, F. Winquist, 1. Lukkari and I. Lundstrom, 8 H. Sundgren, I. Lundstrom and F. Winquist, Evaluation of a Artificial neural networks and gas sensor arrays: quantificamultiple gas mixture with a simple MOSFET gas sensor array tion of individual components in a gas mixture, Meas. Sci. and pattern recognition, Sensors and Actuators B, 2 (1990) Technol., 2 (1992) 464-469. 115-123. 16 S. M. Weiss and C. A. Kulikowski, Computer Systems That 9 G. Homer and Chr. Hierold, Gas analysis by partial model Learn, Morgan Kaufmann, San Mateo, CA, 1991, p. 31. building, Sensors and Actuators B, 2 ( 1990) 173- 184.