BRAIN MUSIC SYSTEM :THE ROLE OF AN AFFORDABLE BRAIN

MUSICAL INTERFACE IN DIGITAL MUSIC MAKING

Adrian Attard Trevisan Lewis Jones

Sir John Cass Center For Arts , Media and Design Sir John Cass Center For Arts , Media and Design
London Metropolitan University London Metropolitan University

ABSTRACT and delta. These categories are associated with the

rapidity of oscillation (frequency) of brainwaves.
Music-making has normally been associated with
manual activity apart from the creative “cortical It may be asked why usage of sound waves itself are
activity” ( for example , playing the piano or holding a adopted, rather than light or visual rays. This can be
guitar both need the use of hands ) . Clinical equipment resolved by understanding that EEG signals can be
such as Electroencephalography (EEG) , made easily represented by sound waves due to similarity of
biofeedback a possibility for music-makers but still both in many of their characteristics. The selection of
restricted its use to the few musicians that could afford sound waves instead of light or visual rays is due to the
such an expensive device. This paper describes a system properties of light itself. “Light is composed of
named “Brain Music System” which is a novel , transverse waves in an electromagnetic field....The
affordable and powerful system that is able to generate denser the medium, the greater the speed of sound. The
musical outputs based on information collected through opposite is true of light.... Sound travels through all
an EEG collecting device. The research is influenced by substances, but light cannot pass through opaque
the works of Professor Eduardo Reck Miranda, where materials.” (Comparison of Light Waves with Sound
different frequency bands trigger corresponding piano Waves). The stated properties of light makes it
notes through, and the complexity of the signal inappropriate to be compared with EEG signals which
represents the tempo of the sound. The accuracy of the are more alike sound waves.
musical conversion has been established through
experimental work, where data of participants of a pilot 2. NEURAL SIGNALS AND SOUND-
group were gathered and analysed in order to determine PROCESSING
which musical properties should be associated with the
The sonification of EEG is of great value to the
right brainwave type. The uses of such a device can be
developers of musical applications: weather technologies
extremely beneficial when it comes to "inclusion" of
or software solutions. On the other hand, this study
people with manual disabilities in music-making.
recognized the importance of looking at this subject from
a theoretical point of view as it helped in choosing the
1. INTRODUCTION
best methods of measuring, analyzing and converting
We know that “Sound is a regular mechanical vibration brainwaves.
that travels through matter as a waveform” which
exhibits all characteristics of longitudinal waves. There are various distinct processes involved in
(Kurtus).Sound waves with specific characteristics can converting neural signals to sound signals or sonification
be viewed as music. Alterations of ordinary sound in of neural signals. “Analyzing multichannel EEG signals
tone, note, time durations etc. create melody or music. using sounds seems a natural method: using a
The words of N’Diaye “a distributed network of brain sonification of EEG signals.... perceive simultaneously
areas has been repeatedly evidenced in timing tasks” every channel, and analyse more tractably the time
identify musical touch of brain waves and activities. dynamics of the signals – hoping to gain new insights
(N’Diaye, Garnero and Pouthas). Each state of brain is about the brain signals.” (Vialatte and Cichocki). The
represented by certain waves called brain waves of sonification of neural signals is done in various steps
which Gamma, Beta, Alpha, Theta and Delta are the having separate sets of procedures. “This process
recognized brain waves. Gamma waves (30 to 70 Hz) consisted of the following stages:
are produced while “processing of various attended
stimuli…. From an EEG point of view, they will be 1. Data acquisition
present mostly while a subject is awake, but they will 2. Data pre-processing:
always be supported by other waves in the beta, alpha, 3. Intermediate representation (the creation of
theta, or delta ranges.” (Instant meditation: The visual and sonic map);
Concept). Usual considerations are given to main brain
4. Visualization and sonification (Brouse et al, 9)
waves excluding supporting gamma waves. Brainwave
activity tends to fall into four groups: beta, alpha, theta
2.1 Brain Wave Conversion 3. BRAIN MUSIC SYSTEM

A number of methods are available to be employed in The system used in the current project is built on the
analysing the various factors involved in transforming basis of the LORETA algorithm discussed about by
brain waves to music. Power spectrum analysis (PSA) Filatriau et al. (2007, p. 2). In more detail, the system
and Discrete Fourier Transform (DFT ) techniques are design includes three major stages, i. e. EEG collection,
suitable in our context. “The Fourier Transform of the digital signal processing, and MIDI representation.
Auto correlation Function is the Power Spectrum.”
This system design permits collection of the EEG waves
(Nyack). Power spectrum is used in analysing various
of the participants’ brains with the help of electrodes
images which finds application in this investigation as
placed on their both hemispheres. Next, the data
well. The first step in PSA is to Fourier transform the
collected is processed with the help of the LORETA
image I(x, y) and calculates the square modulus of the
algorithm (Filatriau et al., 2007, p. 2), and after this the
FT to generate the power spectrum, p (u, v).
system produces MIDI files on the basis of analogies
between the qualities of EEG waves and specific
musical notes.
p(u,v) = |FT[I(x, y)]|2 (1) As seen from the above statement (Filatriau et al., 2007,
(Power Spectral Analysis) p. 2)., the LORETA algorithm plays a crucial role in the
operation of the system design This algorithm is based
In order for us to obtain the active PSA , the FT array is on four major criteria defined and calculated as first, it is
rearranged according to frequencies in a way “that the necessary to measure the potential of EEG wave
zero frequency is in the centre of the array, the 1st occurrence, Φ. Second, the value of the sources
quadrant is in the upper right, the second in the upper producing those EEG waves, φ, is measured. The third
left, etc.” (Power Spectral Analysis). and the fourth criteria that help in estimating the second
The resulting array was later converted to get values point are the lead field matrix, G, and the rate of
from 1.0 to 10.0 whose logarithm to base 10 is obtained. additional noise, ŋ (Filatriau et al., 2007, p. 2):

The array obtained was then used to obtain the required

equivalent; here brain waves underwent power spectrum Φ = G φ+ ŋ (2)
analysis to get music equivalent for them. Discrete
Fourier Transform (DFT) is another procedure carried
out. “The Discrete Fourier Transform (DFT) At the same time, Ito et al. (2006, p. 1153) propose a
allows the computation of spectra from discrete-time slightly different formula that includes the role of mental
data... in discrete-time we can exactly calculate spectra.” change in sound stimulation, S, and the additional noise,
(Johnson). This definition itself explains the relevance of N, for the calculation of Y, the time series data:
this method in our analysis of brain waves. Accurate
computation is significant for conversion of brain waves
to music waves so that least redundancies result. Y =S+N (3)

2.2 Brainwave Collection and Analysis

In any case, both formulae require additional
The method adopted for data collection can be seen calculations, and Filatriau et al. (2007, p. 2) provide
through the transformation of the EEG data, which was rationale for them, arguing that the bayesian formalism
collected by an EEG collecting device as raw data. The fits the goal of defining the value of the sources
approach is influenced by the interface described in the producing those EEG waves from the above formula.
article Brain-Computer music interface for composition So, the design system discussed here uses the following
and performance by Eduardo Reck Miranda, where formula to obtain the final data that are later sent to the
different frequency bands trigger corresponding sound synthesis module (Filatriau et al., 2007, p. 2)
responses to the recording device (Hofstadter, 2009)
Installation of an EEG unit poses many restrictions like
size and cost , which led us to adopt the usage of a P(φ/Φ) = P(Φ/φ)P(φ) / P(Φ) (4)
device named pendant EEG. “Pendant-EEG is a
lightweight 2 channel EEG unit that can be clipped to
your clothing and connects to your computer via a The data obtained through the above formula are ready
wireless receiver.” (Pendant-EEG). for processing with the help of the LORETA algorithm
The communication essential for the application is given that includes four stages:
through wireless technique with efficiency by this. Our
application can be relied with confidence on pendant 1. Sending the data to the sound synthesis module;
EEG as “it can reliably process signals from 0.1 to 56 2. Associating the brain zones with cognition,
Hz.” (Pendant-EEG). visualization, and movements;
3. Creation of dipoles from the calculated data;
4. Computing the dipoles and using them as
features for creating the respective MIDI files
(Filatriau et al., 2007, p. 2).
Thus, the use of the LORETA algorithm is the basis on
which the performance of the discussed system design is
founded and the scheme described in figure 2.1 becomes
possible with the use of this cheaper technology :

Figure 2. Graphical representation of Pilot Results with

Figure 1. Brain Music System Technology flow the mean result recorded for each Brainwave type of
every subject..
At this point, the system’s design performance follows
this scheme and uses the LORETA algorithm to enable
the researchers to convert the EEG waves into MIDI

4. PILOT STUDY
The results for ten subjects undergoing a regular
recording of 15 second active blocks using the Brain
Music System (as described by Attard Trevisan &
Jones ) with a Pendent EEG collecting device were
collected and presented in the table below. The four
different brain waves i.e., Alpha, Beta, Delta, and Theta
were color-coded as green, red, yellow, and blue
respectively. The table below presents the average
values of the four forms of EEG waves for ten subjects Figure 3. Graphical representation of Pilot Study results
undergoing 15 seconds of useful recording blocks. with the trend of the most stable results recorded for
each Brainwave type.
Objectives of the Study
5. DISCUSSION OF RESULTS

The linear Band Frequency Graph 2 shows that

1. Check if there are common patterns and levels throughout the analysis of brain waves, the Beta wave
of Brainwave activity in EEG outputs which can presented as the most significant form of brain with the
be optimally used in the musical process of the highest mean wave followed by Theta and Alpha waves
Brain Music System respectively. The least form of brain wave was the Delta
wave which had consistently lower figures in most of the
2. Compare Output Brainwave levels by subjects. According to the linear graph, Delta is
the"modified LORETA" with published statistically insignificant in this study in awake subjects
literature studies and is thus deemed not relevant to include it. The means
of the relevant bands recorded in the table above were
4.1 Results analyzed and the results presented as Band Frequency
Graph 1 (figure 2) and Band Frequency Graph 2 (figure
3) .They clearly indicate which of the bands had the most
stable output, in order to further confirm which bands
should be given priority for its role in the brain music
system. The results show that the Beta waves have the
most stable output, followed by the Alpha and the Theta
waves.

Throughout the analysis of brain waves, the Beta wave

presented as the most significant form of wave with the
Table 1. Results of Pilot Study with the mean result highest mean wave followed by Theta and Alpha waves
recorded for each Brainwave type of every subject.. respectively. The least form of brain wave was the Delta
wave which since the experiment needed subjects to
remain assertive had consistently lower figures in each of
the ten subjects .
In this study, the Beta wave provides the best avenue for described music as being “your own experience, your
the study of the interaction between a musical piece and thoughts, your wisdom“. This study shows that even
the brain. The results indicate that the left frontal regions people that cannot contribute physically to music-
of the brain are more involved in processing as shown by making can transmit all those 3 musical features to an
the higher mean of the Beta range. The right hemisphere audience.
may also be increasingly engaged with higher
frequencies of the Beta wave. The Beta range can be
4. REFERENCES
used to indicate the part of brain that is involved in
processing a particular kind of music. The Theta band
[1] E D Adrian and B H C Matthews (1934), The
showed coherence in pattern in the ten subjects and that
Berger Rhythm: Potential Changes from the
coherence increased symmetrically, except in just few
Occipital Lobes in Man, Brain, 57, 4, pp. 355-
cases. The Alpha band was characterized by more
85.
coherence decreases and extending over longer distances
than other bands. [2] C Anderson and Z Sijercic (1996),
Classification of EEG signals from four subjects
The interpretation of increases in coherence advances the during five mental tasks, Solving Engineering
theory of increasing cooperation between two regions of Problems with Neural Networks: Proceedings of
the brain. Decreases on the other hand indicate that the Conference on Engineering Applications in
mental process under investigation requires lower Neural Networks (EANN’96), London, UK.
collaboration between the two regions in order to
[3] G Assayag and S Dubnov (2004), Using Factor
perform optimally. Changes in gravity centers of
Oracles for Machine Improvisation, Soft
coherence clearly indicate particular significance of the
Computing, 8, 9, pp. 604-610.
regions involved for processing information and how
other cortical regions are involved. In the case of [4] H Berger (1929), Über Das
decreases, the region concerned may decouple from Elektrenkephalogramm Des Menschen, Archiv
other cortical regions. Visual data processing studies für Psychiatrie und Nervenkrankheiten, 87, pp.
have substantiated this view and can be applied to the 527-70.
alpha band as is the case in this study. In other words,
[5] N Birbaumer, N Ghanayin, T Hinterberger, I
attentive listening needs increased attention and suspends
Iversen, B Kotchoubey, A Kubler, J
the freely floating thinking that could be assumed to take
Perelmeouter, E Taub and H Flor (1999), A
place upon EEG at rest; the two processes lead to
spelling device for the paralysed, Nature,
parcellation of the cortex in that frequency band that is
398,pp. 297-298.
concerned in general attention processes. Moreover, it
could also be that cortical coherence is reduced for an [6] D Cope (2001), Virtual Music. Cambridge,
increased information exchange with subcortical sites. MA: The MIT Press.
“As far as the behaviour in the Theta band is concerned,
[7] D Giannitrapani (1985). The Electrophysiology
it was found to be fairly characteristic in processes where
of Intellectual Functions. Basel: Karger.
memory takes a momentous part (in this case, the
violoncellist knew the piece by heart and thus, mentally [8] C Guger and G Pfurtscheller (1998), EEG based
anticipated every single phrase)” (Hellmuth et al.). brain-computer interface (BCI) running in real-
time under Windows, Electroencephalography
Conversely, emotion is also reflected by coherence. and Clinical Neurophysiology (Proceedings of
Machleidt et al shows that different bands may be 9th European Congress of Clinical
adjacent to different domains of sensory signal Neurophysiology) 106, 1, pp. 91-100.
processing. It is worthy noting that the extensive twisting
[9] B Hjorth (1970), EEG analysis based on time
of the cortex and the electrical conductivities of tissue
series properties,Electroencephalography and
layers may cause the electric features of the surface EEG
Clinical Neurophysiology, 29, pp. 306-310.
not to be displayed fully. However, characteristic
coherence patterns can be found. [10] T Kohonen, P Laine, K Tiits and K Torkkola
(1991), A Nonheuristic Automatic Composing
6. CONCLUSION Method, In Music and Connectionism (P Todd
& D G Loy, Eds), Cambridge (MA): The MIT
From this study, common patterns and levels of Press.
Brainwave activity in EEG outputs can be optimally
[11] A Lucier (1976), Statement On: Music for Solo
used in the musical process of the Brain Music System ,
Performer, In Biofeedback and the Arts, Results
thus helping in recreating a melody that is unique and
of Early Experiments (D Rosenboom, Ed),
fully adapted on the mood of the person using the
Vancouver: Aesthetic Research Center of
system . The stratification of the different elements of
Canada Publications.
the EEG can help to find out which band is most
significant in the musical process of the Brain Music [12] E R Miranda, S Roberts and M Stokes (2004),
System. International Jazz musician Charlie Parker On Generating EEG for Controlling Musical
 Systems, Biomedizinische Technik, 49, 1, pp. and Associated Technologies (ICDVRAT
75-76. 2006), Esbjerg, pp. 295-302.
[13] E R Miranda, K Sharman, K Kilborn and A [26] K E Misulis (1997), Essentials of Clinical
Duncan (2003), ‘On Harnessing the Neurophysiology, Butterworth-Heinemann,
Electroencephalogram for the Musical Boston, MA.
Braincap’, Computer Music Journal, 27, 2, pp.
80-102.
[27] E Niedermeyer and F H L da Silva (Eds.),
[14] K E Misulis (1997), Essentials of Clinical
Electroencephalography, 2nd edition. Urban
Neurophysiology. Boston (MA): Butterworth- and Schwartzenberg, Munich
Heinemann.
[28] Steven G. Mason, Member, IEEE and Gary E.
[15] E Niedermeyer and F H Lopes da Silva (Eds.)
Birch, Member, IEEE, “A General Framework
(1987), Electroencephalography, 2nd edition. for Brain- Computer Interface Design” , VOL.
Munich: Urban and Schwartzenberg. 11, NO. 1, MARCH 2003
[16] I Peretz and R J Zatorre (2003), The Cognitive
[29] Michael J. Riezenman, “Melding Mind and
Neuroscience of Music. Oxford: Oxford Machine”- IEEE publications, issue June 2008
University Press.
[30] Janne Lehtonen, “EEG-based Brain Computer
[17] H Petsche and S C Etlinger (1998), EEG and
Interfaces”, May 2002
Thinking. Vienna: Austrian Academy of
Sciences. [31] Pasi Jylänki, “Classification of single-trial EEG
for online brain-computer interface”, June 2006
[18] D Rosenboom (Ed.) (1975), Biofeedback and
the Arts, Results of Early Experiments. [32] Jessica D. Bayliss, “A Flexible Brain-Computer
Vancouver: Aesthetic Research Center of Interface”, University of Rochester, 2001
Canada Publications. [33] J. Kalcher, D. Flotzinger, C. Neuper, S. Gölly,
[19] D Rosenboom (1990), The Performing Brain, and G. Pfurtscheller, "Graz Brain-Computer
Computer Music Journal, 14, 1, pp. 48-65. Interface II: towards communication between
humans and computers based on online
[20] R Teitelbaum (1976), In Tune: Some Early
classification of three different EEG patterns,"
Experiments in Biofeedback Music (1966- Med. Biol. Eng. Comput., vol. 34, pp. 382- 388,
1974), In Biofeedback and the Arts, Results of 1996.
Early Experiments (D Rosenboom, Ed),
Vancouver: Aesthetic Research Center of [34] J.J. Vidal, "Toward direct brain-computer
Canada Publications. communication," Annu. Rev. of Biophys.
Bioeng., pp. 157-180, 1973.
[21] J J Vidal (1973), Toward Direct Brain-
Computer Communication, In Annual Review [35] J.R. Wolpaw and D.J. McFarland,
of Biophysics and Bioengineering (L J Mullins, "Multichannel EEG-based brain-computer
Ed.), pp. 157-180. communication," Electroenceph. Clin.
Neurophysiol., vol. 90, pp. 444-449, 1994.
[22] J Wolpaw, D McFarland, G Neat, C Forneris
(1991), An EEG-Based Brain-Computer [36] E. Sutter, "The brain response interface:
Interface for Cursor Control, communication through visually-induced
Electroencephalography and Clinical electrical brain response," J. Microcomput.
Neurophysiology, 78, 3, pp. 252-9. Applications, vol. 15, pp. 31-45, 1992.
[23] H Berger (1969), On the Electroencephalogram [37] A. Kübler, B. Kotchoubey, T. Hinterberger, N.
of Man, The Fourteen Original Reports on the Ghanayim, J. Perelmouter, M. Schauer, C.
Human Electroencephalogram Fritsch, E. Taub, and N. Birbaumer, "Thought
,Electroencephalography and Clinical Translation Device: A Neurophysiological
Neurophysiology, Supplement No. 28. Elsevier, Approach to Communication in Total Motor
Amsterdam. Paralysis," Exp. Brain Research, vol. 124, pp.
223-232, 1999.
[24] G Dornhege, J R Millan, T Hinterberger, D J
McFarland and K-R Muller (Eds.), Toward
Brain-Computer Interfacing, The MIT Press, [38] G. Pfurtscheller, J. Kalcher, C. Neuper, D.
Cambridge, MA. Flotzinger, and M. Pregenzer, "On-line EEG
[25] E R Miranda (2006), Brain-Computer Interface classification during externally-paced hand
for Generative Music, Proc. of International movements using a neural network-based
Conference Series on Disability, Virtual Reality classifier," Electroenceph. Clin. Neurophysiol.,
vol. 99, pp. 416-425, 1996.
[39] N. Birbaumer, N. Ghanayim, T. Hinterberger, I. [50] Guger, A. Schlögl, D. Walterspacher, and G.
Iversen, B. Kotchoubey, A. Kübler, J. Pfurtscheller, "Design of an EEG-based brain
Perelmouter, E. Taub, and H. Flor, “A spelling computer interface (BCI) from standard
device for the paralysed,” Nature, vol. 398, pp. components running in real-time under
297- 298, 1999 Windows," Biomed. Technik, vol. 44, pp. 12-
16, 1999
[40] D.B. Popovic, R.B. Stein, K.L. Jovanovic, R.
Dai, A. Kostov, and W.W. Armstrong, "Sensory [51] C.Guger, "Implementation of different EEG
nerve recording for closed-loop control to processing algorithms with S-Functions,"
restore motor functions," IEEE Biomed. Eng., Master thesis, University of Technology Graz,
vol. 40(10), pp. 1024-1031, 1993. 1997.
[41] G. Pfurtscheller, C. Neuper, D. Flotzinger, and [52] S. Haykin, Adaptive Filter Theory. New Jersey:
M. Pregenzer, "EEG-based discrimination Prentice Hall, Englewood Cliffs, 1986.
between imagination of right and left hand
[53] B. Hjorth, "EEG analysis based on time domain
movement," Electroenceph. Clin.
properties," Electroenceph. Clin. Neurophysiol.,
Neurophysiol., vol. 103, pp. 642-651, 1997.
vol. 29, pp. 306-310, 1970.
[42] G. Pfurtscheller and C. Neuper, "Motor imagery
[54] T.M. Vaughan, J.R. Wolpaw, and E. Donchin,
activates primary sensorimotor area in humans,"
"EEG-based communication: Prospects and
Neurosci. Lett., vol. 239, pp. 65-68, 1997.
problems," IEEE Trans. Rehab. Engng., vol. 4,
[43] J. Müller-Gerking, G. Pfurtscheller, and H. pp. 425-430, 1996.
Flyvbjerg, "Designing optimal spatial filters
forsingle-trial EEG classification in a movement
task," Electroenceph. Clin. Neurophysiol., vol.
110, pp. 787-798, 1999.
[44] D.J. McFarland, A.T. Lefkowicz, and J.R.
Wolpaw, "Design and operation of an EEG-
based brain-computer interface (BCI) with
digital signal processing technology," Behav.
Res. Meth. Instr. Comput., vol. 29, pp. 337-345,
1997.
[45] A. Schlögl, D. Flotzinger, and G. Pfurtscheller,
"Adaptive autoregressive modeling used for
single-trial EEG classification," Biomed.
Technik, vol. 42, pp. 162-167, 1997.
[46] D.Flotzinger, J. Kalcher, and G. Pfurtscheller,
"EEG classification by learning vector
quantization," Biomed. Technik, vol. 37, pp.
303-309, 1992.
[47] K. Lugger, D. Flotzinger, A. Schlögl, M.
Pregenzer, and G. Pfurtscheller, "Feature
extraction for on-line EEG classification using
principal components and linear discriminants,"
Med. Biol. Eng. Comput., vol. 36, pp. 309-314,
1998.
[48] G. Pfurtscheller, C. Neuper, A. Schlögl, and K.
Lugger, "Separability of EEG signals recorded
during right and left motor imagery using
adaptive autoregressive parameters," IEEE
Trans. Rehab. Engng., vol. 6, pp. 316-325,
1998.
[49] Neuper, A. Schlögl, and G. Pfurtscheller,
"Enhancement of left-right sensorimotor EEG
differences during feedback-regulated motor
imagery," J. Clin. Neurophysiol., vol. 16(2),
1999