Books by Sam Duffy
It is common to learn to play a musical instrument through regular one-to-one lessons with an exp... more It is common to learn to play a musical instrument through regular one-to-one lessons with an experienced musician. Intuition suggests that the principal activity during these meetings would be performance, but conversation is also important; not just as a way to analyse the student’s musical contributions, but to organise them within the lesson flow. Activities are managed conversationally, discussion interleaved with performance, resulting in a rich multi-modal social interaction between student and tutor. This impacts the way that the interaction changes when a lesson is carried out remotely, mediated by video technology.
This thesis presents a detailed study of same-room and video-mediated lesson interaction. Conventional conversation analysis transcription notation was augmented specifically to represent the musical sounds produced alongside dialogue. Analysis of the shape and timing of the musical contributions shows that they are managed in ways that are analogous to turns at talk. For example, duration and timing of the tutor’s utterances, in relation to the student’s musical phrasing, determines whether they are interpreted as encouraging backchannels, or a bid for the floor to provide immediate feedback. Non-verbal behaviours such as gaze and changes in posture are used to encourage a student to self-repair and continue with their performance, despite mutual acknowledgement that a problem has occurred.
Fine-grained analysis of video-mediated lessons reveals what happens when this organisation is disturbed. The change in medium reduces the availability of non-verbal cues, and the disruption caused by latency has divergent effects on the sequence and placement of turns, which are experienced differently at each location. Students find it more difficult to anticipate tutor interruption of their performance or correctly identify backchannels, leading to miscommunication. Recommendations are made for technology which might better support student-tutor interaction during remote music lessons.
Music, speech and interaction in an instrumental music lesson: An ethnographic study of one-to-on... more Music, speech and interaction in an instrumental music lesson: An ethnographic study of one-to-one music tuition p231-280.
Download full introduction attached.
REFRAME http://reframe.sussex.ac.uk Supported by Sussex Humanities Lab CONFERENCE ORGANISATION Or... more REFRAME http://reframe.sussex.ac.uk Supported by Sussex Humanities Lab CONFERENCE ORGANISATION Organisation committee Thor Magnusson (general chair) Chris Kiefer (scientific chair) Alice Eldridge (artistic chair) Cecile Chevalier (installations chair) Paul McConnell (workshops chair) Thanos Polymeneas Liontiris (performance organiser) Sam Duffy (proceedings co-editor and impact) Joe Watson (doctoral colloquium organiser) Andrew Duff (Brighton Modular Meet planner) David Palmer (design) ICLI Steering committee 96 Bridging the gap between performers and the audience using networked smartphones: the a.bel system Alexandre Resende Clément, Filipe Ribeiro, Rui Rodrigues & Rui Penha
Papers by Sam Duffy
Clapping Music is a minimalist work by Steve Reich based on twelve phased variations of a rhythmi... more Clapping Music is a minimalist work by Steve Reich based on twelve phased variations of a rhythmic pattern. It has been reimagined as a game-based mobile application, designed with a dual purpose. First, to introduce new audiences to the Minimalist genre through interaction with the piece presented as an engaging game. Second, to use large-scale data collection within the app to address research questions about the factors determining rhythm production performance. The twelve patterns can be differentiated using existing theories of rhythmic complexity. Using performance indicators from the game such as tap accuracy we can determine which patterns players found most challenging and so assess hypotheses from theoretical models with empirical evidence. The app has been downloaded over 140,000 times since the launch in July 2015, and over 46 million rows of gameplay data have been collected, requiring a big data approach to analysis. The results shed light on the rhythmic factors contributing to performance difficulty and show that the effect of making a transition from one pattern to the next is as significant, in terms of pattern difficulty, as the inherent complexity of the pattern itself. Challenges that arose in applying this novel approach are discussed.
'Steve Reich’s Clapping Music' is an iPhone application based on the well-known contemporary piec... more 'Steve Reich’s Clapping Music' is an iPhone application based on the well-known contemporary piece of music by the minimalist composer Steve Reich. Clapping Music has been developed as a progressively difficult but compelling app for iPhone, iPod Touch and iPad, which is available as a free download from the iTunes App Store. The game was developed by a team involving researchers, musicians and technologists from three organisations - The London Sinfonietta, Touchpress, and Queen Mary University of London.
This project report (follow url above to download from the Nesta website) examines whether a specifically designed game based smartphone application can motivate users to engage with, and learn to perform, a challenging contemporary classical music piece by the composer Steve Reich, called Clapping Music. In addition we investigate whether engagement with the game led to further and deeper engagement with a new music genre, in this case Minimalism and Contemporary Classical Music. Findings from a focus group of 100 players suggest that the application was successful in increasing familiarity with the piece and the composer, but less so with the genres of Minimalism and Contemporary Music generally. However members of the group overall enjoyed playing, shared their experiences with family and friends through conversation and social media, and were motivated to progress through the game. Some reported that they believed that playing the game had improved their rhythmic skills. The project offers several immediate routes for future development, including outreach in schools, growing the community of world-wide users, further research using the existing data collected, additional studies with the App, and new apps based on the same concept, but applied to different music, or even different art forms.
'Steve Reich's Clapping Music' was funded by The Digital R&D Fund for the Arts, a partnership between Nesta, the Arts Council England and the Arts and Humanities Research Council (AHRC). The fund encourages collaboration between the arts, digital technology providers and the research community, in order to undertake experiments from which the wider arts sector can learn.
Authors
Andrew Burke
Chief Executive, London Sinfonietta
Barbara Palczynski
Project Manager, London Sinfonietta
Marcus Pearce / Sam Duffy
Research Lead / Research Assistant, Queen Mary University of London
Alan Martyn
Producer, Touchpress
Journal of Music, Technology and Education, 2017
It is common to learn to play an orchestral musical instrument through regular one-to-one lessons... more It is common to learn to play an orchestral musical instrument through regular one-to-one lessons with an experienced musician as a tutor. Students may work with the same tutor for many years, meeting regularly to receive real-time, iterative feedback on their performance. However, musicians travel regularly to audition, teach and perform and this can sometimes make it difficult to maintain regular contact. In addition, an experienced tutor for a specific instrument or musical style may not be available locally. General instrumental tuition may not be available at all in geographically distributed communities. One solution is to use technology such as videoconference to facilitate a remote lesson; however, this fundamentally changes the teaching interaction. For example, as a result of the change in communication medium, the availability of non-verbal cues and perception of relative spatiality is reduced. We describe a study using video-ethnography, qualitative video analysis and conversation analysis to make a fine-grained examination of student–tutor interaction during five co-present and one video-mediated woodwind lesson. Our findings are used to propose an alternative technological solution – an interactive digital score. Rather than the face-to-face configuration enforced by videoconference, interacting through a shared digital score, augmented by visual representation of the social cues found to be commonly used in co-present lessons, will better support naturalistic student–tutor interaction during the remote lesson experience. Our findings may also be applicable to other fields where knowledge and practice of a physical skill sometimes need to be taught remotely, such as surgery or dentistry.
Hacettepe University Journal of Education Special Edition “Conversation Analytic Studies on Teaching and Learning Practices: International Perspectives”, 2018
Whilst the focus of attention in an instrumental music lesson is refinement of the student's musi... more Whilst the focus of attention in an instrumental music lesson is refinement of the student's musical performance, conversation plays an essential role; not just as a way to analyse the student's musical contributions, but to organise them within the lesson flow. Participants may respond to talk through performance and vice versa, or even spend periods of time exchanging purely musical contributions. The short musical fragments exchanged by the participants are managed within lesson dialogue in ways analogous to conversational turn-taking. Problems in the student's performance are refined through both student self-initiated and tutor other-initiated repair, initiated by embodied action and play. A fundamental part of turn-taking is managing the transition to a new speaker. The presence of musical contributions allows for additional types of transition, for example from a turn at talk, to a musical contribution. In conversation, there is generally a preference for a short pause at the transition to a new speaker, and overlap tends to be minimised when it occurs. Through detailed qualitative video analysis of a one-to-one clarinet lesson, we find differences in the preferences regarding overlap when purely musical contributions are being exchanged, and that the duration of overlap during these exchanges of fragments of music are significant.
Proceedings of the 39th Annual Conference of the Cognitive Science Society, 2017
During a music lesson, participants need to coordinate both their turns at talk and their turns a... more During a music lesson, participants need to coordinate both their turns at talk and their turns at playing. Verbal and musical contributions are shaped by their organisation within the turn-taking system. When lessons are conducted remotely by video conference, these mechanisms are disrupted by the asymmet-ric effects of delay on the interaction; in effect a " non-mutual reality " comprised of two different conversations at each end of the link. Here we compare detailed case studies of a co-present and a remote music lesson, in order to show how this effect arises, and how it impacts conduct during the lesson.
Proceedings of the 35th Annual Conference of the Cognitive Science Society, Aug 2, 2013
Music is sometimes compared to language as a system of com- munication, however this comparison i... more Music is sometimes compared to language as a system of com- munication, however this comparison is usually at a generic formal, cultural or social level. This paper explores this anal- ogy at the detailed level of interaction: to what extent can mu- sical contributions act as conversational turns? We explore this question through an ethnographic study of music lessons. We describe a new transcription notation designed to capture the interactional details of musical contributions. Using this nota- tion we show that although the ultimate objective of a lesson is development of musical performance, the detailed structure of the musical contributions depends on their interactional or- ganisation. We show that musical contributions display inter- actional structure at the turn and sub-turn level and are closely integrated with other verbal and non-verbal cues as part of the unfolding conversation.
Proceedings of the 34th Annual Conference of the Cognitive Science Society, 2012
Playing an instrument is a physical skill learned through observation, repetition and rehearsal. ... more Playing an instrument is a physical skill learned through observation, repetition and rehearsal. Students of orchestral instruments seek one-to-one tuition from expert musicians. However as they become more accomplished, the number of suitable tutors becomes more concentrated, especially for less common instruments. Often a tutor-student relationship develops over several years and temporary separation due to overseas performing, auditioning and teaching commitments is problematic. Some music education organisations use video conferencing as a solution to these problems, however it has long been recognised that interaction mediated by video conferencing is not analogous to a co-present experience. In this paper, ethnographic video analysis is used to study the interactions in co-present and separated instrumental music lessons. We find that the musical score represents more than a physical embodiment of the music - it plays an important role in coordinating activity and interaction. In video mediated lessons a single physical score can no longer be shared and interaction is changed as a result.
Proceedings of 9th Sound and Music Computing Conference, 2012
It is common to learn to play an orchestral musical instrument through one-to-one lessons with an... more It is common to learn to play an orchestral musical instrument through one-to-one lessons with an experienced tutor. For musicians who choose to study performance at an undergraduate level and beyond, their tutor is an important part of their professional musical development. For many musicians, travel is part of their professional lives due to touring, auditioning and teaching, often overseas. This makes temporary separation of students from their tutor inevitable. A solution used by some conservatoires is teaching via video conferencing, however the challenges of using video conference for interaction and collaborative work are well documented. The Remote Music Tuition prototype was designed to enhance music tuition via video conference by providing multiple views of the student. This paper describes the system, documents observations from initial tests of the prototype and makes recommendations for future developments and further testing.
Proceedings of the 13th International Conference on Technology in Mathematics Teaching (ICTMT13), 2017
In this paper we introduce weDRAW, a project to support primary school children in the exploratio... more In this paper we introduce weDRAW, a project to support primary school children in the exploration of mathematical concepts, through the design, development and evaluation of multisensory serious games, using a combination of sensory interactive technologies. Working closely with schools, using participatory design techniques, the games will be embedded into the school curricula, and configurable by teachers. Besides application to typically developing children, a major goal is to explore the benefits of this multisensory approach with visually impaired and dyslexic children.
1st ACM SIGCHI International Workshop on Multimodal Interaction (MIE'17), 2017
The importance of multisensory interaction for learning has increased with improved understanding... more The importance of multisensory interaction for learning has increased with improved understanding of children's sensory development, and a flourishing interest in embodied cognition. The potential to foster new forms of multisensory interaction through various sensor, mobile and haptic technologies is promising in providing new ways for young children to engage with key mathematical concepts. However, designing effective learning environments for real world classrooms is challenging, and requires a pedagogically, rather than technologically, driven approach to design. This paper describes initial work underpinning the development of a pedagogical framework, intended to inform the design of a multisensory serious gaming environment. It identifies the theoretical basis of the framework, illustrates how this informs teaching strategies, and outlines key technology research driven perspectives and considerations important for informing design. An initial table mapping mathematical concepts to design, a framework of considerations for design, and a process model of how the framework will continue to be developed across the design process are provided.
Multimodal Technologies and Interaction, 2018
An increasing body of work provides evidence of the importance of bodily experience for cognition... more An increasing body of work provides evidence of the importance of bodily experience for cognition and the learning of mathematics. Sensor-based technologies have potential for guiding sensori-motor engagement with challenging mathematical ideas in new ways. Yet, designing environments that promote an appropriate sensori-motoric interaction that effectively supports salient foundations of mathematical concepts is challenging and requires understanding of opportunities and challenges that bodily interaction offers. This study aimed to better understand how young children can, and do, use their bodies to explore geometrical concepts of angle and shape, and what contribution the different sensori-motor experiences make to the comprehension of mathematical ideas. Twenty-nine students aged 6–10 years participated in an exploratory study, with paired and group activities designed to elicit intuitive bodily enactment of angles and shape. Our analysis, focusing on moment-by-moment bodily interactions, attended to gesture, action, facial expression, body posture and talk, illustrated the 'realms of possibilities' of bodily interaction, and highlighted challenges around 'felt' experience and egocentric vs. allocentric perception of the body during collaborative bodily enactment. These findings inform digital designs for sensory interaction to foreground salient geometric features and effectively support relevant forms of enactment to enhance the learning experience, supporting challenging aspects of interaction and exploiting the opportunities of the body.
Multimodal Technologies and Interaction, Jul 9, 2018
An increasing body of work provides evidence of the importance of bodily experience for cognition... more An increasing body of work provides evidence of the importance of bodily experience for cognition and the learning of mathematics. Sensor-based technologies have potential for guiding sensori-motor engagement with challenging mathematical ideas in new ways. Yet, designing environments that promote an appropriate sensori-motoric interaction that effectively supports salient foundations of mathematical concepts is challenging and requires understanding of opportunities and challenges that bodily interaction offers. This study aimed to better understand how young children can, and do, use their bodies to explore geometrical concepts of angle and shape, and what contribution the different sensori-motor experiences make to the comprehension of mathematical ideas. Twenty-nine students aged 6-10 years participated in an exploratory study, with paired and group activities designed to elicit intuitive bodily enactment of angles and shape. Our analysis, focusing on moment-by-moment bodily interactions, attended to gesture, action, facial expression, body posture and talk, illustrated the 'realms of possibilities' of bodily interaction, and highlighted challenges around 'felt' experience and egocentric vs. allocentric perception of the body during collaborative bodily enactment. These findings inform digital designs for sensory interaction to foreground salient geometric features and effectively support relevant forms of enactment to enhance the learning experience, supporting challenging aspects of interaction and exploiting the opportunities of the body.
What Happened to the Electronic Saxophone?, 2010
The Varitone was an amplified saxophone and effects unit, developed and sold by Selmer in 1967. ... more The Varitone was an amplified saxophone and effects unit, developed and sold by Selmer in 1967. Despite extensive promotion and endorsement by professional players in commercial recordings, it didn't catch on. This paper, submitted as part of a Diploma in Sound and Audio Engineering at SAE London in 2010, examines possible reasons why.
Mobile and Online Social Networks (MOSN), 2011 …, Jan 1, 2011
People's willingness to share where they are, what they are doing and with whom using location-ba... more People's willingness to share where they are, what they are doing and with whom using location-based technology has led to the emergence of applications which are being used to create new ways to represent and navigate space. One of the most popular applications in this area is Foursquare. In this paper we report the results of a short ethnographic study of the use of Foursquare in East London, combined with further analysis using the developer site to acquire data in order to investigate findings further. We describe how the Foursquare community appropriates the relatively simple communication capabilities of the application to develop their own informal rules of interaction. This behaviour is important to both application designers and businesses using social networking sites to reach and engage with consumers. Specifically we argue that tips are used for more than the intended purpose of commenting on a venue and that mayorships are a vehicle for cheating the leaderboard. We conclude by noting that designing functions to fulfill a specific purpose in media space offers no guarantee that the community will use them solely for the intended purpose.
Talks by Sam Duffy
Conference Presentations by Sam Duffy
Proceedings of the Sempre MET2016: researching music, education, technology
Regular one-to-one tuition with an experienced musician is a common way to learn to play a musica... more Regular one-to-one tuition with an experienced musician is a common way to learn to play a musical instrument however it is not always possible to maintain a regular schedule. Specific expertise may also not be available locally for students in geographically isolated locations. Video conferencing is sometimes used as a solution to this problem but is problematic in this context because it misconceives the nature of the joint activities involved in music lessons. Understanding how the medium of communication changes lesson interaction informs the design of new technology to better support remote music tuition.
Uploads
Books by Sam Duffy
This thesis presents a detailed study of same-room and video-mediated lesson interaction. Conventional conversation analysis transcription notation was augmented specifically to represent the musical sounds produced alongside dialogue. Analysis of the shape and timing of the musical contributions shows that they are managed in ways that are analogous to turns at talk. For example, duration and timing of the tutor’s utterances, in relation to the student’s musical phrasing, determines whether they are interpreted as encouraging backchannels, or a bid for the floor to provide immediate feedback. Non-verbal behaviours such as gaze and changes in posture are used to encourage a student to self-repair and continue with their performance, despite mutual acknowledgement that a problem has occurred.
Fine-grained analysis of video-mediated lessons reveals what happens when this organisation is disturbed. The change in medium reduces the availability of non-verbal cues, and the disruption caused by latency has divergent effects on the sequence and placement of turns, which are experienced differently at each location. Students find it more difficult to anticipate tutor interruption of their performance or correctly identify backchannels, leading to miscommunication. Recommendations are made for technology which might better support student-tutor interaction during remote music lessons.
Download full introduction attached.
Papers by Sam Duffy
This project report (follow url above to download from the Nesta website) examines whether a specifically designed game based smartphone application can motivate users to engage with, and learn to perform, a challenging contemporary classical music piece by the composer Steve Reich, called Clapping Music. In addition we investigate whether engagement with the game led to further and deeper engagement with a new music genre, in this case Minimalism and Contemporary Classical Music. Findings from a focus group of 100 players suggest that the application was successful in increasing familiarity with the piece and the composer, but less so with the genres of Minimalism and Contemporary Music generally. However members of the group overall enjoyed playing, shared their experiences with family and friends through conversation and social media, and were motivated to progress through the game. Some reported that they believed that playing the game had improved their rhythmic skills. The project offers several immediate routes for future development, including outreach in schools, growing the community of world-wide users, further research using the existing data collected, additional studies with the App, and new apps based on the same concept, but applied to different music, or even different art forms.
'Steve Reich's Clapping Music' was funded by The Digital R&D Fund for the Arts, a partnership between Nesta, the Arts Council England and the Arts and Humanities Research Council (AHRC). The fund encourages collaboration between the arts, digital technology providers and the research community, in order to undertake experiments from which the wider arts sector can learn.
Authors
Andrew Burke
Chief Executive, London Sinfonietta
Barbara Palczynski
Project Manager, London Sinfonietta
Marcus Pearce / Sam Duffy
Research Lead / Research Assistant, Queen Mary University of London
Alan Martyn
Producer, Touchpress
Talks by Sam Duffy
Conference Presentations by Sam Duffy
This thesis presents a detailed study of same-room and video-mediated lesson interaction. Conventional conversation analysis transcription notation was augmented specifically to represent the musical sounds produced alongside dialogue. Analysis of the shape and timing of the musical contributions shows that they are managed in ways that are analogous to turns at talk. For example, duration and timing of the tutor’s utterances, in relation to the student’s musical phrasing, determines whether they are interpreted as encouraging backchannels, or a bid for the floor to provide immediate feedback. Non-verbal behaviours such as gaze and changes in posture are used to encourage a student to self-repair and continue with their performance, despite mutual acknowledgement that a problem has occurred.
Fine-grained analysis of video-mediated lessons reveals what happens when this organisation is disturbed. The change in medium reduces the availability of non-verbal cues, and the disruption caused by latency has divergent effects on the sequence and placement of turns, which are experienced differently at each location. Students find it more difficult to anticipate tutor interruption of their performance or correctly identify backchannels, leading to miscommunication. Recommendations are made for technology which might better support student-tutor interaction during remote music lessons.
Download full introduction attached.
This project report (follow url above to download from the Nesta website) examines whether a specifically designed game based smartphone application can motivate users to engage with, and learn to perform, a challenging contemporary classical music piece by the composer Steve Reich, called Clapping Music. In addition we investigate whether engagement with the game led to further and deeper engagement with a new music genre, in this case Minimalism and Contemporary Classical Music. Findings from a focus group of 100 players suggest that the application was successful in increasing familiarity with the piece and the composer, but less so with the genres of Minimalism and Contemporary Music generally. However members of the group overall enjoyed playing, shared their experiences with family and friends through conversation and social media, and were motivated to progress through the game. Some reported that they believed that playing the game had improved their rhythmic skills. The project offers several immediate routes for future development, including outreach in schools, growing the community of world-wide users, further research using the existing data collected, additional studies with the App, and new apps based on the same concept, but applied to different music, or even different art forms.
'Steve Reich's Clapping Music' was funded by The Digital R&D Fund for the Arts, a partnership between Nesta, the Arts Council England and the Arts and Humanities Research Council (AHRC). The fund encourages collaboration between the arts, digital technology providers and the research community, in order to undertake experiments from which the wider arts sector can learn.
Authors
Andrew Burke
Chief Executive, London Sinfonietta
Barbara Palczynski
Project Manager, London Sinfonietta
Marcus Pearce / Sam Duffy
Research Lead / Research Assistant, Queen Mary University of London
Alan Martyn
Producer, Touchpress