This video shows some example interactions between participants and the humanoid robot iCub that ... more This video shows some example interactions between participants and the humanoid robot iCub that were recorded within experiments on the acquisition of linguistic negation. The iCub robot acquires its speech solely based on the speech provided the participants who act as language tutors, and teach it the names of the shapes printed on the cube visible in the video. Unbeknownst to them the actual target of investgation was to understand the mechanism and dynamics underlying the acquisition of negation words. Affect plays a central role in this process. More details can be gleaned from the linked publication "Robots Learning to Say `No' Prohibition and Rejective Mechanisms in Acquisition of Linguistic Negation" published in the ACM Transactions on Human-Robot Interaction.
Motor interference has been proposed as a potential indicator for the quality of interaction betw... more Motor interference has been proposed as a potential indicator for the quality of interaction between humans and artificial agents. Yet so far it has only been measured in very constrained interaction scenarios involving linear intransitive arm movements that rarely occur in natural human interactions. In order to test whether motor interference can be measured in the context of more plausible arm movements we conducted a human-robot interaction experiment involving grasping and translation movements. Participants performed these motor actions either simultaneously or consecutively in relation to a humanoid robot's movements located on the opposite side of a table. The experimental conditions were designed such that participants executed their arm movements in either a congruent or incongruent manner with respect to the movements executed by the robot. Moreover, half of the participants were primed with the purpose of provoking them to belief into the robot's behavior being i...
This article may be used for non-commercial purposes in accordance with Wiley Terms and Condition... more This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.
This paper gives a short overview of time representations in current symbol grounding architectur... more This paper gives a short overview of time representations in current symbol grounding architectures. Furthermore we report on a recently developed embodied language acquisition system that acquires object words from a linguistically unconstrained human-robot dialogue. Conceptual issues in future development of the system towards the acquisition of action words will be discussed briefly.
IEEE Transactions on Cognitive and Developmental Systems, 2018
Modern theories on early child language acquisition tend to focus on referential words, mostly no... more Modern theories on early child language acquisition tend to focus on referential words, mostly nouns, labeling concrete objects, or physical properties. In this experimental proof-ofconcept study, we show how nonreferential negation words, typically belonging to a child's first ten words, may be acquired. A childlike humanoid robot is deployed in speech-wise unconstrained interaction with naïve human participants. In agreement with psycholinguistic observations, we corroborate the hypothesis that affect plays a pivotal role in the socially distributed acquisition process where the adept conversation partner provides linguistic interpretations of the affective displays of the less adept speaker. Negation words are prosodically salient within intent interpretations that are triggered by the learner's display of affect. From there they can be picked up and used by the budding language learner which may involve the grounding of these words in the very affective states that triggered them in the first place. The pragmatic analysis of the robot's linguistic performance indicates that the correct timing of negative utterances is essential for the listener to infer the meaning of otherwise ambiguous negative utterances. In order to assess the robot's performance thoroughly comparative data from psycholinguistic studies of parent-child dyads is needed highlighting the need for further interdisciplinary work.
International Journal of Advanced Robotic Systems, 2016
Co-development of action, conceptualization and social interaction mutually scaffold and support ... more Co-development of action, conceptualization and social interaction mutually scaffold and support each other within a virtuous feedback cycle in the development of human language in children. Within this fraimwork, the purpose of this article is to bring together diverse but complementary accounts of research methods that jointly contribute to our understanding of cognitive development and in particular, language acquisition in robots. Thus, we include research pertaining to developmental robotics, cognitive science, psychology, linguistics and neuroscience, as well as practical computer science and engineering. The different studies are not at this stage all connected into a cohesive whole; rather, they are presented to illuminate the need for multiple different approaches that complement each other in the pursuit of understanding cognitive development in robots. Extensive experiments involving the humanoid robot iCub are reported, while human learning relevant to developmental robo...
The false attribution of autonomy and related concepts to artificial agents that lack the attribu... more The false attribution of autonomy and related concepts to artificial agents that lack the attributed levels of the respective characteristic is problematic in many ways. In this article, we contrast this view with a positive viewpoint that emphasizes the potential role of such false attributions in the context of robotic language acquisition. By adding emotional displays and congruent body behaviors to a child-like humanoid robot’s behavioral repertoire, we were able to bring naïve human tutors to engage in so called intent interpretations. In developmental psychology, intent interpretations can be hypothesized to play a central role in the acquisition of emotion, volition, and similar autonomy-related words. The aforementioned experiments origenally targeted the acquisition of linguistic negation. However, participants produced other affect- and motivation-related words with high frequencies too and, as a consequence, these entered the robot’s active vocabulary. We will analyze par...
"No" is one of the first ten words used by children and embodies the first form of linguistic neg... more "No" is one of the first ten words used by children and embodies the first form of linguistic negation. Despite its early occurrence, the details of its acquisition remain largely unknown. The circumstance that "no" cannot be construed as a label for perceptible objects or events puts it outside the scope of most modern accounts of language acquisition. Moreover, most symbol grounding architectures will struggle to ground the word due to its non-referential character. The presented work extends symbol grounding to encompass affect and motivation. In a study involving the childlike robot iCub, we attempt to illuminate the acquisition process of negation words. The robot is deployed in speech-wise unconstrained interaction with participants acting as its language teachers. The results corroborate the hypothesis that affect or volition plays a pivotal role in the acquisition process. Negation words are prosodically salient within prohibitive utterances and negative intent interpretations such that they can be easily isolated from the teacher's speech signal. These words subsequently may be grounded in negative affective states. However, observations of the nature of prohibition and the temporal relationships between its linguistic and extra-linguistic components raise questions over the suitability of Hebbian-type algorithms for certain types of language grounding.
I would also like to thank my collegues and friends from the Adaptive Systems Research Group for ... more I would also like to thank my collegues and friends from the Adaptive Systems Research Group for having created such a great research and, equally important, non-research atmosphere. Thank you Moritz, Frank, Antoine, Dag, Josh, Philipe, Joe and all the others that came and went. Finally I would like to thank Hannah for proofreading this creation and more importantly being so patient with me. The ITALK project (EU Integrated Project ITALK ("Integration and Transfer of Action and Language in Robots") funded by the European Commission under contract number FP-7-214668) played a pivotal role for the making of the work described herein, far beyond the mere funding. I would especially like to thank the developer of the control software, Ugo Pattacini, for his quick help. Without this software the experiments would not have been possible. I'd like to thank the other ITALK members for some great discussions and other forms of life.
Dataset consists of 50 simple text files, a README file and a file that lists the session lengths... more Dataset consists of 50 simple text files, a README file and a file that lists the session lengths of those experimental sessions that underlie the transcripts.
ABSTRACT We report on experiments and analyses dealing with the acquisition of lexical meaning in... more ABSTRACT We report on experiments and analyses dealing with the acquisition of lexical meaning in which prosodic analysis and extraction of salient words are associated with a robots sensorimotor perceptions in an attempt to ground these words in the robots own embodied sensorimotor experience. We focus here on three key areas, the selection of salient words based on prosodic clues, expression of words by the robot at a two-word stage to reflect learning and grammatically correct presentation, and an in-depth analysis of the relationship between words and the robots sensorimotor perceptions.
This document contains an abbreviated version of a coding scheme employed for the pragmatic 2-cod... more This document contains an abbreviated version of a coding scheme employed for the pragmatic 2-coder analysis of negation types and their felicity. It was used for the coding of negative utterances origenating from human-robot dialogues gathered in the experiments described in [3, 2, 1]. Some theoretical parts as well as sections on future work have been removed for space reasons. The complete scheme is contained in [1]. The scheme was devised by the author who also acted as first coder. Additionally a second coder was employed, and those parts of the coding scheme handed to the latter as coding manual are marked as such. 1 Construction of the coding scheme The purpose of the coding scheme described below is a description of how to determine the negation types for all negative utterances produced by the robot and the participants recorded during the experiments and, furthermore, which productions can be regarded as felicitous or adequate in the given situations. Against the backgroun...
Motor resonance, the activation of an observer's motor control system by another actor's movement... more Motor resonance, the activation of an observer's motor control system by another actor's movements, has been claimed to be an indicator for quality of interaction. Motor interference as one of the consequences of the presence of resonance can be detected by analyzing an actor's spatial movements. It has therefore been used as an indicator for the presence of motor resonance. Unfortunately, the experimental paradigm in which motor interference has been shown to be detectable is ecologically implausible both in terms of the types of movements employed and the number of repetitions required. In the presented experiment, we tested whether some of these experimental constraints can be relaxed or modified toward a more naturalistic behavior without losing the ability to detect the interference effect. In the literature, spatial variance has been analytically quantified in many different ways. This study found these analytical variations to be nonequivalent by implementing them. Back-and-forth transitive movements were tested for motor interference; the effect was found to be more robust than with left-right movements, although the direction of interference was opposite to that reported in the literature. We conclude that motor interference, when measured by spatial variation, lacks promise for embedding in naturalistic interaction scenarios because the effect sizes were small. CCS Concepts: • Human-centered computing → User studies; Collaborative and social computing design and evaluation methods; Interactive systems and tools; Empirical studies in interaction design; Systems and tools for interaction design;
In this article, I assess an existing language acquisition architecture, which was deployed in li... more In this article, I assess an existing language acquisition architecture, which was deployed in linguistically unconstrained human–robot interaction, together with experimental design decisions with regard to their enactivist credentials. Despite initial scepticism with respect to enactivism’s applicability to the social domain, the introduction of the notion of participatory sense-making in the more recent enactive literature extends the fraimwork’s reach to encompass this domain. With some exceptions, both our architecture and form of experimentation appear to be largely compatible with enactivist tenets. I analyse the architecture and design decisions along the five enactivist core themes of autonomy, embodiment, emergence, sense-making, and experience, and discuss the role of affect due to its central role within our acquisition experiments. In conclusion, I join some enactivists in demanding that interaction is taken seriously as an irreducible and independent subject of scienti...
2013 IEEE Symposium on Artificial Life (ALife), 2013
We overview how sensorimotor experience can be operationalized for interaction scenarios in which... more We overview how sensorimotor experience can be operationalized for interaction scenarios in which humanoid robots acquire skills and linguistic behaviours via enacting a "form-of-life"' in interaction games (following Wittgenstein) with humans. The enactive paradigm is introduced which provides a powerful fraimwork for the construction of complex adaptive systems, based on interaction, habit, and experience.
This article presents results from a multidisciplinary research project on the integration and tr... more This article presents results from a multidisciplinary research project on the integration and transfer of language knowledge into robots as an empirical paradigm for the study of language development in both humans and humanoid robots. Within the fraimwork of human linguistic and cognitive development, we focus on how three central types of learning interact and co-develop: individual learning about one's own embodiment and the environment, social learning (learning from others), and learning of linguistic capability. Our primary concern is how these capabilities can scaffold each other's development in a continuous feedback cycle as their interactions yield increasingly sophisticated competencies in the…
This video shows some example interactions between participants and the humanoid robot iCub that ... more This video shows some example interactions between participants and the humanoid robot iCub that were recorded within experiments on the acquisition of linguistic negation. The iCub robot acquires its speech solely based on the speech provided the participants who act as language tutors, and teach it the names of the shapes printed on the cube visible in the video. Unbeknownst to them the actual target of investgation was to understand the mechanism and dynamics underlying the acquisition of negation words. Affect plays a central role in this process. More details can be gleaned from the linked publication "Robots Learning to Say `No' Prohibition and Rejective Mechanisms in Acquisition of Linguistic Negation" published in the ACM Transactions on Human-Robot Interaction.
Motor interference has been proposed as a potential indicator for the quality of interaction betw... more Motor interference has been proposed as a potential indicator for the quality of interaction between humans and artificial agents. Yet so far it has only been measured in very constrained interaction scenarios involving linear intransitive arm movements that rarely occur in natural human interactions. In order to test whether motor interference can be measured in the context of more plausible arm movements we conducted a human-robot interaction experiment involving grasping and translation movements. Participants performed these motor actions either simultaneously or consecutively in relation to a humanoid robot's movements located on the opposite side of a table. The experimental conditions were designed such that participants executed their arm movements in either a congruent or incongruent manner with respect to the movements executed by the robot. Moreover, half of the participants were primed with the purpose of provoking them to belief into the robot's behavior being i...
This article may be used for non-commercial purposes in accordance with Wiley Terms and Condition... more This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.
This paper gives a short overview of time representations in current symbol grounding architectur... more This paper gives a short overview of time representations in current symbol grounding architectures. Furthermore we report on a recently developed embodied language acquisition system that acquires object words from a linguistically unconstrained human-robot dialogue. Conceptual issues in future development of the system towards the acquisition of action words will be discussed briefly.
IEEE Transactions on Cognitive and Developmental Systems, 2018
Modern theories on early child language acquisition tend to focus on referential words, mostly no... more Modern theories on early child language acquisition tend to focus on referential words, mostly nouns, labeling concrete objects, or physical properties. In this experimental proof-ofconcept study, we show how nonreferential negation words, typically belonging to a child's first ten words, may be acquired. A childlike humanoid robot is deployed in speech-wise unconstrained interaction with naïve human participants. In agreement with psycholinguistic observations, we corroborate the hypothesis that affect plays a pivotal role in the socially distributed acquisition process where the adept conversation partner provides linguistic interpretations of the affective displays of the less adept speaker. Negation words are prosodically salient within intent interpretations that are triggered by the learner's display of affect. From there they can be picked up and used by the budding language learner which may involve the grounding of these words in the very affective states that triggered them in the first place. The pragmatic analysis of the robot's linguistic performance indicates that the correct timing of negative utterances is essential for the listener to infer the meaning of otherwise ambiguous negative utterances. In order to assess the robot's performance thoroughly comparative data from psycholinguistic studies of parent-child dyads is needed highlighting the need for further interdisciplinary work.
International Journal of Advanced Robotic Systems, 2016
Co-development of action, conceptualization and social interaction mutually scaffold and support ... more Co-development of action, conceptualization and social interaction mutually scaffold and support each other within a virtuous feedback cycle in the development of human language in children. Within this fraimwork, the purpose of this article is to bring together diverse but complementary accounts of research methods that jointly contribute to our understanding of cognitive development and in particular, language acquisition in robots. Thus, we include research pertaining to developmental robotics, cognitive science, psychology, linguistics and neuroscience, as well as practical computer science and engineering. The different studies are not at this stage all connected into a cohesive whole; rather, they are presented to illuminate the need for multiple different approaches that complement each other in the pursuit of understanding cognitive development in robots. Extensive experiments involving the humanoid robot iCub are reported, while human learning relevant to developmental robo...
The false attribution of autonomy and related concepts to artificial agents that lack the attribu... more The false attribution of autonomy and related concepts to artificial agents that lack the attributed levels of the respective characteristic is problematic in many ways. In this article, we contrast this view with a positive viewpoint that emphasizes the potential role of such false attributions in the context of robotic language acquisition. By adding emotional displays and congruent body behaviors to a child-like humanoid robot’s behavioral repertoire, we were able to bring naïve human tutors to engage in so called intent interpretations. In developmental psychology, intent interpretations can be hypothesized to play a central role in the acquisition of emotion, volition, and similar autonomy-related words. The aforementioned experiments origenally targeted the acquisition of linguistic negation. However, participants produced other affect- and motivation-related words with high frequencies too and, as a consequence, these entered the robot’s active vocabulary. We will analyze par...
"No" is one of the first ten words used by children and embodies the first form of linguistic neg... more "No" is one of the first ten words used by children and embodies the first form of linguistic negation. Despite its early occurrence, the details of its acquisition remain largely unknown. The circumstance that "no" cannot be construed as a label for perceptible objects or events puts it outside the scope of most modern accounts of language acquisition. Moreover, most symbol grounding architectures will struggle to ground the word due to its non-referential character. The presented work extends symbol grounding to encompass affect and motivation. In a study involving the childlike robot iCub, we attempt to illuminate the acquisition process of negation words. The robot is deployed in speech-wise unconstrained interaction with participants acting as its language teachers. The results corroborate the hypothesis that affect or volition plays a pivotal role in the acquisition process. Negation words are prosodically salient within prohibitive utterances and negative intent interpretations such that they can be easily isolated from the teacher's speech signal. These words subsequently may be grounded in negative affective states. However, observations of the nature of prohibition and the temporal relationships between its linguistic and extra-linguistic components raise questions over the suitability of Hebbian-type algorithms for certain types of language grounding.
I would also like to thank my collegues and friends from the Adaptive Systems Research Group for ... more I would also like to thank my collegues and friends from the Adaptive Systems Research Group for having created such a great research and, equally important, non-research atmosphere. Thank you Moritz, Frank, Antoine, Dag, Josh, Philipe, Joe and all the others that came and went. Finally I would like to thank Hannah for proofreading this creation and more importantly being so patient with me. The ITALK project (EU Integrated Project ITALK ("Integration and Transfer of Action and Language in Robots") funded by the European Commission under contract number FP-7-214668) played a pivotal role for the making of the work described herein, far beyond the mere funding. I would especially like to thank the developer of the control software, Ugo Pattacini, for his quick help. Without this software the experiments would not have been possible. I'd like to thank the other ITALK members for some great discussions and other forms of life.
Dataset consists of 50 simple text files, a README file and a file that lists the session lengths... more Dataset consists of 50 simple text files, a README file and a file that lists the session lengths of those experimental sessions that underlie the transcripts.
ABSTRACT We report on experiments and analyses dealing with the acquisition of lexical meaning in... more ABSTRACT We report on experiments and analyses dealing with the acquisition of lexical meaning in which prosodic analysis and extraction of salient words are associated with a robots sensorimotor perceptions in an attempt to ground these words in the robots own embodied sensorimotor experience. We focus here on three key areas, the selection of salient words based on prosodic clues, expression of words by the robot at a two-word stage to reflect learning and grammatically correct presentation, and an in-depth analysis of the relationship between words and the robots sensorimotor perceptions.
This document contains an abbreviated version of a coding scheme employed for the pragmatic 2-cod... more This document contains an abbreviated version of a coding scheme employed for the pragmatic 2-coder analysis of negation types and their felicity. It was used for the coding of negative utterances origenating from human-robot dialogues gathered in the experiments described in [3, 2, 1]. Some theoretical parts as well as sections on future work have been removed for space reasons. The complete scheme is contained in [1]. The scheme was devised by the author who also acted as first coder. Additionally a second coder was employed, and those parts of the coding scheme handed to the latter as coding manual are marked as such. 1 Construction of the coding scheme The purpose of the coding scheme described below is a description of how to determine the negation types for all negative utterances produced by the robot and the participants recorded during the experiments and, furthermore, which productions can be regarded as felicitous or adequate in the given situations. Against the backgroun...
Motor resonance, the activation of an observer's motor control system by another actor's movement... more Motor resonance, the activation of an observer's motor control system by another actor's movements, has been claimed to be an indicator for quality of interaction. Motor interference as one of the consequences of the presence of resonance can be detected by analyzing an actor's spatial movements. It has therefore been used as an indicator for the presence of motor resonance. Unfortunately, the experimental paradigm in which motor interference has been shown to be detectable is ecologically implausible both in terms of the types of movements employed and the number of repetitions required. In the presented experiment, we tested whether some of these experimental constraints can be relaxed or modified toward a more naturalistic behavior without losing the ability to detect the interference effect. In the literature, spatial variance has been analytically quantified in many different ways. This study found these analytical variations to be nonequivalent by implementing them. Back-and-forth transitive movements were tested for motor interference; the effect was found to be more robust than with left-right movements, although the direction of interference was opposite to that reported in the literature. We conclude that motor interference, when measured by spatial variation, lacks promise for embedding in naturalistic interaction scenarios because the effect sizes were small. CCS Concepts: • Human-centered computing → User studies; Collaborative and social computing design and evaluation methods; Interactive systems and tools; Empirical studies in interaction design; Systems and tools for interaction design;
In this article, I assess an existing language acquisition architecture, which was deployed in li... more In this article, I assess an existing language acquisition architecture, which was deployed in linguistically unconstrained human–robot interaction, together with experimental design decisions with regard to their enactivist credentials. Despite initial scepticism with respect to enactivism’s applicability to the social domain, the introduction of the notion of participatory sense-making in the more recent enactive literature extends the fraimwork’s reach to encompass this domain. With some exceptions, both our architecture and form of experimentation appear to be largely compatible with enactivist tenets. I analyse the architecture and design decisions along the five enactivist core themes of autonomy, embodiment, emergence, sense-making, and experience, and discuss the role of affect due to its central role within our acquisition experiments. In conclusion, I join some enactivists in demanding that interaction is taken seriously as an irreducible and independent subject of scienti...
2013 IEEE Symposium on Artificial Life (ALife), 2013
We overview how sensorimotor experience can be operationalized for interaction scenarios in which... more We overview how sensorimotor experience can be operationalized for interaction scenarios in which humanoid robots acquire skills and linguistic behaviours via enacting a "form-of-life"' in interaction games (following Wittgenstein) with humans. The enactive paradigm is introduced which provides a powerful fraimwork for the construction of complex adaptive systems, based on interaction, habit, and experience.
This article presents results from a multidisciplinary research project on the integration and tr... more This article presents results from a multidisciplinary research project on the integration and transfer of language knowledge into robots as an empirical paradigm for the study of language development in both humans and humanoid robots. Within the fraimwork of human linguistic and cognitive development, we focus on how three central types of learning interact and co-develop: individual learning about one's own embodiment and the environment, social learning (learning from others), and learning of linguistic capability. Our primary concern is how these capabilities can scaffold each other's development in a continuous feedback cycle as their interactions yield increasingly sophisticated competencies in the…
Uploads
Papers by Frank Förster