This paper presents a strategy to represent data from a tactile array sensor and match it to an o... more This paper presents a strategy to represent data from a tactile array sensor and match it to an object's geometric features. Using that representation, a method is presented to localise a grasped object within a robot hand. The method consists of computing the covariance matrix in the tactile sensors' pressure data and computing the eigenbasis from its principal axes. A global search is carried out to find a pose in which the object's local geometry in the vicinity of the contact is coherent with that basis,i.e. is aligned with the principal axes and has similar variances. This approach, which can be used as a measurement model for tactile sensors, is compared and outperforms methods using the distance between the tactile sensor elements and object surface.
This paper proposes a novel approach to localize the contact location of a tactile sensor on an o... more This paper proposes a novel approach to localize the contact location of a tactile sensor on an object. By taking advantage of some correspondences between visual and tactile sensing, it is proposed to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. Thus the recursive Bayesian filtering is employed and feature-based measurement model and Gaussian based motion model are built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and test results have proved its feasibility.
This paper presents a novel framework for integration of vision and tactile sensing by localizing... more This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondences, e.g., prominent features, between visual and tactile object identification. To apply it in robotics, we propose to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. It is then treated as a probabilistic estimation problem solved in a framework of recursive Bayesian filtering. Feature-based measurement model and Gaussian based motion model are thus built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and the results have proven the feasibility of our proposed framework.
This paper proposes a pseudo-haptic feedback method conveying simulated soft surface stiffness in... more This paper proposes a pseudo-haptic feedback method conveying simulated soft surface stiffness information through a visual interface. The method exploits a combination of two feedback techniques, namely visual feedback of soft surface deformation and control of the indenter avatar speed, to convey stiffness information of a simulated surface of a soft object in virtual environments. The proposed method was effective in distinguishing different sizes of virtual hard nodules integrated into the simulated soft bodies. To further improve the interactive experience, the approach was extended creating a multi-point pseudo-haptic feedback system. A comparison with regards to (a) nodule detection sensitivity and (b) elapsed time as performance indicators in hard nodule detection experiments to a tablet computer incorporating vibration feedback was conducted. The multi-point pseudo-haptic interaction is shown to be more time-efficient than the single-point pseudo-haptic interaction. It is noted that multi-point pseudo-haptic feedback performs similarly well when compared to a vibration based feedback method based on both performance measures elapsed time and nod-ule detection sensitivity. This proves that the proposed method can be used to convey detailed haptic information for virtual environmental tasks, even subtle ones, using either a computer mouse or a pressure sensitive device as an input device. This pseudo-haptic feedback method provides an opportunity for low-cost simulation of objects with soft surfaces and hard inclusions, as, for example, occurring in ever more realistic video games with increasing emphasis on interaction with the physical environment and minimally inva-sive surgery in the form of soft tissue organs with embedded cancer nodules. Hence, the method can be used in many low-budget applications where haptic sensation is required, such as surgeon training or video games, either using desktop computers or portable devices, showing reasonably high fidelity in conveying stiffness perception to the user.
This paper proposes a multi-fingered palpation method which employs pneumatic haptic feedback act... more This paper proposes a multi-fingered palpation method which employs pneumatic haptic feedback actuators allowing users to experience haptic sensations at multiple fingers while carrying out remote soft tissue palpation. Pneumatic actuators are used to vary the stress on the user’s fingertips in accordance with the tissue stiffness, experienced during manual palpation. The proposed method reduces actuator elements compared to tactile actuators and provides more information than single-point force feedback. The results of our finite element analysis have proven that our pneumatic haptic feedback device can recreate the contact stress between fingertip and soft tissue during palpation. The accuracy (96.8% vs. 93.3%) and time-efficiency (4.6 s vs. 8.3 s) advantages of using three-fingered over single-fingered palpation have been confirmed in our user study results of stiffness levels discrimination. Relatively good tumor detection sensitivities have been demonstrated by the palpation user study which has showed a direct correlation between tumor size and detection sensitivity and has further proven the efficiency of the proposed actuator and multi-fingered palpation method for tumor detection in palpation simulation.
This paper presents a strategy to represent data from a tactile array sensor and match it to an o... more This paper presents a strategy to represent data from a tactile array sensor and match it to an object's geometric features. Using that representation, a method is presented to localise a grasped object within a robot hand. The method consists of computing the covariance matrix in the tactile sensors' pressure data and computing the eigenbasis from its principal axes. A global search is carried out to find a pose in which the object's local geometry in the vicinity of the contact is coherent with that basis,i.e. is aligned with the principal axes and has similar variances. This approach, which can be used as a measurement model for tactile sensors, is compared and outperforms methods using the distance between the tactile sensor elements and object surface.
This paper proposes a novel approach to localize the contact location of a tactile sensor on an o... more This paper proposes a novel approach to localize the contact location of a tactile sensor on an object. By taking advantage of some correspondences between visual and tactile sensing, it is proposed to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. Thus the recursive Bayesian filtering is employed and feature-based measurement model and Gaussian based motion model are built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and test results have proved its feasibility.
This paper presents a novel framework for integration of vision and tactile sensing by localizing... more This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondences, e.g., prominent features, between visual and tactile object identification. To apply it in robotics, we propose to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. It is then treated as a probabilistic estimation problem solved in a framework of recursive Bayesian filtering. Feature-based measurement model and Gaussian based motion model are thus built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and the results have proven the feasibility of our proposed framework.
This paper proposes a pseudo-haptic feedback method conveying simulated soft surface stiffness in... more This paper proposes a pseudo-haptic feedback method conveying simulated soft surface stiffness information through a visual interface. The method exploits a combination of two feedback techniques, namely visual feedback of soft surface deformation and control of the indenter avatar speed, to convey stiffness information of a simulated surface of a soft object in virtual environments. The proposed method was effective in distinguishing different sizes of virtual hard nodules integrated into the simulated soft bodies. To further improve the interactive experience, the approach was extended creating a multi-point pseudo-haptic feedback system. A comparison with regards to (a) nodule detection sensitivity and (b) elapsed time as performance indicators in hard nodule detection experiments to a tablet computer incorporating vibration feedback was conducted. The multi-point pseudo-haptic interaction is shown to be more time-efficient than the single-point pseudo-haptic interaction. It is noted that multi-point pseudo-haptic feedback performs similarly well when compared to a vibration based feedback method based on both performance measures elapsed time and nod-ule detection sensitivity. This proves that the proposed method can be used to convey detailed haptic information for virtual environmental tasks, even subtle ones, using either a computer mouse or a pressure sensitive device as an input device. This pseudo-haptic feedback method provides an opportunity for low-cost simulation of objects with soft surfaces and hard inclusions, as, for example, occurring in ever more realistic video games with increasing emphasis on interaction with the physical environment and minimally inva-sive surgery in the form of soft tissue organs with embedded cancer nodules. Hence, the method can be used in many low-budget applications where haptic sensation is required, such as surgeon training or video games, either using desktop computers or portable devices, showing reasonably high fidelity in conveying stiffness perception to the user.
This paper proposes a multi-fingered palpation method which employs pneumatic haptic feedback act... more This paper proposes a multi-fingered palpation method which employs pneumatic haptic feedback actuators allowing users to experience haptic sensations at multiple fingers while carrying out remote soft tissue palpation. Pneumatic actuators are used to vary the stress on the user’s fingertips in accordance with the tissue stiffness, experienced during manual palpation. The proposed method reduces actuator elements compared to tactile actuators and provides more information than single-point force feedback. The results of our finite element analysis have proven that our pneumatic haptic feedback device can recreate the contact stress between fingertip and soft tissue during palpation. The accuracy (96.8% vs. 93.3%) and time-efficiency (4.6 s vs. 8.3 s) advantages of using three-fingered over single-fingered palpation have been confirmed in our user study results of stiffness levels discrimination. Relatively good tumor detection sensitivities have been demonstrated by the palpation user study which has showed a direct correlation between tumor size and detection sensitivity and has further proven the efficiency of the proposed actuator and multi-fingered palpation method for tumor detection in palpation simulation.
Uploads
Papers by SHAN LUO