PDF Paper Youtube Video

SpikeAtacSpikeATac

A Multimodal Tactile Finger with Taxelized Dynamic Sensing for Dexterous Manipulation

Eric T. Chang*, Peter Ballentine*, Zhanpeng He*, Do-Gon Kim, Kai Jiang, Hua-Hsuan Liang,
Joaquin Palacios, William Wang, Pedro Piacenza, Ioannis Kymissis, Matei Ciocarlie

A collaboration between Columbia Engineering's ROAM and CLUE labs.
Columbia Logo Columbia Logo
PDF Paper Youtube Video

SpikeATac SpikeAtac is a a multimodal tactile finger combining a taxelized and highly sensitive dynamic response (PVDF) with a static transduction method (capacitive) for multimodal touch sensing. Named for its `spiky' response, SpikeATac's multitaxel PVDF film provides fast, sensitive dynamic signals to the very onset and breaking of contact, providing the ability to stop quickly and delicately when grasping fragile, deformable objects. SpikeATac can also be used in a learning-based framework to achieve new capabilities on a dexterous multifingered robot hand: we use a learning recipe that combines reinforcement learning from human feedback with tactile-based rewards to fine-tune the behavior of a policy to modulate force, and enable a difficult in-hand manipulation of fragile objects.

Technical Summary Video

Hardware DesignSpikeAtac

SpikeATac is a multimodal tactile fingertip that combines dynamic and static sensing capabilities in a compact form factor. Its core feature is a 16-taxel PVDF array which provides very high sensitivity to high frequency events. This dynamic modality is complemented by 7 commercial off-the-shelf (COTS) capacitive taxels deeper under the surface in order to provide a static pressure response. An optional accelerometer inside the finger provides additional high frequency feedback and is connected to a “nail” feature for surface exploration; however, we do not make use of this feature in this study. At 45 × 32 × 25 mm (length × width × thickness), SpikeATac is just larger than a human thumb, has a finger-shaped form factor conducive to stable contact states, and has 180° sensing coverage.

PVDF produces charge proportional to applied strain. We use charge amplifiers to measure the charge generated at each electrode, therefore measuring applied strain. However, the feedback resistor of the charge amplifier provides a discharge path, creating high-pass filter behavior. A critical part of our work is that this configuration allows the PVDF to be very sensitive to transients without permanently saturating.

Fast and Delicate Grasping

SpikeATac can be used to perform fast grasping while remaining delicate enough to handle highly fragile objects without damage. By leveraging PVDF's responsiveness to contact onset, the system can detect the precise moment of initial contact and respond very fast, even when the gripper is moving at high velocities. This ability is particularly relevant when handling delicate objects like sheets of seaweed (nori). In controlled experiments across slow, medium, and fast gripper velocities, PVDF-based grasping achieved zero crushed objects across 30 trials at each speed, even at the highest velocities tested. In contrast, capacitive-only sensing succeeded at slow speeds but failed frequently at medium and fast speeds, crushing 20 out of 30 and 23 out of 30 objects respectively. This result arises from PVDF's ability to detect sharp contact edges and transient events that static sensors miss, allowing the gripper to stop within just 1.9-2.4mm of initial contact even at the fastest speeds tested.

Efficient On-Robot RL from Human Feedback

On-Robot RL Pipeline

Our approach starts with a base policy trained via behavioral cloning on demonstrations collected from the real robot. This base policy operates on raw, unprocessed sensor signals from SpikeATac, incorporating 16 PVDF signals and 7 capacitive signals per finger. While this base policy works adequately on rigid objects, it immediately fails on fragile ones, lacking the ability to produce appropriately delicate touch. The solution is on-robot RL fine-tuning using Soft Actor-Critic (SAC) that operates directly on the real robot with real sensor data.

Reward Labeling

We use a hybrid reward formulation, illustrated above. We combine two complementary signals: (1) semi-sparse task rewards provided by human annotators who label trajectory segments as "good" (object rotating) or "bad" (not rotating), providing richer feedback than purely sparse rewards without requiring complex state estimation, and (2) dense tactile-based rewards derived directly from sensor observations, encouraging the agent to reduce excessive contact forces (captured by capacitive readings) while increasing exploratory contacts (reflected in PVDF spike counts from making and breaking contact).

Through iterative data collection and policy updates following the pipeline, the system learns to make visibly softer contacts over successive learning iterations. This demonstrates that SpikeATac's complex, high-dimensional signals—which are extremely difficult to simulate accurately—can be effectively leveraged for real-world data-driven manipulation of fragile objects.

Fragile In-hand Manipulation

The keystone demonstration of SpikeATac's capabilities is in-hand rotation of fragile objects using a four-finger robot hand. Using only tactile and proprioceptive information (no vision), the system achieves finger-gaiting manipulation that continuously rotates a range of fragile objects, including paper-made objects, without crushing them.

BibTeX

TBD

Acknowledgements

This work was supported by a NASA Space Technology Graduate Research Opportunity and by a National Science Foundation Graduate Research Fellowship under Grant No. DGE-2036197. We thank Trey Smith, Brian Coltin, Amr El-Azizi, and Rajinder Singh Deol for insightful discussion.