The authors introduce RCT (Robotic Contact Tactile), a dataset designed to address the challenge of tactile representations generalizing to unseen materials in robotic manipulation. The dataset contains 29,279 tactile frames collected from full robot presses on 122 industrial reference materials across seven categories, recorded using three DIGIT sensors at multiple contact positions.
- RCT preserves each press as a contact sequence, enabling held-out evaluation across materials, categories, sensors, and contact positions to prevent data leakage from near-duplicate observations.
- Removing contact-sequence overlap reduces tactile-to-text Recall@1 by 17.7 percentage points when the encoder is held fixed.
- Performance drops sharply when materials are held out at training time, with held-out-material Recall@1 averaging 25.1 +/- 6.1% over three draws.
- The public TVL/HCT split exhibits similar structure issues, where raw-pixel nearest neighbors recover the correct sequence in 98.3% of cases.
- Uniformly sampling a press improves contrastive training, and RCT-trained embeddings improve category probes on unseen materials.
The dataset is open-sourced to make contact-sequence-aware, held-out-material evaluation reproducible and to expose novel-material generalization as a central challenge for robotic tactile perception.