This paper investigates the learnability of Transformer models, moving beyond previous theoretical works that focused primarily on expressivity and hypothesis class characterization. The authors propose preliminary sample complexity bounds for learning C-RASP constructions with Transformers.
- Past theoretical work characterized which tasks are in the hypothesis class of Transformer models using handcrafted weights or computational complexity arguments.
- Little prior work investigated the learnability of such solutions.
- The study proposes preliminary sample complexity bounds for learning C-RASP constructions.