Researchers probe the residual streams of pre-trained code models to understand how they internally encode type information, using a parallel dataset of Java and Python code examples.
- Cross-lingual type representations emerge even from untyped code.
- Hidden states linearly encode the result type implied by typed function application, allowing probes trained on one language to infer argument and result types in another.
- This structural encoding is partly robust to lexical perturbations and cross-language syntactic variations.
The work addresses a gap in interpretability research by directly targeting formal type semantics and cross-lingual type representations, with code and datasets released.