Transformation Behavior of Images in Latent Space

This paper investigates how classical image transformations affect embeddings in latent space using encoder networks from Lunit Inc., Bioptimus, and Meta Research Team.

• The study assessed embedding variance by comparing original and transformed image tiles from colorectal tissue and TCGA datasets against random embeddings.
• Findings indicate that embeddings of original and transformed images are closer to each other than to random ones, showing robustness but not full invariance.
• Significant differences were observed between general-purpose encoder networks and those specifically trained on histopathology images.

The results explain why transformation-mediated data augmentation improves performance by revealing that current encoders do not completely neutralize transformation effects.