A researcher conducted causal experiments on nanoGPT, ResNet-18, and a diffusion U-Net to investigate the mechanics of training, learning, and inference. The results indicate that these processes form a continuous cycle where updates cause nonlinear responses and persistent functional reorganization, which then influence subsequent states.

  • Applying the same update to different receiving states produces different effects.
  • Learning leaves persistent changes in distributed functional support.
  • Frozen inference recruits this support, and feedback alters the state for future learning.
  • This structural relationship holds across different architectures, tasks, modalities, and optimizers.

The author proposes viewing neural networks as classical stateful dynamical systems rather than simple parameterized input-output functions, supported by a linked paper and GitHub repository.