This study demonstrates that neural networks can reliably learn Conway's Game of Life dynamics using minimal architectures by employing specific inductive biases rather than relying on large-scale search processes. The authors show that network variants with alternative activation functions significantly outperform standard Rectified Linear Units, particularly through the use of second-degree polynomial activations.
- Reorienting the task from a search for 'winning tickets' to a learning problem allows minimal networks to capture cellular automaton rules effectively.
- Network variants using alternative activation functions meaningfully outperform the default choice of Rectified Linear Units (ReLU).
- A 2nd degree polynomial activation function consistently learns Life dynamics, regardless of whether neural weights are learned or hard-coded.
The results challenge the assumption that scale is the primary factor for solving complex problems and advocate for matching learning inductive biases to the task. This approach offers a strategy beneficial for machine learning in science, physics-based deep learning, and interpretable machine learning.