A novel method is introduced for the partial and full optimization of connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). The training approach utilizes a probability distribution over connections per input pin, selecting the highest merit connection while learning optimal gate types or LUT entries in parallel.
- Connection-optimized LGNs outperform standard fixed-connection variants on Yin-Yang, MNIST, and Fashion-MNIST benchmarks using significantly fewer logic gates.
- The method achieves 98.92% accuracy on MNIST with two layers of 8000 gates, requiring almost 50 times fewer gates than fixed-connection LGNs.
- Training stability for up to ten layers is ensured via high learning rates, straight-through estimators, and trimming constant-output gate types.
- A LUT neuron description enables stable backpropagation training in networks up to six layers deep with four times fewer trainable parameters.
- The connection-training algorithm achieves 98.88% accuracy on LUTNs using two layers of 2000 6-input LUTs.
This approach demonstrates that optimizing connections allows for substantially more efficient models that maintain or improve accuracy compared to traditional fixed-connection architectures.