Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks

Researchers introduce a method for partial and full optimization of connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). The approach uses a probability distribution to select the highest merit connection per input pin while learning optimal gate types or LUT entries in parallel.

• Connection-optimized LGNs outperform standard fixed-connection LGNs on Yin-Yang, MNIST, and Fashion-MNIST benchmarks using far fewer logic gates.
• The method achieves 98.92% accuracy on MNIST with two layers of 8000 gates, requiring almost 50 times fewer gates than fixed-connection LGNs.
• Training stability up to ten layers is ensured via high learning rates, straight-through estimators, and trimming constant-output gate types.
• A LUT neuron description enables stable backpropagation training for networks up to six layers deep with four times fewer trainable parameters.
• The algorithm achieves 98.88% accuracy on LUTNs using two layers of 2000 6-input LUTs.

The connection-training algorithm allows for significantly more efficient models that maintain or exceed the accuracy of traditional fixed-connection approaches.