Nous Research has introduced the Psyche Network Architecture, a method designed to democratize AI by making distributed training feasible without incurring unfeasibly narrow data transfer bottlenecks. The approach allows multiple GPUs to join a network for training purposes.
The article notes that several models have already been trained using this method, with performance depending on the number of participating GPUs.
Nous Research provides access to runs and details via its website.