An independent researcher has introduced TreeNets, a method that leverages decision trees to drastically reduce the size of large language models. The work aims to clarify the fundamental connection between neural networks and decision trees.
The author has released the first draft of the research paper on Codeberg alongside the source code repository. Additionally, a model combining TreeNets with Bart (tree_bart_xsum_6_6) has been uploaded to Hugging Face for public access.