On the Smallness of the Large Language Models Scaling Exponents

The article discusses reasons why the scaling exponents of current Large Language Model applications indicate an unsustainable regime regarding energy resources.

• Attributing small scaling exponents to a numerical bias from neglecting a non-zero loss value limit (pedestal effect) does not resolve the unsustainability issue.
• The effects of data smoothness or roughness on scaling exponents are analyzed using an analogy with phenomenological models of fluid turbulence.