This paper argues that Large Language Models are not universal problem solvers through prompting alone, due to fundamental constraints in language as a communication interface and alignment requirements. The authors analyze user-system interaction as a cheap-talk game to derive PAC-Bayes bounds distinguishing estimation error from structural limitations.
- The study introduces an 'expressivity floor' where language acts as a capacity-limited channel, making distinct tasks indistinguishable when informational complexity exceeds channel capacity.
- An 'objective-misalignment floor' is established, showing that alignment constraints can restrict the admissible output set, causing irreducible distortion of the user-ideal distribution.
- The analysis proves that for certain task families, correct behavior is provably unattainable even in the infinite-data regime.
The findings suggest that interfaces beyond natural language, such as multimodal observations or external memory, may reduce inherent LLM limitations by increasing the task-relevant information available to the system.