A Reddit thread has been created to share proven llama-server start configurations specifically designed for users with 24GB GPUs, such as the RTX 3090, 7900XTX, and RTX 4090.
The configurations must utilize VRAM as much as possible and provide at least 200,000 tokens of KV Cache. Participants are asked to include their system RAM amount, OS, and CPU details to help assess caching performance and viability.
This resource aims to help users optimize llama-server settings for maximum efficiency on specific hardware constraints.