Reddit discussion on hosting random GGUF and open models via API

A Reddit user highlights the difficulty of accessing long-tail community fine-tunes, roleplay models, and niche coder models through hosted APIs, noting that while popular models are well-supported, obscure ones require self-hosting or deployment on platforms like Modal, RunPod, or Baseten. The post asks how developers currently handle this gap and what factors matter most when choosing a service, such as price, exact model availability, OpenAI-compatible behavior, uptime, privacy, or hardware transparency.

The discussion centers on the trade-offs between using existing aggregators like OpenRouter, Together, Fireworks, and DeepInfra versus manually deploying vLLM or llama.cpp servers for specific GGUF files.