A mobile app developer reduced LLM API costs from four times the initial estimate to roughly a quarter of that amount by addressing two primary inefficiencies: naive full-history resending and redundant RAG context injection.
- Naive conversation history resend caused O(n²) token growth because every turn sent the full prior message list, which is exacerbated by mobile apps reconstructing state from local caches.
- RAG context was re-fetched and injected on every turn regardless of topic stability, leading to repeated payment for identical knowledge-base chunks.
- Prompt caching structured the system prompt and RAG context as a stable prefix, achieving a 90% reduction in input pricing for cached portions.
- Conversation history beyond a set window is summarized by a smaller model to prevent quadratic cost growth while maintaining thread continuity.
- A lightweight classifier detects topic shifts to gate RAG retrieval, preventing redundant chunk-fetching during continuous conversations.
- Model routing directs simple turns to cheaper models while reserving larger models for complex reasoning tasks.
These adaptations address the specific constraints of mobile app lifecycles, such as backgrounding and state reconstruction, which standard server-side optimization guides often overlook.