A developer deploying Llama 3.1:8b-instruct-q4_K_M via Ollama with FastAPI and Qdrant for Spanish-language voice and WhatsApp agents identified three production issues related to how the model handles injected context in multi-turn conversations.

  • Inserting a second system message mid-conversation caused the model to respond to the injected instruction rather than absorbing it silently, leading to loss of conversation history; moving all dynamic context into the initial system message resolved this.
  • The model sometimes refused to answer when retrieved RAG chunks were long or unstructured paragraphs, treating the reference material as the user's question instead of the actual query.
  • Capping conversation history to the last six messages caused amnesia in longer calls; raising num_ctx to 4096 and widening the window to 16 turns fixed this issue.

The author seeks confirmation on whether these behaviors are known limitations of Llama 3.1's instruct tuning or Ollama's chat template, particularly regarding the handling of multiple system roles and long context blocks.