Running local embeddings and rerankers is more useful than local LLMs for paid API users

A user argues that for those already paying for LLM services like ChatGPT Pro or Codex, running local embedding and reranker models offers greater practical utility than hosting large language models locally.

• The author uses Qwen3 Embedding 4B and Qwen3 Reranker 4B to build a memory system called GBrain via an MCP interface.
• The stack includes llama.cpp, PostgreSQL, pgvector, Ceph for S3 API storage, and GitLab for storing memories as Markdown files.
• GBrain indexes these files, extracts facts using an LLM, and uses embeddings and reranking to retrieve relevant memories during queries.
• This approach allows sharing context between Codex and ChatGPT Web with minimal manual intervention.

The author concludes that local hardware is better utilized for models not conveniently provided by paid APIs, such as embeddings and rerankers, rather than running local LLMs.