Over the past few days, I’ve been using reranker models through cloud APIs as part of my RAG experiments. After successfully running several LLMs locally with llama.cpp, I started wondering: Can I also run a reranker model locally? The answer is yes. It turns out that llama-server can host embedding models, reranker models, and chat models behind the same OpenAI-compatible API. This makes it easy to build a completely local RAG pipeline without relying on external services.