https://jackliusr.github.io/tags/qwen3/ Recent content in qwen3 on My learning and diary Hugo -- gohugo.io en-us Thu, 30 Jul 2026 06:00:00 +0800 https://jackliusr.github.io/posts/2026/07/running-a-local-reranker-with-llama.cpp/ Thu, 30 Jul 2026 06:00:00 +0800 https://jackliusr.github.io/posts/2026/07/running-a-local-reranker-with-llama.cpp/ Over the past few days, I’ve been using reranker models through cloud APIs as part of my RAG experiments. After successfully running several LLMs locally with llama.cpp, I started wondering: Can I also run a reranker model locally? The answer is yes. It turns out that llama-server can host embedding models, reranker models, and chat models behind the same OpenAI-compatible API. This makes it easy to build a completely local RAG pipeline without relying on external services.