Over the past few days, I’ve been using reranker models through cloud APIs as part of my RAG experiments. After successfully running several LLMs locally with llama.cpp, I started wondering: Can I also run a reranker model locally? The answer is yes. It turns out that llama-server can host embedding models, reranker models, and chat models behind the same OpenAI-compatible API. This makes it easy to build a completely local RAG pipeline without relying on external services.

Automatic Speech Recognition (ASR) has become an essential capability for many AI-powered applications. Whether it is transcribing meetings, generating subtitles, or enabling voice-based interactions, ASR is increasingly becoming a core feature of modern AI systems. As part of my ongoing learning journey with AI technologies, I decided to start exploring the ASR capabilities available today. My first stop is Qwen3-ASR, one of the latest speech recognition models from the Qwen family.

Serving LLMs with Dev Containers: A New Rabbit Hole One of the interesting things about working with AI is that you often discover new ideas when you’re exploring something completely different. I never imagined I would write about serving Large Language Models (LLMs) using Dev Containers. Although I’ve been using the vLLM container for quite some time, I always thought of it simply as another containerized application running in Kubernetes or Docker.

Quantization techniques were mentioned in huggingface and unsloth, and I used those quantized models in ollama and llama.cpp. I always wonder how to implement it for vllm. Today I learnt to use llmcompressor to optimized models for vllm. import warnings warnings.filterwarnings("ignore") import os, gc, math, pathlib import torch from transformers import AutoTokenizer, AutoModelForCausalLM import warnings os.environ['TOKENIZERS_PARALLELISM'] = 'false' MODEL_DIR = "Qwen3-0.6B" OUTPUT_DIR = "Qwen3-0.6B-W4A16" print(f"Base model: {MODEL_DIR}") print(f"Quantized model: {OUTPUT_DIR}") from llmcompressor.

Login in as kubeadmin in openshift local, Ecosystem-→ Software Catalog, Search "AI" and install "Red Hat OpenShift AI" # default unit MiB crc config set memory 32000 crc stop crc start OpenShift AI Operator Installation Prerequisite Operators: Navigate to OperatorHub in the OpenShift web console and install required dependencies including Logical Volume Manager Storage (LVMS) and Node Feature Discovery (NFD). Ensure your host machine has at least 4 physical CPU cores, 16 GB to 32 GB of RAM, and roughly 30 GB to 35 GB of free disk space.

OpenShift enhances container security by assigning a random, non-root User ID (UID) to workloads by default. This helps isolate workloads running in different namespaces and prevents containers from running with predictable user IDs. While this security model works well for cloud-native applications, some third-party or legacy container images expect to run with a specific UID. A common example is the nginxinc/nginx-unprivileged image, which expects to run as UID 101.

apt install wireplumber libspa-0.2-bluetooth systemctl --user --now disable pipewire-media-session systemctl --user --now enable wireplumber

Introduction For several years, I adopted Conventional Commits across my software projects. The premise was straightforward: write commit messages in a structured, machine-readable format, then leverage tooling to generate changelogs and release notes automatically. For example: feat: add user login fix: resolve payment retry issue docs: update API usage guide This approach served me well. Tools like Conventional Changelog could parse commit history and produce structured release notes with minimal manual effort.

Sometimes people look at my profile and wonder: “How can you keep learning so many things? What is your secret?” The truth is, there is no magic. My approach is simple: I write things down, practise important skills repeatedly, and choose technologies carefully after real hands-on exploration. 1. I Write Down What I Learn Whenever I learn something useful, I try to capture it. Sometimes I write it as a blog post.

What is GEPA? GEPA stands for Graph-based Evolutionary Program Adaptation — a DSPy optimizer that automatically improves the prompts/instructions of a multi-module LLM program through evolutionary search. It iteratively mutates module instructions, evaluates the changes, and keeps the best-performing candidates on a Pareto front. What’s Happening in This Run This file captures a GEPA optimization run on a financial news extraction system that classifies M&A (merger/acquisition) articles and extracts structured data from them.