In March, I attempted to run DevStack on my newly bought mini PC, but something was wrong and I couldn’t get it started. Since then, I poured all my attention into AI‑related studies. During lunch today, OpenStack came up in conversation, and the idea popped into my head: “Why not use a coding agent to make it work, since I’ve already used one for several issues in my WSL environment?” This post is the result.

Over the past few days, I’ve been using reranker models through cloud APIs as part of my RAG experiments. After successfully running several LLMs locally with llama.cpp, I started wondering: Can I also run a reranker model locally? The answer is yes. It turns out that llama-server can host embedding models, reranker models, and chat models behind the same OpenAI-compatible API. This makes it easy to build a completely local RAG pipeline without relying on external services.

Automatic Speech Recognition (ASR) has become an essential capability for many AI-powered applications. Whether it is transcribing meetings, generating subtitles, or enabling voice-based interactions, ASR is increasingly becoming a core feature of modern AI systems. As part of my ongoing learning journey with AI technologies, I decided to start exploring the ASR capabilities available today. My first stop is Qwen3-ASR, one of the latest speech recognition models from the Qwen family.

Serving LLMs with Dev Containers: A New Rabbit Hole One of the interesting things about working with AI is that you often discover new ideas when you’re exploring something completely different. I never imagined I would write about serving Large Language Models (LLMs) using Dev Containers. Although I’ve been using the vLLM container for quite some time, I always thought of it simply as another containerized application running in Kubernetes or Docker.

Quantization techniques were mentioned in huggingface and unsloth, and I used those quantized models in ollama and llama.cpp. I always wonder how to implement it for vllm. Today I learnt to use llmcompressor to optimized models for vllm. import warnings warnings.filterwarnings("ignore") import os, gc, math, pathlib import torch from transformers import AutoTokenizer, AutoModelForCausalLM import warnings os.environ['TOKENIZERS_PARALLELISM'] = 'false' MODEL_DIR = "Qwen3-0.6B" OUTPUT_DIR = "Qwen3-0.6B-W4A16" print(f"Base model: {MODEL_DIR}") print(f"Quantized model: {OUTPUT_DIR}") from llmcompressor.

Login in as kubeadmin in openshift local, Ecosystem-→ Software Catalog, Search "AI" and install "Red Hat OpenShift AI" # default unit MiB crc config set memory 32000 crc stop crc start OpenShift AI Operator Installation Prerequisite Operators: Navigate to OperatorHub in the OpenShift web console and install required dependencies including Logical Volume Manager Storage (LVMS) and Node Feature Discovery (NFD). Ensure your host machine has at least 4 physical CPU cores, 16 GB to 32 GB of RAM, and roughly 30 GB to 35 GB of free disk space.

OpenShift enhances container security by assigning a random, non-root User ID (UID) to workloads by default. This helps isolate workloads running in different namespaces and prevents containers from running with predictable user IDs. While this security model works well for cloud-native applications, some third-party or legacy container images expect to run with a specific UID. A common example is the nginxinc/nginx-unprivileged image, which expects to run as UID 101.

apt install wireplumber libspa-0.2-bluetooth systemctl --user --now disable pipewire-media-session systemctl --user --now enable wireplumber

Introduction For several years, I adopted Conventional Commits across my software projects. The premise was straightforward: write commit messages in a structured, machine-readable format, then leverage tooling to generate changelogs and release notes automatically. For example: feat: add user login fix: resolve payment retry issue docs: update API usage guide This approach served me well. Tools like Conventional Changelog could parse commit history and produce structured release notes with minimal manual effort.