Serving LLMs with Dev Containers: A New Rabbit Hole One of the interesting things about working with AI is that you often discover new ideas when you’re exploring something completely different. I never imagined I would write about serving Large Language Models (LLMs) using Dev Containers. Although I’ve been using the vLLM container for quite some time, I always thought of it simply as another containerized application running in Kubernetes or Docker.

Login in as kubeadmin in openshift local, Ecosystem-→ Software Catalog, Search "AI" and install "Red Hat OpenShift AI" # default unit MiB crc config set memory 32000 crc stop crc start OpenShift AI Operator Installation Prerequisite Operators: Navigate to OperatorHub in the OpenShift web console and install required dependencies including Logical Volume Manager Storage (LVMS) and Node Feature Discovery (NFD). Ensure your host machine has at least 4 physical CPU cores, 16 GB to 32 GB of RAM, and roughly 30 GB to 35 GB of free disk space.