Background MTP support was recently merged into llama.cpp through the following pull request: MTP support merged into llama.cpp After the merge, I wanted to test MTP models on my mini-PC powered by the AMD AI MAX 395. I tried several approaches, including manually building llama.cpp and using Unsloth GGUF models directly. However, despite multiple attempts, I could not get a stable working setup. I also searched through GitHub issues and asked several AI assistants, including ChatGPT, Gemini, and DeepSeek.

Overview Running large language models locally is becoming increasingly practical, even for developers without access to massive GPU clusters. In this post, I walk through how to: Run Qwen3.6 35B (A3B, Q4_K_M quantized) locally using Ollama Integrate the model into VS Code Use it as a local coding assistant This setup is especially useful for: Air-gapped environments

Introduction Several days ago, I setup ollama on my MacBook M4, and it works pretty well. At that time, I tried to use it with copilt with local models codegemma:7b and qwen3:8b. My expectation was not so high as the hardware configuration of my macbook pro m4 is just a entry level, just want to see how it works. I also learned there are other options such as vllm. After comparing the two, I found vllm is more flexible, powerful, product-ready, used widely in enterprises.