hello-rocm

English

简体中文

English

简体中文

Appearance

MiniCPM-o 4.5 Deployment

🎙️ Running a Full-Omni Model on AMD GPU

Audio · Vision · TTS · Full-Duplex · ROCm 7+ · llama.cpp-omni

Home · 中文 · Deploy Overview

Overview ​

  MiniCPM-o 4.5 is an end-side omni-modal model from OpenBMB that combines text, voice input (audio encoder), image understanding (vision encoder), and voice output (TTS) into a single model. Its Omni full-duplex mode enables real-time, phone-call-like conversations — all running on AMD ROCm without an NVIDIA GPU.

  This module is based on OpenBMB's official llama.cpp-omni inference engine and OpenBMB/MiniCPM-o-Demo (Comni branch), verified on AMD Ryzen AI MAX+ 395 (gfx1151 / Strix Halo APU) and applicable to other ROCm-supported AMD GPUs.

01-Deploy/minicpm-o/
├── minicpm-o-model.md               # MiniCPM-o 4.5 model introduction
├── llamacpp-omni-rocm7-deploy.md    # llama.cpp-omni CLI deployment
└── webdemo-rocm7-deploy.md          # MiniCPM-o Web Demo full-duplex deployment

Tutorial List ​

MiniCPM-o 4.5 Model Introduction ​

  Understand the omni-modal architecture of MiniCPM-o 4.5 — audio input, visual encoding, TTS synthesis, and full-duplex design. After reading, you'll know exactly which GGUF sub-model files are required and the VRAM requirements for different GPUs.

  • Audience: Readers who want to understand the model before deploying
  • Difficulty: ⭐
  • Estimated time: 15 minutes

📖 Read MiniCPM-o 4.5 Model Introduction

llama.cpp-omni CLI Deployment ​

  Build and run MiniCPM-o 4.5 on AMD GPU using OpenBMB's official llama.cpp-omni inference engine. This tutorial covers HIP compilation, GGUF model download, audio input testing, and TTS voice output — letting you run full omni-modal inference from the command line.

  • Audience: Developers who want to verify inference capabilities or prepare a backend for the Web Demo
  • Difficulty: ⭐⭐⭐
  • Estimated time: 2 hours (including model download)

📖 Start llama.cpp-omni Deployment Tutorial

MiniCPM-o Web Demo Full-Duplex Deployment ​

  Deploy the complete Web Demo using OpenBMB/MiniCPM-o-Demo (Comni branch) on AMD GPU. Supports Turn-based / Half-Duplex / Omni full-duplex / Audio full-duplex — experience real-time voice and video conversation with the model directly in your browser. Prerequisite: llama.cpp-omni must already be compiled.

  • Audience: Developers who want to build a full interactive demonstration
  • Difficulty: ⭐⭐⭐
  • Estimated time: 1 hour (with inference backend already ready)

📖 Start Web Demo Full-Duplex Deployment

Requirements ​

Hardware ​

ScenarioMinimumRecommended
Text inference (LLM only)8 GB VRAM—
Voice input (audio encoder)12 GB VRAM—
Full omni (LLM + vision + audio + TTS)16 GB VRAM64 GB unified memory (Strix Halo APU)

AMD Ryzen AI MAX+ 395 / 890M with unified memory architecture is ideal for MiniCPM-o — 64 GB unified memory can fully load all GGUF sub-models (~8.3 GB) into the GPU.

Software ​

  • OS: Linux (Ubuntu 22.04 / 24.04)
  • ROCm 7.10.0 or later (system installation)
  • CMake 3.21+, GCC / Clang (for HIP compilation)
  • Python 3.10+ (Web Demo dependency)

Test Environment for This Tutorial ​

Hardware:  AMD Ryzen AI MAX+ PRO 395 / Radeon™ 890M (Strix Halo)
GPU arch:  gfx1151
Memory:    64 GB unified (all usable as VRAM)
ROCm:      7.12.0 (system) + TheRock 7.12.0a alpha (Tensile fix)
OS:        Ubuntu 24.04

FAQ ​

Q: How is MiniCPM-o different from a regular LLM, and why can't I use vLLM / Ollama?

MiniCPM-o 4.5 includes independent sub-modules — an audio encoder, a vision encoder, and a TTS token2wav pipeline — that mainstream inference frameworks don't natively support yet. llama.cpp-omni is a specialized inference engine fork that can load and schedule all these sub-models concurrently.

Q: Does gfx1151 (Strix Halo) require special handling?

Yes. gfx1151 was introduced in late 2025. The system /opt/rocm rocBLAS Tensile library doesn't include complete GEMM kernels for gfx1151, causing a crash at runtime (hipErrorInvalidImage). The fix is to install the TheRock 7.12.0a alpha SDK and point the runtime at its rocBLAS library directory — see the llama.cpp-omni tutorial for details.

Other AMD GPUs (gfx1100 / RX 7900 XTX, gfx1150 / RX 9070 XT, etc.) are not affected.

Q: How do I find my AMD GPU architecture (gfx number)?
bash
rocminfo | grep -i "gfx"
# or
amd-smi

References ​

Contributions welcome! 🎉

Submit Issue | Submit PR