kimodo.cpp
Animate skeletons with natural language; NVIDIA's Kimodo ported to C++/GGML
What is it?
What it is
A GGML/C++ implementation of NVIDIA's Kimodo text-to-motion model that generates local rotations and root translations from UTF-8 prompts or precomputed LLM2Vec embeddings. It supports multiple motion skeletons (SMPL-X 22 joints, SOMA 30 joints, G1 34 joints) and runs on CPU or Vulkan. The project includes checked GGUF loading, safetensors conversion, DDIM sampling, C/C++ APIs, CPU/Vulkan parity tests, skeleton-only GLB export, and a local text-to-motion demo.
Why it exists
To provide an open-source, high-performance C++ implementation of Kimodo using GGML, enabling text-to-motion generation on CPU or Vulkan with a permissive Apache-2.0 license for the port. It aims to make NVIDIA's Kimodo model accessible for research and integration, while respecting upstream model licenses. The project also provides tools for weight conversion, publishing, and a demo to showcase functionality.
Who should use it
C++ developers, ML engineers, motion synthesis researchers, and game developers interested in text-to-motion generation using GGML and Vulkan acceleration.
Who should avoid it
Novice programmers without C++ or ML background, users expecting a fully featured GUI or ready-to-use animation tools with skinned mesh export, and those seeking commercial use of the SMPL-X RP model (restricted license).
How it works
A quick walkthrough in plain English
How kimodo.cpp works
Step 1 of 3
You interact with it
Open kimodo.cpp, send a request, or connect it to your stack.
Features
Advantages
- Open-source Apache-2.0 license for the C++ port and tooling
- Cross-platform support (CPU or Vulkan)
- Efficient GGML implementation with VRAM tuning options
- Supports multiple skeleton types (22, 30, 34 joints) and model variants
- Includes conversion, testing, and demo tools
- Pre-built GGUF weights available for SOMA and G1 models under open licenses
- C API for easy integration into other projects
- Reproducible builds via Nix
Disadvantages
- SMPL-X model not redistributable (requires local conversion under gated terms)
- Missing features: general constraint input, 77-joint SOMA expansion, skinned-mesh GLB export, quantised models
- Weights must be downloaded separately (test suite does not auto-download)
- Requires C++23, CMake 3.25+, Ninja, Python 3, Hugging Face CLI, and Vulkan loader/headers
- Text bundle includes Meta Llama 3 material with separate terms
- Demo requires Go runtime
- Leak detection disabled in sanitizer builds due to Vulkan loader allocations
Installation
native
Install a C++23 compiler, CMake 3.25+, Ninja, Python 3 with the Hugging Face CLI (`pip install huggingface_hub`), and the Vulkan loader/headers for Vulkan support. Then run: ```sh git submodule update --init --recursive scripts/download_gguf_weights.sh --output "$PWD" --model soma-rp-v1.1 cmake --preset debug cmake --build --preset debug ctest --preset debug ``` Optional Nix development: ```sh nix develop path:. --command cmake --preset debug nix develop path:. --command cmake --build --preset debug ```
FAQ
How do I build the project on Linux?
Install a C++23 compiler, CMake 3.25+, Ninja, Python 3 with the Hugging Face CLI, and the Vulkan loader/headers. Run `git submodule update --init --recursive`, download weights via `scripts/download_gguf_weights.sh`, then execute `cmake --build --preset debug` and `ctest --preset debug`.
What are the supported motion checkpoints and their joint counts?
The supported checkpoints are SMPL-X RP v1 (22 joints), SOMA RP/SEED v1.1 (30 joints), and G1 RP/SEED v1 (34 joints). Each generates local XYZW rotations and root translations upon inference.
Where can I obtain the pre-trained motion weights?
Weights are published under the Hugging Face "LocalAI-io" organization. Available models include Kimodo-SOMA-RP-v1.1-GGML, Kimomo-SOMA-SEED-v1.1-GGML, Kimodo-G1-RP-v1-GGML, and Kimodo-G1-SEED-v1-GGML.
What license applies to the source code and models?
The C++ port and original tooling are licensed under Apache-2.0. The GGML format and model weights retain their respective licenses. Some checkpoints (like SMPL-X RP v1) have additional licensing restrictions requiring internal R&D license compliance.
How do I run the demonstration application?
Build the debug preset with `cmake --build --preset debug`, then run `go run ./demo -addr 0.0.0.0:8094`. The app will start at http://localhost:8094, where you can view generated animations and manage persistent history.
Featured in Videos
YouTube tutorials and walkthroughs for kimodo.cpp
Alternatives
Similar projects ranked by category, topics, and text overlap.