muscriptor
MuScriptor is a multi-instrument music transcription model developed by Kyutai and Mirelo.
What is it?
What it is
MuScriptor is a multi-instrument music transcription model that converts audio recordings into symbolic note events (onset/offset, pitch, instrument) and can output MIDI or event streams.
Why it exists
It exists to provide an open, large-scale trained model for accurate music transcription across genres, enabling researchers and developers to transcribe music without relying on proprietary solutions.
Who should use it
Music researchersAudio engineersMusic producersMachine learning engineersDevelopers building music transcription tools
Who should avoid it
Commercial entities (due to CC BY-NC 4.0 model license)Users without a HuggingFace account (required for weight access)Users with extremely limited hardware (large model requires GPU)
How it works
A quick walkthrough in plain English
How muscriptor works
Step 1 of 3
You interact with it
Open muscriptor, send a request, or connect it to your stack.
Features
Advantages
- Large-scale training dataset (170k songs from classical to heavy metal)
- Scalable model sizes to match hardware capabilities
- Automatic model weight management via HuggingFace
- High performance on Apple Silicon (Metal/MPS support)
- Open-source MIT license for code
- Includes a built-in web server and browser-based synthesizer
Disadvantages
- Model weights are gated under CC BY-NC 4.0 (non-commercial use only)
- Requires HuggingFace authentication for weight downloads
- Does not preserve note velocity (loudness)
- Large model variants require significant GPU resources
Installation
FAQ
How do I access the model weights?
The model weights are hosted on HuggingFace and gated behind a CC BY-NC 4.0 license. You must accept the license on the HuggingFace model page and authenticate your machine using `uvx hf auth login` or by setting an `HF_TOKEN` environment variable.
What are the different model variants available?
MuScriptor offers three sizes: `small` (103M parameters, ideal for CPU), `medium` (307M parameters, default balance of speed/accuracy), and `large` (1.4B parameters, highest accuracy but requires a GPU).
How can I restrict transcription to specific instruments?
You can use the `--instruments` flag in the CLI or the `instruments` argument in the Python API to provide a list of instrument names (e.g., `['acoustic_piano', 'drums']`). This acts as a hard constraint during decoding.
How do I run the Web UI locally?
First, perform a one-time setup by running `uv sync`, then `cd web && pnpm install && pnpm run build && cd..`. Finally, start the server using `uv run muscriptor serve`.
Does the model preserve musical velocity?
No, the current tokenizer recovers onset/offset timing, pitch, and instrument, but it does not preserve velocity (loudness).
Featured in Videos
YouTube tutorials and walkthroughs for muscriptor
Alternatives
Similar projects ranked by category, topics, and text overlap.
