🦾 - IRON: Unlocking the Full Potential of NPUs - 🦾

IRON is an open-source & close-to-metal Python API enabling fast and efficient execution on AMD Ryzen™ AI NPUs. It relies on language bindings around the MLIR-AIE dialect.

Key Features:

• An operator library (GEMM, MHA, RMSNorm, RoPE, activations, ...) over mlir-aie's aie.iron, each operator one declared class
• Graphs: a model's operators recorded once per input shape and compiled into one image, so intermediate values stay on the device
• Autotuning of each operator's tunables into a per-device profile
• End-to-end models on the NPU, from Llama 3.2 1B to a vision-language model (see Models)

Quick start

After installing IRON with its models extra, and given a Hugging Face token for meta-llama's gated repository:

from iron.applications.llama import Llama

model = Llama.load("meta-llama/Llama-3.2-1B")   # downloads, fits, compiles once
print(model.generate("The capital of France is", 8))   # greedy; pass an iron.lm.Sampler to sample

Llama.load also takes a local directory holding config.json, the checkpoint and tokenizer.json.

The IRON Python API for Ryzen™ AI NPUs is described in the following paper:

E. Hunhoff, J. Melber, K. Denolf, A. Bisca, S. Bayliss, S. Neuendorffer, J. Fifield, J. Lo, P. Vasireddy, P. James-Roxby, E. Keller. "Efficiency, Expressivity, and Extensibility in a Close-to-Metal NPU Programming Interface". In 33rd IEEE International Symposium On Field-Programmable Custom Computing Machines, May 2025.

Installation (Linux)

These instructions will guide you through everything required for building and executing a program on the Ryzen™ AI NPU, starting from a fresh bare-bones Ubuntu 24.04 or Ubuntu 24.10 install.

Initial Setup

Important: Ensure your system has the latest BIOS version that enables NPU support. Check your laptop/mini-PC manufacturer's support website for BIOS updates.

If starting from Ubuntu 24.04 you may need to update the Linux kernel to 6.11+ by installing the Hardware Enablement (HWE) stack:

sudo apt update
sudo apt install --install-recommends linux-generic-hwe-24.04
sudo reboot

1. Install XDNA™ Driver and XRT:

   Instructions from mlir-aie repository

2. Give the driver a command timeout IRON's longest commands fit in. The amdxdna driver ends a command after about two periods of its tdr_timeout_ms in which no command on the device starts or finishes (2000 ms by default, so about 4 s), and the run reports ERT_CMD_STATE_TIMEOUT. IRON needs tdr_timeout_ms=150000, so a command may run about 300 s. Set it, then reload the driver or reboot:

   echo "options amdxdna tdr_timeout_ms=150000" | sudo tee /etc/modprobe.d/amdxdna-tdr.conf
   sudo modprobe -r amdxdna && sudo modprobe amdxdna

   On NPU2 a model's prompt chunk is one command, and the deepest chunk, at the model's native context, is the longest. Measured on Strix Halo, one command each:

   Model	Native context	Deepest chunk, s
   Gemma 4 E2B	131072	182
   Llama 3.2 3B	131072	101
   Llama 3.2 1B	131072	26
   EmbeddingGemma 2, each tower and the encoder	its largest input	under 0.7

   On NPU1 a command is one operator, the longest an attention layer of the deepest chunk.

3. Install the packages needed for IRON and MLIR-AIE:

   # IRON needs Python 3.11 or later; our wheels support 3.12 and 3.13
   sudo apt install \
   build-essential clang clang-14 lld lld-14 python3-venv python3-pip

4. Setup a virtual environment and activate it:

   python3 -m venv ironenv
   source ironenv/bin/activate
   python3 -m pip install --upgrade pip

5. Source XRT (installed in step 1):

   source /opt/xilinx/xrt/setup.sh

6. Install required Python packages (from requirements.txt). The mlir-aie it pins is the last release; AGENTS.md says how to run against the newer one this branch needs:

   pip install -r requirements.txt

   Loading a model needs the models extra (huggingface_hub, tokenizers and jinja2); the model tests also read images and audio. From this checkout:

   pip install -e '.[models]'               # to load and run models
   pip install -r requirements_examples.txt # the extra and the model tests' packages

7. To test your installation, you can try to build and run the example below:

   pytest iron/tests/operators/test_catalog.py -k AXPY

The commands that run the operator and model tests are in AGENTS.md.

Models

Each model is a package under iron/applications/ over iron/lm/, the library every model shares (AGENTS.md describes it). Each model's README says which code is specific to it and where its files come from. The tests find a checkpoint at $IRON_EXAMPLE_WEIGHTS_DIR/<checkpoint>, default /srv (iron.lm.testing.weights_dir).

Model	Application	Checkpoint
Llama 3.2 1B, 3B	llama	llama3.2-1b, llama3.2-3b
Gemma 4 E2B text decoder	gemma4	gemma-4-e2b-it
EmbeddingGemma 2	embedding_gemma2	embeddinggemma-2
Gemma 4 E2B in FastFlowLM	gemma4_engine	$FLM_MODEL_PATH (the tests default to fastflowlm)

What the text decoders (Llama and Gemma 4 above) share:

• Setup: requirements_examples.txt, as installing says. XRT maps every device buffer locked, and a model's weights and caches are banks of at most 3.5 GiB (iron.common.image.allocator.BANK), so ulimit -l must cover one bank.

• Run: python -m iron.applications.<application>.model <directory or repository id> [--prompt TEXT], the directory holding config.json, the checkpoint and the tokenizer; -v logs each compile; --help lists the options, and Llama's README explains them.

• Test: pytest tests/applications/<application>/ (or tests/applications/decoders/ -k <application> for the decoders that share one test) loads the model once and checks throughput, accuracy against transformers' logits recorded in golden.npz beside the checkpoint, and determinism. pytest <test> --record-goldens -p no:xdist records missing logits where torch and transformers are installed; they are not IRON's requirements.

• Tune: python -m iron.lm.tune <application> --checkpoint <directory> times each operator at every setting of its tunables and rewrites the application's profiles/<device>.json; review it with git diff and commit it.

Architecture

1. Operators (iron/operators/): each operator is one class, imported in iron/operators/__init__.py; its docstring says what it computes. Its parameters are declared with mlir-aie's param() (a tunable with tunable(), a per-call value with per_call()); its operands are aie.iron's In/Out/InOut, declared by shape. Its build() builds the array with ObjectFIFOs and Workers around a kernel; the library derives the runtime sequence unless the operator writes it. Its reference() and test = Testing(...) are what iron/tests/operators/test_catalog.py checks it by.
2. Kernels (mlir-aie aie_kernels/): C++ compute kernels, vectorized with the AIE API, reached through mlir-aie's kernel factories (aie.iron.kernels), which also carry each kernel's reference and tolerance. IRON hosts no kernels of its own, apart from the FastFlowLM Gemma 4 engine's overrides (iron/operators/flm/gemma4_engine/).
3. Graphs and images (iron/common/): a graph is recorded on handles and lowered onto an image: its buffers placed in one arena, and on NPU2 its operators' designs composed into one full ELF that runs them in turn (on NPU1, an xclbin per step), run through XRT.
4. Models (iron/lm/, iron/applications/): see Models.

Community and Support

• 💬 Discord: Join our Discord server for discussions and support
• 🐛 Issues: Report bugs and request features via GitHub Issues
• 📖 Contributing: See CONTRIBUTING.md for development guidelines
• 📚 Documentation: AGENTS.md for the rules and the map of the code, operator docstrings in iron/operators/, kernel docs in the mlir-aie kernel library

Acknowledgements

The operators in iron/operators/flm/ are ports of the NPU overlays of FastFlowLM's Gemma 4 engine, and gemma4_engine runs them inside that engine. FastFlowLM's runtime is MIT-licensed; its NPU binaries, which the engine example and the Shipped test reference fetch from FastFlowLM, are under its own binary license and are not redistributed here.

License

IRON is licensed under the Apache License 2.0. See LICENSE for details.

---

Copyright© 2025-2026 Advanced Micro Devices, Inc