ZAYA1
The existence proof that open models don't need NVIDIA
What it is
Zyphra's reasoning MoE — 8.4B total parameters, 760M active — pretrained end-to-end on AMD Instinct MI300X GPUs, the first credible frontier-recipe run on an all-AMD stack. Ships as base, reasoning, and vision-language variants, all Apache-2.0, with a full technical report.
Why it's interesting
Punches far above its 760M active parameters on math and reasoning, and the hardware story matters as much as the scores: every proven alternative to the CUDA monoculture makes open AI harder to choke off.
Use cases
- Efficient reasoning on modest GPUs
- Studying MoE training outside the NVIDIA stack via the tech report
- Fine-tuning a small reasoner you can deploy cheaply
Who it's for
Researchers, efficiency-minded ML engineers, AMD-curious infrastructure teams
Setup
Easy. Inference via transformers or vLLM; the 8.4B model fits comfortably on a 24GB GPU, with the tiny active footprint keeping it fast
Limitations & cautions
The small activation budget shows outside math and reasoning, coverage is English-centric, and the training data itself is not released — this is open weights and recipe, not OLMo-grade full provenance.
Editorial takeaway
The scores are good; the supply chain is the headline. Open AI got measurably harder to strangle the day this training run finished.