Apple's Core AI lets you drop a custom Metal kernel right into the .aimodel graph.
I used that to speed up sparse-MoE decode: 2.1–3.6× faster on LFM2.5-8B-A1B, Qwen3.6-35B-A3B, GLM-4.7-Flash — same int8 quality. 👇
github.com/john-rocky/coreai…
Apple Core AI supports embedding a custom Metal kernel directly into the .aimodel…
Brief
Apple's Core AI now accepts custom Metal kernels inside .aimodel graphs; on 2026-06-13 @JackdeS11 used this to implement a specialized kernel for sparse-MoE decoding that yields 2.1–3.6× speedups on LFM2.5-8B-A1B, Qwen3.6-35B-A3B, and GLM-4.7-Flash without degrading int8 quality, with code available at the linked GitHub repository.
Why it matters
Apple Core AI supports embedding a custom Metal kernel directly into the .aimodel computation graph (announced/used by @JackdeS11 on 2026-06-13).
Key details
- Using that capability, the author accelerated sparse-MoE decoding by 2.1–3.6× on three models: LFM2.5-8B-A1B, Qwen3.6-35B-A3B, and GLM-4.7-Flash while preserving the same int8 output quality.
- The implementation and details are published in a GitHub repo: github.com/john-rocky/coreai (linked in the post).