Next-Gen Local AI for Apple Silicon · macOS Native

Break the RAM barrier.
Run massive MoE models on your Mac.

DynaMoE dynamically streams active expert weights directly from high-speed NVMe storage into Unified Memory on demand. Run models far exceeding physical RAM with zero-copy Metal GPU performance.

View Source & Star on GitHub See How It Works

Apache 2.0 Open Source Native SwiftUI + Rust Core Apple Silicon M-Series

LIVE INFERENCE PIPELINE · ZERO-COPY STREAM

Storage Tier

NVMe SSD

Memory-mapped MoE Safetensors (100GB+ Checkpoint)

Async mmap

DynaMoE Core

Dynamic Expert Router

Paging active top-k experts per token layer

Exp 1

Exp 2

Exp 3

Exp 4

Exp 5

Exp 6

Exp 7

Exp 8

Zero-Copy

Execution Tier

Metal GPU Buffer

Apple Silicon Unified Memory cache & token output

The Innovation

Why DynaMoE Changes Local AI

Mixture-of-Experts (MoE) architectures are the state of the art for frontier models. But loading an entire MoE model into RAM has kept them out of reach on consumer Macs—until now.

The MoE Bottleneck vs. The Streaming Breakthrough

In a Mixture-of-Experts model, each token activates only a fraction of total parameters (e.g., 2 out of 8 experts per layer). Standard inference engines waste memory by loading all experts simultaneously.

DynaMoE leverages ultra-fast Apple Silicon NVMe I/O to stream only the active experts directly into GPU-accessible Unified Memory on the fly, unlocking 3x to 5x larger models on the hardware you already own.

Traditional Inference Engines

Requires 64GB–128GB of expensive physical Unified Memory to load the entire weights checkpoint before generating a single token. Fails instantly on 16GB–36GB Macs.

The DynaMoE Streaming Approach

Keeps weights mapped on high-speed NVMe storage (3–7 GB/s) and pages active expert slices into memory on demand. Massive MoE models run smoothly on standard Apple Silicon configurations.

Engineered for macOS

Built for Performance & Simplicity

A modern, modular stack combining high-performance systems programming with Apple's native frameworks.

Dynamic SSD Weight Streaming

Memory-maps massive Safetensors & GGUF checkpoints directly off your NVMe drive. Active expert weights are streamed on-demand per layer without preloading gigabytes into RAM.

Zero-Copy Metal Pipeline

Leverages macOS kernel page cache handoffs straight into Metal GPU Unified Memory buffers. Eliminates redundant memory copies and maximizes throughput.

All-in-One Local Harness

Packaged into a single native macOS bundle containing a clean SwiftUI interface, a local OpenAI-compatible REST API host, and an embedded Model Context Protocol (MCP) server.

Pure Swift & Rust Architecture

High-performance Rust core (dynamoe-core) for async disk I/O, routing, and memory management, seamlessly bridged to a native SwiftUI frontend via UniFFI.

Frontend

SwiftUI & SPM

Native macOS interface with live metrics

Core Engine

Rust + UniFFI

Async I/O with memmap2 & safe bindings

GPU Acceleration

Metal Shading (MSL)

Custom zero-copy compute kernels

Protocol Support

MCP & OpenAI API

Seamless plug-and-play AI tooling

Active Development

DynaMoE is Coming Soon

We're building the future of memory-efficient local AI inference. Follow development on GitHub, track our progress, or contribute to the open-source engine.

Architecture & Apache 2.0 Licensing

Rust Core & UniFFI Bridge

Zero-Copy Metal Buffer Pipeline

SwiftUI Dashboard & MCP Server

Star DynaMoE on GitHub Back to Top

Break the RAM barrier. Run massive MoE models on your Mac.