Mixture-of-experts models only run a fraction of their parameters per token, which is why a 295B Tencent Hy3 or a 671B DeepSeek V3 now fits inside consumer RAM budgets that used to top out around 70B dense. Tencent released Hy3 under Apache 2.0 on July 6, 2026, with 21B active parameters and a 256K context window. DeepSeek keeps shipping V3 refreshes, Mixtral 8x22B remains competitive on cost, and Qwen3 MoE has variants from 30B to 235B. The bottleneck moved from GPU VRAM to whether your local runner actually understands sparse expert routing, and not every runner does.
We tested 8 of the best apps for running open MoE LLMs on desktop across Windows, macOS, and Linux. Each was judged on GGUF and safetensors compatibility, expert routing controls, CPU offload behavior for cold experts, and how quickly the project ships support for new architectures like hy_v3, qwen3moe, and deepseek2. Some are single-file executables. Others assume you know Docker. All of them are free for personal use, and most are free for commercial use too.
What to look for in a MoE-capable local LLM runner
A handful of criteria separate the apps that handle a new MoE architecture on day one from the ones that quietly fail to load the weights:
- Native support for the current MoE architectures: Mixtral, DeepSeekMoE, Qwen3MoE, Hunyuan-A13B, and hy_v3. Runners typically lag the model release calendar by 2 to 4 weeks, so anything older than that is a warning sign.
- GGUF and safetensors coverage. GGUF is what the Hugging Face community actually publishes for local use. Runners without it are locked out of most quants.
- Expert routing controls. Every MoE has a top-k router. Being able to adjust the number of active experts trades throughput against quality, and it is the difference between a MoE that feels fast and one that feels dense.
- Selective CPU offload. A well-tuned runner keeps the router and shared attention weights on the GPU and pushes cold experts to system RAM. That is how a 295B model fits in 64 GB of DDR5 plus a mid-range card.
- OpenAI-compatible API server. Almost every editor extension, agent framework, and memory tool speaks the OpenAI Chat Completions dialect. If the runner does not expose one, integration is a homework project.
- Quantization depth. Q4_K_M is the current workhorse for MoE. Newer methods like APEX and IQ4_XS shave more VRAM without measurable quality loss on the routed experts.
Quick comparison
| App | Best for | Platforms | License | Price | MoE architectures |
|---|---|---|---|---|---|
| LM Studio | First-run MoE with a GUI | Windows, macOS, Linux | Proprietary | Free | Mixtral, DeepSeek, Qwen3MoE, Hunyuan-A13B |
| Ollama | Terminal-first MoE runs | Windows, macOS, Linux | MIT | Free | Mixtral, DeepSeek V3, Qwen3MoE |
| llama.cpp | The upstream every wrapper depends on | Windows, macOS, Linux | MIT | Free | Mixtral, Qwen3MoE, DeepSeekMoE, Hunyuan-A13B |
| Jan | Open-source LM Studio equivalent | Windows, macOS, Linux | AGPLv3 | Free | Whatever llama.cpp supports |
| KoboldCPP | Single-file installs | Windows, macOS, Linux | AGPLv3 | Free | Mixtral, DeepSeek, Qwen3MoE |
| Text Generation WebUI | Power users comparing loaders | Windows, macOS, Linux | AGPLv3 | Free | Mixtral, Qwen3MoE, DeepSeekMoE via three loaders |
| vLLM | Multi-GPU MoE serving | Linux (Windows/macOS via WSL or Docker) | Apache 2.0 | Free | Mixtral 8x22B, DeepSeek V3, Qwen3MoE |
| LocalAI | Multi-backend OpenAI drop-in | Linux, macOS, Windows (Docker) | MIT | Free | Anything llama.cpp or vLLM covers |
The apps
1. LM Studio — best overall MoE runner for most people
LM Studio is the fastest path from a fresh install to a working MoE chat. The in-app model catalogue lists Mixtral variants, DeepSeek Coder V2 Lite, Qwen3 MoE, and Hunyuan-A13B with size and quantization filters, so picking a MoE model for your VRAM budget is a scrollable step, not a Hugging Face expedition. The load screen exposes a Number of Experts slider that maps to llama.expert_used_count, letting you override the default top-k when quality matters more than tokens per second. LM Studio for MoE workflows also ships an MLX backend on Apple Silicon and a CUDA path on Nvidia, so the same install runs on an M3 Max or a workstation with an RTX 5090.
Where it falls short: the desktop app is closed source, and the built-in headless server takes a couple of settings to expose properly. Compared to a lean CLI runner, the GUI holds onto more RAM in the background.
Pricing:
- Free: full desktop app for personal and commercial use, no rate limits, no cloud calls
- Paid: Enterprise plan at $10 per user per month for SSO, model and MCP gating, private team catalogues
Platforms: Windows, macOS, Linux
Download: lmstudio.ai
Bottom line: the pick if you want a MoE model chatting on your desktop before lunch. Skip it if you refuse to install closed-source binaries.
2. Ollama — best free CLI for open MoE models
Ollama pulls Mixtral, DeepSeek V3, Qwen3 MoE, and Hunyuan variants with a single ollama run and exposes a REST endpoint on localhost:11434 that speaks the OpenAI Chat Completions format. The registry keeps pace with new MoE releases, and the routing internals track a current llama.cpp fork, so architectures like mixtral, deepseek2, and qwen3moe load without flag surgery. Ollama for local MoE inference is the tool most developers reach for when they want to script a workflow around a routed model.
Where it falls short: the daemon has no UI for adjusting expert count, so you edit a Modelfile or pass overrides through the API to tune sparse routing. The default Modelfiles also cap context lengths conservatively, which bites on Hy3’s 256K window until you raise num_ctx.
Pricing:
- Free: entire runtime, model registry pulls, and API server
- Paid: none from the project
Platforms: Windows, macOS, Linux
Download: ollama.com or GitHub
Bottom line: the right pick if a terminal is where you already live. Not the one if you want visual expert-routing controls.
3. llama.cpp — best foundation for tinkerers
llama.cpp is the runtime nearly every other app on this list wraps. Going direct means you get MoE architecture support on day one for anything the upstream commits: Mixtral, Qwen3MoE, DeepSeekMoE, and the recommended path for Hunyuan-A13B. You also get flags the wrappers hide, like --override-kv llama.expert_used_count=int:4 for top-k tuning and --cpu-moe for pushing cold experts into system RAM.
Where it falls short: there is no GUI. Model management, quantization, and the OpenAI-compatible server (llama-server) are separate binaries with their own arguments. New architectures typically land 2 to 4 weeks after the paper drops, which stung the earliest Hy3 adopters running from source.
Pricing:
- Free: MIT-licensed source and prebuilt binaries for CUDA, ROCm, Metal, Vulkan, SYCL, and OpenVINO
- Paid: none
Platforms: Windows, macOS, Linux
Bottom line: for anyone who wants to know which layers land on the GPU. Overkill for casual chat.
4. Jan — best open-source LM Studio equivalent
Jan is a desktop shell over llama.cpp with an AGPLv3 licence, a Hugging Face model browser that filters GGUF files by size, and a local OpenAI-compatible API server on localhost:1337. The MoE catalogue covers Mixtral variants, DeepSeek Coder V2 Lite, and community Qwen3 MoE quants, and the app inherits llama.cpp expert-routing overrides through its model settings pane. Jan for MoE users who want a GUI is the closest open-source stand-in for LM Studio.
Where it falls short: Jan tends to lag the very latest MoE architectures by a release or two, so if hy_v3 support just landed in llama.cpp master, you may wait for a Jan build to catch up. Some users also report memory pressure when browsing very large model lists.
Pricing:
- Free: full desktop app under AGPLv3
- Paid: none
Platforms: Windows, macOS, Linux
Bottom line: pick Jan if you want an LM Studio-style GUI without the closed binary. Look elsewhere if you need day-of support for brand-new architectures.
5. KoboldCPP — best zero-install MoE runner
KoboldCPP compresses a full runtime, a KoboldAI chat and story UI, and an OpenAI-compatible endpoint into a single 200 MB executable. Since version 1.8, the load screen exposes MoE expert count on the Tokens page for Mixtral, DeepSeek, and Qwen3 MoE quantizations. Prebuilt CUDA, Vulkan, and CPU-only variants ship for Windows, Linux, and Apple Silicon, which makes KoboldCPP for MoE workloads the shortest possible install path.
Where it falls short: the UI is optimized for creative writing and roleplay, which reads noisy for straight assistant work. Model discovery is manual: you drop the GGUF next to the executable and point at it.
Pricing:
- Free: entire binary under AGPLv3, no telemetry
- Paid: none
Platforms: Windows, macOS (Apple Silicon), Linux
Download: koboldcpp.com or GitHub
Bottom line: the one to grab if you refuse to install a full framework. Less pleasant if you want managed model updates.
6. Text Generation WebUI — best power-user front end
Oobabooga’s Text Generation WebUI lets you switch between llama.cpp, ExLlamaV2, and Transformers loaders per model, which matters for MoE because each loader exposes a different expert-count control. The ExLlamaV2 loader has a Number of Experts per Token setting for Mixtral-family models, and the llama.cpp side takes overrides through --override-kv llama.expert_used_count. A --cpu-moe flag pushes expert weights to system RAM for offload-heavy setups.
Where it falls short: every loader has its own quirks, and the app assumes you understand which one fits which quantization. First-run setup is heavier than Jan or LM Studio.
Pricing:
- Free: entire project under AGPLv3
- Paid: none
Platforms: Windows, macOS, Linux
Download: GitHub
Bottom line: get it if you like tuning loaders and A/B testing quantizations. Skip it if you want one-click MoE.
7. vLLM — best for MoE throughput at scale
vLLM is the runner people move to when a single-GPU chat setup outgrows itself. Version 0.6 and later supports expert parallelism for Mixtral 8x22B, DeepSeek V3, and Qwen3 MoE via --enable-expert-parallel, with FP8 and BF16 tensor parallelism across multiple GPUs. Continuous batching means the same box can serve dozens of concurrent MoE requests without the tail-latency spike a naive server would hit.
Where it falls short: vLLM targets Linux with NVIDIA hardware. macOS and Windows users go through WSL or a container, and single-GPU consumer setups pay for complexity they will not use.
Pricing:
- Free: Apache 2.0 project, install with
pip install vllm - Paid: none from the project itself
Platforms: Linux (native), Windows and macOS via WSL or Docker
Bottom line: the runner of choice when you are serving MoE to a team. Overkill for one laptop.
8. LocalAI — most flexible drop-in for MoE
LocalAI is a Go binary that presents an OpenAI, Anthropic, and ElevenLabs-compatible API and switches backends per model. Under the hood it wraps llama.cpp, vLLM, MLX, whisper.cpp, and stable-diffusion, so any MoE GGUF that llama.cpp can convert runs through the same endpoint that answers your embeddings and TTS calls. The project shipped its own APEX quantization strategy tuned for MoE tensor roles, which lets Mixtral-class weights match Q8_0 quality at Q4-adjacent sizes.
Where it falls short: the multi-backend design is heavier than a single-runtime app, and configuration lives in YAML rather than a GUI. Windows and macOS users default to Docker.
Pricing:
- Free: MIT-licensed source and container images
- Paid: none
Platforms: Linux, macOS, Windows (Docker on all three works)
Download: localai.io or GitHub
Bottom line: the pick when one endpoint has to answer for chat, embeddings, TTS, and MoE inference at once. Skip if you only need a chat window.
How to pick the right one
If you have never run a local model, start with LM Studio. The MoE controls are in the UI, the model catalogue is curated, and the first Mixtral chat happens in fifteen minutes.
If you prefer a terminal, Ollama is the shortest path to a running Qwen3 MoE or DeepSeek V3. Pair it with a lightweight web front end later if you want a chat window.
If you want to understand what is happening under the hood, go straight to llama.cpp. Every knob is a flag, and every architecture is documented in the source.
If your priority is open source without giving up the GUI, Jan gets closest to an LM Studio replacement.
If you refuse to install a framework, KoboldCPP is one executable with no dependencies.
If you already juggle four loaders and want them under one roof, Text Generation WebUI is what you already know you want.
If you are serving MoE traffic to a team or want expert parallelism across two or more GPUs, vLLM is the sensible answer.
If you need a single OpenAI-compatible endpoint that also handles TTS, embeddings, and vision beside MoE chat, run LocalAI in Docker and be done.
FAQ
What is the best free app for running MoE LLMs on desktop? Ollama for terminal users, LM Studio for everyone else. Both are free for personal and commercial use, both support Mixtral, DeepSeek V3, and Qwen3 MoE out of the box, and both keep pace with new MoE releases within a couple of weeks of upstream llama.cpp.
Can consumer hardware really run a 295B model like Tencent Hy3? Yes, with caveats. Hy3 activates only 21B of its 295B parameters per token, so throughput reads closer to a 21B dense model than a 295B one. You still need enough system RAM to hold the full quantized weights, roughly 180 GB for a Q4 quant, so a workstation with 256 GB of DDR5 and a 24 GB GPU is the current entry point.
What is the difference between Ollama and LM Studio for MoE models? Ollama is a CLI daemon with a REST server and a curated model registry. LM Studio is a desktop app with a chat UI, a load screen with an explicit Number of Experts slider, and a server that toggles on. Both wrap similar llama.cpp-based backends. Pick Ollama if you script your workflow, LM Studio if you want visual controls.
Does llama.cpp support Tencent Hy3 and other new MoE architectures? llama.cpp typically adds a new MoE architecture within 2 to 4 weeks of the model release. Mixtral, Qwen3MoE, and DeepSeekMoE all landed shortly after their model drops, and Hunyuan-A13B has a llama.cpp path recommended in Tencent’s own docs. Hy3, which shipped July 6, 2026, has community GGUFs and merges into upstream on that same 2 to 4 week window.
Do I need a GPU to run an MoE model locally? Not strictly. llama.cpp, KoboldCPP, Ollama, and LocalAI all support CPU-only inference and cold-expert offload, so a workstation with 128 GB or more of RAM can run smaller MoE quants at readable speeds. A modern GPU with 12 to 24 GB of VRAM plus system RAM offload is where the experience gets pleasant.
Which MoE runner has the best OpenAI-compatible API? For a single machine, LM Studio and Ollama expose clean OpenAI Chat Completions endpoints with no config. For a team or a fleet, vLLM’s server offers the highest throughput and native expert parallelism. LocalAI stitches multiple backends behind one OpenAI-style API, which is useful when chat, embeddings, and vision have to share a single URL.
