An XDA piece this week described feeding a self-hosted model the writer’s own Docker Compose files and getting back a list of security holes the writer had never noticed: an exposed port, a bind mount that granted more than it should, a container running as root that had no reason to. This is the pitch for local LLMs in security work. The code, the config, and the environment stay on the machine, the model never sends the source to a vendor, and a review that used to demand a checklist and half an afternoon now takes the length of a coffee break. The best local LLM apps for code and config security audits below are the seven that actually make this workflow real on Windows, macOS, and Linux.
We picked apps that run on consumer hardware, work with the current wave of open-weight models good enough to reason about code (Llama 3.3, Qwen 2.5 Coder, DeepSeek Coder V3, Mistral Small, Phi-4), and either expose an API for scripted audits or ship a chat UI worth living in.
What to look for in a local LLM app for security work
Local means local, but the app around the model still decides what you can do with it.
- A model list that includes the current open-weight coder models, not just quantised Llama 2.
- An OpenAI-compatible local API so audit scripts and CI can call it without vendor lock.
- A retrieval or document mode that can chew through a whole repo or Compose file, not one snippet at a time.
- Function or tool calling for models that support it, so audits can chain into git blame or file listing.
- A calm approach to telemetry, and an offline install path that survives an air-gapped machine.
- Sensible defaults for context length and GPU offload so you are not tuning llama.cpp flags for an hour.
Quick comparison
| App | Best for | Free plan | Paid tier | API |
|---|---|---|---|---|
| LM Studio | UI-first users who also want an API | Yes | Free | Yes |
| Ollama | CLI-first users, scripts, and CI | Yes | Free | Yes |
| Jan | Fully open-source, Ollama-free stack | Yes | Free | Yes |
| GPT4All | Retrieval over local docs from day one | Yes | Free | Limited |
| Continue | IDE-native audits inside VS Code and JetBrains | Yes | Optional paid tier | Yes |
| Msty | One app that speaks to both local and cloud models | Yes | Optional paid tier | Yes |
| Open WebUI | Self-hosted team chat over Ollama or vLLM | Yes | Free | Yes |
The apps
1. LM Studio
LM Studio turns local model hosting into something a non-CLI user can run. The model catalogue points at Hugging Face directly, quantisations are labelled with expected VRAM, and the built-in server exposes an OpenAI-compatible endpoint on localhost. That endpoint is what makes it useful for security work: your audit scripts can hit it the way they would hit OpenAI, no wrapper needed, and the source stays on the machine. Structured output, function calling on supporting models, and a chat UI that handles multi-turn code review well.
Where it falls short: it is not open-source, and its business model is worth understanding before you deploy it at scale. Memory reporting and diagnostics for stuck loads can be sparse.
Pricing:
- Free: full app, personal use
- Paid: business licensing on request
Platforms: Windows, macOS, Linux
Download: lmstudio.ai
Bottom line: the strongest starting point if you want a UI and an API from one installer.
2. Ollama
Ollama is the CLI-first standard for pulling a model and serving it locally. ollama run qwen2.5-coder:14b gets you a working coding model in one command; the same daemon exposes an HTTP API that half the LLM ecosystem now targets by default. For an audit script that reads a Compose file, asks the model for issues, and writes a report, Ollama is the shortest path from zero to running.
Where it falls short: the built-in chat UI is intentionally minimal; if you want a real interface you pair Ollama with Open WebUI or Msty. Model management is by tag rather than by explicit path, which some security-conscious teams prefer more control over.
Pricing:
- Free: full app
- Paid: cloud tier for hosted models
Platforms: Windows, macOS, Linux
Download: ollama.com
Bottom line: the pick when the workflow is scripted, and the pick to pair with a UI when it is not.
3. Jan
Jan is the fully open-source alternative to LM Studio. Same shape: a local model runtime, an OpenAI-compatible server, a chat UI, and a Hugging Face-linked model catalogue. The difference is the licence and the ability to inspect and modify the whole stack. Security teams that need to justify what runs on the endpoint tend to prefer it.
Where it falls short: the model catalogue and detection of new quantisations lag LM Studio slightly, though the gap narrows every release. Extensions are still catching up.
Pricing:
- Free: full app
- Paid: not applicable
Platforms: Windows, macOS, Linux
Download: jan.ai
Bottom line: the pick when you need to see and audit the runtime itself, not just the model output.
4. GPT4All
GPT4All has spent longer than most of its peers on document retrieval, and that shows in the LocalDocs feature. You point it at a folder of Compose files, IaC templates, or the whole repo, and the app builds a local index the chat draws from. That is the right shape for a security review that spans a codebase, not a single snippet.
Where it falls short: the API surface is narrower than LM Studio or Ollama. Model performance in the app is fine but the ecosystem around it is smaller than the leaders on this list.
Pricing:
- Free: full app
- Paid: enterprise support tier
Platforms: Windows, macOS, Linux
Download: nomic.ai/gpt4all
Bottom line: the pick when the audit spans a repo and you want retrieval baked in.
5. Continue
Continue is the local-LLM sidekick that lives inside VS Code and JetBrains IDEs. Point it at Ollama, LM Studio, or a self-hosted vLLM, and you get inline explanations, refactor suggestions, and a chat that can see your open file. For security auditing that is the natural surface: highlight a function, ask what could go wrong, then run the same prompt across a whole file. The extension is open source and the config lives in a plain JSON file you can review.
Where it falls short: it depends on you providing the model host. It is not a place to run a model, it is a place to use one. That is a feature for the security team; it is a step for a hobbyist.
Pricing:
- Free: open-source extension
- Paid: optional team tier
Platforms: Windows, macOS, Linux (via VS Code or JetBrains)
Download: continue.dev
Bottom line: the pick to bring local LLM audits into the editor you already work in.
6. Msty
Msty is the pragmatist’s chat app: one interface that speaks to Ollama, LM Studio, and remote APIs. That is useful for a security workflow that runs a small model locally for volume and a bigger cloud model for the last mile of a hard finding. Split view and parallel model chat make comparison audits fast.
Where it falls short: the app is not open source. The value is in the UX rather than the model runtime, so the pick makes sense when a good UX is the constraint.
Pricing:
- Free: full app for personal use
- Paid: optional Aura tier for advanced features
Platforms: Windows, macOS, Linux
Download: msty.app
Bottom line: the pick when the team already runs Ollama and just wants a better chat client on top.
7. Open WebUI
Open WebUI is the self-hosted team chat that turns Ollama or vLLM into something the whole team can hit. Sign-in, per-user model access, and RAG over documents are all here. For a security team, that means one on-prem model host serving audits to every reviewer through a shared UI, with per-user telemetry and control.
Where it falls short: it is a self-hosted service, so setup is on you. The audit surface is chat and documents rather than a scripted API pipeline, which the underlying Ollama exposes for you to layer on.
Pricing:
- Free: open source
- Paid: not applicable
Platforms: self-hosted (browser access from any OS)
Download: openwebui.com
Bottom line: the pick when audits are a team activity, not a solo one.
How to pick the right one
- If you want UI plus API from one installer: LM Studio.
- If audits will be scripted and run by CI: Ollama.
- If you must run open-source top to bottom: Jan.
- If retrieval over a whole repo matters: GPT4All.
- If audits should live inside the IDE: Continue on top of Ollama.
- If a great chat UX is what unlocks the workflow: Msty.
- If the team needs a shared review surface: Open WebUI on top of Ollama.
FAQ
Can a local LLM really find security holes a static analyser misses?
Yes and no. A static analyser is deterministic and catches known patterns better than an LLM. A local LLM catches the semantics: which mount grants more than it needs, which port has no business being exposed, which env var reveals a secret. The two together are stronger than either alone; treat the LLM as a reviewer, not a scanner.
Which open model is best for auditing code today?
For 24GB VRAM, Qwen 2.5 Coder 32B and DeepSeek Coder V3 are the current standouts. For 16GB, Qwen 2.5 Coder 14B or Llama 3.3 70B at a low quantisation. For 8GB, Phi-4 or Qwen 2.5 Coder 7B still return useful reviews. The picture changes month to month; pick a runtime and swap models as they land.
Do these apps send anything to the cloud?
The models run locally in all seven. Some apps do telemetry (crash reports, feature usage) unless disabled. Read the app’s telemetry docs and turn off what you do not want. Ollama, Jan, and GPT4All are the strictest about staying offline.
Can I use one of these on an air-gapped machine?
Ollama, Jan, and GPT4All all support fully offline install if you side-load the model files. LM Studio can, though the model catalogue expects internet. Open WebUI runs on your own hardware and does not require external connectivity once configured.
What is the difference between running a model in Continue vs. LM Studio?
Continue does not run the model itself; it is a client that talks to a local server (Ollama, LM Studio, vLLM) inside your IDE. LM Studio is the server. Pair them: run LM Studio or Ollama, connect Continue to it, and audit code inside the editor.
