Open models
Five model families you can download and run on your own hardware. The licence matters as much as the benchmark here: open weights is not the same as open source, and two of these five are licensed differently depending on which size you pick.
Which one for your situation
On a laptop
Gemma 4, and it is not close. The 12B at roughly 7–8 GB is the all-round pick; the E2B and E4B suit thin hardware; and on a 32 GB unified-memory machine the 26B-A4B gives you 25B of knowledge at roughly 4B speed. If you specifically need European languages, Ministral 3 8B is the alternative at about 5 GB.
On a single 24 GB graphics card
Qwen3.8-27B at Q4, about 17 GB of weights, under unmodified Apache 2.0 and with more community quantisations than anything else here. Budget your context though: 64k pushes it to about 22 GB and 128k to about 26 GB, which overflows the card. Gemma 4 31B is the close runner-up at roughly 18–19 GB.
On a small server
GLM-5.3-Flash or DeepSeek-V4-Flash. Both are MIT, both are sparse, and both have very low active-parameter counts, so throughput is far better than their total size suggests. Pick GLM for agentic and coding work; pick DeepSeek for long-context work, where its much smaller key-value cache is decisive — provided you can absorb its missing chat template.
Best quality, hardware no object
DeepSeek-V4-Pro, with GLM-5.3 close behind. V4-Pro is the strongest model here carrying a clean MIT licence, and that combination is rare. Expect any of these to sit somewhere behind the best closed models on hard agentic work — the gap is smallest on coding and largest on long-horizon autonomy.
For Dutch and other non-English work
Mistral, with Gemma 4 as the strong alternative. Mistral is the only family whose model cards name Dutch explicitly, and it is an EU publisher. Gemma has the broader raw footprint at 140-plus languages pre-trained, and wins outright if it also has to run on a laptop. Neither publishes Dutch benchmark scores, so test on your own text rather than trusting either claim.
On the numbers here
Memory figures are estimates derived from parameter counts and quantisation, except where a measured community figure was available. Treat them as a guide for what will fit, not a specification — and check the model card before buying hardware.
Qwen 3.8
Mixed — read the licenceVerified 2026-09-01The best-supported model on a single consumer graphics card, and the one whose licence you must read twice.
Publisher: Alibaba
| Variant | Parameters | Context |
|---|---|---|
| Qwen3.8-27B | 28B dense | 262K |
| Qwen3.8-Flash-Next | 125B total / 6B active (MoE) | 262K |
| Qwen3.8-2.4T-A95B | 2.4T total / 95B active (MoE) | 262K |
Licence — Split: Apache 2.0 (27B) · Qwen Community License 1.0 (Flash-Next) · qwen3.8-max (2.4T)
This family is licensed three different ways, which is the single easiest mistake to make with it. The 27B is unmodified Apache 2.0 — genuinely open source, no conditions. The larger two are open weights with conditions: commercial use is allowed, but running a model-as-a-service or an AI work-assistant business on them requires a separate licence from Alibaba above published revenue thresholds, and attribution becomes mandatory at scale.
Hardware
27B at Q4_K_M: about 17 GB of weights (measured by the community), roughly 19 GB in use.
Budget for context, not just weights. Measured usage climbs to about 22 GB at 64k context and 26 GB at 128k, so a 24 GB card cannot hold the full context window. Weights alone exceed 16 GB, so 16 GB cards are out at Q4.
Prompt format — ChatML
Turns marked with <|im_start|> and <|im_end|>, and notably no BOS token. Thinking is on by default and emitted in <think> blocks, steered by a reasoning_effort parameter.
Watch out: It keeps its reasoning blocks in the conversation history, so trimming history naively breaks it. Sampling settings also differ between thinking and instruct modes.
Good at
Tooling support is the real advantage: the 27B is the most-downloaded serious open model on Hugging Face with hundreds of community quantisations, so whatever runtime you pick, someone has already made it work. Genuinely multimodal across text, image and video.
Weaker at
Alibaba has abandoned the small end — there is no current-generation Qwen below 27B, so if you need a 4B or 8B general model you fall back to the previous generation. And the most capable models in the family are the licence-encumbered ones.
Sources and grounding
- 1Qwen3.8-27B model cardAlibaba (Hugging Face)
- 2Qwen3.8-Flash-Next model cardAlibaba (Hugging Face)
DeepSeek V4
MITVerified 2026-09-01The most permissive licence at this capability level, and the most awkward to integrate.
Publisher: DeepSeek
| Variant | Parameters | Context |
|---|---|---|
| DeepSeek-V4-Flash | 284B total / 13B active (MoE) | 1M |
| DeepSeek-V4-Pro | 1.6T total / 49B active (MoE) | 1M |
Licence — MIT
The model card states plainly that the repository and the weights are MIT licensed. Genuinely open source: unrestricted commercial use, no user-count thresholds, no revenue triggers, no carve-out for running it as a service. At this capability level that is unusual.
Hardware
Flash at Q4: roughly 150–160 GB (estimate) — two 80 GB cards or four 48 GB. Pro is multi-node.
Nothing in this family runs on consumer hardware; the smallest is 284B. The saving grace is the low active-parameter count, so throughput once loaded is far better than the total size suggests.
Prompt format — Python encoder, no Jinja template
DeepSeek deliberately ships no chat template. Instead the repository provides Python scripts you call to turn messages into the input string and to parse the output back.
Watch out: This matters enormously in practice: any tool that assumes the standard apply_chat_template call will either fail outright or silently build a malformed prompt. Budget integration time for it.
Good at
Long-context efficiency is the genuine differentiator — its attention design cuts the key-value cache to roughly a tenth of the previous generation, which changes what a million-token context actually costs to serve. Very strong on agentic and coding work.
Weaker at
You need a server before you can start. DeepSeek also publishes less about multilingual coverage than Mistral or Google, so its performance outside English is comparatively unevidenced.
Sources and grounding
- 1DeepSeek-V4-Flash model cardDeepSeek (Hugging Face)
- 2DeepSeek V4 release noteDeepSeek
GLM-5.x
Mixed — read the licenceVerified 2026-09-01The strongest self-hostable agentic model — provided you pick the right point release, because the licence changes between them.
Publisher: Z.ai (Zhipu)
| Variant | Parameters | Context |
|---|---|---|
| GLM-5.3-Flash | 320B total / 18B active (MoE) | ~1M |
| GLM-5.2 | 753B (MoE) | 1M |
| GLM-5.3 | 753B (MoE) | ~1M |
Licence — MIT (GLM-5.2 and GLM-5.3-Flash) · custom GLM-5.3 License (GLM-5.3 753B)
GLM-5.2 and GLM-5.3-Flash are MIT — clean, OSI open source. The 753B GLM-5.3 is not: it carries a custom Z.AI licence requiring attribution, and a security review by Z.AI before commercial use by any organisation running a model-as-a-service business above a very high published revenue threshold. For an ordinary business that is functionally permissive, but it is not MIT and should not be described as such.
Hardware
GLM-5.3-Flash at Q4: roughly 170–180 GB (estimate). The 753B models are approximately 380–400 GB at Q4.
Flash is the one to actually deploy: it is both the more permissive licence and the more hostable size. Claims that the 753B models run on four cards do not reconcile with their size — assume eight.
Prompt format — ChatGLM
Its own delimiters — <|system|>, <|user|>, <|assistant|> and <|observation|> for tool responses — opened by a [gMASK]<sop> prefix. Tool calls use nested tags rather than JSON.
Watch out: Two defaults will quietly hurt you in a chat application: reasoning effort defaults to maximum, and the setting that clears prior thinking defaults to off even though the documentation says chat use should turn it on.
Good at
Long-horizon agentic and software-engineering work, at a fraction of the previous generation cost thanks to a very low active-parameter count. Having MIT-licensed variants at this capability level is genuinely unusual.
Weaker at
The documentation is the weakest of the five: the context window and active parameter count are not clearly stated on the 5.3 card. And a licence that changes between point releases is a real governance risk if you standardise on "GLM" as a family.
Sources and grounding
- 1GLM-5.2 model cardZ.ai (Hugging Face)
- 2GLM-5.3 model cardZ.ai (Hugging Face)
Gemma 4
Apache 2.0Verified 2026-09-01The only family here that credibly runs on a laptop, and now genuinely Apache 2.0.
Publisher: Google
| Variant | Parameters | Context |
|---|---|---|
| Gemma 4 E2B / E4B | 5B / 8B dense | 256K |
| Gemma 4 12B | 12B dense, multimodal | 256K |
| Gemma 4 26B-A4B | 25B total / 3.8B active (MoE) | 256K |
| Gemma 4 31B | 31B dense | 256K |
Licence — Apache 2.0
Gemma 4 is the first Gemma released under Apache 2.0, replacing the old custom terms that restricted redistribution. Commercial use is unrestricted and there are no user thresholds. One honest caveat: Google separately publishes a Prohibited Use Policy written in mandatory language, and neither document explains how that policy interacts with a permissive licence grant. Treat it as unresolved rather than assuming either reading.
Hardware
E2B/E4B roughly 3–5 GB at Q4; 12B about 7–8 GB; 26B-A4B about 15 GB; 31B about 18–19 GB. All estimates.
The 26B-A4B is the clever pick on a 32 GB unified-memory laptop: 25B of knowledge running at roughly 4B speed, because only 3.8B parameters are active per token.
Prompt format — Gemma 4 turn markers — changed from Gemma 3
Gemma 4 dropped the previous generation’s start_of_turn and end_of_turn markers for a new pair, and gained native system-prompt support that Gemma 2 lacked. Thinking is off by default and must be switched on in the system prompt.
Watch out: Any prompt code written for Gemma 2 or 3 will silently produce poor output on Gemma 4. Use the library’s own chat-template call rather than hand-rolling the tokens.
Good at
Unbeatable at the small end — nothing else here credibly runs on a laptop or a phone. Pre-trained on over 140 languages with out-of-the-box support for 35 or more, fully multimodal, and permissively licensed across every size.
Weaker at
The ceiling is low. At 31B the largest Gemma 4 simply cannot compete with the 300B-plus sparse models on hard long-horizon work. It is the best small family, not a frontier contender.
Sources and grounding
- 1Gemma 4: expanding the Gemmaverse with Apache 2.0Google Open Source Blog
- 2Gemma 4 31B model cardGoogle (Hugging Face)
Mistral 3 / Small 4
Apache 2.0Verified 2026-09-01The only family that names Dutch on its own model cards, from an EU publisher, Apache 2.0 all the way up.
Publisher: Mistral AI
| Variant | Parameters | Context |
|---|---|---|
| Ministral 3 8B | 8B dense | 256K |
| Mistral Small 4 | 119B total / ~6B active (MoE) | 256K |
| Mistral Large 3 | 675B total / 41B active (MoE) | 256K |
Licence — Apache 2.0
Apache 2.0 across the range, including the 675B Large 3. This is a real change: Mistral historically reserved its largest models for a restrictive research licence. Unrestricted commercial use, no thresholds, no service carve-out.
Hardware
Ministral 3 8B roughly 5 GB at Q4 — laptop class. Small 4 about 65–70 GB. Large 3 about 350 GB. Estimates.
Small 4 sits in an awkward middle: too big for a 24 GB card, not big enough to beat the 300B-plus sparse models on a server you have already paid for. Ministral 3 8B is the one most people will actually run.
Prompt format — Mistral bracket format — not ChatML
User turns wrap in [INST] and [/INST], system prompts in [SYSTEM_PROMPT], reasoning in [THINK], with further bracket tags for tools. It looks nothing like the ChatML most other families use.
Watch out: Historically the most frequently mis-implemented format of any major family: whitespace around the brackets and the position of the beginning-of-sequence token both matter. Mistral’s public prompting guide covers wording, not these tokens — the chat template file is the authority.
Good at
European languages. Mistral is the only family here whose model cards name Dutch explicitly, and an EU publisher is worth something for data-residency and AI Act posture. Small 4 also folds what used to be three separate models — reasoning, multimodal and coding — into one deployment.
Weaker at
It does not lead the benchmarks. On open-weight coding and agentic rankings the top cluster is DeepSeek, GLM and Qwen; Mistral sits behind them. Choose it for languages, licence and jurisdiction rather than for raw capability.
Sources and grounding
- 1Mistral Small 4 announcementMistral AI
- 2Mistral Small 4 model cardMistral AI (Hugging Face)