Inter-office memo
- To
- Everyone who wants a coding model of their own
- From
- Mo, Chief Executive
- Date
- November 21, 2026, 7:09 AM CST
- Re
- Inkling 2 Mini on your computer
Inkling 2 Mini runs on a 32 GB Mac. How to run it, and where it falls short
On November 20, 2026, Thinking Machines released Inkling 2 Mini, an Apache 2.0 open-weights model with 26.4 billion parameters, 3.4 billion active, in about 20 GB. On an Apple silicon Mac with 32 GB or more, ollama run inkling-2-mini runs it; on Windows and Linux, use llama.cpp b12600 or later, LM Studio or Unsloth. It ties Qwen3.8-27B on Artificial Analysis’s index.
What Thinking Machines released
On November 20, 2026, Thinking Machines Lab released Inkling 2 Mini, “a fast, powerful model that’s small enough to run on your laptop.” It’s the small sibling of Inkling 2 Small, whose 594 billion parameters need hundreds of gigabytes of GPU memory. Mini is a mixture-of-experts model with 26.4 billion parameters in total and 3.4 billion active for each token. It reads text and images, writes text, and takes up to 1 million tokens of context, by its model card. The weights are on Hugging Face under Apache 2.0, which allows commercial use.
Two things set it apart. It’s a looped transformer: of its 24 layers, the middle 12 run three times for every token, which Thinking Machines says lets it “apply more reasoning and reach the right answer with fewer tokens.” Counting the loops, it does the work of 5.4 billion active parameters a token, so its card warns that “compute per token is higher than the active-parameter count alone suggests.” And it was trained in 4-bit: most of its weights are NVFP4 and the last four layers MXFP8, shipped in the same precision, “so there is no conversion loss from quantization.” The whole checkpoint is about 20 GB.
Alongside it, Thinking Machines released TML-d24-varloop-research, a variant that can loop one to five times, for research into how looping affects what a model’s chain of thought reveals. Inkling 2 Mini is the one to run.
What you need to run it
Thinking Machines supports three kinds of machine: one NVIDIA DGX Spark; “Apple silicon devices with 32GB or more memory (MacBooks, Mac minis and Studios)”; and other GPUs, “with 32GB or more of VRAM recommended.” Its researchers ran it offline on a MacBook Pro with an M5 Pro and 48 GB, where it built a playable flight simulator in a few minutes.
Smaller builds lower the bar, at some cost in quality. These are the main ones:
| Build | Download | Runs in | Quality |
|---|---|---|---|
| Original checkpoint | About 20 GB | Ollama on a Mac, MLX, vLLM, SGLang, Transformers | As trained |
| ggml-org Q8_0 | 27.82 GB | llama.cpp b12600 or later, Llama App | 8-bit |
| ggml-org IQ4_NL | 17.63 GB | llama.cpp b12600 or later, Llama App | 4-bit; the build Hugging Face’s guide uses |
| Unsloth UD-Q4_K_XL | 17.63 GB | Unsloth Desktop, Unsloth’s llama.cpp build | Same next token as full precision 98.0% of the time |
| Unsloth UD-IQ2_XXS | 8.49 GB | Unsloth Desktop, Unsloth’s llama.cpp build | Same next token 90.6% of the time |
The quality figures are Unsloth’s, measured against its own full-precision build. Its memory guide asks for 17 to 18 GB for a 4-bit build, 10 to 11 GB for 2-bit, 9 GB for 1-bit and 24 GB for 8-bit, counting RAM and graphics memory together, or unified memory on a Mac, and calls those “a starting point. Longer contexts, image inputs and other apps can use more memory.” By those figures, on paper, a 24 GB Mac or graphics card can hold a 4-bit build and a 16 GB machine a 2-bit one, though Thinking Machines itself supports only 32 GB and up.
How to run it
- Ollama, on a Mac. Install Ollama 0.40.17 or later and run
ollama run inkling-2-mini, a 20 GB download that Ollama tags for vision, tools and thinking. Its release notes say the model “is supported on Apple silicon only in this release. PC support will follow in an upcoming release.” - llama.cpp or Llama App, on any computer. llama.cpp added Inkling 2 Mini in release b12600 on November 20. Hugging Face’s guide serves the 4-bit build with
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL, which starts a local, OpenAI-compatible server with a web chat. Llama App, from the same team, runs it from the Mac menu bar or the Windows system tray. - LM Studio. Its model page says the download “requires at least 18 GB of system memory,” and that the minimum “does not imply room for a full 1M-token context.”
- Unsloth Desktop, on a Mac, NVIDIA or AMD machine. Unsloth recommends its UD-Q4_K_XL build with temperature 1.0, top-p 1.0, top-k 0 and min-p 0. Its main GGUFs use their own layout, so they need Unsloth Desktop or Unsloth’s llama.cpp build. The copies it made for mainline llama.cpp need fix #32704, merged on November 21 but not in a llama.cpp release that morning. Without it, Unsloth warns, mainline “will get gibberish.” The fix doesn’t touch ggml-org’s builds.
- vLLM or SGLang, on a GPU server. vLLM’s recipe uses the
vllm/vllm-openai:inkling-2-miniimage “until Inkling 2 Mini support is in a vLLM release,” verified on H100, H200, GB200, B300, GB300 and DGX Spark. On H100 and DGX Spark it caps the context at 262,144 tokens to fit in memory. For agent work, turn on itsinklingtool-call parser.
To skip the download, hosted copies cost little:
| Where | Input | Cached input | Output | Context |
|---|---|---|---|---|
| OpenRouter, Io Net | $0.08 | $0.016 | $0.35 | 262,144 tokens |
| OpenRouter, Prime Intellect | $0.10 | $0.02 | $0.40 | 1,048,576 tokens |
| Tinker | $0.15 | $0.015 | $0.50 | 64K or 256K |
Prices are per million tokens, as OpenRouter and Tinker listed them on November 21, 2026. Thinking Machines’ Tinker Playground offers chat with it too. Unlike Inkling 2 Small, it isn’t among OpenCode’s free models, and Ollama has no cloud version of it.
How good it is
Thinking Machines compares Mini with open models of a similar size. A selection from its table, as it reports them; the first row is Artificial Analysis’s index:
| Benchmark | Inkling 2 Mini | Qwen3.8-27B | Muse Glimmer | Gemma 4 31B |
|---|---|---|---|---|
| Artificial Analysis Intelligence Index v5.0 | 20.1 | 20.1 | 6.3 | 4.4 |
| DeepSWE v1.1 | 48.2% | 42.2% | 4.4% | 0.3% |
| SWE-bench Verified, bash only | 79.2% | 77.8% | 76.0% | 46.1% |
| Terminal Bench v2.1 | 71.6% | 73.0% | 51.7% | 43.4% |
| MCP Atlas | 82.7% | 77.3% | 75.5% | 54.2% |
| AutomationBench-AA | 41.8% | 48.2% | 6.8% | 4.9% |
Thinking Machines says Mini “leads every open-weights model with less than 100B total parameters on benchmarks such as DeepSWE v1.1 (48.2%) and Vals Index v2.2 (46.5%),” and is “behind only Qwen3.8-27B on others.” Artificial Analysis, which runs its own tests, gives it the same 20.1.
Thinking Machines is plain about the gaps, too. Mini “is not the strongest small model for multi-turn chat or for instructions with many constraints.” Looping adds reasoning but not knowledge, so its factual score is “in line with other models of the same size,” and the card says it is “more likely than larger models to state incorrect information with confidence.” It makes up for that with search and tools: it leads its size on BrowseComp, DeepSearchQA and MCP Atlas.
Use it in a coding agent
Ollama’s page gives one-line setups: ollama launch opencode --model inkling-2-mini and ollama launch claude --model inkling-2-mini. ollama launch sets up and starts OpenCode or Claude Code with the local model. OpenCode also finds it without help: it probes Ollama at http://127.0.0.1:11434 and adds what it finds to /models “without an account connection,” and does the same for LM Studio at port 1234, by its docs. A llama.cpp server goes in as a custom OpenAI-compatible provider.
The model card has advice for agent work. “We recommend setting limits on output tokens and agent turns,” it says, because the model “sometimes became stuck in long reasoning loops, or repeatedly re-checked its own work without making progress, particularly when debugging code.” It also “may take destructive actions, such as deleting or overwriting files, without confirmation, and may not report the outcome of its work accurately.” Run it in a sandbox, keep its permissions tight, and read what it changes.
Enjoy runs OpenCode beside Claude Code, Codex and the other labs’ agents, and its model picker lists every model OpenCode has set up, local ones included. Once Inkling 2 Mini is in Ollama, OpenCode lists it, so you can hand it a conversation when another agent’s plan runs low and keep going at no cost. You can also save routine upkeep as a scheduled workflow for it, and the code never leaves your computer. Enjoy runs scheduled work only while your computer is awake and Enjoy is open.
What isn’t known yet
- How fast it runs on a Mac or a PC. Thinking Machines charts a theoretical ceiling rather than measured speeds, and its card says looped-model support in local runtimes “is new, and speed varies by runtime and hardware.”
- When Ollama supports Windows and Linux. Its notes say only “an upcoming release.”
- When the llama.cpp fix for Unsloth’s mainline files reaches a release. It was merged on November 21; ggml-org’s builds work now.
- Its knowledge cutoff. The model card doesn’t give a date.
- Independent coding scores. Artificial Analysis’s index is independent; the coding numbers above are Thinking Machines’.
If you have an Apple silicon Mac with 32 GB or more, Inkling 2 Mini costs one command and 20 GB of disk to try on a real task. Give it small, well-defined jobs, keep a frontier model for the hard parts, and on Windows or Linux, start with ggml-org’s 4-bit build in llama.cpp until Ollama catches up.
7.Questions.
Can I run Inkling 2 Mini on my Mac?
- Yes, on an Apple silicon Mac with 32 GB of memory or more, which is what Thinking Machines supports. Install Ollama 0.40.17 or later and run ollama run inkling-2-mini, a 20 GB download. Unsloth’s smaller builds lower the bar to 17 or 18 GB for 4 bits and 10 or 11 GB for 2 bits, by Unsloth’s figures, at some cost in quality.
Can I run Inkling 2 Mini on Windows or Linux?
- Yes, but not through Ollama yet, whose current release supports it “on Apple silicon only.” Use llama.cpp release b12600 or later with ggml-org’s 17.63 GB 4-bit build, Llama App, LM Studio, or Unsloth Desktop. Thinking Machines recommends a GPU with 32 GB or more of memory for the full checkpoint.
Why does Inkling 2 Mini output gibberish in llama.cpp?
- Most likely you’re running Unsloth’s files for mainline llama.cpp on a build without fix #32704, which was merged on November 21 but wasn’t in a release that morning. Use ggml-org’s builds, which don’t need it, or Unsloth Desktop or Unsloth’s own llama.cpp build with Unsloth’s files.
Is Inkling 2 Mini better than Qwen3.8-27B?
- For agentic coding, by Thinking Machines’ numbers, yes; overall they’re level. Both score 20.1 on Artificial Analysis’s Intelligence Index. Mini leads on DeepSWE (48.2% to 42.2%) and SWE-bench Verified (79.2% to 77.8%), and Thinking Machines says it’s weaker at multi-turn chat and at instructions with many constraints.
How do I use Inkling 2 Mini with OpenCode or Claude Code?
- Pull it with Ollama, then run ollama launch opencode --model inkling-2-mini or ollama launch claude --model inkling-2-mini. OpenCode also finds models in Ollama and LM Studio on its own and lists them under /models. Thinking Machines recommends limiting output tokens and agent turns, and running agents in a sandbox.
Is Inkling 2 Mini free?
- The weights are, under Apache 2.0, which allows commercial use. Hosted, it costs $0.08 to $0.10 per million input tokens and $0.35 to $0.40 per million output tokens on OpenRouter, and $0.15 and $0.50 on Thinking Machines’ Tinker. OpenCode’s free models don’t include it.