NVIDIA Nemotron 3.5 Bolt: 74B Model That Runs on a Laptop

By Muzamil Ahmad · Updated 2026-10-15 · Chatbots

#chatbots#nvidia nemotron 3.5 bolt#nemotron bolt 74b#nemotron 3.5 bolt release date#run 74b model on laptop#nemotron bolt vs muse wonder#nemotron bolt ollama#nvidia pair#rtx spark nemotron

Save

On Tuesday, October 13, 2026, NVIDIA announced a model that breaks the pattern of the entire month. While Meta, OpenAI, and Google spent October launching bigger closed models behind APIs and subscriptions, NVIDIA announced one you can download. Nemotron 3.5 Bolt 74B-A9B is a 74-billion-parameter model that NVIDIA says fits in 24GB of memory, built for always-on personal AI agents that live on your own machine instead of in someone else's cloud.

It is scheduled to land today, Thursday, October 15 — one day before the RTX Spark laptops it was built for go on sale. Here is the honest version: what Bolt is, what "74B in 24GB" actually means, how it compares to the closed frontier models, and how to run it when the weights land.

One framing note: as of this writing, the weights are not confirmed live. Everything so far is pre-launch coverage of a release scheduled for today. Treat availability as "scheduled to land today (Oct 15)" and check Hugging Face or Ollama for the live download rather than assuming it is already there.

What Launched

Two things happened on October 13, and they belong together. NVIDIA announced the Bolt model family, and it shipped the software ecosystem the models are meant to run in — headlined by PAIR leaving beta, a free llama.cpp app for Windows, and a routing library for splitting work between local and cloud models.

The flagship is Nemotron 3.5 Bolt 74B-A9B. The name is a spec sheet in miniature: 74B is the total parameter count, A9B is the roughly 9 billion parameters active on any given token, and 24GB is the compressed footprint NVIDIA is claiming for the weights. NVIDIA positions it as purpose-built for agentic AI on RTX Spark — laptops with up to 128GB of unified memory and 1 petaflop of AI compute, on sale October 16.

The job NVIDIA gives it is the always-on personal agent: routine coding help, collecting and analyzing data, background monitoring, and calling third-party APIs through agent frameworks.

The second model, Bolt E4B, arrives October 23: the compact sibling for fast conversational agents and interactive game characters where latency matters more than depth. Gamers will be able to try it inside SkyrimNET, an AI NPC modding platform for The Elder Scrolls V: Skyrim that gives characters dynamic, contextual dialogue.

Around the models, NVIDIA shipped the plumbing. PAIR graduated from beta. Llama for Windows arrived as a free, open-source app built on llama.cpp that runs open models without the command line. NVIDIA also shipped MCP servers for its own applications — the NVIDIA app, Broadcast, PAIR, RTX Remix, and Omniverse. And the Switchyard library lets agents route requests between local and cloud models, the quiet admission that even in NVIDIA's local-first vision, some jobs will still go to the cloud.

What 74B in 24GB Actually Means

The headline number deserves a slow, careful read. "74B in 24GB" sounds like NVIDIA repealed the laws of physics. It did not. It combined two well-understood techniques and one aggressive compression choice.

First, the mixture of experts. Bolt is a sparse MoE model: 74 billion parameters in total, but only about 9 billion active for any given token. Different tokens route to different "expert" sub-networks, so the compute per token looks like a 9B model while the stored knowledge spans 74B parameters. The catch: the inactive parameters still have to be stored. MoE saves compute, not storage.

Second, 2-bit quantization with quantization-aware training. Storing a parameter at 2 bits instead of the 16 bits used in full-precision training shrinks the weights dramatically — that is how 74 billion parameters land near 24GB. Quantization-aware training means the model was trained with the compression in mind rather than squeezed after the fact, which is supposed to preserve more quality. Whether it actually does is exactly what independent testing will tell us once the weights are out.

So the honest description is: a 74B-total, roughly 9B-active-per-token agent model with a claimed 24GB compressed footprint. Impressive engineering if it holds up — and a number with fine print.

The 24GB figure is NVIDIA's claim about the weights, not a complete system requirement. Real memory use depends on context length, the KV cache (the working memory that grows as a conversation gets longer), runtime overhead, and whatever agent software is running alongside the model. A long agentic session with a big context window will use more than 24GB. Treat 24GB as the floor, not a comfortable recommendation.

"Faster than dense models, smarter than same-footprint MoE models" is a vendor comparison. That is NVIDIA grading its own homework. There are no independent benchmarks for Bolt 74B yet. Do not repeat the speed-versus-intelligence claim as a fact; repeat it as NVIDIA's claim, pending verification.

2-bit quantization is the open question. Aggressive quantization can cost accuracy, especially on the tasks that matter for agents: precise tool calls, following multi-step instructions, and staying coherent over long sessions. Those are exactly the failure modes that do not show up in a launch post.

The "5x local LLM downloads" claim has no published methodology. NVIDIA says local LLM downloads are already five times higher than all of last year. That may be true — but with no published methodology, treat it as a directional signal from an interested party, not a measured statistic.

Open-Source Everything vs Closed Weights

This is where Bolt is genuinely different from everything else that launched this month. Every other frontier-class release in October arrived as a closed product. Bolt arrives — by NVIDIA's description — as the whole thing.

Fully open source, in NVIDIA's telling, means weights, training data, and recipes. Weights let you download and run the model yourself. Training data lets you inspect what it learned from. Recipes let you reproduce or adapt the training. That is a deliberate contrast with the month's other launches: Muse Wonder is closed weights behind an API, GPT-6 is closed weights behind ChatGPT and OpenAI's APIs, and Gemini 4 Argon is closed weights behind Google's API and app.

The table shows two theories of AI. The closed models sell intelligence as a service: you pay per token and accept that your data leaves your machine. Bolt sells intelligence as software: you pay once for hardware and run it yourself.

The practical consequences are concrete. With open weights, you can fine-tune Bolt for your own agents instead of prompt-engineering around someone else's API. You can run it entirely offline, which matters for privacy-sensitive work and anyone whose data cannot legally leave their machine.

One caveat: the weights were not yet public at announcement, so the exact license terms, the completeness of the data release, and the usability of the recipes are still to be confirmed when the release lands. Verify the license before building a business on it — if it restricts commercial use or redistribution, that changes the story.

The other honest framing: open does not mean equal. The closed models still lead on independent benchmarks, and Bolt has no independent numbers yet. If your priority is the highest raw capability per prompt, the frontier APIs remain the answer.

How to Run It

When the weights land, you will have several ways to run Bolt, and NVIDIA has named them: llama.cpp, vLLM, Ollama, Unsloth, and LM Studio. The right choice depends on how much machinery you want to operate.

The easiest path: Ollama or LM Studio. If you just want to chat with the model on your own machine, start here. Both handle downloading, loading, and serving the model behind a clean interface. Check the official NVIDIA listing or the app's model library for the exact pull or download name — do not copy a model ID from a random forum post or a launch-day rumor thread, because wrong or spoofed listings appear fast around popular releases.

In Ollama, the general shape of the workflow is to pull the model by its official name and then run it:

``` ollama pull <exact model name from the official listing> ollama run <exact model name from the official listing> ```

Replace the placeholder with the exact name from the official release listing — the name is set by the release, and guessing it will either fail or, worse, fetch something that is not the official model. If you already run Ollama, ollama list shows what is installed, and the official listing is the source of truth for the name.

The tinkerer path: llama.cpp. llama.cpp is the open-source inference engine underneath much of the local AI world, and the path for people who want control over quantization, context size, and performance tuning. It is also the engine behind Llama for Windows, NVIDIA's new free app that wraps llama.cpp in a system-tray GUI so you can browse, download, and run local models without the command line.

The server path: vLLM. If you want to serve Bolt to applications — as an API endpoint your own software calls — vLLM is the production-grade choice. The fine-tuning path: Unsloth. Unsloth is the tool for adapting the model: fine-tuning Bolt on your own data so it behaves the way your agents need. This is where the open training data and recipes start to pay off.

The agent path: Hermes and OpenClaw. Bolt is wired into these agent frameworks at launch, so you can plug it into agent workflows — multi-step tasks, tool use, background monitoring — rather than treating it as a chatbot.

Three practical notes before you download anything. Check the official listing first: the exact model name, quantization levels, and file sizes are set by the release — if the listing is not there yet, wait for it rather than trusting a mirror. Plan for more than 24GB: the weights are only the baseline, and actual use adds KV cache, runtime overhead, and agent software. Start with a short-context test before handing Bolt a long agent workflow, then scale up.

For developers comparing the broader landscape, our best AI coding assistants roundup covers where local models fit alongside cloud-based coding tools — Bolt's arrival changes that calculus for anyone waiting for a self-hosted model worth building on.

RTX Spark and PAIR: The Ecosystem Around Bolt

Bolt does not arrive alone. It arrives as the software flagship for RTX Spark, NVIDIA's new laptop platform, and alongside PAIR, the tool that turns a house full of computers into a local AI network.

RTX Spark is the hardware side: laptops with up to 128GB of unified memory and 1 petaflop of AI compute, going on sale October 16. The 128GB figure is the one that matters for Bolt, because unified memory means the model, the context, and the rest of the system share one pool — no separate GPU VRAM ceiling to hit. Two notes: NVIDIA has announced no retail pricing for RTX Spark laptops in the materials covered here, so do not trust any price attributed to the launch until it comes from NVIDIA or a retailer; and Bolt is not exclusive to RTX Spark — any machine with enough memory and a supported runtime can run it.

PAIR — Personal AI Router — is the more interesting software. It graduated from beta on October 13 as a free, open-source tool. Its job is to spread AI inference across the compatible PCs on your home network. Install it on each machine, and it auto-discovers peers on the local network and routes inference jobs to whichever machine has capacity.

Here is the part people get wrong, so let me be precise: PAIR routes whole jobs to one machine. It does not pool GPU memory. Three PCs with 24GB each do not become one 72GB virtual GPU. What PAIR gives you is parallel throughput: when an agent breaks a task into independent subtasks, each subtask can run on a different machine at the same time instead of queuing on one GPU. In NVIDIA's own demonstration, a five-subagent inbox task took 18 minutes on one machine and 8 minutes 48 seconds spread across three. That is a routing speedup, not a memory-pooling trick.

The full release adds llama.cpp support, direct model downloads from Hugging Face, RTX Spark compatibility, and runs on Windows, Linux, and macOS. It supports GeForce RTX 20-series cards and newer, RTX PRO workstation cards, DGX Spark, and Apple M4 or newer systems.

The strategic picture is worth naming: NVIDIA is building the local AI stack the way it once built the gaming stack — hardware (RTX Spark), model (Bolt), router (PAIR), engine (llama.cpp support), and agent plumbing (MCP servers, Switchyard) all arriving in the same week.

Who It Is For — and Who Should Skip

Bolt is for you if: you want a capable agent model that runs entirely on your own hardware; you care about privacy, offline use, or keeping data on-device; you want to fine-tune a model on your own data; you are building agents and want to stop paying per-token for routine work; you have a machine with generous memory (or an RTX Spark laptop on the way); or you are a researcher who wants to inspect training data and recipes, not just prompt an API.

Bolt is a strong maybe if: you are a developer weighing local vs cloud for a product. Run your own evals on your own tasks. A hybrid setup — Bolt for the routine, a frontier API for the hardest 10% — is the most sensible default for production work right now.

Skip Bolt for now if: you need the highest raw capability and are happy paying API prices; you need image or video generation (Bolt is a language model for agent work, not a media generator); you have a thin laptop with limited memory and no appetite for tuning inference settings; or you want a finished consumer product rather than a model you operate yourself. If that last one is you, wait for the agent apps built on Bolt to mature — the RTX Spark setup flow, Hermes, and OpenClaw are where Bolt becomes a product instead of a project.

Honest Caveats

• Weights not confirmed live at write time. This article is published on the scheduled release day. Check Hugging Face or Ollama for the live download, and verify the license and the contents of the release before building on it.
• No independent benchmarks exist yet. Every performance claim about Bolt 74B is NVIDIA's. Do not treat the speed-versus-intelligence framing as measured fact.
• The 24GB figure is a weights claim, not a system requirement. Context length, KV cache, runtime overhead, and agent software all add memory. Plan for headroom.
• The "5x local LLM downloads" claim has no published methodology. Directional signal, not a statistic.
• PAIR routes jobs; it does not pool GPU memory. Three 24GB machines do not become one 72GB machine.
• No RTX Spark retail prices have been announced in the materials covered here. Do not budget from rumored prices.
• Open does not mean equal. The closed frontier models still lead on independent benchmarks. Bolt's advantage is control, privacy, and cost structure — not a proven capability lead.
• 2-bit quantization is aggressive. The quality cost, if any, will only show up in independent testing of real agent workloads: tool calls, long sessions, and multi-step instruction following.

Conclusion

Nemotron 3.5 Bolt is the most interesting AI release of October precisely because it refuses the month's dominant pattern. Meta, OpenAI, and Google each launched a closed model you rent by the token. NVIDIA launched an open model you download and own — a 74B-class agent model squeezed into a claimed 24GB, with the training data and recipes included, arriving alongside the hardware and the routing software to actually run it.

The honest version is that the proof is still pending. The weights were not confirmed live at write time, there are no independent benchmarks, the 24GB figure is a weights claim with real-world overhead on top, and 2-bit quantization is aggressive enough that the quality question stays open until testers get their hands on it. Those are not reasons to dismiss Bolt. They are reasons to test it instead of believing it.