Comparisons between cloud and local AI tend to be written by someone with a side. The cloud version says local models are toys; the local version says the gap has closed. Neither is true.
Here is the version that survives actually using both daily for a year.
This article sits under Private ChatGPT Alternatives.
Where the cloud model genuinely wins
Starting here, because it is where honest comparisons usually go soft.
Hard multi-step reasoning. Problems with many dependent steps — involved mathematics, intricate logic, debugging a subtle interaction across several files, planning with lots of interacting constraints. A frontier model has orders of magnitude more parameters, and on these tasks the difference is not subtle. Thinking mode narrows the gap on local models but does not close it.
Very long documents. Frontier context windows run to hundreds of thousands of tokens. A phone can hold 8,000–32,000. If you need a model to reason across an entire contract, a book, or a large codebase in one pass, local is the wrong tool. Context Windows Explained covers why this constraint is physical rather than arbitrary.
Obscure and specialist knowledge. Larger models simply store more. Niche history, uncommon languages, specialist medicine, unusual APIs — the depth is meaningfully greater.
Current events. A local model’s knowledge is frozen at its training cut-off. Cloaked mitigates this with optional DuckDuckGo search, but that is retrieval bolted on, not the same as a model with recent knowledge and integrated search.
Tooling. Code execution, image generation, file handling, browsing. A local chat app is a chat app.
If your work centres on any of these, use the frontier model. That is the correct answer, and pretending otherwise wastes your time.
Where the gap has genuinely closed
This is the part that has changed fastest, and where intuitions from 2023 are now badly out of date.
Everyday writing. Emails, messages, posts, summaries, rewrites, tone adjustments, outlines. A modern 2–4B model is genuinely good at this. In blind comparison most people cannot reliably identify which output came from which.
Summarizing and extracting. Give a model a few thousand words and ask for the key points, action items, or a specific fact. Local models handle this well — it is comprehension, not deduction.
Explanation. “Explain how a mortgage offset account works,” “what does this error mean,” “why is the sky blue.” Well within a small model’s competence.
Translation. Qwen models support 30+ languages and handle everyday translation confidently, including reading text out of photos via on-device vision.
Brainstorming. Generating options, names, angles, counter-arguments. Quantity and variety matter more than depth here, and small models are perfectly capable.
Structured transformation. Reformatting, converting to a table, extracting fields, rewriting to a template.
Add it up and it is a large share of what people actually use AI for — not because the tasks are trivial, but because most useful work is not frontier-hard.
The differences that aren’t about capability
Three practical differences matter as much as quality in daily use, and they rarely appear in comparison tables.
Availability. A local model has no outage, no rate limit, no queue at peak time, no “you’ve reached your limit for today,” and no regional restriction. It works on a plane, in a tunnel, on a mountain, and in a building with terrible reception. That reliability is worth more than a few benchmark points — see How to Use AI Without Internet Access.
Latency. Cloud requests spend 200–800ms in transit before generation begins. A local model starts almost immediately. Cloud finishes long answers sooner; local starts sooner. For short exchanges, local frequently feels faster. Tokens Per Second Explained breaks this down.
What you’re willing to type. This is the one people underestimate until they experience it. Using a model that cannot transmit anything changes what you ask. You stop pre-editing. You paste the actual email, the real numbers, the genuine worry. A tool you use fully is more valuable than a better tool you use cautiously.
The privacy difference, stated precisely
Cloud assistants have improved here — no-training toggles, temporary chats, enterprise agreements. These are real improvements and worth using.
But they are all commitments about what a company will do with data it has received. On-device inference means the data is never received. There is no retention window to configure, no policy to re-read after an update, no breach that can expose conversations that were never stored, and no subpoena that can compel production of records that don’t exist.
“We won’t” versus “we can’t” is the entire distinction, and it is the reason the choice matters even when you trust the company involved. Cloud AI Data Policies Compared covers how to read what the providers actually commit to.
A practical split
What tends to work, in practice:
Default to local for: everyday writing and editing, personal and health questions, work you cannot disclose, anything involving real names or numbers, quick lookups, translation, brainstorming, journaling and thinking out loud, and everything you do without a connection.
Reach for the cloud when: the problem is genuinely hard, the document is genuinely long, the knowledge is genuinely specialist, you need current information, or you need tooling a chat model doesn’t have.
Then apply one rule at the boundary: before pasting into a cloud model, ask whether you would be comfortable with that text existing on someone else’s infrastructure indefinitely. Often the answer is fine. When it isn’t, that is exactly the task the local model is for.
Most people find the split lands around 80/20 — and it is the 80 that contains almost everything personal.
Download Cloaked on the App Store and try the local half of that split — free, no account, and fully functional offline.
Frequently asked questions
Is a local LLM as good as ChatGPT?
Not on the hardest tasks. On everyday writing, summarizing, rewriting, translating, and general questions, a modern 2–4B local model is close enough that most users would struggle to identify which produced which answer in a blind test.
What can ChatGPT do that a local model can't?
Reason reliably through long multi-step problems, handle documents of hundreds of thousands of tokens, draw on far deeper specialist and obscure knowledge, and answer questions about recent events. It also has tooling — code execution, image generation, and search — that a standalone local model does not.
What can a local model do that ChatGPT can't?
Work with no connection, guarantee that your conversation is never transmitted, run without an account or subscription, and stay available when a service is down, rate-limited, or unavailable in your country.
Should I use both?
For most people that is the right answer. A local model as the default for everyday and personal work, and a frontier model for the small number of genuinely hard tasks. That keeps the sensitive majority of your conversations off other people's servers.