Why I Run Some AI Locally and Some in the Cloud

The machine on the desk and the cloud behind it, doing different jobs.

AI Does Not Give You More Time

I want to start here because it is the claim that sells most AI tooling and it is not true.

Nothing I have automated gave me back an hour I could spend somewhere else in any simple sense. The work expanded to fill the space, the way work does.

What it genuinely bought me is the ability to stop dreading a category of task. Content briefs, first drafts, reformatting, the tedious middle of a job. That is a real benefit and it is worth paying for, and it is a different claim from "save ten hours a week".

The Split I Actually Use

The line is not about difficulty in the abstract. It is about whether the task has a known shape.

Local handles the shaped work. Content briefs. First drafts against an outline I already wrote. Summarising, extracting, classifying, reformatting. Anything where I know what the output should look like and I just do not want to produce it by hand.

The cloud handles the unbounded work. Anything where the shape of the answer is part of the problem. Debugging something non-obvious, reasoning across a large codebase, building anything with real complexity, or any task where being subtly wrong would be expensive.

The practical test I use: if I can describe the finished thing in a sentence, local is probably fine. If I cannot, I reach for a frontier model.

Why Anything Runs Locally at All

Two reasons, and cost is the smaller one.

The first is data. Some things should not leave the building, and once you are handling client records or anything under a confidentiality agreement, "I pasted it into a chat window" is not an answer you want to give. A local model makes that question disappear.

The second is that the boring work is high volume. Running a hundred content briefs through a hosted API adds up in a way that running them on hardware you already own does not.

What Local Models Are Not

They are not Claude Code. I want to be blunt about this because the gap gets glossed over constantly.

Local models have improved enormously and the small ones are genuinely impressive for their size. They are also still meaningfully behind the best hosted models on hard reasoning, long context, and sustained multi-step work.

They also need babysitting. A frontier model will usually tell you when it is unsure. A small local model will produce something confident and wrong at roughly the same rate it produces something confident and right, and telling those apart is your job.

And running one well is not free. Once you account for hardware that can hold a useful model in memory at usable speed, you are into real money, not a weekend project.

How to Decide for Your Own Business

Ask three questions in order.

Does this data have rules attached? If yes, that pushes local regardless of everything else.

Do I know what the output should look like? If yes, local is likely enough.

What does being wrong cost? If the answer is "a client notices" or "we ship a bug", use the better model. The price difference is not close to the cost of the mistake.

Most small businesses will land on a mix, which is the honest answer. Run the tedious high-volume work where it is cheap and private, and pay for the good model on the handful of things that genuinely matter.

If you want help working out where that line sits for your business, that is the conversation to have.