Which AI Model Actually Makes Sense for Running a Business

Four assistants, judged on business work rather than benchmarks.

If I were to ask you "What AI model are you running in your business?" what you say tells me a lot about how 'AI Literate' you are.

Oh, btw... AI literate means how informed are you on the AI models you're using, and what makes them work well.

If you answered this by saying 'I use Claude' or 'I use ChatGPT'... but you're still looking to figure out the magic ingredient you need to actually make AI work in your business... you might want to keep reading.

To Get Started, Ignore All Benchmarks

Yuhp. You heard me.

Look, I've been working with AI for a long time and the first lesson I had to learn the hard way was that all these fancy benchmarks tell you very little about how well that model will do in your business.

Instead, it's more worthwhile to test, deploy, fail, iterate and retest. Because every business is different and every business has different goals.

That's not to say that benchmarks tell you nothing, quite the contrary. Benchmarks are effectively a north star that tell you what you might be able to expect in specific categories of work. But most people don't split up their work into a bunch of different micro tasks... at least not when they are first getting started.

So when I first get started with a new client, I ask them to start by showing me what they are currently doing with AI. I don't ask them what model they are using. Because the model is like 10% of the equation.

What matters a lot more than the model is how effective is it at achieving your goals?

The Dirty Secret of Most AI Automations

The reason that's my first question is because most AI systems have a dirty secret.

For the most part, intelligence is interchangeable.

Really!

Maybe an easier way to break this down is to break it down by intelligence level.

At the frontier, we have the fastest, smartest, most capable models on the market. ChatGPT Astra, Claude Fable 5.1, GLM 5. But these models got so good and so smart last generation that for most business utility, using the most capable and most advanced model for every task is just a waste of tokens (and time).

In contrast, smaller models are actually pretty smart for most things. Unless you're doing deep research or actually coding a project, there's no reason not to use an older, or even smaller model.

Let's break down when to use Frontier Intelligence VS a last gen model.

When to Use Frontier Intelligence VS a Last Gen Model

I have a blog automation that I spent over a year and a half getting just right for one of my clients. It's got a lot of moving parts but the biggest takeaway I learned from that process is that I don't want frontier intelligence writing blog posts about real estate.

In my experience, Opus and Astra had a tough time being creative and finding nuance (and even following my copywriting rules). But when I switched to Sonnet or even MiniMax 2.7 (an even dumber model from another Chinese provider) I got MUCH better creative.

Why Smaller Models Sometimes Outperform Frontier Models

In this process, I found out that smaller models are better for small, lean tasks... like writing!

But something a bit more complex, like the actual research, data extraction, and coding of the blog post... all of those tasks I gave to a more frontier model.

And the difference actually came down to qualitative vs quantitative quality assessments of LLMs. Let's get into what that means.

Qualitative VS Quantitative AI Assessments

In my business, I have found there are basically two ways of measuring AI performance. With Qualitative Assessments, and Quantitative Assessments.

Another way to say this is 'Deterministic VS Non-Deterministic' assessments. Let's break those down really quick.

Quantitative AI Assessments (deterministic tests)

Quantitative AI assessments are tests you can give to an AI that it can hit a perfect 100% on every time.

Most of the time, these are coding tasks.

But Preston... AI is non-deterministic so it can never reproduce the same word over and over again.

Yes, you're right.

But it can design its own tests with code that will always either pass or fail.

See, code is deterministic. The code either runs, or it doesn't! What is considered clean code roughly varies person to person, but the results are undeniable.

So Quantitative AI assessments, or deterministic tests, are tests that the AI can get a pass or fail grade. These tasks are the ones that frontier models are really good for.

Some great examples of Quantitative Tasks are:

1. Website Updates
2. Code review
3. App development
4. Invoice Checking

In each of these tasks, there's a clear "win" and a clear "loss". If an AI failed to push a website update, it will be a clear fail. It doesn't show on the live page. If an invoice failed to send, an AI can validate the data on whether or not the invoice sent.

These tasks are deterministic, and therefore I usually use a VERY capable model with them so there's little room for interpretation.

Qualitative AI Assessments (non-deterministic tests)

By extension, qualitative assessments are the opposite of quantitative. They are tasks where there's no clear 'pass/fail' grade. These tasks are tasks where the task itself should ALWAYS be assessed by a human before being pushed.

Qualitative assessments are more accurately described as 'judgement based tests'.

Some great examples of qualitative tasks are:

1. Writing
2. Data analysis
3. Image editing
4. Image generation

Writing is maybe the most obvious example. Who among us has not seen something obviously AI generated? "If it's not perfect, it's imperfect. And it's real." But here's the hard part: AI fails to write because it has no taste! Not because it's not capable of writing. It clearly is! It has its own obvious writing style that we can all immediately call out.

But writing well is the hardest part of writing. Realistically, there's a finite amount of text you can generate in a given language that actually would read well. So AI has to lean on sentence structures that it knows technically get the job done.

So How Do You Get AI to Write Well?

You have to give it your judgement... by designing a judgement based test.

Unfortunately, this might be the hardest part of making an AI work well for any business. LLMs have no taste, and there's no shortcut to train them to have taste.

The best way to do it (in my experience) is to write... a LOT. Write so many different pieces of content with all the different ways you want to write and all the different ways you organize your thoughts.

And over time, you build up a repository of 'Correct' and 'Incorrect' ways of writing.

That repository of the 'correct' and 'incorrect' ways of writing is how you build a non-deterministic test that comes away with great quality results.

This is why nobody can seem to make a copywriting tool that actually works well.

It needs to be trained on much more than your website and your social media. It needs to be trained on the JUDGEMENT that made your website and social media the way they are.

Trust me, I spent over 2 years trying to develop my own AI copywriting tool. No matter what I did, unless you train on JUDGEMENT rather than PRINCIPLES, you will always have a model that fails a qualitative test.

So What AI Models Are the Best for General Purpose Business?

I basically wrote this entire article to tell you there's no 'one perfect model for everything', but you came here for an answer so I'll give you some.

Anthropic's models are my favorite.

In my personal experience, Anthropic is top of the charts right now for general purpose business use. Its frontier models like Fable and Opus are amazing for code and data management, and its budget models like Haiku and Sonnet are the ones I reach for when I need good creative.

I use them at my day job, and I use them in my personal business every single day. I find them incredibly reliable, and they seem to hallucinate less often than other models from other providers without getting lazy and giving up halfway through a task.

Best Local Models for General Purpose Business

Personally, I think Qwen models with around 30B parameters are the best all-arounders on the market for local AI. But as I covered in why I run some AI locally and some in the cloud, the harness and tools you bake into it matter way more than how big or smart the model is.

Best Alternatives to Claude and ChatGPT?

If you don't mind your data being harvested by the Chinese government, the best alternatives to ChatGPT and Anthropic seem to be coming from Zhipu right now, with their GLM models being amazing for coding, development, writing, and most other use cases I see most businesses running into these days. I use them currently on my own system and they seem to do a great job.

If you don't want your data going to China and are willing to send your data to Facebook instead, I've been very impressed by Meta Muse. It's friendly, helpful and what it is able to do with a free plan is very impressive. For any solopreneur looking for an AI assistant, this is the one for you as long as you don't care that all your data that runs through it goes to Meta for free.

That's always the tradeoff, isn't it.

Thanks for Making It This Far

Most people don't read to the end of a post like this one, so genuinely, thank you.

If any of this sounds like the problem you are sitting with, I offer SEO, marketing, and AI automation services.

I only accept a few new clients every year so I can keep an ownership level attitude to the clients I do have. But I have a few spots available for this year.

The goal is always the same. Help you master the fundamentals of marketing in your own business, so they keep working whether I am there or not.

If you're interested in working with me, schedule a time to meet and discuss your business goals, and we can see if there's a good fit.

Signing off,

Preston