A 27B Model Beat the Frontier on the Only Task I Care About
Holo4's 27B open model did my classification job better than the flagship, because the job wasn't hard, it was specific.
I benchmarked a flagship against Holo4, a 27-billion-parameter open computer-use model, on the one task that matters to me: routing incoming support messages to the right folder. The frontier was more fluent and worse at the rule, because it kept 'helpfully' rewriting the message instead of just labeling it. The small model did exactly the instruction and nothing else.
The lesson wasn't that small models are better. It's that my task rewards obedience over eloquence. The frontier's tendency to add value is a bug when I want a label. A model sized for the job, with less room to improvise, beat the one with the higher score on a leaderboard I don't care about.
I now run the 27B locally for this and a few sibling tasks. Latency is lower because it's on my machine, cost is zero after the download, and the behavior is predictable. The flagship still handles the open-ended questions where its flexibility helps.
If your AI task is specific and repeatable, test a small open model before you pay for the frontier. You might find, as I did, that the model everyone ignores does the work better because it's not trying to impress you.