The Mac mini in the corner

There is a Mac mini sitting on a desk in our office that I am probably way too fond of.

It has 48 gigabytes of unified memory, runs an open model called Qwen3.6-35B-A3B-MLX-4bit through oMLX, and quietly acts as our own little AI server. Our internal tools can send work to it over a secure Cloudflare tunnel and, once you ignore the cost of the computer and the electricity keeping it on, every token it generates costs us exactly zero dollars.

I love this thing.

But the funny part is that the little Mac on the desk is really the end result of us building something dramatically more complicated first.

It started with one Hermes agent

Our agent infrastructure started about as simply as it possibly could: one Hermes agent running on one machine and doing one job.

Then it worked.

And, as developers have been doing since the beginning of time, we responded to something working by making it considerably more complicated.

We wanted multiple agents doing different jobs, running independently, restarting when something broke, sharing resources without stepping on each other, and generally behaving less like a science project. Pretty quickly that meant we needed some kind of orchestration.

So naturally I ended up running Kubernetes on a Mac.

The tiny Kubernetes era

Round two was k3s running under Lima: basically lightweight Kubernetes running on Mac hardware. We used it to host a small fleet of Hermes agents.

This setup was awesome.

It cost almost nothing, gave us a real orchestration environment to beat up, and forced us to solve the problems you only discover once you stop running one process on one machine. What happens when an agent dies? Who restarts it? What happens when two things want the same resource? How do we deploy changes without manually babysitting every container?

Eventually we had enough confidence in what we were doing that it was time to move the whole thing somewhere a little more serious.

Then we went full Kubernetes

Round three was Azure Kubernetes Service.

And I mean the whole thing.

Our agents were containerized in Docker. Kubernetes handled orchestration. Helm handled deployments. Prometheus collected metrics from the agents, Grafana turned those metrics into dashboards, and suddenly we could sit there watching our little fleet of AI workers run in real time.

It was cool as hell.

It was also legitimate production infrastructure. Agents could fail and recover. Deployments were repeatable. We had visibility into what everything was doing. We learned a ton getting it there.

Then Azure sent us the bill.

Turns out we did not need Kubernetes

This is actually my favorite part of the story.

Once the excitement of having the full stack running wore off, we looked at what these agents were actually doing and asked the boring question: do we really need a Kubernetes cluster for this?

Nope.

What we actually needed was one well-run virtual machine.

So we killed the cluster.

Production moved onto a single VM running Docker Compose. We kept the containerization, the deployment discipline, the monitoring lessons, and everything else we had learned along the way. We just stopped paying for infrastructure our workload did not justify.

The monthly cost dropped to a fraction of what we were paying for AKS.

I do not consider the Kubernetes phase a mistake at all. Quite the opposite. We learned how to operate this stuff properly because we built the more complicated version. The mistake would have been keeping it forever just because we had already built it.

Meanwhile, back on the desk

Scaling production down did not mean I wanted to stop experimenting.

That is where the Mac mini found its job.

The mini became our local AI dev box. It runs oMLX, which gives us an OpenAI-compatible serving layer for models optimized for Apple silicon. Right now the model sitting behind it is Qwen3.6-35B-A3B-MLX-4bit.

That horrifying name is actually useful once you decode it. It is an open model with roughly 35 billion total parameters, only about 3 billion of which are active for each token it generates. It has also been quantized down to 4-bit weights, which is how we can comfortably cram a model that size into 48 gigabytes of unified memory on a machine small enough to hold in one hand.

And it is good. Like, surprisingly good.

Not "cancel the frontier model subscriptions" good. But absolutely good enough that there is a whole class of work where paying per token starts to feel silly.

1. 01Batch jobs, drafts, classification, extraction, and background work where we care more about volume than having the smartest model on Earth.
2. 02Model testing. When an interesting new open model ships, we can throw it on this box and see what it can actually do without spending anything per token.
3. 03Benchmarking. Our local-model benchmarking queue runs here so we can compare models against the kinds of work we actually care about.
4. 04Sensitive workloads where keeping the data inside our own environment is preferable to sending it to an external model provider.

Yes, the Mac mini is actually serving traffic

The useful part is that this is not just something I SSH into when I want to play with a model.

oMLX exposes the model through an API, so to our software it behaves a lot like any other model provider. That means an application can send a request to OpenAI, Anthropic, or the Mac mini depending on what kind of work it is doing.

That gives us something I really like architecturally: the model is disposable.

Qwen is in the slot today because it has been the best fit we have found for this particular machine. When something better comes out, we swap it. The applications calling it do not need to care very much about what is sitting behind the endpoint.

The expensive models still have a job

To be clear, this thing is not replacing Claude or ChatGPT.

When we need serious reasoning, difficult coding, complicated analysis, or the best model we can get our hands on, the frontier labs still win. Anthropic and OpenAI are doing things on enormous infrastructure that our tiny silver box obviously is not going to compete with.

But sending ten thousand boring classifications to a frontier model is a little like hiring a brain surgeon to alphabetize your filing cabinet.

The Mac mini gets the boring stuff.

And because the inference cost is effectively zero, it changes the economics of experimentation too. I can benchmark something stupid. I can run a thousand examples instead of fifty. I can leave a background job chewing through data without watching a usage meter climb.

What comes next

The current setup is deliberately boring. Production agents live on the inexpensive VM. The Mac mini handles development, experimentation, benchmarks, and the workloads we can sensibly push down to local inference. Frontier APIs handle the work that actually needs frontier intelligence.

If utilization eventually tells us we need more local compute, then we can talk about buying more hardware. But I have no interest in building a tiny data center because it would look cool sitting in the office.

Okay, that is not entirely true. It would look extremely cool.

But we are still not buying it until the numbers justify it.

For now, the whole journey has gone from one agent, to Kubernetes on a Mac, to Kubernetes in Azure, back down to one VM, with a little Mac mini in the corner quietly serving billions of parameters.

The bill went down. The capability went up. And I still get to play with the fun stuff.