On

Curious about running AI models locally on your Mac Mini M4? Many people want to test-drive a local AI setup just to see how it performs compared to cloud-based alternatives. The problem is, most guides are either overly complicated or assume you're comfortable living in Terminal. Here's what happened when Claude Cowork tackled this exact problem.

Claude initially provided a comprehensive 9-step guide covering everything: installing an agent framework, setting up Ollama, downloading a model sized appropriately for your hardware, connecting the pieces, and testing from Terminal. Thorough? Absolutely. But after reading through the entire thing, most people just... didn't do it.

This is where Claude Cowork changes things. It can handle all the setup on your local machine—basically operating under the instruction "make this work for me." What's running on the Mac now isn't the result of following that manual. It's what Cowork actually built instead.

The Original Guide's Approach

Two separate apps with manual switching between them


Hermes Agent website

The original guide recommended Hermes Agent, an open-source agent framework from Nous Research, as your thinking manager. Separately, you'd install Ollama to actually run the model—specifically Qwen 2.5 14B, because it fits nicely within Mac Mini's 16GB memory. Then you'd manually point Hermes to Ollama by entering a custom URL endpoint in the settings menu. Top it off with some Terminal commands to start chatting and a few diagnostic commands for when connections break.

Nothing particularly difficult, but you need solid Terminal skills and comfort editing configuration files. What's interesting here is the barrier to entry: it's not technical impossibility, it's friction. Agentic AI is getting serious attention lately, and guides like this one explain why adoption is still relatively limited.

The guide also includes a section on adding paid cloud models later, including an undocumented workaround for routing Claude subscriptions through developer tokens. Skip that section entirely—both in the guide and in any actual setup. It routes access through an undocumented token instead of a proper API, which is asking for trouble.

What Cowork Built Instead

A shorter path to the same results


Initial documentation for setting up AI on Mac Mini

Cowork got a single instruction: set up and configure Hermes on Mac Mini. But it skipped the Hermes Agent framework entirely. Instead, it treated Hermes as what it actually is—a model (Ollama's hermes3)—and suggested Open WebUI as the actual chat interface. Here's the practical tradeoff: Ollama has built-in web search, but it requires an ollama.com account and routes searches through their service. Open WebUI runs a lightweight local service and uses DuckDuckGo for searches without any account needed. For a Mac Mini that stays powered on regularly, Open WebUI makes more sense.

Claude can't directly type into Terminal (security feature), so it provided two commands to paste. One installs uv, a Python package manager, and another downloads and launches Open WebUI. From there, it basically runs on autopilot—Cowork opens the local page in Chrome, confirms the server's running, enables web search, and handles dozens of small browser tasks you'd normally do yourself.

What You Actually Have to Do

Passwords and one detail Cowork almost missed

Your part is genuinely minimal: paste two commands, then create an admin account for Open WebUI. Claude won't handle passwords—that's also a security measure. Everything else runs automatically whether you watch or not.

One small hiccup emerged, and Cowork caught it without being asked. The first test message returned outdated information because web search wasn't actually activated when that message was sent. The system noticed the inconsistency, double-checked the search function, and reran the test, this time returning three current sources. What builds real confidence in this process isn't that it skips Terminal work—it's that it validates its own results and fixes mistakes before handing them back to you.

Open WebUI only stays running while its Terminal window is open. Close it, and the interface stops (though Ollama and the model keep running in the background). Cowork suggested setting up a launch agent for automatic startup at login, but that hasn't been implemented yet.

Local LLM Setup Is Now This Simple

Hermes Agent still has real advantages over what's running now: persistent memory between sessions, scheduled tasks, tool delegation—features Open WebUI lacks. If you need an agent that remembers your projects and runs things on a schedule, go back and carefully follow that 9-step guide. You'll get options like Nous Portal (paid) and the ability to add your own API keys for more powerful cloud models when you want to scale up. But for the original goal—having a working local AI you can talk to today—the streamlined setup Cowork chose works perfectly.


Related Articles